A Data Science Central Community
This is an interesting data science conjecture, inspired by the well known six degrees of separation problem, stating that there is a link involving no more than 6 connections between any two people on Earth, say between you and anyone living (say) in North Korea.
Here the link is between any two univariate data sets…Continue
Added by Vincent Granville on September 9, 2019 at 10:30am — No Comments
The material discussed here is also of interest to machine learning, AI, big data, and data science practitioners, as much of the work is based on heavy data processing, algorithms, efficient coding, testing, and experimentation. Also, it's not just two new conjectures, but paths and suggestions to solve these problems. The last section contains a few new, original exercises, some with solutions, and may be useful to students, researchers, and instructors offering math and statistics classes…Continue
Added by Vincent Granville on September 8, 2019 at 4:09am — No Comments
Machine learning is a hot topic in research and industry, with new methodologies developed all the time. The speed and complexity of the field makes keeping up with new techniques difficult even for experts — and potentially overwhelming for beginners.
To demystify machine learning and to offer a learning path for those who are new to the core…Continue
Added by Vincent Granville on August 30, 2019 at 11:08am — No Comments
I introduce here a family of very peculiar statistical distributions governed by two parameters: p, a real number in [0, 1], and b, an integer > 1.
Potential applications are found in cryptography, Fintech (stock market modeling), Bitcoin, number theory, random number…Continue
Added by Vincent Granville on August 30, 2019 at 10:11am — No Comments
Continued fractions are usually considered as a beautiful, curious mathematical topic, but with applications mostly theoretical and limited to math and number theory. Here we show how it can be used in applied business and economics contexts, leveraging the mathematical theory developed for continued fraction, to model and explain natural phenomena. …Continue
Added by Vincent Granville on August 30, 2019 at 9:42am — No Comments
In my previous posts, I compared model evaluation techniques using Statistical Tools & Tests and commonly used Classification and Clustering evaluation techniques
In this post, I'll take a look at how you can compare regression models. Comparing regression models is perhaps one of the trickiest tasks to complete in the "comparing models" arena; The reason is that there are literally dozens of statistics you can calculate to compare regression models, including:
Added by Vincent Granville on August 8, 2019 at 10:37am — No Comments
A brief explanation is:
Added by Vincent Granville on August 8, 2019 at 10:29am — No Comments
Decision Trees, Random Forests and Boosting are among the top 16 data science and machine learning tools used by data scientists. The three methods are similar, with a significant amount of overlap. In a nutshell:
Added by Vincent Granville on August 8, 2019 at 10:25am — No Comments
In financial markets, two of the most common trading strategies used by investors are the momentum and mean reversion strategies. If a stock exhibits momentum (or trending behavior as shown in the figure below), its price on the current period is more likely to increase (decrease) if it has already increased (decreased) on the previous period.
When the return of a stock at time t depends in some way on the return at the previous time t-1, the returns are said to be autocorrelated. In…Continue
Added by Vincent Granville on July 8, 2019 at 10:25am — No Comments
Summary: The annual Burtch Works salary survey tells us a lot about which industries are using the most data scientists and the difference between higher and lower skilled data scientists. Salary increases show us whether demand is increasing, and finally we take a shot at determining which skills are most in demand.
What a difference a few years can make. We used to say that everyone loves a data scientist – and wants to be one. …Continue
Added by Vincent Granville on July 8, 2019 at 10:18am — No Comments
By Ajit Jaokar. This post is a part of my forthcoming book on Mathematical foundations of Data Science. In this post, we use the Perceptron algorithm to bridge the gap between high school maths and deep learning.
As part of my role as course director of the Artificial Intelligence: Cloud and Edge Computing at the University of Oxford, I see more students who are familiar with programming than with mathematics.
They have last learnt maths…Continue
Added by Vincent Granville on June 27, 2019 at 12:22pm — No Comments
Originally published in 2014 and viewed more than 200,000 times, this is the oldest data science cheat sheet - the mother of all the numerous cheat sheets that are so popular nowadays. I decided to update it in June 2019. While the first half, dealing with installing components on your laptop and learning UNIX, regular expressions, and file management hasn't changed much, the second half, dealing with machine learning, was rewritten entirely from scratch. It is amazing how things changed in…Continue
Added by Vincent Granville on June 6, 2019 at 8:27pm — No Comments
We propose simple solutions to important problems that all data scientists face almost every day. In short, a toolbox for the handyman, useful to busy professionals in any field.
1. Eliminating sample size effects. Many statistics, such as correlations or R-squared, depend on the sample size, making it difficult to…Continue
Added by Vincent Granville on June 4, 2019 at 12:00pm — No Comments
This simple introduction to matrix theory offers a refreshing perspective on the subject. Using a basic concept that leads to a simple formula for the power of a matrix, we see how it can solve time series, Markov chains, linear regression, data reduction, principal components analysis (PCA) and other machine learning problems. These problems are usually solved with more advanced matrix calculus, including eigenvalues, diagonalization, generalized inverse matrices, and other types of…Continue
Added by Vincent Granville on May 28, 2019 at 9:00pm — No Comments
We have added a new free book in our selection exclusively for DSC members. See the first entry below, to get started with machine learning with Python.
1. Book: Classification and Regression In a Weekend
This tutorial began as a series of weekend workshops created by Ajit Jaokar and Dan Howarth. The idea was to work with a specific (longish) program such that we explore as much of it as possible in one weekend. This book is an attempt to take this idea online.…Continue
Added by Vincent Granville on May 16, 2019 at 6:24pm — No Comments
We propose a simple model-free solution to compute any confidence interval and to extrapolate these intervals beyond the observations available in your data set. In addition we propose a mechanism to sharpen the confidence intervals, to reduce their width by an order of magnitude. The methodology works with any estimator (mean, median, variance, quantile, correlation and so on) even when the data set violates the classical requirements necessary to make traditional statistical techniques…Continue
Added by Vincent Granville on May 9, 2019 at 11:30am — No Comments
This crash course features a new fundamental statistics theorem -- even more important than the central limit theorem -- and a new set of statistical rules and recipes. We discuss concepts related to determining the optimum sample size, the optimum k in k-fold cross-validation, bootstrapping, new re-sampling techniques, simulations, tests of hypotheses, confidence intervals, and statistical inference using a unified, robust, simple…Continue
Added by Vincent Granville on May 4, 2019 at 12:30pm — No Comments
So many fascinating and deep results have been written about the number (1 + SQRT(5)) / 2 and its related sequence - the Fibonacci numbers - that it would take years to read all of them. This number has been studied both for its applications (population growth, architecture) and its mathematical properties, for over 2,000 years. It is still a topic of active research.…Continue
Added by Vincent Granville on April 25, 2019 at 7:30am — No Comments
Summary: Finally there are tools that let us transcend ‘correlation is not causation’ and identify true causal factors and their relative strengths in our models. This is what prescriptive analytics was meant to be.
Just when I thought we’d figured it all out,…Continue
Added by Vincent Granville on April 24, 2019 at 7:30pm — No Comments
I describe here the ultimate number guessing game, played with real money. It is a new trading and gaming system, based on state-of-the-art mathematical engineering, robust architecture, and patent-pending technology. It offers an alternative to the stock market and traditional gaming. This system is also far more transparent than the stock market, and can not be manipulated, as formulas to win the biggest returns (with real money) are made public. Also, it simulates a neutral,…Continue
Added by Vincent Granville on April 15, 2019 at 10:00am — No Comments