Can Data Science Predict the Stock Market?

Video Description

Subscribe to RichardOnData here: https://www.youtube.com/channel/UCKPyg5gsnt6h0aA8EBw3i6A?sub_confirmation=1 In this video, I discuss one of the oldest and most popular challenges in the entire data science community as well as in the financial industry, which is predicting the stock market. A couple things are true at once: most stock trades are done automatically by bots, but also a monkey is thought to have a better chance of out-performing the market than a lot of experts. Usually when people in the data community ask this question, they are trying to predict that an individual stock will out-perform the whole market. So we can ask how close can a prediction get to a real stock's value at close over a short term, but also then if you're an investor over a one-year or even longer horizon, can you predict how a stock will perform? From a data perspective, there are a lot of data sources that can be leveraged. These "alternative" data sources include social media activity, credit card purchases, product reviews, Google search queries, etc. One study was able to predict quarterly earnings using alternative data, better than Wall Street 57% of the time. Read more here: https://blog.galvanize.com/can-data-science-predict-the-stock-market/ And see the study here: http://news.mit.edu/2019/model-beats-wall-street-forecasts-business-sales-1219 If we look at how well people do with this exercise, the results are pretty interesting. The data scientist Jere Xu published an article in Medium about his efforts using the Stocker module in Python which utilizes Artificial Neural Networks (ANN). He also experimented with regression, KNN, and Multilayer Perceptron for this problem. The results are not bad in the short term, but they have lots of variability in the long term, and it would be impractical to use them to make a concrete buy/sell decision. See his work here: https://towardsdatascience.com/how-to-use-machine-learning-to-possibly-become-a-millionaire-predicting-the-stock-market-33861916e9c5 The challenges of this problem can be described as at least 3-fold: 1) The "efficient market hypothesis" 2) It is very difficult to predict future shocks 3) Solving the problem presents a conflict of interest PayPal: [email protected] Patreon: https://www.patreon.com/richardondata BTC: 3LM5d1vibhp1F7pcxAFX8Ys1DM6XLUoNVL ETH: 0x3CfC599C4c1040963B644780a0E62d45999bE9D8 LTC: MH8yPjvSmKvpmRRmufofjRB9hnRAFHfx32