Riding the Market Waves: Predicting the Ups and Downs of Stock Market
Overview
Predicting stock market trends is a complex task due to the highly stochastic nature of financial data. This project compared multiple models, including k-NN, Savitzky-Golay filter, and LSTM networks, to analyze daily stock price movements using S&P 500 data.
Background & Objectives
With daily stock data from the S&P 500 companies, sourced from Yahoo Finance, this project aimed to:
- Understand daily market dynamics by analyzing "Adj Close" price changes and volume trends.
- Classify price changes into Positive, Negative, and Neutral categories for trend analysis.
- Identify the most effective model for predicting price movements.
Results
- k-NN: Achieved 37.6% accuracy using cosine similarity with k=5.
- Savitzky-Golay Filter: Explained 99.62% of variance and achieved 45.6% accuracy with an F1-score of 0.45.
- LSTM: The best-performing model with 57.7% accuracy and an F1-score of 0.572, explaining 76.55% of variance.
- Baseline Models: Majority and Random Classifiers served as benchmarks for comparison.
Recommendations
- Incorporate additional factors like market sentiments and political events for improved predictions.
- Adjust thresholds in k-NN to address class imbalance issues.
- Refine LSTM models to reduce overfitting and improve generalization.
Conclusion
This project highlighted the challenges of predicting stock market trends and the strengths and weaknesses of various models. The findings serve as a foundation for further research in stock market analytics and emphasize the need for comprehensive data analysis.