A ground-up modernization of the classic (archived, TensorFlow 1.x) Stock-Prediction-Models repo: PyTorch forecasters, RL trading agents on a custom Gymnasium environment, and Monte Carlo market simulations — all visualized in a FastAPI dashboard. The differentiator is honest evaluation: walk-forward scoring, bootstrap confidence intervals, Diebold-Mariano significance tests, and out-of-sample backtests with transaction costs.
Most stock-prediction repos claim ~95% accuracy by leaking the future into their metrics via recursive error hiding and smoothing tricks.
Models are rarely tested against a naive baseline with statistical significance, so 'model X wins' claims carry no evidence.
Single-ticker results are cherry-picking — a strategy that shines on one stock can lose money on another.
Trading backtests routinely ignore transaction costs and test in-sample, inflating agent performance.
The reference repo everyone learns from is archived on TensorFlow 1.x with leakage-prone evaluation baked in.
System Overview
yfinance data + cache→Forecasting zoo (PyTorch/ARIMA/XGBoost)→Walk-forward evaluation with DM tests→RL & rule-based trading agents→Monte Carlo simulations→FastAPI dashboard
Forecasting zoo spans LSTM, GRU, Transformer, N-BEATS, and PatchTST in PyTorch plus ARIMA, XGBoost, and a drift baseline — all predicting next-day log returns, scored one-step-ahead walk-forward.
Statistical rigor layer attaches bootstrap 95% CIs to every RMSE and Diebold-Mariano tests (HLN-corrected) against the drift baseline, reported across three sectors (GOOG, JPM, XOM); the honest finding is that no model beats drift at p < 0.05.
Trading agents — DQN and PPO via stable-baselines3 on a custom Gymnasium environment, an evolution-strategy agent, and rule-based baselines — train on the first 80% of history and backtest out-of-sample with 10 bps costs per side.
Simulation module runs Monte Carlo scenarios (GBM, EWMA dynamic volatility, correlated multi-asset) and efficient-frontier portfolio optimization (random search + SLSQP).
A 22-test pytest suite guards the failure modes quant projects hide: leakage in chronological splits, cost accounting, and the statistics implementations themselves.
FastAPI dashboard visualizes forecasting tables, agent backtests, simulations, and a live quick-forecast widget; deployed on Render with bundled data for offline operation.
What I Built
A forecasting zoo — LSTM, GRU, Transformer, N-BEATS, PatchTST, ARIMA, XGBoost, and a drift baseline — predicting next-day log returns, scored one-step-ahead walk-forward.
Statistical rigor layer: bootstrap 95% CIs on every RMSE and a Diebold-Mariano test (HLN small-sample correction) against the drift baseline for every model, on three tickers from different sectors (GOOG, JPM, XOM).
RL trading agents (DQN and PPO via stable-baselines3) on a custom Gymnasium environment, plus an evolution-strategy agent and rule-based baselines (turtle, SMA crossover, RSI).
Cost-aware backtesting: agents train on the first 80% of history and are evaluated out-of-sample on the last 20% with 10 bps fees per side — the evolution-strategy agent was profitable on 3/3 tickers.
Monte Carlo simulations (GBM, EWMA dynamic volatility, correlated multi-asset) and efficient-frontier portfolio optimization (20k random portfolios + SLSQP).
A FastAPI dashboard visualizing forecasting tables with CIs and p-values, agent backtests, simulations, and a live ~2s quick-forecast widget.
A 22-test pytest suite guarding the two places quant projects silently lie: data leakage (poisoning a future value leaves earlier predictions bit-identical) and cost accounting (exact cash arithmetic with fees).
Live Render deployment via render.yaml — slim runtime build with bundled price data so the demo works where Yahoo Finance blocks cloud IPs.
Screenshots
Forecasting dashboard — walk-forward RMSE with bootstrap 95% CIs and Diebold-Mariano p-values per modelTrading agents dashboard — out-of-sample backtests with transaction costs across GOOG, JPM, and XOMLive quick-forecast widget — ARIMA forecast for any ticker in about 2 secondsEfficient frontier over a 5-asset universe — 20k random portfolios with the SLSQP max-Sharpe optimum
Key Decisions & Tradeoffs
Publish the honest result: across 3 tickers × 7 models, not one forecaster beats the drift baseline at p < 0.05 — exactly what an efficient market predicts for daily data, and the opposite of the inflated claims this project set out to correct.
Report on three tickers from different sectors rather than one cherry-picked chart — turtle breakout gains +25.4% on GOOG and loses money on JPM.
Keep the original repo's one-unit position sizing so all agents are comparable, and explain why that structurally caps ROI versus buy-and-hold rather than hiding the gap.
Label 30-day recursive forecasts as scenarios, not predictions, since uncertainty compounds.
Enforce evaluation integrity in CI: leakage tests assert chronological splits are disjoint and walk-forward never fits on test days.
Why It Matters
It demonstrates that rigorous ML evaluation — significance tests, out-of-sample backtests, leakage tests — matters more than model architecture, and has the integrity to report a null forecasting result instead of a fabricated 95%.
What I'd Improve Next
Add exogenous signals (macro indicators, news sentiment, options-implied volatility) to test whether any information source beats the drift baseline on daily data.
Move to intraday or weekly horizons where predictability structure may differ from the near-efficient daily close.
Give trading agents realistic position sizing (fractional capital allocation) instead of the one-unit convention so returns are comparable to buy-and-hold.
Add walk-forward retraining and regime-detection for the RL agents, which currently show instability across market regimes.
Extend statistical testing with multiple-comparison corrections and probabilistic (quantile) forecasts with calibration checks.
