diff --git a/README.md b/README.md index a18db60..0cf1fe9 100644 --- a/README.md +++ b/README.md @@ -1,160 +1,135 @@ -# BTC ML Trading Strategy Optimizer +# Bitcoin Accumulation Zone Monitor -An automated optimization loop that trains ML models on BTC/USDT data, backtests trading strategies, and uses an LLM to iteratively improve the configuration. +> On-chain metrics dashboard with historical backtesting for long-term BTC holders. No ML, no black box — pure signal monitoring from proven indicators. + +![Dashboard](screenshots/dashboard-main.png) + +## What It Does + +Monitors 10 proven Bitcoin on-chain metrics that have historically identified optimal buying zones for long-term holders. Each metric scores 0-10, producing a composite accumulation score of 0-100. + +**Current reading example:** Fear & Greed at 11 (Extreme Fear), MVRV Z-Score at 0.52 (undervalued), Puell Multiple at 0.66 — the kind of conditions that preceded every major BTC rally. + +## Screenshots + +### Main Dashboard +![Main Dashboard](screenshots/dashboard-main.png) +*Live accumulation score with all 10 metrics, current BTC price, and individual metric breakdowns* + +### Historical Backtest +![Backtest](screenshots/dashboard-backtest.png) +*Historical score vs BTC price overlay, score bracket performance table, and major signal events* + +### Settings +![Settings](screenshots/dashboard-settings.png) +*LLM provider configuration for optional AI-powered signal commentary* + +## Metrics + +| # | Metric | Source | Accumulation Signal | +|---|--------|--------|-------------------| +| 1 | Fear & Greed Index | alternative.me API | Extreme Fear (< 10) | +| 2 | Puell Multiple | LookIntoBitcoin (scraped) | Miner capitulation (< 0.5) | +| 3 | MVRV Z-Score | LookIntoBitcoin (scraped) | Below realized value (< 0) | +| 4 | Drawdown from ATH | Calculated | Deep correction (> 50%) | +| 5 | Price vs 200W SMA | LookIntoBitcoin (scraped) | Below 200-week average | +| 6 | Reserve Risk | LookIntoBitcoin (scraped) | High holder confidence (< 0.002) | +| 7 | RHODL Ratio | LookIntoBitcoin (scraped) | Long-term holder dominance (< 100) | +| 8 | NUPL | LookIntoBitcoin (scraped) | Market capitulation (< 0) | +| 9 | LTH Realized Price | LookIntoBitcoin (scraped) | Price below LTH cost basis | +| 10 | Hash Ribbons | LookIntoBitcoin (scraped) | Miner capitulation recovery | + +## Score Interpretation + +| Score | Assessment | Historical Outcome | +|-------|-----------|-------------------| +| 85-100 | 🟢 Extreme Accumulation | Rare (~4x per decade). Historically: 200%+ 1yr returns | +| 70-84 | 🟢 Strong Accumulation | Excellent long-term entry point | +| 55-69 | 🟡 Moderate Opportunity | Decent entry, DCA appropriate | +| 40-54 | 🟡 Neutral | Hold — not compelling either way | +| 25-39 | 🔴 Caution | Market heating up | +| 0-24 | 🔴 Extreme Caution | Historically worst times to buy | + +## Tech Stack + +| Component | Technology | +|-----------|-----------| +| Backend | Python 3.13 + FastAPI | +| Frontend | Inline HTML/CSS/JS (dark trading terminal theme) | +| Charts | Chart.js | +| Scraping | Playwright (headless Chromium) | +| Data APIs | alternative.me (F&G), CoinGecko (price) | +| Process Manager | pm2 | +| Port | 3088 | + +## How Data Is Collected + +All data is scraped from free, public sources — **no API keys required**. + +LookIntoBitcoin charts use Plotly Dash. We intercept the chart data XHR response which contains full historical time series (5000+ points back to 2010). The scraper runs every 15 minutes for live data and weekly for full historical updates. + +## Project Structure + +``` +├── dashboard/ +│ └── server.py # FastAPI server + inline dashboard HTML +├── scrapers/ +│ ├── lookintobitcoin.py # Playwright scraper for on-chain charts +│ ├── history_collector.py # Full historical data collection +│ ├── fear_greed.py # Fear & Greed Index API +│ └── price.py # BTC price API +├── scoring/ +│ └── engine.py # Metric scoring logic (0-10 per metric) +├── backtesting/ +│ └── engine.py # Historical backtest engine +├── data/ +│ ├── cache.json # Live metric cache (auto-generated) +│ └── history.json # Historical data (auto-generated) +├── config/ +│ └── thresholds.json # Scoring thresholds (customizable) +├── screenshots/ # Dashboard screenshots +├── ARCHITECTURE.md # Detailed architecture & scoring logic +└── README.md +``` + +## Running + +```bash +# Install dependencies +pip install fastapi uvicorn playwright requests + +# Install Playwright browsers (first time only) +playwright install chromium + +# Start the dashboard +cd /opt/apps/btc-ml-optimizer +python3 -m uvicorn dashboard.server:app --host 0.0.0.0 --port 3088 + +# Or with pm2 +pm2 start "python3 -m uvicorn dashboard.server:app --host 0.0.0.0 --port 3088" --name btc-ml-optimizer +``` + +### First Run +1. Visit `http://localhost:3088` — the dashboard will auto-scrape current metrics +2. Visit `http://localhost:3088/backtest` — triggers historical data collection (takes ~5 min first time) +3. Data auto-refreshes every 15 minutes after initial scrape + +## Backtest Methodology + +The backtest engine reconstructs the composite score for every historical day and compares against actual BTC forward returns. + +**Key feature: Recency weighting** — Bitcoin's cycle returns diminish over time (100x → 30x → 8x → 3-4x). The backtest weights recent cycles more heavily: +- 2022-present: 4x weight +- 2020-2021: 3x weight +- 2018-2019: 2x weight +- Pre-2018: 1x weight + +Results are shown per-cycle so you see realistic expectations for the current cycle, not averages inflated by early moonshots. ## Architecture -``` -┌─────────────────────────────────────────────────────────────────┐ -│ Optimization Loop │ -│ │ -│ ┌──────────┐ ┌───────────────┐ ┌──────────────────────┐ │ -│ │ VPS │───>│ Windows PC │───>│ Mac Mini │ │ -│ │ (Orch.) │<───│ (GPU/ML) │ │ (LLM) │ │ -│ │ │<───────────────────────>│ │ │ -│ │ - Fetch │ │ - XGBoost │ │ - Ollama │ │ -│ │ data │ │ - LightGBM │ │ - qwen3.5:27b │ │ -│ │ - Coord │ │ - CatBoost │ │ - Analyze results │ │ -│ │ - Store │ │ - RTX 4070 Ti │ │ - Suggest changes │ │ -│ └──────────┘ └───────────────┘ └──────────────────────┘ │ -│ ▲ │ │ -│ └────────────────────────────────────────┘ │ -│ Modified config │ -└─────────────────────────────────────────────────────────────────┘ -``` +See [ARCHITECTURE.md](ARCHITECTURE.md) for detailed documentation of every metric's scoring logic, data pipeline, and backtest methodology. -### Machines (Tailscale) +## License -| Machine | Role | Address | Key Resources | -|------------|-------------|-------------------|---------------------| -| VPS | Orchestrator | localhost | Coordination, data | -| Windows PC | ML Engine | 100.76.218.38 | RTX 4070 Ti GPU | -| Mac Mini | LLM | 100.100.242.21 | Ollama, qwen3.5:27b | - -## Directory Structure - -``` -btc-ml-optimizer/ -├── orchestrator.py # Main loop — coordinates everything -├── ml_engine/ -│ └── train_and_backtest.py # Self-contained ML script (runs on Windows) -├── llm_client/ -│ └── analyzer.py # LLM strategy analyzer (calls Mac Mini) -├── scripts/ -│ ├── fetch_data.py # BTC/USDT data fetcher (ccxt) -│ └── setup_windows.sh # Install deps on Windows PC -├── config/ -│ └── initial_config.json # Starting configuration -├── data/ # OHLCV CSV files -├── results/ # Iteration results + logs -├── requirements_vps.txt # VPS Python dependencies -└── requirements_windows.txt # Windows PC Python dependencies -``` - -## Setup - -### 1. VPS (this machine) - -```bash -pip install -r requirements_vps.txt -``` - -### 2. Windows PC - -```bash -# From VPS — installs all ML deps on Windows via SSH -bash scripts/setup_windows.sh -``` - -Or manually on Windows: -```bash -pip install -r requirements_windows.txt -``` - -### 3. Mac Mini - -Ensure Ollama is running with the qwen3.5:27b model: -```bash -ollama pull qwen3.5:27b -ollama serve # should already be running -``` - -## Usage - -### Fetch Data - -```bash -python3 scripts/fetch_data.py -``` - -Downloads 2 years of BTC/USDT 1h and 4h OHLCV data from Binance. - -### Run the Optimizer - -```bash -python3 orchestrator.py -``` - -The optimizer will: -1. Ensure data is fetched -2. Upload ML engine + data to Windows PC -3. Train model and backtest on GPU -4. Send results to LLM for analysis -5. Apply LLM-suggested config changes -6. Repeat until convergence (or 50 iterations) - -### Run ML Engine Standalone (on Windows) - -```bash -python train_and_backtest.py --config config.json --data btc_4h.csv --output results.json -``` - -## Configuration Reference - -### `model_type` -- `xgboost` — XGBoost with GPU (default, generally best) -- `lightgbm` — LightGBM with GPU (faster training) -- `catboost` — CatBoost with GPU (handles interactions well) -- `ensemble` — Soft voting of all three - -### `features` -- `technical_indicators` — List of indicators to compute -- `lookback_periods` — Windows for return/volatility features -- `use_volume_features` — Include volume-derived features -- `use_volatility_features` — Include volatility features -- `use_candle_patterns` — Include candlestick pattern features -- `use_lag_features` — Include lagged feature values -- `lag_periods` — Specific lag periods to use - -### `target` -- `direction` — `"long"` or `"both"` -- `horizon_candles` — Forward-looking prediction window -- `threshold_pct` — Minimum % move to label as positive - -### `hyperparameters` -Standard gradient boosting params: `learning_rate`, `max_depth`, `n_estimators`, `subsample`, `colsample_bytree`, `min_child_weight`, `gamma`, `reg_alpha`, `reg_lambda` - -### `strategy` -- `entry_threshold` — Min probability to enter trade (0.5-0.8) -- `stop_loss_pct` — Stop loss percentage -- `take_profit_pct` — Take profit percentage -- `trailing_stop_pct` — Trailing stop distance -- `position_sizing` — `"confidence_scaled"` or `"fixed"` -- `min_confidence_to_trade` — Absolute minimum confidence - -### `training` -- `walk_forward_windows` — Number of walk-forward splits (3-10) -- `train_pct` / `validation_pct` / `test_pct` — Data split ratios - -## Convergence Criteria - -The optimizer stops when: -- Sharpe ratio exceeds 3.0 -- Sharpe improvement < 1% over 5 consecutive iterations -- Maximum 50 iterations reached - -## Output - -- `config/best_config.json` — Best configuration found -- `results/iterations.jsonl` — Full log of every iteration -- `results/results_iter_N.json` — Detailed results per iteration +Private — not for public distribution. diff --git a/screenshots/dashboard-backtest.png b/screenshots/dashboard-backtest.png new file mode 100644 index 0000000..bd7bfd1 Binary files /dev/null and b/screenshots/dashboard-backtest.png differ diff --git a/screenshots/dashboard-main.png b/screenshots/dashboard-main.png new file mode 100644 index 0000000..9e0de22 Binary files /dev/null and b/screenshots/dashboard-main.png differ diff --git a/screenshots/dashboard-settings.png b/screenshots/dashboard-settings.png new file mode 100644 index 0000000..17689ee Binary files /dev/null and b/screenshots/dashboard-settings.png differ