diff --git a/README.md b/README.md index 0cf1fe9..a42fcc6 100644 --- a/README.md +++ b/README.md @@ -1,134 +1,221 @@ # Bitcoin Accumulation Zone Monitor -> On-chain metrics dashboard with historical backtesting for long-term BTC holders. No ML, no black box — pure signal monitoring from proven indicators. +> Bitcoin on-chain metrics dashboard with classic equal-weight scoring, ML-optimized scoring, historical backtesting, and click-to-select metric context for long-term BTC accumulation decisions. ![Dashboard](screenshots/dashboard-main.png) ## What It Does -Monitors 10 proven Bitcoin on-chain metrics that have historically identified optimal buying zones for long-term holders. Each metric scores 0-10, producing a composite accumulation score of 0-100. +Monitors Bitcoin accumulation conditions using 16 scored market/on-chain indicators plus optional informational cycle metrics. Each scored metric receives a 0-10 score and rolls into a 0-100 accumulation score. -**Current reading example:** Fear & Greed at 11 (Extreme Fear), MVRV Z-Score at 0.52 (undervalued), Puell Multiple at 0.66 — the kind of conditions that preceded every major BTC rally. +The dashboard now supports two scoring modes: + +- **Classic** — transparent equal-weight scoring across every active metric. +- **ML** — feature-importance weights trained against historical 365-day forward returns, with displayed per-metric weights and point contributions. + +Historical backtests show score-vs-price behavior, score bracket performance, major signal events, and current-score context. Metric cards are clickable: selecting a metric overlays its historical series on the score chart and shows comparable historical periods with forward returns. ## Screenshots -### Main Dashboard +### Main Dashboard — ML mode + metric context ![Main Dashboard](screenshots/dashboard-main.png) -*Live accumulation score with all 10 metrics, current BTC price, and individual metric breakdowns* +*Live BTC price, Classic/ML scoring toggle, 16 active scored metrics, ML weights/contributions, metric sparklines, and click-to-select historical context.* ### Historical Backtest ![Backtest](screenshots/dashboard-backtest.png) -*Historical score vs BTC price overlay, score bracket performance table, and major signal events* +*Current signal percentile, comparable historical periods by cycle, score-vs-BTC chart, bracket performance, and major signal events.* ### Settings ![Settings](screenshots/dashboard-settings.png) -*LLM provider configuration for optional AI-powered signal commentary* +*LLM provider configuration for optional AI-powered signal commentary and local/cloud model selection.* + +## Feature Highlights + +- **16 scored metrics** from market sentiment, miner stress, valuation, holder behavior, network activity, and velocity signals. +- **Classic vs ML scoring toggle** on the dashboard and backtest API. +- **ML score explainability**: metric cards show learned weight and contribution in points. +- **Leakage-resistant ML validation**: training uses purged time-series splits so 365-day forward-return labels do not overlap validation windows. +- **Historical context panel**: compares the current composite score against historical periods and forward returns. +- **Clickable metric cards**: select any metric to see percentile, similar historical levels, forward returns, example dates by market cycle, and highlighted chart periods. +- **Score history chart** with BTC price overlay, range controls, and selected-metric overlay. +- **Backtest dashboard** with current signal context, score bracket performance, and signal-crossing events. +- **Quick vs full refresh**: quick refresh updates BTC price and Fear & Greed; full refresh re-scrapes on-chain sources. +- **LLM settings UI** for Ollama, LM Studio, OpenAI, Anthropic, and OpenRouter. ## Metrics | # | Metric | Source | Accumulation Signal | |---|--------|--------|-------------------| -| 1 | Fear & Greed Index | alternative.me API | Extreme Fear (< 10) | -| 2 | Puell Multiple | LookIntoBitcoin (scraped) | Miner capitulation (< 0.5) | -| 3 | MVRV Z-Score | LookIntoBitcoin (scraped) | Below realized value (< 0) | -| 4 | Drawdown from ATH | Calculated | Deep correction (> 50%) | -| 5 | Price vs 200W SMA | LookIntoBitcoin (scraped) | Below 200-week average | -| 6 | Reserve Risk | LookIntoBitcoin (scraped) | High holder confidence (< 0.002) | -| 7 | RHODL Ratio | LookIntoBitcoin (scraped) | Long-term holder dominance (< 100) | -| 8 | NUPL | LookIntoBitcoin (scraped) | Market capitulation (< 0) | -| 9 | LTH Realized Price | LookIntoBitcoin (scraped) | Price below LTH cost basis | -| 10 | Hash Ribbons | LookIntoBitcoin (scraped) | Miner capitulation recovery | +| 1 | Fear & Greed Index | alternative.me API | Extreme fear / capitulation sentiment | +| 2 | Puell Multiple | LookIntoBitcoin | Miner revenue stress | +| 3 | MVRV Z-Score | LookIntoBitcoin | Market near/below realized value | +| 4 | Drawdown from ATH | Calculated from BTC price | Deep correction from cycle high | +| 5 | Price vs 200W SMA | LookIntoBitcoin + BTC price | Price near/below long-term trend | +| 6 | Reserve Risk | LookIntoBitcoin | High holder confidence relative to price | +| 7 | RHODL Ratio | LookIntoBitcoin | Long-term holder dominance | +| 8 | Net Unrealized Profit/Loss (NUPL) | LookIntoBitcoin | Capitulation / early recovery zones | +| 9 | LTH Realized Price | LookIntoBitcoin | Price near long-term holder cost basis | +| 10 | Hash Ribbons | LookIntoBitcoin | Miner capitulation/recovery signal | +| 11 | SOPR | CheckOnChain | Spent outputs near loss / reset territory | +| 12 | Sell-side Risk Ratio | CheckOnChain | Low realized profit/loss pressure | +| 13 | Active Address Momentum | CheckOnChain | Network activity momentum extremes | +| 14 | Transaction Count Momentum | CheckOnChain | Transaction activity momentum extremes | +| 15 | NVT Price | CheckOnChain | Network-value valuation discount/premium | +| 16 | VDD Multiple | CheckOnChain | Coin-days/velocity reset conditions | + +Informational cards may also appear when data is available, such as **Long-Term Holder Supply** and **Pi Cycle Bottom**. These are displayed for context and are not included in the composite score. ## Score Interpretation -| Score | Assessment | Historical Outcome | -|-------|-----------|-------------------| -| 85-100 | 🟢 Extreme Accumulation | Rare (~4x per decade). Historically: 200%+ 1yr returns | -| 70-84 | 🟢 Strong Accumulation | Excellent long-term entry point | -| 55-69 | 🟡 Moderate Opportunity | Decent entry, DCA appropriate | -| 40-54 | 🟡 Neutral | Hold — not compelling either way | -| 25-39 | 🔴 Caution | Market heating up | -| 0-24 | 🔴 Extreme Caution | Historically worst times to buy | +| Score | Assessment | Interpretation | +|-------|-----------|----------------| +| 80-100 | 🟢 Extreme Accumulation Zone | Broad capitulation/value conditions across active metrics | +| 65-79 | 🟢 Strong Accumulation Zone | Historically attractive long-term entry territory | +| 50-64 | 🟡 Moderate Opportunity | DCA-friendly, but not maximum-signal conditions | +| 35-49 | 🟡 Neutral | Mixed signals; not compelling either direction | +| 20-34 | 🔴 Caution — Overheated | Market conditions becoming less favorable | +| 0-19 | 🔴 Extreme Caution | Historically poor accumulation setup | + +Backtest tables provide actual historical forward-return statistics per score bracket, including 30d/90d/180d/1yr averages, win rate, max gain/loss, and average max drawdown. + +## ML-Optimized Scoring + +The ML mode uses a `GradientBoostingClassifier` trained on historical feature rows to predict whether a day was a good long-term buy based on 365-day forward return. Training features include: + +- Classic metric scores. +- Raw metric values. +- 30-day metric deltas. +- Interaction features such as MVRV × NUPL and Puell × Reserve Risk. +- Cycle-position context such as days since ATH. + +The resulting feature importances are aggregated back into transparent metric weights stored in `config/ml_weights.json`. The UI displays normalized weight and contribution for each active metric. + +Validation uses purged expanding time-series splits: because each label uses a 365-day forward-return window, training rows whose label windows overlap validation are removed before scoring validation folds. ## Tech Stack | Component | Technology | |-----------|-----------| | Backend | Python 3.13 + FastAPI | -| Frontend | Inline HTML/CSS/JS (dark trading terminal theme) | +| Frontend | Inline HTML/CSS/JS dark trading-terminal UI | | Charts | Chart.js | -| Scraping | Playwright (headless Chromium) | -| Data APIs | alternative.me (F&G), CoinGecko (price) | -| Process Manager | pm2 | -| Port | 3088 | +| Scraping | requests + Playwright-style browser scraping where needed | +| Data APIs | alternative.me, CoinGecko, LookIntoBitcoin, CheckOnChain | +| ML | NumPy + pandas + scikit-learn GradientBoostingClassifier | +| Process Manager | pm2 or uvicorn | +| Default Port | 3088 | ## How Data Is Collected -All data is scraped from free, public sources — **no API keys required**. +Data is collected from free/public sources and cached locally under `data/`. -LookIntoBitcoin charts use Plotly Dash. We intercept the chart data XHR response which contains full historical time series (5000+ points back to 2010). The scraper runs every 15 minutes for live data and weekly for full historical updates. +- Fast live refreshes update BTC price, ATH/drawdown, 200D SMA/Mayer where possible, and Fear & Greed. +- On-chain metrics are cached and reused because they update slowly. +- Full refresh re-scrapes on-chain metrics from LookIntoBitcoin/CheckOnChain. +- Historical backtest data lives in `data/history.json` and supports charting, backtests, and metric-context lookups. +- Score history appends to `data/score_history.jsonl`. ## Project Structure ``` ├── dashboard/ -│ └── server.py # FastAPI server + inline dashboard HTML +│ └── server.py # FastAPI server + inline dashboard/backtest/settings UI ├── scrapers/ -│ ├── lookintobitcoin.py # Playwright scraper for on-chain charts -│ ├── history_collector.py # Full historical data collection -│ ├── fear_greed.py # Fear & Greed Index API -│ └── price.py # BTC price API +│ ├── lookintobitcoin.py # LookIntoBitcoin metric scraping +│ ├── checkonchain.py # CheckOnChain metric scraping +│ ├── history_collector.py # Full historical data collection +│ ├── history_updater.py # Incremental historical updates +│ ├── fear_greed.py # Fear & Greed Index API +│ └── price.py # BTC price, ATH, drawdown, SMA helpers ├── scoring/ -│ └── engine.py # Metric scoring logic (0-10 per metric) +│ └── engine.py # Classic + ML-weighted scoring logic ├── backtesting/ -│ └── engine.py # Historical backtest engine +│ └── engine.py # Historical backtest engine +├── ml/ +│ └── optimizer.py # ML training, purged CV, weight export +├── tests/ +│ ├── test_ml_optimizer_validation.py +│ └── test_scoring_engine_ml.py ├── data/ -│ ├── cache.json # Live metric cache (auto-generated) -│ └── history.json # Historical data (auto-generated) +│ ├── cache.json # Live metric cache +│ ├── history.json # Historical metric/time-series data +│ └── score_history.jsonl # Live score history ├── config/ -│ └── thresholds.json # Scoring thresholds (customizable) -├── screenshots/ # Dashboard screenshots -├── ARCHITECTURE.md # Detailed architecture & scoring logic +│ ├── thresholds.json # Classic scoring thresholds +│ ├── ml_weights.json # Learned ML metric weights +│ └── llm_settings.json # Optional AI commentary provider config +├── screenshots/ # README screenshots +├── ARCHITECTURE.md └── README.md ``` ## Running +### Local / ad-hoc with uv + ```bash -# Install dependencies -pip install fastapi uvicorn playwright requests +cd /opt/data/btc-accumulation-monitor +PYTHONPATH=. uv run \ + --with fastapi \ + --with uvicorn \ + --with requests \ + --with pandas \ + --with numpy \ + --with scikit-learn \ + python -m uvicorn dashboard.server:app --host 0.0.0.0 --port 3088 +``` -# Install Playwright browsers (first time only) -playwright install chromium +### VPS-style install -# Start the dashboard +```bash cd /opt/apps/btc-ml-optimizer -python3 -m uvicorn dashboard.server:app --host 0.0.0.0 --port 3088 +python3 -m venv .venv +. .venv/bin/activate +pip install -r requirements_vps.txt pandas numpy scikit-learn +python -m uvicorn dashboard.server:app --host 0.0.0.0 --port 3088 +``` -# Or with pm2 +### pm2 + +```bash pm2 start "python3 -m uvicorn dashboard.server:app --host 0.0.0.0 --port 3088" --name btc-ml-optimizer ``` -### First Run -1. Visit `http://localhost:3088` — the dashboard will auto-scrape current metrics -2. Visit `http://localhost:3088/backtest` — triggers historical data collection (takes ~5 min first time) -3. Data auto-refreshes every 15 minutes after initial scrape +## First Run -## Backtest Methodology +1. Visit `http://localhost:3088` for the live dashboard. +2. Use **Quick Refresh** for fast price/Fear & Greed updates. +3. Use **Full Refresh** to re-scrape on-chain metrics. +4. Visit `http://localhost:3088/backtest` to view historical score performance. +5. If historical data is missing, use the backtest page's collection flow to populate `data/history.json`. -The backtest engine reconstructs the composite score for every historical day and compares against actual BTC forward returns. +## Useful API Endpoints -**Key feature: Recency weighting** — Bitcoin's cycle returns diminish over time (100x → 30x → 8x → 3-4x). The backtest weights recent cycles more heavily: -- 2022-present: 4x weight -- 2020-2021: 3x weight -- 2018-2019: 2x weight -- Pre-2018: 1x weight +| Endpoint | Description | +|----------|-------------| +| `GET /api/data?mode=classic` | Current metrics using equal-weight scoring | +| `GET /api/data?mode=ml` | Current metrics using ML-optimized weights | +| `GET /api/history` | Recent live score history | +| `POST /api/refresh` | Quick refresh | +| `POST /api/refresh?full=true` | Full on-chain refresh | +| `GET /api/backtest?mode=classic` | Historical backtest with classic scoring | +| `GET /api/backtest?mode=ml` | Historical backtest with ML scoring | +| `GET /api/metric-context?metric=mvrv_zscore&mode=ml` | Similar historical levels and forward returns for one metric | +| `GET /api/settings` | Safe LLM settings payload | -Results are shown per-cycle so you see realistic expectations for the current cycle, not averages inflated by early moonshots. +## Testing + +Focused tests can be run with uv: + +```bash +cd /opt/data/btc-accumulation-monitor +PYTHONPATH=. uv run --with pytest --with numpy --with scikit-learn --with pandas \ + pytest -q tests/test_ml_optimizer_validation.py tests/test_scoring_engine_ml.py +``` ## Architecture -See [ARCHITECTURE.md](ARCHITECTURE.md) for detailed documentation of every metric's scoring logic, data pipeline, and backtest methodology. +See [ARCHITECTURE.md](ARCHITECTURE.md) for deeper implementation details on scoring, data collection, and backtesting. ## License diff --git a/screenshots/dashboard-backtest.png b/screenshots/dashboard-backtest.png index df22290..04858e6 100644 Binary files a/screenshots/dashboard-backtest.png and b/screenshots/dashboard-backtest.png differ diff --git a/screenshots/dashboard-main.png b/screenshots/dashboard-main.png index 9e0de22..7a4556a 100644 Binary files a/screenshots/dashboard-main.png and b/screenshots/dashboard-main.png differ diff --git a/screenshots/dashboard-settings.png b/screenshots/dashboard-settings.png index 17689ee..7a8f021 100644 Binary files a/screenshots/dashboard-settings.png and b/screenshots/dashboard-settings.png differ