fix: address pre-push review findings

- Click any metric card to see historical periods where it was at a similar level
- Purple dot highlighting on chart shows matching periods
- Metric overlay line plotted on chart (dashed purple)
- Metric Context panel shows percentile, comparable days, avg forward returns,
  and historical examples from different market cycles
- New /api/metric-context endpoint for per-metric similarity analysis
- Backtest chart_data now includes per-metric raw values
- score_day() returns raw metric values alongside scores
- Fixed JS SyntaxError from broken inline onclick escaping (uses addEventListener)

Co-Authored-By: Claude Opus 4.6 <<EMAIL>>

.dockerignore

@@ -0,0 +1,11 @@
+.git
+.gitea
+.github
+.venv
+.playwright
+__pycache__
+*.py[cod]
+*.log
+.env
+results
+screenshots

.gitea/workflows/ci.yml

@@ -0,0 +1,28 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out repository
+        uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v6
+        with:
+          version: "0.11.6"
+          enable-cache: true
+
+      - name: Validate lockfile and install dependencies
+        run: uv sync --locked --group runtime --group ml --group dev
+
+      - name: Compile Python sources
+        run: uv run --frozen python -m compileall -q dashboard scrapers scoring backtesting ml ml_engine llm_client scripts orchestrator.py
+
+      - name: Run tests
+        run: uv run --frozen pytest

.gitignore

@@ -1,7 +1,13 @@
 __pycache__/
 *.pyc
+.venv/
+.playwright/
+.pytest_cache/
 data/cache.json
 data/history.json
+data/score_history.jsonl
+data/jobs.json
+data/*.lock
 config/llm_settings.json
 results/
 *.log

Dockerfile

@@ -0,0 +1,31 @@
+# syntax=docker/dockerfile:1.7
+FROM ghcr.io/astral-sh/uv:0.11.6 AS uv
+FROM python:3.13.5-slim-bookworm
+
+ENV PYTHONDONTWRITEBYTECODE=1 \
+    PYTHONUNBUFFERED=1 \
+    PLAYWRIGHT_BROWSERS_PATH=/ms-playwright \
+    UV_COMPILE_BYTECODE=1 \
+    UV_LINK_MODE=copy
+
+COPY --from=uv /uv /uvx /bin/
+
+WORKDIR /app
+
+COPY pyproject.toml uv.lock ./
+RUN uv sync --frozen --no-install-project --no-dev --group runtime --group ml \
+    && uv run --frozen --no-dev --group runtime --group ml \
+       playwright install --with-deps chromium \
+    && chmod -R a+rX /ms-playwright
+
+COPY --chown=10001:10001 . .
+RUN mkdir -p /app/data /app/config \
+    && chown -R 10001:10001 /app/data /app/config
+
+USER 10001:10001
+EXPOSE 3088
+
+HEALTHCHECK --interval=30s --timeout=5s --start-period=30s --retries=3 \
+  CMD ["/app/.venv/bin/python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:3088/health/live', timeout=3)"]
+
+CMD ["/app/.venv/bin/python", "-m", "uvicorn", "dashboard.server:app", "--host", "0.0.0.0", "--port", "3088"]

README.md

@@ -1,160 +1,269 @@
-# BTC ML Trading Strategy Optimizer
+# Bitcoin Accumulation Zone Monitor
 
-An automated optimization loop that trains ML models on BTC/USDT data, backtests trading strategies, and uses an LLM to iteratively improve the configuration.
+> Bitcoin on-chain metrics dashboard with classic equal-weight scoring, ML-optimized scoring, historical backtesting, and click-to-select metric context for long-term BTC accumulation decisions.
+
+![Dashboard](screenshots/dashboard-main.png)
+
+## What It Does
+
+Monitors Bitcoin accumulation conditions using 16 scored market/on-chain indicators plus optional informational cycle metrics. Each scored metric receives a 0-10 score and rolls into a 0-100 accumulation score.
+
+The dashboard now supports two scoring modes:
+
+- **Classic** — transparent equal-weight scoring across every active metric.
+- **ML** — feature-importance weights trained against historical 365-day forward returns, with displayed per-metric weights and point contributions.
+
+Historical backtests show score-vs-price behavior, score bracket performance, major signal events, and current-score context. Metric cards are clickable: selecting a metric overlays its historical series on the score chart and shows comparable historical periods with forward returns.
+
+## Screenshots
+
+### Main Dashboard — ML mode + metric context
+![Main Dashboard](screenshots/dashboard-main.png)
+*Live BTC price, Classic/ML scoring toggle, 16 active scored metrics, ML weights/contributions, metric sparklines, and click-to-select historical context.*
+
+### Historical Backtest
+![Backtest](screenshots/dashboard-backtest.png)
+*Current signal percentile, comparable historical periods by cycle, score-vs-BTC chart, bracket performance, and major signal events.*
+
+### Settings
+![Settings](screenshots/dashboard-settings.png)
+*LLM provider configuration for optional AI-powered signal commentary and local/cloud model selection.*
+
+## Feature Highlights
+
+- **16 scored metrics** from market sentiment, miner stress, valuation, holder behavior, network activity, and velocity signals.
+- **Classic vs ML scoring toggle** on the dashboard and backtest API.
+- **ML score explainability**: metric cards show learned weight and contribution in points.
+- **Leakage-resistant ML validation**: training uses purged time-series splits so 365-day forward-return labels do not overlap validation windows.
+- **Historical context panel**: compares the current composite score against historical periods and forward returns.
+- **Clickable metric cards**: select any metric to see percentile, similar historical levels, forward returns, example dates by market cycle, and highlighted chart periods.
+- **Score history chart** with BTC price overlay, range controls, and selected-metric overlay.
+- **Backtest dashboard** with current signal context, score bracket performance, and signal-crossing events.
+- **Quick vs full refresh**: quick refresh updates BTC price and Fear & Greed; full refresh re-scrapes on-chain sources.
+- **LLM settings UI** for Ollama, LM Studio, OpenAI, Anthropic, and OpenRouter.
+
+## Metrics
+
+| # | Metric | Source | Accumulation Signal |
+|---|--------|--------|-------------------|
+| 1 | Fear & Greed Index | alternative.me API | Extreme fear / capitulation sentiment |
+| 2 | Puell Multiple | LookIntoBitcoin | Miner revenue stress |
+| 3 | MVRV Z-Score | LookIntoBitcoin | Market near/below realized value |
+| 4 | Drawdown from ATH | Calculated from BTC price | Deep correction from cycle high |
+| 5 | Price vs 200W SMA | LookIntoBitcoin + BTC price | Price near/below long-term trend |
+| 6 | Reserve Risk | LookIntoBitcoin | High holder confidence relative to price |
+| 7 | RHODL Ratio | LookIntoBitcoin | Long-term holder dominance |
+| 8 | Net Unrealized Profit/Loss (NUPL) | LookIntoBitcoin | Capitulation / early recovery zones |
+| 9 | LTH Realized Price | LookIntoBitcoin | Price near long-term holder cost basis |
+| 10 | Hash Ribbons | LookIntoBitcoin | Miner capitulation/recovery signal |
+| 11 | SOPR | CheckOnChain | Spent outputs near loss / reset territory |
+| 12 | Sell-side Risk Ratio | CheckOnChain | Low realized profit/loss pressure |
+| 13 | Active Address Momentum | CheckOnChain | Network activity momentum extremes |
+| 14 | Transaction Count Momentum | CheckOnChain | Transaction activity momentum extremes |
+| 15 | NVT Price | CheckOnChain | Network-value valuation discount/premium |
+| 16 | VDD Multiple | CheckOnChain | Coin-days/velocity reset conditions |
+
+Informational cards may also appear when data is available, such as **Long-Term Holder Supply** and **Pi Cycle Bottom**. These are displayed for context and are not included in the composite score.
+
+## Score Interpretation
+
+| Score | Assessment | Interpretation |
+|-------|-----------|----------------|
+| 80-100 | 🟢 Extreme Accumulation Zone | Broad capitulation/value conditions across active metrics |
+| 65-79 | 🟢 Strong Accumulation Zone | Historically attractive long-term entry territory |
+| 50-64 | 🟡 Moderate Opportunity | DCA-friendly, but not maximum-signal conditions |
+| 35-49 | 🟡 Neutral | Mixed signals; not compelling either direction |
+| 20-34 | 🔴 Caution — Overheated | Market conditions becoming less favorable |
+| 0-19 | 🔴 Extreme Caution | Historically poor accumulation setup |
+
+Backtest tables provide actual historical forward-return statistics per score bracket, including 30d/90d/180d/1yr averages, win rate, max gain/loss, and average max drawdown.
+
+## ML-Optimized Scoring
+
+The ML mode uses a `GradientBoostingClassifier` trained on historical feature rows to predict whether a day was a good long-term buy based on 365-day forward return. Training features include:
+
+- Classic metric scores.
+- Raw metric values.
+- 30-day metric deltas.
+- Interaction features such as MVRV × NUPL and Puell × Reserve Risk.
+- Cycle-position context such as days since ATH.
+
+The resulting feature importances are aggregated back into transparent metric weights stored in `config/ml_weights.json`. The UI displays normalized weight and contribution for each active metric.
+
+Validation uses purged expanding time-series splits: because each label uses a 365-day forward-return window, training rows whose label windows overlap validation are removed before scoring validation folds.
+
+## Tech Stack
+
+| Component | Technology |
+|-----------|-----------|
+| Backend | Python 3.13 + FastAPI |
+| Frontend | Inline HTML/CSS/JS dark trading-terminal UI |
+| Charts | Chart.js |
+| Scraping | requests + Playwright-style browser scraping where needed |
+| Data APIs | alternative.me, CoinGecko, LookIntoBitcoin, CheckOnChain |
+| ML | NumPy + pandas + scikit-learn GradientBoostingClassifier |
+| Process Manager | pm2 or uvicorn |
+| Default Port | 3088 |
+
+## How Data Is Collected
+
+Data is collected from free/public sources and cached locally under `data/`.
+
+- Fast live refreshes update BTC price, ATH/drawdown, 200D SMA/Mayer where possible, and Fear & Greed.
+- On-chain metrics are cached and reused because they update slowly.
+- Full refresh re-scrapes on-chain metrics from LookIntoBitcoin/CheckOnChain.
+- Historical backtest data lives in `data/history.json` and supports charting, backtests, and metric-context lookups.
+- Score history appends to `data/score_history.jsonl`.
+
+## Project Structure
+
+```
+├── dashboard/
+│   └── server.py                  # FastAPI server + inline dashboard/backtest/settings UI
+├── scrapers/
+│   ├── lookintobitcoin.py         # LookIntoBitcoin metric scraping
+│   ├── checkonchain.py            # CheckOnChain metric scraping
+│   ├── history_collector.py       # Full historical data collection
+│   ├── history_updater.py         # Incremental historical updates
+│   ├── fear_greed.py              # Fear & Greed Index API
+│   └── price.py                   # BTC price, ATH, drawdown, SMA helpers
+├── scoring/
+│   └── engine.py                  # Classic + ML-weighted scoring logic
+├── backtesting/
+│   └── engine.py                  # Historical backtest engine
+├── ml/
+│   └── optimizer.py               # ML training, purged CV, weight export
+├── tests/
+│   ├── test_ml_optimizer_validation.py
+│   └── test_scoring_engine_ml.py
+├── data/
+│   ├── cache.json                 # Live metric cache
+│   ├── history.json               # Historical metric/time-series data
+│   └── score_history.jsonl        # Live score history
+├── config/
+│   ├── thresholds.json            # Classic scoring thresholds
+│   ├── ml_weights.json            # Learned ML metric weights
+│   └── llm_settings.json          # Optional AI commentary provider config
+├── screenshots/                   # README screenshots
+├── scripts/run.sh                 # Locked local launcher with Playwright path
+├── .gitea/workflows/ci.yml        # Gitea Actions test/compile gates
+├── Dockerfile                     # Non-root Chromium-enabled image
+├── docker-compose.yml             # Port, healthcheck, restart, persistent volumes
+├── pyproject.toml                 # Runtime, ML, and development dependency groups
+├── uv.lock                        # Exact reproducible dependency resolution
+├── ARCHITECTURE.md
+└── README.md
+```
+
+## Reproducible Setup
+
+Install [uv](https://docs.astral.sh/uv/) and use Python 3.11-3.13. Dependencies are declared in explicit `runtime`, `ml`, and `dev` groups in `pyproject.toml`; exact cross-platform resolutions are committed in `uv.lock`.
+
+```bash
+git clone <repository-url>
+cd btc-accumulation-monitor
+uv sync --locked --group runtime --group ml --group dev
+```
+
+Install the Chromium binary once for full on-chain refreshes. Keep its path explicit so installation and runtime use the same browser cache:
+
+```bash
+export PLAYWRIGHT_BROWSERS_PATH="$PWD/.playwright"
+uv run --frozen playwright install chromium
+```
+
+`requirements_vps.txt` is a lock-derived, hash-pinned compatibility export for pip-based hosts. `pyproject.toml` and `uv.lock` remain authoritative; regenerate the compatibility file after dependency changes with:
+
+```bash
+uv export --frozen --no-dev --group runtime --group ml \
+  --no-emit-project --no-header --output-file requirements_vps.txt
+```
+
+## Running
+
+The executable launcher fixes `PYTHONPATH`, preserves an explicitly supplied `PLAYWRIGHT_BROWSERS_PATH`, and starts port 3088 from the locked environment:
+
+```bash
+./scripts/run.sh
+```
+
+Equivalent exact command:
+
+```bash
+PLAYWRIGHT_BROWSERS_PATH="$PWD/.playwright" PYTHONPATH=. \
+  uv run --frozen --no-dev --group runtime --group ml \
+  python -m uvicorn dashboard.server:app --host 0.0.0.0 --port 3088
+```
+
+Then visit `http://localhost:3088`.
+
+## Container Deployment
+
+The image uses a multi-architecture Python base, installs Playwright Chromium and its OS libraries during the build, and runs the application as non-root UID `10001`. Compose publishes port 3088, restarts unless stopped, and persists `/app/data` and `/app/config` in named volumes.
+
+```bash
+docker compose build
+docker compose up -d
+```
+
+The Docker and Compose healthchecks probe `GET /health/live`. The deployment must include the reliability revision that supplies that endpoint; without it, Docker correctly reports the container unhealthy even if the older application server is accepting requests.
+
+Named volumes are initialized from the image on first use. Back up both before replacing or deleting them:
+
+```bash
+docker volume inspect btc-accumulation-monitor_btc-monitor-data
+docker volume inspect btc-accumulation-monitor_btc-monitor-config
+```
+
+For bind-mounted deployments, ensure the host directories are writable by UID `10001` and do not replace `config/` with an empty directory.
+
+## First Run and Data Freshness
+
+1. Visit `http://localhost:3088` for the dashboard.
+2. Use **Quick Refresh** for price and Fear & Greed updates while retaining cached slow-moving on-chain metrics.
+3. Use **Full Refresh** when on-chain source data must be re-scraped; this requires the installed Playwright Chromium browser and external source availability.
+4. Visit `http://localhost:3088/backtest` for historical analysis.
+5. If historical data is missing, populate `data/history.json` through the existing collection flow.
+
+Freshness is metric-specific. Price and sentiment APIs can update frequently, while public on-chain chart sources commonly update daily and may be reused from cache. A successful refresh is not proof that every upstream metric has a new observation. Check source timestamps/status exposed by the running revision, and treat missing, stale, or scrape-failed metrics as unavailable rather than silently current. `data/` is operational state and should be persisted and backed up.
+
+## ML and Backtest Caveats
+
+ML weights and backtest output are research artifacts, not investment advice or evidence of future performance. Any reported ML result must retain its provenance: source-data snapshot/range, feature and label definitions, training window, purge/embargo policy, code revision, dependency lock, random seed (when applicable), and generated weight/config artifact.
+
+Model selection and threshold tuning must use training/validation data only. Report final performance on a genuinely untouched out-of-sample (OOS) period; do not describe in-sample fit, cross-validation used for selection, or the best result from repeated experiments as OOS. Forward-return labels require purging overlapping label horizons, but purged cross-validation alone does not create an untouched final test set. Results without reproducible provenance and a reserved OOS evaluation should be labeled exploratory.
+
+## Useful API Endpoints
+
+| Endpoint | Description |
+|----------|-------------|
+| `GET /api/data?mode=classic` | Current metrics using equal-weight scoring |
+| `GET /api/data?mode=ml` | Current metrics using ML-optimized weights |
+| `GET /api/history` | Recent live score history |
+| `POST /api/refresh` | Quick refresh |
+| `POST /api/refresh?full=true` | Full on-chain refresh |
+| `GET /api/backtest?mode=classic` | Historical backtest with classic scoring |
+| `GET /api/backtest?mode=ml` | Historical backtest with ML scoring |
+| `GET /api/metric-context?metric=mvrv_zscore&mode=ml` | Similar historical levels and forward returns for one metric |
+| `GET /api/settings` | Safe LLM settings payload |
+
+## Testing and CI
+
+Run the committed test suite and the same static compilation gate used by Gitea Actions:
+
+```bash
+uv sync --locked --group runtime --group ml --group dev
+uv run --frozen python -m compileall -q \
+  dashboard scrapers scoring backtesting ml ml_engine llm_client scripts orchestrator.py
+uv run --frozen pytest
+```
+
+`.gitea/workflows/ci.yml` runs lock validation/install, static compilation, and tests for pull requests and pushes to `main`.
 
 ## Architecture
 
-```
-┌─────────────────────────────────────────────────────────────────┐
-│                        Optimization Loop                        │
-│                                                                 │
-│  ┌──────────┐    ┌───────────────┐    ┌──────────────────────┐  │
-│  │   VPS    │───>│  Windows PC   │───>│     Mac Mini         │  │
-│  │ (Orch.)  │<───│  (GPU/ML)     │    │     (LLM)           │  │
-│  │          │<───────────────────────>│                      │  │
-│  │ - Fetch  │    │ - XGBoost     │    │ - Ollama             │  │
-│  │   data   │    │ - LightGBM    │    │ - qwen3.5:27b        │  │
-│  │ - Coord  │    │ - CatBoost    │    │ - Analyze results    │  │
-│  │ - Store  │    │ - RTX 4070 Ti │    │ - Suggest changes    │  │
-│  └──────────┘    └───────────────┘    └──────────────────────┘  │
-│       ▲                                        │                │
-│       └────────────────────────────────────────┘                │
-│                    Modified config                              │
-└─────────────────────────────────────────────────────────────────┘
-```
+See [ARCHITECTURE.md](ARCHITECTURE.md) for deeper implementation details on scoring, data collection, and backtesting.
 
-### Machines (Tailscale)
+## License
 
-| Machine    | Role         | Address           | Key Resources        |
-|------------|-------------|-------------------|---------------------|
-| VPS        | Orchestrator | localhost          | Coordination, data   |
-| Windows PC | ML Engine    | 100.76.218.38     | RTX 4070 Ti GPU      |
-| Mac Mini   | LLM         | 100.100.242.21    | Ollama, qwen3.5:27b  |
-
-## Directory Structure
-
-```
-btc-ml-optimizer/
-├── orchestrator.py              # Main loop — coordinates everything
-├── ml_engine/
-│   └── train_and_backtest.py    # Self-contained ML script (runs on Windows)
-├── llm_client/
-│   └── analyzer.py              # LLM strategy analyzer (calls Mac Mini)
-├── scripts/
-│   ├── fetch_data.py            # BTC/USDT data fetcher (ccxt)
-│   └── setup_windows.sh         # Install deps on Windows PC
-├── config/
-│   └── initial_config.json      # Starting configuration
-├── data/                        # OHLCV CSV files
-├── results/                     # Iteration results + logs
-├── requirements_vps.txt         # VPS Python dependencies
-└── requirements_windows.txt     # Windows PC Python dependencies
-```
-
-## Setup
-
-### 1. VPS (this machine)
-
-```bash
-pip install -r requirements_vps.txt
-```
-
-### 2. Windows PC
-
-```bash
-# From VPS — installs all ML deps on Windows via SSH
-bash scripts/setup_windows.sh
-```
-
-Or manually on Windows:
-```bash
-pip install -r requirements_windows.txt
-```
-
-### 3. Mac Mini
-
-Ensure Ollama is running with the qwen3.5:27b model:
-```bash
-ollama pull qwen3.5:27b
-ollama serve  # should already be running
-```
-
-## Usage
-
-### Fetch Data
-
-```bash
-python3 scripts/fetch_data.py
-```
-
-Downloads 2 years of BTC/USDT 1h and 4h OHLCV data from Binance.
-
-### Run the Optimizer
-
-```bash
-python3 orchestrator.py
-```
-
-The optimizer will:
-1. Ensure data is fetched
-2. Upload ML engine + data to Windows PC
-3. Train model and backtest on GPU
-4. Send results to LLM for analysis
-5. Apply LLM-suggested config changes
-6. Repeat until convergence (or 50 iterations)
-
-### Run ML Engine Standalone (on Windows)
-
-```bash
-python train_and_backtest.py --config config.json --data btc_4h.csv --output results.json
-```
-
-## Configuration Reference
-
-### `model_type`
-- `xgboost` — XGBoost with GPU (default, generally best)
-- `lightgbm` — LightGBM with GPU (faster training)
-- `catboost` — CatBoost with GPU (handles interactions well)
-- `ensemble` — Soft voting of all three
-
-### `features`
-- `technical_indicators` — List of indicators to compute
-- `lookback_periods` — Windows for return/volatility features
-- `use_volume_features` — Include volume-derived features
-- `use_volatility_features` — Include volatility features
-- `use_candle_patterns` — Include candlestick pattern features
-- `use_lag_features` — Include lagged feature values
-- `lag_periods` — Specific lag periods to use
-
-### `target`
-- `direction` — `"long"` or `"both"`
-- `horizon_candles` — Forward-looking prediction window
-- `threshold_pct` — Minimum % move to label as positive
-
-### `hyperparameters`
-Standard gradient boosting params: `learning_rate`, `max_depth`, `n_estimators`, `subsample`, `colsample_bytree`, `min_child_weight`, `gamma`, `reg_alpha`, `reg_lambda`
-
-### `strategy`
-- `entry_threshold` — Min probability to enter trade (0.5-0.8)
-- `stop_loss_pct` — Stop loss percentage
-- `take_profit_pct` — Take profit percentage
-- `trailing_stop_pct` — Trailing stop distance
-- `position_sizing` — `"confidence_scaled"` or `"fixed"`
-- `min_confidence_to_trade` — Absolute minimum confidence
-
-### `training`
-- `walk_forward_windows` — Number of walk-forward splits (3-10)
-- `train_pct` / `validation_pct` / `test_pct` — Data split ratios
-
-## Convergence Criteria
-
-The optimizer stops when:
-- Sharpe ratio exceeds 3.0
-- Sharpe improvement < 1% over 5 consecutive iterations
-- Maximum 50 iterations reached
-
-## Output
-
-- `config/best_config.json` — Best configuration found
-- `results/iterations.jsonl` — Full log of every iteration
-- `results/results_iter_N.json` — Detailed results per iteration
+Private — not for public distribution.

backtesting/engine.py

@@ -1,12 +1,18 @@
 """Historical backtest engine for Bitcoin Accumulation Zone scoring."""
 
+import copy
 import json
 import logging
 import os
 import sys
+import threading
 from collections import defaultdict
 from datetime import datetime, timedelta
 
+from scoring.policy import SCORE_BRACKETS, SCORE_VERSION, score_in_bracket
+from ml.artifacts import validate_ml_artifact
+from backtesting.statistics import summarize_returns
+
 log = logging.getLogger(__name__)
 
 BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
@@ -14,56 +20,65 @@ sys.path.insert(0, BASE_DIR)
 
 HISTORY_PATH = os.path.join(BASE_DIR, "data", "history.json")
 CACHE_PATH = os.path.join(BASE_DIR, "data", "cache.json")
+ML_WEIGHTS_PATH = os.path.join(BASE_DIR, "config", "ml_weights.json")
+
+_BACKTEST_CACHE = {}
+_BACKTEST_CACHE_LOCK = threading.Lock()
+_BACKTEST_CACHE_LIMIT = 4
 
 # Score brackets matching the dashboard assessment levels
-BRACKETS = [
-    (0, 20, "Extreme Caution"),
-    (21, 40, "Caution"),
-    (41, 55, "Neutral"),
-    (56, 70, "Moderate Opportunity"),
-    (71, 85, "Strong Accumulation"),
-    (86, 100, "Extreme Accumulation"),
-]
+BRACKETS = SCORE_BRACKETS
+
+# Scoring thresholds — load from config/thresholds.json (single source of truth)
+import os as _os
+import json as _json
+
+_THRESH_PATH = _os.path.join(_os.path.dirname(_os.path.dirname(_os.path.abspath(__file__))), "config", "thresholds.json")
+try:
+    with open(_THRESH_PATH) as _f:
+        _THRESH = _json.load(_f)
+except Exception:
+    _THRESH = {}
 
-# Scoring thresholds — replicated from scoring/engine.py for standalone use
 METRIC_SCORERS = {
-    "fear_greed": {
-        "ranges": [[None, 10, 10], [10, 25, 7], [25, 45, 4], [45, 55, 2], [55, 75, 1], [75, None, 0]],
-    },
-    "puell_multiple": {
-        "ranges": [[None, 0.3, 10], [0.3, 0.5, 8], [0.5, 0.8, 5], [0.8, 1.2, 3], [1.2, 2.0, 1], [2.0, None, 0]],
-    },
-    "mvrv_zscore": {
-        "ranges": [[None, 0, 10], [0, 0.5, 8], [0.5, 1.5, 5], [1.5, 3, 2], [3, 5, 1], [5, None, 0]],
-    },
-    "reserve_risk": {
-        "ranges": [[None, 0.002, 10], [0.002, 0.005, 7], [0.005, 0.01, 4], [0.01, 0.02, 2], [0.02, None, 0]],
-    },
-    "rhodl_ratio": {
-        "ranges": [[None, 100, 10], [100, 500, 7], [500, 2000, 4], [2000, 10000, 1], [10000, None, 0]],
-    },
-    "nupl": {
-        "ranges": [[None, 0, 10], [0, 0.25, 7], [0.25, 0.5, 4], [0.5, 0.75, 1], [0.75, None, 0]],
-    },
+    "fear_greed": {"ranges": _THRESH.get("fear_greed", {}).get("ranges", [[0, 15, 10], [15, 30, 8], [30, 45, 5], [45, 55, 3], [55, 75, 1], [75, None, 0]])},
+    "puell_multiple": {"ranges": _THRESH.get("puell_multiple", {}).get("ranges", [[None, 0.4, 10], [0.4, 0.7, 8], [0.7, 1.0, 5], [1.0, 1.5, 3], [1.5, 2.0, 1], [2.0, None, 0]])},
+    "mvrv_zscore": {"ranges": _THRESH.get("mvrv_zscore", {}).get("ranges", [[None, 0, 10], [0, 1.0, 8], [1.0, 2.0, 5], [2.0, 3.0, 3], [3.0, 5.0, 1], [5.0, None, 0]])},
+    "reserve_risk": {"ranges": _THRESH.get("reserve_risk", {}).get("ranges", [[None, 0.002, 10], [0.002, 0.005, 7], [0.005, 0.01, 4], [0.01, 0.02, 2], [0.02, None, 0]])},
+    "rhodl_ratio": {"ranges": _THRESH.get("rhodl_ratio", {}).get("ranges", [[None, 200, 10], [200, 1000, 7], [1000, 5000, 4], [5000, 20000, 1], [20000, None, 0]])},
+    "nupl": {"ranges": _THRESH.get("nupl", {}).get("ranges", [[None, 0, 10], [0, 0.3, 8], [0.3, 0.5, 4], [0.5, 0.75, 1], [0.75, None, 0]])},
 }
 
-# Ratio-based metrics: score based on price vs reference value
 RATIO_SCORERS = {
     "price_vs_200w_sma": {
-        # pct_above ranges
-        "ranges": [[None, 0, 10], [0, 20, 6], [20, 50, 3], [50, 100, 1], [100, None, 0]],
+        "ranges": _THRESH.get("price_vs_200w_sma", {}).get("ranges", [[None, 0, 10], [0, 30, 7], [30, 60, 5], [60, 100, 2], [100, None, 0]]),
         "price_key": "btc_price",
         "ref_key": "200w_sma",
     },
     "lth_realized_price": {
-        "ranges": [[None, 0, 10], [0, 20, 6], [20, 50, 3], [50, None, 1]],
+        "ranges": _THRESH.get("lth_realized_price", {}).get("ranges", [[None, 0, 10], [0, 30, 7], [30, 80, 5], [80, 150, 3], [150, None, 1]]),
         "price_key": "btc_price",
         "ref_key": "lth_realized_price",
     },
 }
 
-# Drawdown scoring
-DRAWDOWN_RANGES = [[70, None, 10], [50, 70, 8], [30, 50, 6], [20, 30, 4], [10, 20, 2], [None, 10, 0]]
+BACKTEST_METRIC_PANEL = tuple(METRIC_SCORERS) + tuple(RATIO_SCORERS) + ("drawdown",)
+METRIC_MAX_AGE_DAYS = {
+    "fear_greed": 2,
+    "puell_multiple": 7,
+    "mvrv_zscore": 7,
+    "reserve_risk": 7,
+    "rhodl_ratio": 7,
+    "nupl": 7,
+    "btc_price": 3,
+    "btc_price_coingecko": 3,
+    "btc_price_sma": 3,
+    "btc_price_lth": 3,
+    "200w_sma": 7,
+    "lth_realized_price": 7,
+}
+
+DRAWDOWN_RANGES = _THRESH.get("drawdown", {}).get("ranges", [[60, None, 10], [40, 60, 8], [25, 40, 6], [15, 25, 4], [5, 15, 2], [None, 5, 0]])
 
 
 def _score_range(value, ranges):
@@ -99,8 +114,8 @@ def _get_all_dates(index):
     return sorted(all_dates)
 
 
-def _last_known_value(lookup, date, max_lookback=30):
-    """Get value for date, or most recent prior value within lookback window."""
+def _last_known_value(lookup, date, max_lookback=0):
+    """Get value for date, or a prior value within an explicit lookback."""
     if date in lookup:
         return lookup[date]
     d = datetime.strptime(date, "%Y-%m-%d")
@@ -111,6 +126,17 @@ def _last_known_value(lookup, date, max_lookback=30):
     return None
 
 
+def _metric_observation(lookup, date, metric_key):
+    """Return value, source date, and age under a metric-specific freshness rule."""
+    max_age = METRIC_MAX_AGE_DAYS.get(metric_key, 0)
+    target = datetime.strptime(date, "%Y-%m-%d")
+    for age in range(max_age + 1):
+        source_date = (target - timedelta(days=age)).strftime("%Y-%m-%d")
+        if source_date in lookup:
+            return lookup[source_date], source_date, age
+    return None, None, None
+
+
 def _compute_ath_series(price_lookup, dates):
     """Compute running ATH and drawdown for each date."""
     ath = 0
@@ -126,36 +152,189 @@ def _compute_ath_series(price_lookup, dates):
     return drawdowns
 
 
-def score_day(date, index, drawdowns):
-    """Score a single day using all available metrics. Returns (composite_score, individual_scores, n_metrics)."""
+def _load_ml_artifact():
+    """Load an ML artifact and return it with validation status."""
+    ml_path = _os.path.join(_os.path.dirname(_os.path.dirname(_os.path.abspath(__file__))), "config", "ml_weights.json")
+    try:
+        with open(ml_path) as f:
+            data = _json.load(f)
+        status = validate_ml_artifact(data)
+        if not status["valid"]:
+            log.error("Rejected invalid ML artifact: %s", ", ".join(status["errors"]))
+            return None, status
+        return data, status
+    except Exception as exc:
+        return None, {"valid": False, "errors": [f"load_error:{exc}"]}
+
+
+def _build_ml_backtest_plan(artifact):
+    """Choose OOS fold weights when available; otherwise mark final weights in-sample."""
+    status = validate_ml_artifact(artifact)
+    if not status["valid"]:
+        raise ValueError("invalid ML artifact: " + ", ".join(status["errors"]))
+
+    if status["has_oos_fold_weights"]:
+        folds = []
+        for fold in artifact["cv_results"]["folds"]:
+            start, separator, end = fold["date_ranges"]["validation"].partition(" to ")
+            if not separator:
+                raise ValueError("invalid validation date range")
+            folds.append({
+                "fold": fold.get("fold"),
+                "start": start,
+                "end": end,
+                "weights": fold["weights"],
+            })
+        return {
+            "evaluation_scope": "out_of_sample_validation_folds",
+            "is_out_of_sample": True,
+            "weighting_source": "fold_specific_weights",
+            "folds": folds,
+            "weights": None,
+        }
+
+    return {
+        "evaluation_scope": "in_sample_full_history_weights",
+        "is_out_of_sample": False,
+        "weighting_source": "final_full_history_weights",
+        "folds": [],
+        "weights": artifact["weights"],
+    }
+
+
+def _weights_for_backtest_date(date, plan):
+    """Return date-appropriate weights and fold number for an ML plan."""
+    if plan["is_out_of_sample"]:
+        for fold in plan["folds"]:
+            if fold["start"] <= date <= fold["end"]:
+                return fold["weights"], fold["fold"]
+        return None, None
+    return plan["weights"], None
+
+
+def _load_ml_weights():
+    """Compatibility helper returning valid final weights only."""
+    artifact, _ = _load_ml_artifact()
+    return artifact.get("weights", {}) if artifact else {}
+
+# ML weight key mapping (backtest metric keys -> ML weight keys)
+_BT_ML_KEY_MAP = {
+    "fear_greed": "fear_greed",
+    "puell_multiple": "puell_multiple",
+    "mvrv_zscore": "mvrv_zscore",
+    "reserve_risk": "reserve_risk",
+    "rhodl_ratio": "rhodl_ratio",
+    "nupl": "nupl",
+    "price_vs_200w_sma": "pct_above_200w_sma",
+    "lth_realized_price": "pct_above_lth_rp",
+    "drawdown": "drawdown",
+}
+
+
+def _common_panel_current_score(scored, ml_weights=None):
+    """Recompute the current score using only metrics present historically."""
+    by_key = {
+        metric.get("key"): metric.get("score")
+        for metric in scored.get("metrics", [])
+        if metric.get("key") in BACKTEST_METRIC_PANEL and metric.get("score") is not None
+    }
+    available_keys = [key for key in BACKTEST_METRIC_PANEL if key in by_key]
+    coverage = {
+        "available_count": len(available_keys),
+        "panel_count": len(BACKTEST_METRIC_PANEL),
+        "available_keys": available_keys,
+    }
+    if not available_keys:
+        return None, coverage
+
+    if ml_weights:
+        weighted = [
+            (by_key[key], ml_weights.get(_BT_ML_KEY_MAP[key], 0.0))
+            for key in available_keys
+        ]
+        weight_total = sum(weight for _, weight in weighted)
+        if weight_total > 0:
+            return round(sum(score * weight for score, weight in weighted) / weight_total * 10, 1), coverage
+
+    return round(sum(by_key[key] for key in available_keys) / len(available_keys) * 10, 1), coverage
+
+
+def _backtest_data_quality_metadata(metric_counts):
+    """Describe historical panel, coverage, and freshness assumptions."""
+    coverage = {
+        "minimum_metrics": min(metric_counts),
+        "maximum_metrics": max(metric_counts),
+        "average_metrics": round(sum(metric_counts) / len(metric_counts), 1),
+        "panel_count": len(BACKTEST_METRIC_PANEL),
+    } if metric_counts else {
+        "minimum_metrics": 0,
+        "maximum_metrics": 0,
+        "average_metrics": 0,
+        "panel_count": len(BACKTEST_METRIC_PANEL),
+    }
+    return {
+        "metric_panel": {
+            "id": "historical-common-v1",
+            "keys": list(BACKTEST_METRIC_PANEL),
+            "count": len(BACKTEST_METRIC_PANEL),
+        },
+        "coverage": coverage,
+        "staleness_days": dict(METRIC_MAX_AGE_DAYS),
+    }
+
+
+def score_day(date, index, drawdowns, ml_weights=None):
+    """Score a single day using all available metrics. Returns (composite_score, details, n_metrics).
+
+    If ml_weights is provided, uses ML-optimized weighting instead of equal weights.
+    details includes both "score" and "raw" (the actual metric value before scoring).
+    """
     scores = []
     details = {}
 
     # Simple range-based metrics
     for metric_key, cfg in METRIC_SCORERS.items():
-        val = _last_known_value(index.get(metric_key, {}), date)
+        val, observed_date, age_days = _metric_observation(
+            index.get(metric_key, {}), date, metric_key
+        )
         if val is not None:
             s = _score_range(val, cfg["ranges"])
             if s is not None:
                 scores.append(s)
-                details[metric_key] = {"value": val, "score": s}
+                details[metric_key] = {
+                    "value": val,
+                    "score": s,
+                    "raw": val,
+                    "observed_date": observed_date,
+                    "age_days": age_days,
+                }
 
     # Ratio-based metrics (price vs reference)
     for metric_key, cfg in RATIO_SCORERS.items():
-        price_val = _last_known_value(index.get(cfg["price_key"], {}), date)
-        # Try alternate price keys
+        price_val, price_date, price_age = _metric_observation(
+            index.get(cfg["price_key"], {}), date, cfg["price_key"]
+        )
+        # Try alternate price keys, each with an explicit freshness rule.
         if price_val is None:
             for pk in ["btc_price_coingecko", "btc_price_sma", "btc_price_lth"]:
-                price_val = _last_known_value(index.get(pk, {}), date)
+                price_val, price_date, price_age = _metric_observation(index.get(pk, {}), date, pk)
                 if price_val is not None:
                     break
-        ref_val = _last_known_value(index.get(cfg["ref_key"], {}), date)
+        ref_val, ref_date, ref_age = _metric_observation(
+            index.get(cfg["ref_key"], {}), date, cfg["ref_key"]
+        )
         if price_val is not None and ref_val is not None and ref_val > 0:
             pct_above = ((price_val - ref_val) / ref_val) * 100
             s = _score_range(pct_above, cfg["ranges"])
             if s is not None:
                 scores.append(s)
-                details[metric_key] = {"value": pct_above, "score": s}
+                details[metric_key] = {
+                    "value": pct_above,
+                    "score": s,
+                    "raw": pct_above,
+                    "observed_date": min(price_date, ref_date),
+                    "age_days": max(price_age, ref_age),
+                }
 
     # Drawdown
     dd = drawdowns.get(date)
@@ -163,12 +342,26 @@ def score_day(date, index, drawdowns):
         s = _score_range(dd, DRAWDOWN_RANGES)
         if s is not None:
             scores.append(s)
-            details["drawdown"] = {"value": dd, "score": s}
+            details["drawdown"] = {"value": dd, "score": s, "raw": dd}
 
     if not scores:
         return None, details, 0
 
-    composite = sum(scores) / len(scores) * 10
+    if ml_weights:
+        # ML-weighted composite
+        weighted_sum = 0.0
+        weight_total = 0.0
+        for metric_key, info in details.items():
+            ml_key = _BT_ML_KEY_MAP.get(metric_key, metric_key)
+            w = ml_weights.get(ml_key, 0.0)
+            weighted_sum += info["score"] * w
+            weight_total += w
+        if weight_total > 0:
+            composite = weighted_sum / weight_total * 10
+        else:
+            composite = sum(scores) / len(scores) * 10
+    else:
+        composite = sum(scores) / len(scores) * 10
     return round(composite, 1), details, len(scores)
 
 
@@ -213,9 +406,67 @@ def compute_max_drawdown_forward(price_lookup, date, window=90):
     return round(max_dd, 2) if max_dd > 0 else 0
 
 
-def run_backtest():
-    """Run the full backtest and return comprehensive results."""
-    log.info("Loading historical data...")
+def _file_signature(path):
+    """Return a cheap signature that invalidates when an input file changes."""
+    try:
+        stat = os.stat(path)
+        return path, stat.st_mtime_ns, stat.st_size
+    except OSError:
+        return path, None, None
+
+
+def clear_backtest_cache():
+    """Clear memoized backtest results (primarily for explicit refreshes/tests)."""
+    with _BACKTEST_CACHE_LOCK:
+        _BACKTEST_CACHE.clear()
+
+
+def _add_return_statistics(stats, period, returns):
+    """Add return summaries and a moving-block-bootstrap mean interval."""
+    horizon_days = int(period.removesuffix("d"))
+    summary = summarize_returns(
+        returns,
+        block_size=min(horizon_days, len(returns)),
+        n_resamples=400,
+    )
+    stats[f"avg_{period}"] = summary["mean"]
+    stats[f"median_{period}"] = summary["median"]
+    stats[f"win_rate_{period}"] = summary["win_rate"]
+    stats[f"avg_{period}_ci_low"] = summary["mean_ci_low"]
+    stats[f"avg_{period}_ci_high"] = summary["mean_ci_high"]
+    stats[f"max_gain_{period}"] = round(max(returns), 2)
+    stats[f"max_loss_{period}"] = round(min(returns), 2)
+    stats[f"n_{period}"] = summary["n"]
+
+
+def run_backtest(ml_mode=False):
+    """Return an isolated cached result keyed by all material input files."""
+    signature = (
+        bool(ml_mode),
+        _file_signature(HISTORY_PATH),
+        _file_signature(_THRESH_PATH),
+        _file_signature(ML_WEIGHTS_PATH),
+        _file_signature(CACHE_PATH),
+    )
+    with _BACKTEST_CACHE_LOCK:
+        cached = _BACKTEST_CACHE.get(signature)
+    if cached is not None:
+        return copy.deepcopy(cached)
+
+    result = _compute_backtest(ml_mode=ml_mode)
+    with _BACKTEST_CACHE_LOCK:
+        _BACKTEST_CACHE[signature] = copy.deepcopy(result)
+        while len(_BACKTEST_CACHE) > _BACKTEST_CACHE_LIMIT:
+            _BACKTEST_CACHE.pop(next(iter(_BACKTEST_CACHE)))
+    return copy.deepcopy(result)
+
+
+def _compute_backtest(ml_mode=False):
+    """Run the full backtest and return comprehensive results.
+
+    If ml_mode=True, uses ML-optimized metric weights instead of equal weights.
+    """
+    log.info("Loading historical data... (ml_mode=%s)", ml_mode)
     if not os.path.exists(HISTORY_PATH):
         return {"error": "No historical data found. Run history collector first."}
 
@@ -245,20 +496,47 @@ def run_backtest():
     log.info("Computing forward returns...")
     fwd_returns = compute_forward_returns(price_lookup, all_dates)
 
+    # Build an explicit evaluation plan. Fold-specific validation weights are OOS;
+    # final weights fitted on full history are never represented as OOS.
+    ml_plan = None
+    ml_artifact = None
+    ml_artifact_status = None
+    if ml_mode:
+        ml_artifact, ml_artifact_status = _load_ml_artifact()
+        if ml_artifact:
+            ml_plan = _build_ml_backtest_plan(ml_artifact)
+        else:
+            log.warning("ML mode requested with invalid artifact — falling back to equal weights")
+
     # Score each day
     log.info("Scoring %d days...", len(all_dates))
     daily_scores = []
     for d in all_dates:
-        composite, details, n_metrics = score_day(d, index, drawdowns)
+        ml_weights = None
+        ml_fold = None
+        if ml_plan:
+            ml_weights, ml_fold = _weights_for_backtest_date(d, ml_plan)
+            if ml_plan["is_out_of_sample"] and ml_weights is None:
+                continue
+        composite, details, n_metrics = score_day(d, index, drawdowns, ml_weights=ml_weights)
         if composite is not None and n_metrics >= 3:  # Require at least 3 metrics
             price = price_lookup.get(d)
+            # Collect raw metric values for per-metric historical exploration
+            metric_values = {}
+            for mk, info in details.items():
+                raw = info.get("raw")
+                if raw is not None:
+                    metric_values[mk] = round(raw, 6) if isinstance(raw, float) else raw
             entry = {
                 "date": d,
                 "score": composite,
                 "n_metrics": n_metrics,
                 "price": price,
                 "forward_returns": fwd_returns.get(d, {}),
+                "metric_values": metric_values,
             }
+            if ml_fold is not None:
+                entry["ml_fold"] = ml_fold
             daily_scores.append(entry)
 
     if not daily_scores:
@@ -269,7 +547,7 @@ def run_backtest():
     # --- Bracket statistics ---
     bracket_stats = []
     for low, high, label in BRACKETS:
-        days_in = [d for d in daily_scores if low <= d["score"] <= high]
+        days_in = [d for d in daily_scores if score_in_bracket(d["score"], (low, high, label))]
         if not days_in:
             bracket_stats.append({
                 "range": f"{low}-{high}", "label": label, "days": 0,
@@ -280,13 +558,7 @@ def run_backtest():
         for period in ["30d", "90d", "180d", "365d"]:
             returns = [d["forward_returns"][period] for d in days_in if period in d["forward_returns"]]
             if returns:
-                returns_sorted = sorted(returns)
-                stats[f"avg_{period}"] = round(sum(returns) / len(returns), 2)
-                stats[f"median_{period}"] = round(returns_sorted[len(returns_sorted) // 2], 2)
-                stats[f"win_rate_{period}"] = round(len([r for r in returns if r > 0]) / len(returns) * 100, 1)
-                stats[f"max_gain_{period}"] = round(max(returns), 2)
-                stats[f"max_loss_{period}"] = round(min(returns), 2)
-                stats[f"n_{period}"] = len(returns)
+                _add_return_statistics(stats, period, returns)
 
         # Average max drawdown within 90 days
         dd_list = []
@@ -330,23 +602,30 @@ def run_backtest():
     all_scores_list = [d["score"] for d in daily_scores]
     all_scores_list.sort()
 
-    # Get current score from cache
+    # Get current score from cache, recomputed on the common historical panel.
     current_score = None
     current_price = None
+    current_coverage = None
     if os.path.exists(CACHE_PATH):
         try:
             with open(CACHE_PATH) as f:
                 cache = json.load(f)
             scored = cache.get("_scored", {})
-            current_score = scored.get("composite_score")
+            current_ml_weights = ml_artifact.get("weights") if ml_mode and ml_artifact else None
+            current_score, current_coverage = _common_panel_current_score(scored, current_ml_weights)
             current_price = cache.get("price", {}).get("price")
         except Exception:
             pass
 
-    # If no cache, use latest daily score
+    # If no comparable cache panel is available, use latest historical score.
     if current_score is None and daily_scores:
         current_score = daily_scores[-1]["score"]
         current_price = daily_scores[-1].get("price")
+        current_coverage = {
+            "available_count": daily_scores[-1]["n_metrics"],
+            "panel_count": len(BACKTEST_METRIC_PANEL),
+            "available_keys": list(daily_scores[-1].get("metric_values", {})),
+        }
 
     current_context = None
     if current_score is not None:
@@ -361,50 +640,131 @@ def run_backtest():
             if abs(d["score"] - current_score) <= margin and d["forward_returns"]:
                 comparable.append(d)
 
-        avg_1yr = None
+        avg_returns = {}
         if comparable:
-            yr_returns = [d["forward_returns"]["365d"] for d in comparable if "365d" in d["forward_returns"]]
-            if yr_returns:
-                avg_1yr = round(sum(yr_returns) / len(yr_returns), 2)
+            for period in ["30d", "90d", "180d", "365d"]:
+                vals = [d["forward_returns"][period] for d in comparable if period in d["forward_returns"]]
+                if vals:
+                    avg_returns[period] = round(sum(vals) / len(vals), 2)
+        avg_1yr = avg_returns.get("365d")
 
-        # Best comparable examples (most recent 5)
+        # Best comparable examples — one per market cycle for diversity
+        # Cycles: pre-2016, 2016-2017 bull, 2018-2019 bear, 2020-2021 bull, 2022-2023 bear, 2024+
+        cycle_bins = [
+            ("pre-2016", "2010-01-01", "2015-12-31"),
+            ("2016-17 Bull", "2016-01-01", "2017-12-31"),
+            ("2018-19 Bear", "2018-01-01", "2019-12-31"),
+            ("2020-21 Bull", "2020-01-01", "2021-12-31"),
+            ("2022-23 Bear", "2022-01-01", "2023-12-31"),
+            ("2024+", "2024-01-01", "2099-12-31"),
+        ]
         examples = []
-        for d in comparable[-5:]:
-            examples.append({
-                "date": d["date"],
-                "score": d["score"],
-                "price": d["price"],
-                "forward_returns": d["forward_returns"],
-            })
+        used_cycles = set()
+        # Sort comparable by closest score first, then pick one per cycle
+        sorted_comp = sorted(comparable, key=lambda d: abs(d["score"] - current_score))
+        for d in sorted_comp:
+            cycle_label = None
+            for label, start, end in cycle_bins:
+                if start <= d["date"] <= end:
+                    cycle_label = label
+                    break
+            if cycle_label and cycle_label not in used_cycles:
+                used_cycles.add(cycle_label)
+                examples.append({
+                    "date": d["date"],
+                    "score": d["score"],
+                    "price": d["price"],
+                    "forward_returns": d["forward_returns"],
+                    "cycle": cycle_label,
+                })
+            if len(examples) >= 6:
+                break
+        # Sort examples chronologically
+        examples.sort(key=lambda d: d["date"])
 
         current_context = {
             "current_score": current_score,
             "current_price": current_price,
+            "score_version": SCORE_VERSION,
+            "metric_panel_id": "historical-common-v1",
+            "coverage": current_coverage,
+            "current_weighting_source": (
+                "final_full_history_weights" if ml_mode and ml_artifact else "equal_weight"
+            ),
             "percentile": percentile,
             "comparable_days": len(comparable),
             "avg_1yr_return": avg_1yr,
+            "avg_30d_return": avg_returns.get("30d"),
+            "avg_90d_return": avg_returns.get("90d"),
+            "avg_180d_return": avg_returns.get("180d"),
             "examples": examples,
         }
 
     # --- Build time series for charting ---
-    # Downsample to weekly for chart efficiency
+    # Smart downsampling: daily for last 2 years, weekly before that
+    # Include per-metric values so the frontend can plot any metric.
     chart_data = []
+    import datetime as _dt
+    try:
+        last_date = _dt.datetime.strptime(daily_scores[-1]["date"], "%Y-%m-%d")
+        cutoff_date = (last_date - _dt.timedelta(days=730)).strftime("%Y-%m-%d")
+    except Exception:
+        cutoff_date = "2024-01-01"
+
+    # Collect all metric keys that were ever scored (for per-metric series)
+    all_metric_keys = set()
+    for d in daily_scores:
+        all_metric_keys.update(d.get("metric_values", {}).keys())
+
     for i, d in enumerate(daily_scores):
-        # Include every 7th day + last day
-        if i % 7 == 0 or i == len(daily_scores) - 1:
-            chart_data.append({
+        is_recent = d["date"] >= cutoff_date
+        if is_recent or i % 7 == 0 or i == len(daily_scores) - 1:
+            entry = {
                 "date": d["date"],
                 "score": d["score"],
                 "price": d["price"],
-            })
+            }
+            # Include per-metric values (raw metric value, not score)
+            metric_vals = d.get("metric_values", {})
+            if metric_vals:
+                entry["metric_values"] = metric_vals
+            chart_data.append(entry)
 
+    if not ml_mode:
+        ml_evaluation = {"requested": False, "is_out_of_sample": False}
+    elif ml_plan:
+        ml_evaluation = {
+            "requested": True,
+            "evaluation_scope": ml_plan["evaluation_scope"],
+            "is_out_of_sample": ml_plan["is_out_of_sample"],
+            "weighting_source": ml_plan["weighting_source"],
+            "folds": len(ml_plan["folds"]),
+            "artifact": ml_artifact_status,
+        }
+    else:
+        ml_evaluation = {
+            "requested": True,
+            "evaluation_scope": "equal_weight_fallback",
+            "is_out_of_sample": False,
+            "weighting_source": "none_invalid_artifact",
+            "folds": 0,
+            "artifact": ml_artifact_status,
+        }
+
+    data_quality = _backtest_data_quality_metadata([day["n_metrics"] for day in daily_scores])
     result = {
         "date_range": {"start": daily_scores[0]["date"], "end": daily_scores[-1]["date"]},
         "total_days_scored": len(daily_scores),
+        "metric_panel": data_quality["metric_panel"],
+        "coverage": data_quality["coverage"],
+        "staleness_days": data_quality["staleness_days"],
         "bracket_stats": bracket_stats,
         "signal_events": signal_events,
         "current_context": current_context,
         "chart_data": chart_data,
+        "ml_mode": ml_mode,
+        "ml_evaluation": ml_evaluation,
+        "score_version": SCORE_VERSION,
         "computed_at": datetime.utcnow().isoformat() + "Z",
     }
 

backtesting/statistics.py

@@ -0,0 +1,85 @@
+"""Statistical helpers for honest time-series backtest reporting."""
+
+from __future__ import annotations
+
+import math
+import random
+import statistics as stdlib_statistics
+from collections.abc import Iterable
+
+
+def _quantile(sorted_values: list[float], probability: float) -> float:
+    position = (len(sorted_values) - 1) * probability
+    lower = math.floor(position)
+    upper = math.ceil(position)
+    if lower == upper:
+        return sorted_values[lower]
+    fraction = position - lower
+    return sorted_values[lower] * (1 - fraction) + sorted_values[upper] * fraction
+
+
+def moving_block_bootstrap_ci(
+    values: Iterable[float],
+    *,
+    block_size: int = 30,
+    n_resamples: int = 1_000,
+    confidence: float = 0.95,
+    seed: int = 42,
+) -> dict[str, float | int]:
+    """Estimate a mean and CI while preserving local serial dependence."""
+    series = [float(value) for value in values]
+    if not series:
+        raise ValueError("values must not be empty")
+    if block_size < 1 or block_size > len(series):
+        raise ValueError("block_size must be between 1 and the number of values")
+    if n_resamples < 2:
+        raise ValueError("n_resamples must be at least 2")
+    if not 0 < confidence < 1:
+        raise ValueError("confidence must be between 0 and 1")
+
+    rng = random.Random(seed)
+    sample_means: list[float] = []
+    final_start = len(series) - block_size
+    for _ in range(n_resamples):
+        sample: list[float] = []
+        while len(sample) < len(series):
+            start = rng.randint(0, final_start)
+            sample.extend(series[start:start + block_size])
+        sample = sample[:len(series)]
+        sample_means.append(sum(sample) / len(sample))
+
+    sample_means.sort()
+    tail = (1 - confidence) / 2
+    return {
+        "estimate": sum(series) / len(series),
+        "ci_low": _quantile(sample_means, tail),
+        "ci_high": _quantile(sample_means, 1 - tail),
+        "n": len(series),
+    }
+
+
+def summarize_returns(
+    values: Iterable[float],
+    *,
+    block_size: int = 30,
+    n_resamples: int = 1_000,
+    confidence: float = 0.95,
+    seed: int = 42,
+) -> dict[str, float | int]:
+    """Summarize realized returns with an autocorrelation-aware mean CI."""
+    series = [float(value) for value in values]
+    interval = moving_block_bootstrap_ci(
+        series,
+        block_size=min(block_size, len(series)),
+        n_resamples=n_resamples,
+        confidence=confidence,
+        seed=seed,
+    )
+    return {
+        "n": len(series),
+        "mean": round(float(interval["estimate"]), 2),
+        "median": round(stdlib_statistics.median(series), 2),
+        "win_rate": round(sum(value > 0 for value in series) / len(series) * 100, 1),
+        "mean_ci_low": round(float(interval["ci_low"]), 2),
+        "mean_ci_high": round(float(interval["ci_high"]), 2),
+    }

config/best_config.json

@@ -27,6 +27,11 @@
       0.3,
       0.5
     ],
+    "return_scales_pct": [
+      10.0,
+      30.0,
+      60.0
+    ],
     "score_range": [
       0,
       100
@@ -61,7 +66,7 @@
     "rolling_test_size": 300,
     "walk_forward_windows": 5,
     "train_pct": 0.7,
-    "validation_pct": 0.15,
+    "validation_pct": 0.3,
     "test_pct": 0.15
   },
   "timeframe": "4h"

config/current_config.json

@@ -27,6 +27,11 @@
       0.3,
       0.5
     ],
+    "return_scales_pct": [
+      10.0,
+      30.0,
+      60.0
+    ],
     "score_range": [
       0,
       100
@@ -61,7 +66,7 @@
     "rolling_test_size": 300,
     "walk_forward_windows": 5,
     "train_pct": 0.7,
-    "validation_pct": 0.15,
+    "validation_pct": 0.3,
     "test_pct": 0.15
   },
   "timeframe": "4h"

config/initial_config.json

@@ -27,6 +27,11 @@
       0.3,
       0.5
     ],
+    "return_scales_pct": [
+      10.0,
+      30.0,
+      60.0
+    ],
     "score_range": [
       0,
       100
@@ -61,7 +66,7 @@
     "rolling_test_size": 300,
     "walk_forward_windows": 5,
     "train_pct": 0.7,
-    "validation_pct": 0.15,
+    "validation_pct": 0.3,
     "test_pct": 0.15
   },
   "timeframe": "4h"

config/llm_settings.example.json

@@ -3,10 +3,10 @@
   "model": "qwen3.5:27b",
   "providers": {
     "ollama": {
-      "base_url": "http://100.100.242.21:11434"
+      "base_url": "http://127.0.0.1:11434"
     },
     "lmstudio": {
-      "base_url": "http://100.100.242.21:1234"
+      "base_url": "http://127.0.0.1:1234"
     },
     "openai": {
       "api_key": ""

config/ml_weights.json

@@ -0,0 +1,383 @@
+{
+  "artifact_schema_version": 2,
+  "score_version": "accumulation-score-v2",
+  "weights": {
+    "pct_above_200w_sma": 0.5075,
+    "drawdown": 0.1716,
+    "pct_above_lth_rp": 0.0934,
+    "rhodl_ratio": 0.08,
+    "fear_greed": 0.0459,
+    "puell_multiple": 0.0384,
+    "reserve_risk": 0.0335,
+    "mvrv_zscore": 0.018,
+    "nupl": 0.0116
+  },
+  "feature_importances": {
+    "raw_pct_above_200w_sma": 0.448165,
+    "days_since_ath": 0.138053,
+    "raw_pct_above_lth_rp": 0.093426,
+    "raw_rhodl_ratio": 0.080038,
+    "score_pct_above_200w_sma": 0.059336,
+    "raw_fear_greed": 0.04573,
+    "puell_x_reserve": 0.034997,
+    "raw_drawdown": 0.033552,
+    "raw_puell_multiple": 0.02023,
+    "raw_reserve_risk": 0.012477,
+    "raw_mvrv_zscore": 0.011366,
+    "mvrv_x_nupl": 0.008204,
+    "raw_nupl": 0.003843,
+    "delta_30d_nupl": 0.003624,
+    "delta_30d_reserve_risk": 0.003541,
+    "delta_30d_mvrv_zscore": 0.002548,
+    "delta_30d_puell_multiple": 0.000677,
+    "score_fear_greed": 0.000182,
+    "score_rhodl_ratio": 1.1e-05,
+    "score_puell_multiple": 0.0,
+    "score_mvrv_zscore": 0.0,
+    "score_reserve_risk": 0.0,
+    "score_nupl": 0.0,
+    "score_drawdown": 0.0,
+    "score_pct_above_lth_rp": 0.0
+  },
+  "cv_results": {
+    "mean_auc": 0.7667,
+    "std_auc": 0.1734,
+    "mean_f1": 0.4085,
+    "mean_precision": 0.3708,
+    "mean_recall": 0.4898,
+    "validation_method": "purged_expanding_window",
+    "label_horizon_days": 365,
+    "folds": [
+      {
+        "fold": 1,
+        "weights": {
+          "pct_above_lth_rp": 0.7418,
+          "drawdown": 0.1523,
+          "reserve_risk": 0.0339,
+          "mvrv_zscore": 0.0211,
+          "puell_multiple": 0.0164,
+          "rhodl_ratio": 0.0141,
+          "nupl": 0.0128,
+          "pct_above_200w_sma": 0.0043,
+          "fear_greed": 0.0033
+        },
+        "metrics": {
+          "auc": 0.7583,
+          "f1": 0.0,
+          "precision": 0.0,
+          "recall": 0.0
+        },
+        "date_ranges": {
+          "train": "2018-02-01 to 2019-08-05",
+          "validation": "2020-08-04 to 2021-10-31"
+        },
+        "n_train": 548,
+        "n_validation": 454
+      },
+      {
+        "fold": 2,
+        "weights": {
+          "reserve_risk": 0.4125,
+          "puell_multiple": 0.3942,
+          "drawdown": 0.085,
+          "pct_above_lth_rp": 0.0429,
+          "pct_above_200w_sma": 0.0324,
+          "rhodl_ratio": 0.0135,
+          "nupl": 0.0131,
+          "mvrv_zscore": 0.0053,
+          "fear_greed": 0.001
+        },
+        "metrics": {
+          "auc": 0.8346,
+          "f1": 0.6582,
+          "precision": 0.4906,
+          "recall": 1.0
+        },
+        "date_ranges": {
+          "train": "2018-02-01 to 2020-11-01",
+          "validation": "2021-11-01 to 2023-01-28"
+        },
+        "n_train": 1002,
+        "n_validation": 454
+      },
+      {
+        "fold": 3,
+        "weights": {
+          "reserve_risk": 0.2919,
+          "puell_multiple": 0.2495,
+          "drawdown": 0.1619,
+          "pct_above_lth_rp": 0.1028,
+          "pct_above_200w_sma": 0.089,
+          "fear_greed": 0.0754,
+          "nupl": 0.0103,
+          "mvrv_zscore": 0.0099,
+          "rhodl_ratio": 0.0091
+        },
+        "metrics": {
+          "auc": 0.9755,
+          "f1": 0.9757,
+          "precision": 0.9926,
+          "recall": 0.9593
+        },
+        "date_ranges": {
+          "train": "2018-02-01 to 2022-01-29",
+          "validation": "2023-01-29 to 2024-04-26"
+        },
+        "n_train": 1456,
+        "n_validation": 454
+      },
+      {
+        "fold": 4,
+        "weights": {
+          "drawdown": 0.6216,
+          "reserve_risk": 0.1302,
+          "puell_multiple": 0.1161,
+          "fear_greed": 0.0554,
+          "rhodl_ratio": 0.0322,
+          "pct_above_lth_rp": 0.0163,
+          "mvrv_zscore": 0.0117,
+          "pct_above_200w_sma": 0.0094,
+          "nupl": 0.0071
+        },
+        "metrics": {
+          "auc": 0.4983,
+          "f1": 0.0,
+          "precision": 0.0,
+          "recall": 0.0
+        },
+        "date_ranges": {
+          "train": "2018-02-01 to 2023-04-28",
+          "validation": "2024-04-27 to 2025-07-26"
+        },
+        "n_train": 1910,
+        "n_validation": 454
+      }
+    ]
+  },
+  "training_info": {
+    "n_samples": 2728,
+    "n_positive": 1554,
+    "positive_rate": 0.5696,
+    "n_features": 25,
+    "target_threshold": 30.0,
+    "date_range": "2018-02-01 to 2025-07-26",
+    "model": "GradientBoostingClassifier"
+  },
+  "provenance": {
+    "validation_method": "purged_expanding_window",
+    "label_horizon_days": 365,
+    "weight_scope": "full_history_fit",
+    "training_date_range": {
+      "start": "2018-02-01",
+      "end": "2025-07-26"
+    },
+    "trained_at": "2026-07-26T23:19:01.231759+00:00"
+  },
+  "comparison": {
+    "equal_weight": [
+      {
+        "range": "0-20",
+        "label": "EXTREME CAUTION",
+        "days": 286,
+        "avg_365d": -7.25,
+        "median_365d": -13.89,
+        "win_rate_365d": 34.3
+      },
+      {
+        "range": "20-35",
+        "label": "CAUTION \u2014 OVERHEATED",
+        "days": 537,
+        "avg_365d": 8.85,
+        "median_365d": -21.49,
+        "win_rate_365d": 35.6
+      },
+      {
+        "range": "35-50",
+        "label": "NEUTRAL",
+        "days": 660,
+        "avg_365d": 85.33,
+        "median_365d": 16.54,
+        "win_rate_365d": 60.8
+      },
+      {
+        "range": "50-65",
+        "label": "MODERATE OPPORTUNITY",
+        "days": 575,
+        "avg_365d": 113.54,
+        "median_365d": 88.64,
+        "win_rate_365d": 88.5
+      },
+      {
+        "range": "65-80",
+        "label": "STRONG ACCUMULATION ZONE",
+        "days": 339,
+        "avg_365d": 183.9,
+        "median_365d": 128.5,
+        "win_rate_365d": 89.7
+      },
+      {
+        "range": "80-100",
+        "label": "EXTREME ACCUMULATION ZONE",
+        "days": 331,
+        "avg_365d": 120.82,
+        "median_365d": 91.85,
+        "win_rate_365d": 99.7
+      }
+    ],
+    "ml_weighted": [
+      {
+        "range": "0-20",
+        "label": "EXTREME CAUTION",
+        "days": 642,
+        "avg_365d": -7.11,
+        "median_365d": -26.48,
+        "win_rate_365d": 25.1
+      },
+      {
+        "range": "20-35",
+        "label": "CAUTION \u2014 OVERHEATED",
+        "days": 679,
+        "avg_365d": 18.61,
+        "median_365d": 4.67,
+        "win_rate_365d": 53.2
+      },
+      {
+        "range": "35-50",
+        "label": "NEUTRAL",
+        "days": 462,
+        "avg_365d": 138.2,
+        "median_365d": 95.1,
+        "win_rate_365d": 87.0
+      },
+      {
+        "range": "50-65",
+        "label": "MODERATE OPPORTUNITY",
+        "days": 277,
+        "avg_365d": 206.31,
+        "median_365d": 163.37,
+        "win_rate_365d": 87.7
+      },
+      {
+        "range": "65-80",
+        "label": "STRONG ACCUMULATION ZONE",
+        "days": 285,
+        "avg_365d": 182.42,
+        "median_365d": 123.75,
+        "win_rate_365d": 99.3
+      },
+      {
+        "range": "80-100",
+        "label": "EXTREME ACCUMULATION ZONE",
+        "days": 383,
+        "avg_365d": 118.93,
+        "median_365d": 119.03,
+        "win_rate_365d": 100.0
+      }
+    ]
+  },
+  "out_of_sample_comparison": {
+    "folds": 4,
+    "validation_days": 1816,
+    "equal_weight": [
+      {
+        "range": "0-20",
+        "label": "EXTREME CAUTION",
+        "days": 281,
+        "avg_365d": -6.2,
+        "median_365d": -13.66,
+        "win_rate_365d": 34.9
+      },
+      {
+        "range": "20-35",
+        "label": "CAUTION \u2014 OVERHEATED",
+        "days": 432,
+        "avg_365d": 20.09,
+        "median_365d": -14.6,
+        "win_rate_365d": 43.1
+      },
+      {
+        "range": "35-50",
+        "label": "NEUTRAL",
+        "days": 419,
+        "avg_365d": 71.09,
+        "median_365d": -9.19,
+        "win_rate_365d": 48.9
+      },
+      {
+        "range": "50-65",
+        "label": "MODERATE OPPORTUNITY",
+        "days": 255,
+        "avg_365d": 101.47,
+        "median_365d": 103.63,
+        "win_rate_365d": 76.9
+      },
+      {
+        "range": "65-80",
+        "label": "STRONG ACCUMULATION ZONE",
+        "days": 237,
+        "avg_365d": 100.95,
+        "median_365d": 122.53,
+        "win_rate_365d": 85.2
+      },
+      {
+        "range": "80-100",
+        "label": "EXTREME ACCUMULATION ZONE",
+        "days": 192,
+        "avg_365d": 76.91,
+        "median_365d": 51.03,
+        "win_rate_365d": 99.5
+      }
+    ],
+    "ml_weighted": [
+      {
+        "range": "0-20",
+        "label": "EXTREME CAUTION",
+        "days": 371,
+        "avg_365d": 9.72,
+        "median_365d": -19.7,
+        "win_rate_365d": 31.3
+      },
+      {
+        "range": "20-35",
+        "label": "CAUTION \u2014 OVERHEATED",
+        "days": 372,
+        "avg_365d": 13.85,
+        "median_365d": -10.98,
+        "win_rate_365d": 42.2
+      },
+      {
+        "range": "35-50",
+        "label": "NEUTRAL",
+        "days": 263,
+        "avg_365d": 107.81,
+        "median_365d": 76.78,
+        "win_rate_365d": 59.7
+      },
+      {
+        "range": "50-65",
+        "label": "MODERATE OPPORTUNITY",
+        "days": 348,
+        "avg_365d": 64.05,
+        "median_365d": 99.55,
+        "win_rate_365d": 64.9
+      },
+      {
+        "range": "65-80",
+        "label": "STRONG ACCUMULATION ZONE",
+        "days": 223,
+        "avg_365d": 121.71,
+        "median_365d": 126.27,
+        "win_rate_365d": 93.7
+      },
+      {
+        "range": "80-100",
+        "label": "EXTREME ACCUMULATION ZONE",
+        "days": 239,
+        "avg_365d": 61.69,
+        "median_365d": 42.02,
+        "win_rate_365d": 89.1
+      }
+    ]
+  },
+  "trained_at": "2026-07-26T23:19:01.231759+00:00"
+}

config/thresholds.json

@@ -1,30 +1,38 @@
 {
+  "_comment": "Cycle-aware thresholds — widened ranges to account for BTC maturing and diminishing cycle extremes",
   "fear_greed": {
-    "ranges": [[0, 10, 10], [11, 25, 7], [26, 45, 4], [46, 55, 2], [56, 75, 1], [76, 100, 0]]
+    "ranges": [[0, 15, 10], [15, 30, 8], [30, 45, 5], [45, 55, 3], [55, 75, 1], [75, 100, 0]]
   },
   "puell_multiple": {
-    "ranges": [[null, 0.3, 10], [0.3, 0.5, 8], [0.5, 0.8, 5], [0.8, 1.2, 3], [1.2, 2.0, 1], [2.0, null, 0]]
+    "_note": "Post-halving floors rising: 2016=0.15, 2020=0.3, 2024=0.5+",
+    "ranges": [[null, 0.4, 10], [0.4, 0.7, 8], [0.7, 1.0, 5], [1.0, 1.5, 3], [1.5, 2.0, 1], [2.0, null, 0]]
   },
   "mvrv_zscore": {
-    "ranges": [[null, 0, 10], [0, 0.5, 8], [0.5, 1.5, 5], [1.5, 3, 2], [3, 5, 1], [5, null, 0]]
+    "_note": "Bottoms getting shallower: 2015=-0.6, 2018=-0.4, 2022=-0.3, next may be ~0",
+    "ranges": [[null, 0, 10], [0, 1.0, 8], [1.0, 2.0, 5], [2.0, 3.0, 3], [3.0, 5.0, 1], [5.0, null, 0]]
   },
   "drawdown": {
-    "ranges": [[70, null, 10], [50, 70, 8], [30, 50, 6], [20, 30, 4], [10, 20, 2], [null, 10, 0]]
+    "_note": "Drawdowns compressing: 2014=86%, 2018=84%, 2022=77%, future may max at 50-60%",
+    "ranges": [[60, null, 10], [40, 60, 8], [25, 40, 6], [15, 25, 4], [5, 15, 2], [null, 5, 0]]
   },
   "price_vs_200w_sma": {
-    "ranges": [[null, 0, 10], [0, 20, 6], [20, 50, 3], [50, 100, 1], [100, null, 0]]
+    "_note": "BTC spends more time above 200W SMA as it matures",
+    "ranges": [[null, 0, 10], [0, 30, 7], [30, 60, 5], [60, 100, 2], [100, null, 0]]
   },
   "reserve_risk": {
     "ranges": [[null, 0.002, 10], [0.002, 0.005, 7], [0.005, 0.01, 4], [0.01, 0.02, 2], [0.02, null, 0]]
   },
   "rhodl_ratio": {
-    "ranges": [[null, 100, 10], [100, 500, 7], [500, 2000, 4], [2000, 10000, 1], [10000, null, 0]]
+    "_note": "RHODL baseline rising with institutional adoption",
+    "ranges": [[null, 200, 10], [200, 1000, 7], [1000, 5000, 4], [5000, 20000, 1], [20000, null, 0]]
   },
   "nupl": {
-    "ranges": [[null, 0, 10], [0, 0.25, 7], [0.25, 0.5, 4], [0.5, 0.75, 1], [0.75, null, 0]]
+    "_note": "NUPL bottoms getting shallower as BTC matures",
+    "ranges": [[null, 0, 10], [0, 0.3, 8], [0.3, 0.5, 4], [0.5, 0.75, 1], [0.75, null, 0]]
   },
   "lth_realized_price": {
-    "ranges": [[null, 0, 10], [0, 20, 6], [20, 50, 3], [50, null, 1]]
+    "_note": "Price stays further above LTH RP as BTC matures — 60% above is still a good entry in 2024+",
+    "ranges": [[null, 0, 10], [0, 30, 7], [30, 80, 5], [80, 150, 3], [150, null, 1]]
   },
   "hash_ribbons": {
     "buy_signal": 10,

dashboard/jobs.py

@@ -0,0 +1,111 @@
+"""Persistent, thread-safe background job state."""
+
+from __future__ import annotations
+
+import threading
+import uuid
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any, Callable
+
+from dashboard.persistence import atomic_write_json, load_json
+
+_ACTIVE = {"queued", "running"}
+
+
+def _now() -> str:
+    return datetime.now(timezone.utc).isoformat()
+
+
+class JobRegistry:
+    """Reserve jobs before spawning and persist their lifecycle."""
+
+    def __init__(self, path: str | Path, *, history_limit: int = 100):
+        self.path = Path(path)
+        self.history_limit = history_limit
+        self._lock = threading.RLock()
+        loaded = load_json(self.path, {"jobs": []}) or {"jobs": []}
+        self._jobs = {
+            job["id"]: dict(job)
+            for job in loaded.get("jobs", [])
+            if isinstance(job, dict) and job.get("id")
+        }
+        changed = False
+        for job in self._jobs.values():
+            if job.get("status") in _ACTIVE:
+                job.update(
+                    status="interrupted",
+                    finished_at=_now(),
+                    error="process restarted before job completed",
+                )
+                changed = True
+        if changed:
+            self._save_locked()
+
+    def _save_locked(self) -> None:
+        jobs = sorted(self._jobs.values(), key=lambda job: job.get("created_at", ""))
+        if len(jobs) > self.history_limit:
+            keep = jobs[-self.history_limit :]
+            self._jobs = {job["id"]: job for job in keep}
+            jobs = keep
+        atomic_write_json(self.path, {"jobs": jobs})
+
+    def reserve(self, kind: str, *, details: dict[str, Any] | None = None) -> dict[str, Any] | None:
+        with self._lock:
+            if any(
+                job.get("kind") == kind and job.get("status") in _ACTIVE
+                for job in self._jobs.values()
+            ):
+                return None
+            job = {
+                "id": uuid.uuid4().hex,
+                "kind": kind,
+                "status": "queued",
+                "created_at": _now(),
+                "started_at": None,
+                "finished_at": None,
+                "progress": {},
+                "details": details or {},
+                "result": None,
+                "error": None,
+            }
+            self._jobs[job["id"]] = job
+            self._save_locked()
+            return dict(job)
+
+    def get(self, job_id: str) -> dict[str, Any] | None:
+        with self._lock:
+            job = self._jobs.get(job_id)
+            return dict(job) if job else None
+
+    def active(self, kind: str) -> dict[str, Any] | None:
+        with self._lock:
+            for job in self._jobs.values():
+                if job.get("kind") == kind and job.get("status") in _ACTIVE:
+                    return dict(job)
+        return None
+
+    def update_progress(self, job_id: str, progress: dict[str, Any]) -> None:
+        with self._lock:
+            job = self._jobs[job_id]
+            job["progress"] = dict(progress)
+            self._save_locked()
+
+    def run(self, job_id: str, operation: Callable[[], Any]) -> Any:
+        with self._lock:
+            job = self._jobs[job_id]
+            if job["status"] != "queued":
+                raise RuntimeError(f"job {job_id} is not queued")
+            job.update(status="running", started_at=_now())
+            self._save_locked()
+        try:
+            result = operation()
+        except Exception as exc:
+            with self._lock:
+                job.update(status="error", error=str(exc), finished_at=_now())
+                self._save_locked()
+            raise
+        with self._lock:
+            job.update(status="complete", result=result, finished_at=_now())
+            self._save_locked()
+        return result

dashboard/persistence.py

@@ -0,0 +1,222 @@
+"""Small, dependency-free persistence primitives for dashboard state."""
+
+from __future__ import annotations
+
+import json
+import os
+import tempfile
+import threading
+from contextlib import contextmanager
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any, Iterator
+
+try:
+    import fcntl
+except ImportError:  # pragma: no cover - Windows fallback uses the process lock
+    fcntl = None
+
+
+_LOCKS: dict[str, threading.RLock] = {}
+_LOCKS_GUARD = threading.Lock()
+_METADATA_KEYS = {"observed_at", "source", "stale", "last_error", "error"}
+
+
+def _thread_lock(path: Path) -> threading.RLock:
+    key = str(path.resolve())
+    with _LOCKS_GUARD:
+        return _LOCKS.setdefault(key, threading.RLock())
+
+
+@contextmanager
+def file_lock(path: str | os.PathLike[str]) -> Iterator[None]:
+    """Serialize readers/writers across threads and, on POSIX, processes."""
+    target = Path(path)
+    target.parent.mkdir(parents=True, exist_ok=True)
+    lock_path = target.with_name(f".{target.name}.lock")
+    with _thread_lock(target):
+        with lock_path.open("a+b") as lock_file:
+            if fcntl is not None:
+                fcntl.flock(lock_file.fileno(), fcntl.LOCK_EX)
+            try:
+                yield
+            finally:
+                if fcntl is not None:
+                    fcntl.flock(lock_file.fileno(), fcntl.LOCK_UN)
+
+
+def atomic_write_json(path: str | os.PathLike[str], data: Any, *, indent: int = 2) -> None:
+    """Durably replace a JSON file without exposing a partial document."""
+    target = Path(path)
+    target.parent.mkdir(parents=True, exist_ok=True)
+    with file_lock(target):
+        fd, temporary = tempfile.mkstemp(
+            prefix=f".{target.name}.", suffix=".tmp", dir=target.parent
+        )
+        try:
+            with os.fdopen(fd, "w", encoding="utf-8") as handle:
+                json.dump(data, handle, indent=indent, default=str)
+                handle.write("\n")
+                handle.flush()
+                os.fsync(handle.fileno())
+            os.replace(temporary, target)
+            try:
+                directory_fd = os.open(target.parent, os.O_DIRECTORY)
+                try:
+                    os.fsync(directory_fd)
+                finally:
+                    os.close(directory_fd)
+            except (AttributeError, OSError):
+                pass
+        finally:
+            try:
+                os.unlink(temporary)
+            except FileNotFoundError:
+                pass
+
+
+def load_json(path: str | os.PathLike[str], default: Any = None) -> Any:
+    target = Path(path)
+    if not target.exists():
+        return default
+    with file_lock(target):
+        try:
+            with target.open(encoding="utf-8") as handle:
+                return json.load(handle)
+        except (OSError, ValueError):
+            return default
+
+
+def _tail_bytes(target: Path, *, line_hint: int, chunk_size: int) -> bytes:
+    with target.open("rb") as handle:
+        handle.seek(0, os.SEEK_END)
+        position = handle.tell()
+        blocks: list[bytes] = []
+        newlines = 0
+        while position > 0 and newlines <= line_hint:
+            size = min(chunk_size, position)
+            position -= size
+            handle.seek(position)
+            block = handle.read(size)
+            blocks.append(block)
+            newlines += block.count(b"\n")
+        return b"".join(reversed(blocks))
+
+
+def load_jsonl_tail(
+    path: str | os.PathLike[str], *, limit: int = 90, chunk_size: int = 8192
+) -> list[dict[str, Any]]:
+    """Read only enough of a JSONL file to return its last valid entries."""
+    if limit <= 0:
+        return []
+    target = Path(path)
+    if not target.exists():
+        return []
+    with file_lock(target):
+        raw = _tail_bytes(target, line_hint=limit + 8, chunk_size=max(chunk_size, 32))
+    entries: list[dict[str, Any]] = []
+    for line in raw.splitlines():
+        try:
+            value = json.loads(line)
+        except (UnicodeDecodeError, ValueError):
+            continue
+        if isinstance(value, dict):
+            entries.append(value)
+    return entries[-limit:]
+
+
+def _utc_day(timestamp: Any) -> str | None:
+    if not isinstance(timestamp, str):
+        return None
+    try:
+        parsed = datetime.fromisoformat(timestamp.replace("Z", "+00:00"))
+        if parsed.tzinfo is None:
+            parsed = parsed.replace(tzinfo=timezone.utc)
+        return parsed.astimezone(timezone.utc).date().isoformat()
+    except ValueError:
+        return None
+
+
+def append_daily_jsonl(path: str | os.PathLike[str], entry: dict[str, Any]) -> bool:
+    """Append at most one record per UTC day, inspecting only the bounded tail."""
+    target = Path(path)
+    target.parent.mkdir(parents=True, exist_ok=True)
+    entry_day = _utc_day(entry.get("timestamp"))
+    if entry_day is None:
+        raise ValueError("entry timestamp must be an ISO-8601 datetime")
+    with file_lock(target):
+        if target.exists():
+            raw = _tail_bytes(target, line_hint=8, chunk_size=4096)
+            for line in reversed(raw.splitlines()):
+                try:
+                    previous = json.loads(line)
+                except (UnicodeDecodeError, ValueError):
+                    continue
+                if _utc_day(previous.get("timestamp")) == entry_day:
+                    return False
+                break
+        payload = (json.dumps(entry, default=str) + "\n").encode("utf-8")
+        fd = os.open(target, os.O_WRONLY | os.O_CREAT | os.O_APPEND, 0o644)
+        try:
+            os.write(fd, payload)
+            os.fsync(fd)
+        finally:
+            os.close(fd)
+    return True
+
+
+def has_observation(payload: Any) -> bool:
+    if not isinstance(payload, dict):
+        return payload is not None
+    return any(value is not None for key, value in payload.items() if key not in _METADATA_KEYS)
+
+
+def merge_observation(
+    previous: Any,
+    observed: Any,
+    *,
+    source: str,
+    observed_at: str | None = None,
+    error: str | None = None,
+) -> dict[str, Any]:
+    """Annotate a fresh observation or retain the last-known-good value as stale."""
+    if has_observation(observed):
+        merged = dict(observed) if isinstance(observed, dict) else {"value": observed}
+        merged.update(
+            observed_at=observed_at or datetime.now(timezone.utc).isoformat(),
+            source=source,
+            stale=False,
+            last_error=None,
+        )
+        return merged
+
+    merged = dict(previous) if isinstance(previous, dict) else {}
+    observed_error = observed.get("error") if isinstance(observed, dict) else None
+    merged.update(
+        source=merged.get("source") or source,
+        stale=True,
+        last_error=observed_error or error or "metric was not observed",
+    )
+    merged.setdefault("observed_at", None)
+    return merged
+
+
+def onchain_refresh_due(
+    timestamp: Any,
+    *,
+    now: datetime | None = None,
+    ttl_seconds: int = 6 * 60 * 60,
+) -> bool:
+    """Return whether the last successful on-chain observation exceeded its TTL."""
+    if not isinstance(timestamp, str) or not timestamp:
+        return True
+    try:
+        observed = datetime.fromisoformat(timestamp.replace("Z", "+00:00"))
+        if observed.tzinfo is None:
+            observed = observed.replace(tzinfo=timezone.utc)
+    except ValueError:
+        return True
+    current = now or datetime.now(timezone.utc)
+    if current.tzinfo is None:
+        current = current.replace(tzinfo=timezone.utc)
+    return (current.astimezone(timezone.utc) - observed.astimezone(timezone.utc)).total_seconds() >= ttl_seconds

data/score_history.jsonl

@@ -1,4 +0,0 @@
-{"timestamp": "2026-03-20T22:26:50.475811+00:00", "composite_score": 32.5, "scored_count": 8, "metrics": {"fear_greed": {"score": 7, "value": 11}, "puell_multiple": {"score": 5, "value": 0.6602699608966011}, "mvrv_zscore": {"score": 5, "value": 0.5211180167687892}, "drawdown": {"score": 6, "value": 43.891180203045685}, "price_vs_200w_sma": {"score": null, "value": 0.0}, "reserve_risk": {"score": 0, "value": 69871.0}, "rhodl_ratio": {"score": 0, "value": 69871.0}, "nupl": {"score": 0, "value": 69871.0}, "lth_realized_price": {"score": null, "value": null}, "hash_ribbons": {"score": 3, "value": null}}}
-{"timestamp": "2026-03-20T22:30:13.547149+00:00", "composite_score": 51.0, "scored_count": 10, "metrics": {"fear_greed": {"score": 7, "value": 11}, "puell_multiple": {"score": 5, "value": 0.6602699608966011}, "mvrv_zscore": {"score": 5, "value": 0.5211180167687892}, "drawdown": {"score": 6, "value": 43.910215736040605}, "price_vs_200w_sma": {"score": 3, "value": 58895.78086828114}, "reserve_risk": {"score": 10, "value": 0.0012985709697654493}, "rhodl_ratio": {"score": 4, "value": 1230.6243545314708}, "nupl": {"score": 7, "value": 0.22243290955405431}, "lth_realized_price": {"score": 1, "value": 43346.58756410873}, "hash_ribbons": {"score": 3, "value": null}}}
-{"timestamp": "2026-03-20T22:46:34.952569+00:00", "composite_score": 51.0, "scored_count": 10, "metrics": {"fear_greed": {"score": 7, "value": 11}, "puell_multiple": {"score": 5, "value": 0.6602699608966011}, "mvrv_zscore": {"score": 5, "value": 0.5211180167687892}, "drawdown": {"score": 6, "value": 43.931630710659896}, "price_vs_200w_sma": {"score": 3, "value": 58895.78086828114}, "reserve_risk": {"score": 10, "value": 0.0012985709697654493}, "rhodl_ratio": {"score": 4, "value": 1230.6243545314708}, "nupl": {"score": 7, "value": 0.22243290955405431}, "lth_realized_price": {"score": 1, "value": 43346.58756410873}, "hash_ribbons": {"score": 3, "value": null}}}
-{"timestamp": "2026-03-20T22:51:27.724327+00:00", "composite_score": 54.0, "scored_count": 10, "metrics": {"fear_greed": {"score": 7, "value": 11}, "puell_multiple": {"score": 5, "value": 0.6602699608966011}, "mvrv_zscore": {"score": 5, "value": 0.5211180167687892}, "drawdown": {"score": 6, "value": 43.94907994923858}, "price_vs_200w_sma": {"score": 6, "value": 58895.78086828114}, "reserve_risk": {"score": 10, "value": 0.0012985709697654493}, "rhodl_ratio": {"score": 4, "value": 1230.6243545314708}, "nupl": {"score": 7, "value": 0.22243290955405431}, "lth_realized_price": {"score": 1, "value": 43346.58756410873}, "hash_ribbons": {"score": 3, "value": null}}}

docker-compose.yml

@@ -0,0 +1,27 @@
+services:
+  btc-monitor:
+    build:
+      context: .
+    image: btc-accumulation-monitor:local
+    ports:
+      - "3088:3088"
+    restart: unless-stopped
+    environment:
+      PLAYWRIGHT_BROWSERS_PATH: /ms-playwright
+    volumes:
+      - btc-monitor-data:/app/data
+      - btc-monitor-config:/app/config
+    healthcheck:
+      test:
+        - CMD
+        - /app/.venv/bin/python
+        - -c
+        - "import urllib.request; urllib.request.urlopen('http://127.0.0.1:3088/health/live', timeout=3)"
+      interval: 30s
+      timeout: 5s
+      start_period: 30s
+      retries: 3
+
+volumes:
+  btc-monitor-data:
+  btc-monitor-config:

ml/artifacts.py

@@ -0,0 +1,91 @@
+"""Validation for persisted ML scoring artifacts."""
+
+import math
+
+from scoring.policy import SCORE_VERSION
+
+ML_ARTIFACT_SCHEMA_VERSION = 2
+REQUIRED_WEIGHT_KEYS = frozenset({
+    "puell_multiple",
+    "mvrv_zscore",
+    "reserve_risk",
+    "rhodl_ratio",
+    "nupl",
+    "fear_greed",
+    "drawdown",
+    "pct_above_200w_sma",
+    "pct_above_lth_rp",
+})
+
+
+def _weights_valid(weights):
+    if not isinstance(weights, dict) or not REQUIRED_WEIGHT_KEYS.issubset(weights):
+        return False
+    values = [weights[key] for key in REQUIRED_WEIGHT_KEYS]
+    return all(
+        isinstance(value, (int, float))
+        and not isinstance(value, bool)
+        and math.isfinite(value)
+        and value >= 0
+        for value in values
+    ) and sum(values) > 0
+
+
+def _has_oos_fold_weights(artifact):
+    folds = artifact.get("cv_results", {}).get("folds", [])
+    if not isinstance(folds, list) or not folds:
+        return False
+    for fold in folds:
+        validation_range = fold.get("date_ranges", {}).get("validation")
+        if not validation_range or not _weights_valid(fold.get("weights")):
+            return False
+    return True
+
+
+def validate_ml_artifact(artifact):
+    """Return machine-readable validity and provenance for an ML artifact."""
+    errors = []
+    if not isinstance(artifact, dict):
+        artifact = {}
+        errors.append("artifact_object")
+
+    schema_version = artifact.get("artifact_schema_version")
+    if schema_version != ML_ARTIFACT_SCHEMA_VERSION:
+        errors.append("artifact_schema_version")
+
+    score_version = artifact.get("score_version")
+    if score_version != SCORE_VERSION:
+        errors.append("score_version")
+
+    if not _weights_valid(artifact.get("weights")):
+        errors.append("weights")
+
+    provenance = artifact.get("provenance")
+    if not isinstance(provenance, dict):
+        provenance = {}
+        errors.append("provenance")
+    else:
+        required_provenance = {
+            "validation_method",
+            "label_horizon_days",
+            "weight_scope",
+            "training_date_range",
+            "trained_at",
+        }
+        if not required_provenance.issubset(provenance):
+            errors.append("provenance")
+        if provenance.get("validation_method") != "purged_expanding_window":
+            errors.append("purged_validation")
+        if provenance.get("label_horizon_days") != 365:
+            errors.append("label_horizon_days")
+        if provenance.get("weight_scope") != "full_history_fit":
+            errors.append("weight_scope")
+
+    return {
+        "valid": not errors,
+        "schema_version": schema_version,
+        "score_version": score_version,
+        "weight_scope": provenance.get("weight_scope"),
+        "has_oos_fold_weights": _has_oos_fold_weights(artifact),
+        "errors": list(dict.fromkeys(errors)),
+    }

ml/optimizer.py

@@ -0,0 +1,682 @@
+#!/usr/bin/env python3
+"""
+ML Optimizer for Bitcoin Accumulation Zone Scoring.
+
+Trains a gradient boosted tree model on historical on-chain metrics to find
+optimal metric weights for identifying the best long-term buying opportunities.
+
+Output: config/ml_weights.json with optimized weights and feature importances.
+"""
+
+import json
+import logging
+import os
+import sys
+from datetime import datetime, timedelta
+
+import numpy as np
+from sklearn.ensemble import GradientBoostingClassifier
+from sklearn.metrics import (
+    classification_report,
+    f1_score,
+    precision_score,
+    recall_score,
+    roc_auc_score,
+)
+from sklearn.model_selection import TimeSeriesSplit
+from sklearn.preprocessing import StandardScaler
+
+from scoring.policy import SCORE_BRACKETS, SCORE_VERSION, score_in_bracket
+from ml.artifacts import ML_ARTIFACT_SCHEMA_VERSION
+
+logging.basicConfig(
+    level=logging.INFO,
+    format="%(asctime)s [%(name)s] %(levelname)s: %(message)s",
+)
+log = logging.getLogger("ml-optimizer")
+
+BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+HISTORY_PATH = os.path.join(BASE_DIR, "data", "history.json")
+OUTPUT_PATH = os.path.join(BASE_DIR, "config", "ml_weights.json")
+THRESHOLDS_PATH = os.path.join(BASE_DIR, "config", "thresholds.json")
+
+# Date range: 2018-02-01 onward (when all 8 metrics + fear_greed available)
+START_DATE = "2018-02-01"
+# Training cutoff: need 1yr forward data for labels
+TRAIN_CUTOFF_DAYS = 365
+# Target: forward 365d return > 30% = "good time to buy"
+GOOD_BUY_THRESHOLD = 30.0
+# Validation embargo/purge horizon: labels use 365-day forward returns.
+LABEL_HORIZON_DAYS = 365
+VALIDATION_SPLITS = 5
+
+# The 8 core metrics we score
+METRIC_KEYS = [
+    "puell_multiple",
+    "mvrv_zscore",
+    "reserve_risk",
+    "rhodl_ratio",
+    "nupl",
+    "fear_greed",
+]
+# Ratio-based metrics (derived from price vs reference)
+RATIO_METRICS = {
+    "pct_above_200w_sma": {"price_key": "btc_price", "ref_key": "200w_sma"},
+    "pct_above_lth_rp": {"price_key": "btc_price", "ref_key": "lth_realized_price"},
+}
+
+
+def load_history():
+    """Load historical data and build date-aligned lookup."""
+    with open(HISTORY_PATH) as f:
+        raw = json.load(f)
+
+    index = {}
+    for key, data in raw.items():
+        if not isinstance(data, dict) or "dates" not in data:
+            continue
+        lookup = {}
+        for d, v in zip(data["dates"], data["values"]):
+            if v is not None:
+                lookup[d] = v
+        index[key] = lookup
+    return index
+
+
+def load_thresholds():
+    """Load scoring thresholds for converting raw values to 0-10 scores."""
+    with open(THRESHOLDS_PATH) as f:
+        return json.load(f)
+
+
+def score_range(value, ranges):
+    """Score a value using range-based thresholds (same logic as scoring/engine.py)."""
+    if value is None:
+        return None
+    for low, high, score in ranges:
+        low_ok = low is None or value >= low
+        high_ok = high is None or value < high
+        if low_ok and high_ok:
+            return score
+    return 0
+
+
+SCORE_KEYS = [
+    "puell_multiple", "mvrv_zscore", "reserve_risk", "rhodl_ratio",
+    "nupl", "fear_greed", "drawdown", "pct_above_200w_sma", "pct_above_lth_rp",
+]
+
+SCORE_FEATURES = [f"score_{k}" for k in SCORE_KEYS]
+RAW_FEATURES = [
+    "raw_puell_multiple", "raw_mvrv_zscore", "raw_reserve_risk",
+    "raw_rhodl_ratio", "raw_nupl", "raw_fear_greed",
+    "raw_pct_above_200w_sma", "raw_pct_above_lth_rp", "raw_drawdown",
+]
+DELTA_FEATURES = [
+    "delta_30d_mvrv_zscore", "delta_30d_nupl",
+    "delta_30d_puell_multiple", "delta_30d_reserve_risk",
+]
+INTERACTION_FEATURES = ["mvrv_x_nupl", "puell_x_reserve"]
+CYCLE_FEATURES = ["days_since_ath"]
+FEATURE_COLS = SCORE_FEATURES + RAW_FEATURES + DELTA_FEATURES + INTERACTION_FEATURES + CYCLE_FEATURES
+
+BRACKETS = SCORE_BRACKETS
+
+
+def _row_date(row):
+    return datetime.strptime(row["date"], "%Y-%m-%d")
+
+
+def purged_time_series_splits(rows, n_splits=VALIDATION_SPLITS,
+                              label_horizon_days=LABEL_HORIZON_DAYS,
+                              embargo_days=0):
+    """Yield expanding-window splits with overlapping forward-label windows removed.
+
+    A row dated T with a 365-day forward-return label consumes information up to
+    T+365. For validation beginning at V, any training row whose label window
+    reaches V is removed. This keeps validation metrics out-of-sample for the
+    forward-return label, not just for features.
+    """
+    base_splitter = TimeSeriesSplit(n_splits=n_splits)
+    row_dates = [_row_date(r) for r in rows]
+    horizon = timedelta(days=label_horizon_days)
+    embargo = timedelta(days=embargo_days)
+
+    for train_idx, val_idx in base_splitter.split(np.arange(len(rows))):
+        val_start = row_dates[val_idx[0]]
+        val_end = row_dates[val_idx[-1]]
+        purged_train = []
+        for idx in train_idx:
+            label_end = row_dates[idx] + horizon
+            before_validation_label_window = label_end <= val_start - embargo
+            after_validation_embargo = row_dates[idx] > val_end + embargo
+            if before_validation_label_window or after_validation_embargo:
+                purged_train.append(idx)
+        if purged_train:
+            yield np.array(purged_train, dtype=int), np.array(val_idx, dtype=int)
+
+
+def viable_classification_splits(y, splits):
+    """Yield only folds whose training window contains both target classes."""
+    for train_idx, val_idx in splits:
+        if len(np.unique(y[train_idx])) < 2:
+            continue
+        yield train_idx, val_idx
+
+
+def artifact_fold_results(fold_results):
+    """Strip training-only row indexes from the persisted ML artifact."""
+    return [
+        {key: value for key, value in fold.items() if key not in {"train_idx", "val_idx"}}
+        for fold in fold_results
+    ]
+
+
+def _build_model():
+    return GradientBoostingClassifier(
+        n_estimators=300,
+        learning_rate=0.05,
+        max_depth=4,
+        subsample=0.8,
+        min_samples_leaf=20,
+        random_state=42,
+    )
+
+
+def derive_metric_weights(feature_cols, importances):
+    """Aggregate feature importances back to transparent score metric weights."""
+    metric_names = list(SCORE_KEYS)
+    feature_to_metric = {}
+    for m in metric_names:
+        feature_to_metric[f"score_{m}"] = m
+        feature_to_metric[f"raw_{m}"] = m
+    feature_to_metric["delta_30d_mvrv_zscore"] = "mvrv_zscore"
+    feature_to_metric["delta_30d_nupl"] = "nupl"
+    feature_to_metric["delta_30d_puell_multiple"] = "puell_multiple"
+    feature_to_metric["delta_30d_reserve_risk"] = "reserve_risk"
+
+    metric_importances = {m: 0.0 for m in metric_names}
+    for name, imp in zip(feature_cols, importances):
+        if name in feature_to_metric:
+            metric_importances[feature_to_metric[name]] += float(imp)
+        elif name == "mvrv_x_nupl":
+            metric_importances["mvrv_zscore"] += float(imp) / 2
+            metric_importances["nupl"] += float(imp) / 2
+        elif name == "puell_x_reserve":
+            metric_importances["puell_multiple"] += float(imp) / 2
+            metric_importances["reserve_risk"] += float(imp) / 2
+        elif name == "days_since_ath":
+            metric_importances["drawdown"] += float(imp)
+
+    total_imp = sum(metric_importances.values())
+    if total_imp > 0:
+        weights = {k: round(v / total_imp, 4) for k, v in metric_importances.items()}
+    else:
+        weights = {k: round(1 / len(metric_importances), 4) for k in metric_importances}
+    return dict(sorted(weights.items(), key=lambda x: x[1], reverse=True))
+
+
+def build_dataset(index, thresholds):
+    """Build aligned training dataset: metric scores + forward returns."""
+    # Get all dates from 2018-02-01 onward
+    all_dates = set()
+    for lookup in index.values():
+        all_dates.update(lookup.keys())
+    dates = sorted(d for d in all_dates if d >= START_DATE)
+
+    # Build price lookup for forward returns
+    price_lookup = {}
+    for pk in ["btc_price", "btc_price_sma", "btc_price_lth"]:
+        if pk in index:
+            for d, v in index[pk].items():
+                if d not in price_lookup:
+                    price_lookup[d] = v
+
+    # Compute ATH series for drawdown
+    all_dates_sorted = sorted(all_dates)
+    ath = 0
+    drawdowns = {}
+    for d in all_dates_sorted:
+        p = price_lookup.get(d)
+        if p is None:
+            continue
+        if p > ath:
+            ath = p
+        if ath > 0:
+            drawdowns[d] = ((ath - p) / ath) * 100
+
+    # Get threshold ranges for scoring raw values
+    metric_ranges = {
+        "puell_multiple": thresholds.get("puell_multiple", {}).get("ranges", []),
+        "mvrv_zscore": thresholds.get("mvrv_zscore", {}).get("ranges", []),
+        "reserve_risk": thresholds.get("reserve_risk", {}).get("ranges", []),
+        "rhodl_ratio": thresholds.get("rhodl_ratio", {}).get("ranges", []),
+        "nupl": thresholds.get("nupl", {}).get("ranges", []),
+        "fear_greed": thresholds.get("fear_greed", {}).get("ranges", []),
+        "drawdown": thresholds.get("drawdown", {}).get("ranges", []),
+        "price_vs_200w_sma": thresholds.get("price_vs_200w_sma", {}).get("ranges", []),
+        "lth_realized_price": thresholds.get("lth_realized_price", {}).get("ranges", []),
+    }
+
+    log.info("Building dataset from %d dates (%s to %s)", len(dates), dates[0], dates[-1])
+
+    rows = []
+    for d in dates:
+        # Get raw metric values
+        vals = {}
+        skip = False
+        for key in METRIC_KEYS:
+            v = index.get(key, {}).get(d)
+            if v is None:
+                skip = True
+                break
+            vals[key] = v
+        if skip:
+            continue
+
+        # Compute ratio metrics
+        price = price_lookup.get(d)
+        sma_200w = index.get("200w_sma", {}).get(d)
+        lth_rp = index.get("lth_realized_price", {}).get(d)
+
+        if price is None or sma_200w is None or lth_rp is None:
+            continue
+        if sma_200w == 0 or lth_rp == 0:
+            continue
+
+        pct_200w = ((price - sma_200w) / sma_200w) * 100
+        pct_lth = ((price - lth_rp) / lth_rp) * 100
+        dd = drawdowns.get(d, 0)
+
+        vals["pct_above_200w_sma"] = pct_200w
+        vals["pct_above_lth_rp"] = pct_lth
+        vals["drawdown"] = dd
+
+        # Score each metric (0-10) using existing thresholds
+        scores = {}
+        scores["puell_multiple"] = score_range(vals["puell_multiple"], metric_ranges["puell_multiple"])
+        scores["mvrv_zscore"] = score_range(vals["mvrv_zscore"], metric_ranges["mvrv_zscore"])
+        scores["reserve_risk"] = score_range(vals["reserve_risk"], metric_ranges["reserve_risk"])
+        scores["rhodl_ratio"] = score_range(vals["rhodl_ratio"], metric_ranges["rhodl_ratio"])
+        scores["nupl"] = score_range(vals["nupl"], metric_ranges["nupl"])
+        scores["fear_greed"] = score_range(vals["fear_greed"], metric_ranges["fear_greed"])
+        scores["drawdown"] = score_range(dd, metric_ranges["drawdown"])
+        scores["pct_above_200w_sma"] = score_range(pct_200w, metric_ranges["price_vs_200w_sma"])
+        scores["pct_above_lth_rp"] = score_range(pct_lth, metric_ranges["lth_realized_price"])
+
+        if any(s is None for s in scores.values()):
+            continue
+
+        # Forward returns
+        dt = datetime.strptime(d, "%Y-%m-%d")
+        fwd = {}
+        for days in [30, 90, 180, 365]:
+            future_d = (dt + timedelta(days=days)).strftime("%Y-%m-%d")
+            fp = price_lookup.get(future_d)
+            if fp is not None and price > 0:
+                fwd[f"fwd_{days}d"] = ((fp - price) / price) * 100
+
+        # Compute rate-of-change features (30d deltas)
+        deltas = {}
+        d_30ago = (dt - timedelta(days=30)).strftime("%Y-%m-%d")
+        for key in ["mvrv_zscore", "nupl", "puell_multiple", "reserve_risk"]:
+            v_now = vals[key]
+            v_prev = index.get(key, {}).get(d_30ago)
+            if v_prev is not None and v_prev != 0:
+                deltas[f"delta_30d_{key}"] = v_now - v_prev
+            else:
+                deltas[f"delta_30d_{key}"] = 0.0
+
+        # Interaction terms
+        interactions = {
+            "mvrv_x_nupl": vals["mvrv_zscore"] * vals["nupl"],
+            "puell_x_reserve": vals["puell_multiple"] * vals["reserve_risk"],
+        }
+
+        # Days since last ATH
+        days_since_ath = 0
+        for i in range(1, 2000):
+            check_d = (dt - timedelta(days=i)).strftime("%Y-%m-%d")
+            check_dd = drawdowns.get(check_d, 100)
+            if check_dd < 0.1:  # essentially at ATH
+                days_since_ath = i
+                break
+        else:
+            days_since_ath = 2000
+
+        row = {
+            "date": d,
+            "price": price,
+            **{f"score_{k}": v for k, v in scores.items()},
+            **{f"raw_{k}": v for k, v in vals.items()},
+            **deltas,
+            **interactions,
+            "days_since_ath": days_since_ath,
+            **fwd,
+        }
+        rows.append(row)
+
+    log.info("Built %d complete data rows", len(rows))
+    return rows
+
+
+def train_model(rows):
+    """Train gradient boosted classifier to identify good buying opportunities."""
+    # Filter to rows that have 365d forward return (for labeling)
+    labeled = [r for r in rows if "fwd_365d" in r]
+    log.info("Rows with 365d forward data: %d", len(labeled))
+
+    if len(labeled) < 100:
+        log.error("Not enough labeled data. Need at least 100 rows, got %d", len(labeled))
+        return None
+
+    # Create binary target: forward 365d return > threshold
+    for r in labeled:
+        r["target"] = 1 if r["fwd_365d"] > GOOD_BUY_THRESHOLD else 0
+
+    positive = sum(r["target"] for r in labeled)
+    log.info("Target distribution: %d positive (%.1f%%), %d negative",
+             positive, positive / len(labeled) * 100, len(labeled) - positive)
+
+    feature_cols = FEATURE_COLS
+
+    X = np.array([[r[f] for f in feature_cols] for r in labeled])
+    y = np.array([r["target"] for r in labeled])
+
+    log.info("Feature matrix: %d samples x %d features", X.shape[0], X.shape[1])
+
+    # Purged time-series cross-validation. Standard TimeSeriesSplit is not
+    # enough here because each label consumes the next 365 days of returns.
+    cv_scores = []
+    cv_f1 = []
+    cv_precision = []
+    cv_recall = []
+    fold_results = []
+
+    splits = list(viable_classification_splits(y, purged_time_series_splits(
+        labeled,
+        n_splits=VALIDATION_SPLITS,
+        label_horizon_days=LABEL_HORIZON_DAYS,
+        embargo_days=0,
+    )))
+    if not splits:
+        log.error("No viable purged validation splits. Need more history for %dd label horizon.",
+                  LABEL_HORIZON_DAYS)
+        return None
+
+    for fold, (train_idx, val_idx) in enumerate(splits):
+        X_train, X_val = X[train_idx], X[val_idx]
+        y_train, y_val = y[train_idx], y[val_idx]
+
+        scaler = StandardScaler()
+        X_train_s = scaler.fit_transform(X_train)
+        X_val_s = scaler.transform(X_val)
+
+        model = _build_model()
+        model.fit(X_train_s, y_train)
+
+        y_pred = model.predict(X_val_s)
+        y_prob = model.predict_proba(X_val_s)[:, 1]
+
+        auc = roc_auc_score(y_val, y_prob) if len(np.unique(y_val)) > 1 else 0
+        f1 = f1_score(y_val, y_pred, zero_division=0)
+        prec = precision_score(y_val, y_pred, zero_division=0)
+        rec = recall_score(y_val, y_pred, zero_division=0)
+
+        cv_scores.append(auc)
+        cv_f1.append(f1)
+        cv_precision.append(prec)
+        cv_recall.append(rec)
+
+        fold_weights = derive_metric_weights(feature_cols, model.feature_importances_)
+        fold_results.append({
+            "fold": fold + 1,
+            "train_idx": train_idx.tolist(),
+            "val_idx": val_idx.tolist(),
+            "weights": fold_weights,
+            "metrics": {
+                "auc": round(float(auc), 4),
+                "f1": round(float(f1), 4),
+                "precision": round(float(prec), 4),
+                "recall": round(float(rec), 4),
+            },
+            "date_ranges": {
+                "train": f"{labeled[train_idx[0]]['date']} to {labeled[train_idx[-1]]['date']}",
+                "validation": f"{labeled[val_idx[0]]['date']} to {labeled[val_idx[-1]]['date']}",
+            },
+            "n_train": len(train_idx),
+            "n_validation": len(val_idx),
+        })
+
+        train_dates = fold_results[-1]["date_ranges"]["train"]
+        val_dates = fold_results[-1]["date_ranges"]["validation"]
+        log.info("Fold %d: Train %s | Val %s | AUC=%.3f F1=%.3f P=%.3f R=%.3f",
+                 fold + 1, train_dates, val_dates, auc, f1, prec, rec)
+
+    log.info("Purged CV Mean AUC: %.3f (+/- %.3f)", np.mean(cv_scores), np.std(cv_scores))
+    log.info("Purged CV Mean F1:  %.3f (+/- %.3f)", np.mean(cv_f1), np.std(cv_f1))
+
+    # Train final model on all labeled data
+    log.info("Training final model on all %d labeled samples...", len(labeled))
+    scaler = StandardScaler()
+    X_scaled = scaler.fit_transform(X)
+
+    final_model = _build_model()
+    final_model.fit(X_scaled, y)
+
+    # Feature importances
+    importances = final_model.feature_importances_
+    feat_imp = sorted(
+        zip(feature_cols, importances),
+        key=lambda x: x[1],
+        reverse=True,
+    )
+
+    log.info("\nFeature Importance Ranking:")
+    log.info("-" * 50)
+    for name, imp in feat_imp:
+        bar = "#" * int(imp * 200)
+        log.info("  %-30s %.4f %s", name, imp, bar)
+
+    weights = derive_metric_weights(feature_cols, importances)
+
+    log.info("\nOptimal Metric Weights:")
+    log.info("-" * 50)
+    equal_weight = round(1 / len(weights), 4)
+    for metric, w in weights.items():
+        change = "+" if w > equal_weight else ""
+        diff = (w - equal_weight) / equal_weight * 100
+        log.info("  %-25s %.4f (%s%.0f%% vs equal)", metric, w, change, diff)
+
+    # Run comparison backtest: ML-weighted vs equal-weight
+    log.info("\n" + "=" * 60)
+    log.info("COMPARISON BACKTEST: ML-Weighted vs Equal-Weight")
+    log.info("=" * 60)
+    comparison = run_comparison(rows, weights)
+    out_of_sample_comparison = run_out_of_sample_comparison(labeled, fold_results)
+
+    # Build output. Final weights are fitted on all labeled history for live use;
+    # only the fold weights below are valid for OOS comparisons.
+    trained_at = datetime.now(tz=__import__('datetime').timezone.utc).isoformat()
+    result = {
+        "artifact_schema_version": ML_ARTIFACT_SCHEMA_VERSION,
+        "score_version": SCORE_VERSION,
+        "weights": weights,
+        "feature_importances": {name: round(float(imp), 6) for name, imp in feat_imp},
+        "cv_results": {
+            "mean_auc": round(float(np.mean(cv_scores)), 4),
+            "std_auc": round(float(np.std(cv_scores)), 4),
+            "mean_f1": round(float(np.mean(cv_f1)), 4),
+            "mean_precision": round(float(np.mean(cv_precision)), 4),
+            "mean_recall": round(float(np.mean(cv_recall)), 4),
+            "validation_method": "purged_expanding_window",
+            "label_horizon_days": LABEL_HORIZON_DAYS,
+            "folds": artifact_fold_results(fold_results),
+        },
+        "training_info": {
+            "n_samples": len(labeled),
+            "n_positive": int(positive),
+            "positive_rate": round(positive / len(labeled), 4),
+            "n_features": len(feature_cols),
+            "target_threshold": GOOD_BUY_THRESHOLD,
+            "date_range": f"{labeled[0]['date']} to {labeled[-1]['date']}",
+            "model": "GradientBoostingClassifier",
+        },
+        "provenance": {
+            "validation_method": "purged_expanding_window",
+            "label_horizon_days": LABEL_HORIZON_DAYS,
+            "weight_scope": "full_history_fit",
+            "training_date_range": {
+                "start": labeled[0]["date"],
+                "end": labeled[-1]["date"],
+            },
+            "trained_at": trained_at,
+        },
+        "comparison": comparison,
+        "out_of_sample_comparison": out_of_sample_comparison,
+        "trained_at": trained_at,
+    }
+
+    return result
+
+
+def _composite_score(row, mode, ml_weights=None):
+    scores = [row[f"score_{k}"] for k in SCORE_KEYS]
+    if mode == "equal_weight" or not ml_weights:
+        return sum(scores) / len(SCORE_KEYS) * 10
+    equal_weight = 1.0 / len(SCORE_KEYS)
+    weighted_sum = sum(row[f"score_{k}"] * ml_weights.get(k, equal_weight) for k in SCORE_KEYS)
+    return weighted_sum * 10
+
+
+def _summarize_brackets(scored_rows, score_key):
+    results = []
+    for low, high, label in BRACKETS:
+        days_in = [r for r in scored_rows if score_in_bracket(r[score_key], (low, high, label))]
+        if not days_in:
+            results.append({
+                "range": f"{low}-{high}", "label": label,
+                "days": 0, "avg_365d": None,
+            })
+            continue
+        returns_365 = [r["fwd_365d"] for r in days_in]
+        returns_sorted = sorted(returns_365)
+        win_rate = len([r for r in returns_365 if r > 0]) / len(returns_365) * 100
+        results.append({
+            "range": f"{low}-{high}",
+            "label": label,
+            "days": len(days_in),
+            "avg_365d": round(sum(returns_365) / len(returns_365), 2),
+            "median_365d": round(returns_sorted[len(returns_sorted) // 2], 2),
+            "win_rate_365d": round(win_rate, 1),
+        })
+    return results
+
+
+def _log_comparison_table(results):
+    log.info("\n%-18s | %-8s %-8s %-8s | %-8s %-8s %-8s",
+             "Bracket", "EQ Avg", "EQ Med", "EQ Win%", "ML Avg", "ML Med", "ML Win%")
+    log.info("-" * 80)
+    for eq, ml in zip(results["equal_weight"], results["ml_weighted"]):
+        eq_avg = f"{eq['avg_365d']:.1f}%" if eq["avg_365d"] is not None else "--"
+        eq_med = f"{eq['median_365d']:.1f}%" if eq.get("median_365d") is not None else "--"
+        eq_win = f"{eq['win_rate_365d']:.0f}%" if eq.get("win_rate_365d") is not None else "--"
+        ml_avg = f"{ml['avg_365d']:.1f}%" if ml["avg_365d"] is not None else "--"
+        ml_med = f"{ml['median_365d']:.1f}%" if ml.get("median_365d") is not None else "--"
+        ml_win = f"{ml['win_rate_365d']:.0f}%" if ml.get("win_rate_365d") is not None else "--"
+        log.info("%-18s | %-8s %-8s %-8s | %-8s %-8s %-8s",
+                 eq["label"], eq_avg, eq_med, eq_win, ml_avg, ml_med, ml_win)
+
+
+def run_comparison(rows, ml_weights):
+    """Compare final ML-weighted scoring vs equal-weight scoring across all labeled rows.
+
+    This is retained for backwards compatibility with existing output. It is an
+    in-sample/full-history comparison; prefer out_of_sample_comparison for model
+    selection decisions.
+    """
+    scored_rows = [dict(r) for r in rows if "fwd_365d" in r]
+    for r in scored_rows:
+        r["composite_equal_weight"] = _composite_score(r, "equal_weight")
+        r["composite_ml_weighted"] = _composite_score(r, "ml_weighted", ml_weights)
+
+    results = {
+        "equal_weight": _summarize_brackets(scored_rows, "composite_equal_weight"),
+        "ml_weighted": _summarize_brackets(scored_rows, "composite_ml_weighted"),
+    }
+    _log_comparison_table(results)
+    return results
+
+
+def run_out_of_sample_comparison(rows, fold_results):
+    """Compare fold-specific ML weights on validation rows only."""
+    validation_rows = []
+    for fold in fold_results:
+        weights = fold.get("weights", {})
+        for idx in fold.get("val_idx", []):
+            if idx >= len(rows) or "fwd_365d" not in rows[idx]:
+                continue
+            r = dict(rows[idx])
+            r["fold"] = fold.get("fold")
+            r["composite_equal_weight"] = _composite_score(r, "equal_weight")
+            r["composite_ml_weighted"] = _composite_score(r, "ml_weighted", weights)
+            validation_rows.append(r)
+
+    results = {
+        "folds": len(fold_results),
+        "validation_days": len(validation_rows),
+        "equal_weight": _summarize_brackets(validation_rows, "composite_equal_weight"),
+        "ml_weighted": _summarize_brackets(validation_rows, "composite_ml_weighted"),
+    }
+    return results
+
+
+def main():
+    log.info("=" * 60)
+    log.info("Bitcoin Accumulation Zone ML Optimizer")
+    log.info("=" * 60)
+
+    if not os.path.exists(HISTORY_PATH):
+        log.error("No historical data at %s. Run history collector first.", HISTORY_PATH)
+        sys.exit(1)
+
+    # Load data
+    log.info("Loading historical data...")
+    index = load_history()
+    thresholds = load_thresholds()
+
+    # Build dataset
+    log.info("Building training dataset...")
+    rows = build_dataset(index, thresholds)
+
+    # Train model
+    log.info("Training ML model...")
+    result = train_model(rows)
+
+    if result is None:
+        log.error("Training failed.")
+        sys.exit(1)
+
+    # Save weights
+    with open(OUTPUT_PATH, "w") as f:
+        json.dump(result, f, indent=2)
+    log.info("\nSaved ML weights to %s", OUTPUT_PATH)
+
+    # Print summary
+    log.info("\n" + "=" * 60)
+    log.info("SUMMARY")
+    log.info("=" * 60)
+    log.info("Model: %s", result["training_info"]["model"])
+    log.info("Samples: %d (%d positive)", result["training_info"]["n_samples"], result["training_info"]["n_positive"])
+    log.info("CV AUC: %.3f (+/- %.3f)", result["cv_results"]["mean_auc"], result["cv_results"]["std_auc"])
+    log.info("CV F1:  %.3f", result["cv_results"]["mean_f1"])
+    log.info("\nTop 5 Feature Importances:")
+    for name, imp in list(result["feature_importances"].items())[:5]:
+        log.info("  %-30s %.4f", name, imp)
+    log.info("\nMetric Weights (ML-Optimized):")
+    for metric, weight in result["weights"].items():
+        log.info("  %-25s %.1f%%", metric, weight * 100)
+
+
+if __name__ == "__main__":
+    main()

ml_engine/train_and_backtest.py

@@ -161,8 +161,8 @@ def compute_features(df, config):
 def create_accumulation_target(df, config):
     """Create accumulation score target based on forward returns.
 
-    For each candle, compute actual forward returns at multiple horizons,
-    rank them, and create a weighted accumulation score (0-100).
+    For each candle, compute actual forward returns at multiple horizons and
+    map them through fixed, configured return scales to a weighted 0-100 score.
     Times when buying led to the best long-term returns get highest scores.
     """
     tgt = config.get("target", {})
@@ -190,30 +190,22 @@ def create_accumulation_target(df, config):
             fwd[i] = (close[i + period] - close[i]) / close[i] * 100
         forward_returns.append(fwd)
 
-    # Rank each forward return (percentile rank, 0-1)
-    # Higher rank = better buy point (higher future return)
-    ranked = []
-    for fwd in forward_returns:
-        valid_mask = ~np.isnan(fwd)
-        ranks = np.full(n, np.nan)
-        valid_vals = fwd[valid_mask]
-        if len(valid_vals) > 0:
-            from scipy.stats import rankdata
-            r = rankdata(valid_vals, method="average") / len(valid_vals)
-            ranks[valid_mask] = r
-        ranked.append(ranks)
+    # Convert each return to a deterministic 0-100 quality score. Global
+    # percentile ranks leak the distribution of future validation/test rows into
+    # earlier training labels; a fixed tanh transform is invariant to rows added
+    # outside the observation's own forward horizons.
+    scales = tgt.get("return_scales_pct", [10.0, 30.0, 60.0])
+    if len(scales) != len(forward_periods) or any(scale <= 0 for scale in scales):
+        raise ValueError("target.return_scales_pct must contain one positive scale per forward period")
 
-    # Weighted combination of ranks -> accumulation score (0-100)
     score = np.zeros(n)
     valid = np.ones(n, dtype=bool)
-    for r, w in zip(ranked, weights):
-        nan_mask = np.isnan(r)
+    for fwd, weight, scale in zip(forward_returns, weights, scales):
+        nan_mask = np.isnan(fwd)
         valid &= ~nan_mask
-        r_filled = np.where(nan_mask, 0, r)
-        score += w * r_filled
+        quality = 50.0 + 50.0 * np.tanh(np.where(nan_mask, 0.0, fwd) / scale)
+        score += weight * quality
 
-    # Scale to 0-100
-    score = score * 100
     score[~valid] = np.nan
 
     return pd.Series(score, index=df.index, name="target")
@@ -488,6 +480,27 @@ def apply_scaling_pca(X_train, X_val, X_test, config):
     return X_train, X_val, X_test, scaler, pca
 
 
+# ---------------------------------------------------------------------------
+# Walk-forward split helpers
+# ---------------------------------------------------------------------------
+
+def _max_forward_horizon(config):
+    """Return the longest forward-label horizon in candles."""
+    target = config.get("target", {})
+    key = "forward_periods_1h" if config.get("timeframe", "4h") == "1h" else "forward_periods_4h"
+    periods = target.get(key, [168, 720, 2160] if key.endswith("1h") else [42, 180, 540])
+    return max(int(period) for period in periods)
+
+
+def _purge_label_overlap(frame, horizon):
+    """Remove rows whose forward-return labels cross the next split boundary."""
+    if horizon <= 0:
+        return frame
+    if len(frame) <= horizon:
+        return frame.iloc[0:0]
+    return frame.iloc[:-horizon]
+
+
 # ---------------------------------------------------------------------------
 # Rolling Window Validation
 # ---------------------------------------------------------------------------
@@ -499,6 +512,7 @@ def rolling_window_train_test(df, feature_cols, config):
     test_size = training_cfg.get("rolling_test_size", 300)
     val_pct = training_cfg.get("validation_pct", 0.15)
     model_type = config.get("model_type", "xgboost")
+    purge_horizon = _max_forward_horizon(config)
 
     n = len(df)
     all_predictions = []  # list of (predicted_score, actual_score, close_price)
@@ -527,10 +541,16 @@ def rolling_window_train_test(df, feature_cols, config):
             start += test_size
             continue
 
-        # Split train into train/val
+        # Purge labels whose longest forward-return horizon overlaps the next
+        # split. Without this embargo, training and validation targets consume
+        # prices from the following validation/test partition.
         val_split = int(len(train_full) * (1.0 - val_pct))
-        train_df = train_full.iloc[:val_split]
-        val_df = train_full.iloc[val_split:]
+        train_df = _purge_label_overlap(train_full.iloc[:val_split], purge_horizon)
+        val_df = _purge_label_overlap(train_full.iloc[val_split:], purge_horizon)
+
+        if len(train_df) < 10 or len(val_df) < 1:
+            start += test_size
+            continue
 
         X_train = train_df[feature_cols].values
         y_train = train_df["target"].values
@@ -632,6 +652,7 @@ def walk_forward_train_test(df, feature_cols, config):
     n_windows = training_cfg.get("walk_forward_windows", 5)
     train_pct = training_cfg.get("train_pct", 0.7)
     val_pct = training_cfg.get("validation_pct", 0.15)
+    purge_horizon = _max_forward_horizon(config)
 
     n = len(df)
     window_size = n // n_windows
@@ -655,8 +676,8 @@ def walk_forward_train_test(df, feature_cols, config):
         train_end = int(wn * train_pct)
         val_end = int(wn * (train_pct + val_pct))
 
-        train_df = window_data.iloc[:train_end]
-        val_df = window_data.iloc[train_end:val_end]
+        train_df = _purge_label_overlap(window_data.iloc[:train_end], purge_horizon)
+        val_df = _purge_label_overlap(window_data.iloc[train_end:val_end], purge_horizon)
         test_df = window_data.iloc[val_end:]
 
         if len(test_df) < 10:
@@ -842,6 +863,46 @@ def _extract_feature_importances(model, n_features):
 # Results Compilation
 # ---------------------------------------------------------------------------
 
+def simulate_periodic_accumulation(predicted_scores, close_prices, buy_threshold, contribution=1.0):
+    """Compare DCA and signal strategies with equal periodic contributions.
+
+    Both strategies receive the same cash on every observation. DCA invests the
+    contribution immediately; the signal strategy retains cash until a buy
+    signal, then deploys its available balance. Terminal wealth includes cash.
+    """
+    scores = np.asarray(predicted_scores, dtype=float)
+    prices = np.asarray(close_prices, dtype=float)
+    if len(scores) != len(prices):
+        raise ValueError("predicted_scores and close_prices must have equal length")
+    if len(prices) == 0 or contribution <= 0 or np.any(prices <= 0):
+        raise ValueError("prices must be positive and contribution must be greater than zero")
+
+    dca_btc = float(np.sum(contribution / prices))
+    model_btc = 0.0
+    model_cash = 0.0
+    for score, price in zip(scores, prices):
+        model_cash += contribution
+        if score >= buy_threshold:
+            model_btc += model_cash / price
+            model_cash = 0.0
+
+    contributed = float(len(prices) * contribution)
+    terminal_price = float(prices[-1])
+    dca_terminal = dca_btc * terminal_price
+    model_terminal = model_btc * terminal_price + model_cash
+    improvement = (model_terminal - dca_terminal) / dca_terminal * 100 if dca_terminal else 0.0
+    return {
+        "dca_contributed": contributed,
+        "model_contributed": contributed,
+        "dca_btc": dca_btc,
+        "model_btc": model_btc,
+        "model_cash": model_cash,
+        "dca_terminal_value": dca_terminal,
+        "model_terminal_value": model_terminal,
+        "terminal_wealth_improvement_pct": improvement,
+    }
+
+
 def compile_results(predictions, per_window_cost_improvement,
                     fi_sum, fi_count, feature_cols, config):
     """Compile accumulation signal results into output JSON."""
@@ -870,10 +931,8 @@ def compile_results(predictions, per_window_cost_improvement,
     else:
         avg_actual_strong = 0.0
 
-    # We need forward return info. Since actual_score is a rank-based measure (0-100),
-    # and we want to report real forward returns, we approximate:
-    # actual_score > 80 means the buy was in the top 20% of quality.
-    # For actual forward return stats, we use actual score as a proxy.
+    # The target is a bounded return-quality score, not a realized return.
+    # Report it explicitly as quality rather than approximating a return.
 
     # Profitable signals: those where actual score is also above median (50)
     if strong_buy_count > 0:
@@ -896,18 +955,25 @@ def compile_results(predictions, per_window_cost_improvement,
         model_avg = dca_avg
         cost_basis_improvement = 0.0
 
+    portfolio = simulate_periodic_accumulation(
+        pred_scores,
+        close_prices,
+        buy_threshold=good_threshold,
+        contribution=1.0,
+    )
+
     # --- Signal Frequency ---
     signal_frequency = strong_buy_count / total_candles * 100 if total_candles > 0 else 0
 
     # --- Score at actual extremes ---
-    # "Actual bottoms" = candles with actual score > 85 (top 15% buy opportunities)
+    # "Actual bottoms" = candles with a high realized return-quality score.
     actual_bottom_mask = actual_scores > 85
     if np.any(actual_bottom_mask):
         avg_score_at_bottoms = float(np.mean(pred_scores[actual_bottom_mask]))
     else:
         avg_score_at_bottoms = 0.0
 
-    # "Actual tops" = candles with actual score < 15 (worst 15% buy times)
+    # "Actual tops" = candles with a low realized return-quality score.
     actual_top_mask = actual_scores < 15
     if np.any(actual_top_mask):
         avg_score_at_tops = float(np.mean(pred_scores[actual_top_mask]))
@@ -932,10 +998,7 @@ def compile_results(predictions, per_window_cost_improvement,
         count = int(np.sum((pred_scores >= lo) & (pred_scores < (hi if hi < 100 else 101))))
         score_distribution[key] = count
 
-    # --- Forward return approximation from actual scores ---
-    # Map actual score to approximate return quality
-    # Score 90+ = historically best 10% buys, score 10- = worst 10%
-    # Use actual score as proxy for "quality rank"
+    # --- Realized return-quality summary ---
     if strong_buy_count > 0:
         # Average actual quality score for strong buy signals
         avg_quality_strong = float(np.mean(actual_scores[strong_buy_mask]))
@@ -946,7 +1009,14 @@ def compile_results(predictions, per_window_cost_improvement,
         quality_good = False
 
     return {
+        # Retained for backward compatibility; model selection uses the equal-
+        # capital terminal wealth metric below.
         "cost_basis_improvement_pct": round(cost_basis_improvement, 2),
+        "terminal_wealth_improvement_pct": round(portfolio["terminal_wealth_improvement_pct"], 2),
+        "model_terminal_value": round(portfolio["model_terminal_value"], 6),
+        "dca_terminal_value": round(portfolio["dca_terminal_value"], 6),
+        "model_cash": round(portfolio["model_cash"], 6),
+        "backtest_objective": "equal_periodic_contribution_terminal_wealth",
         "avg_cost_basis_model": round(model_avg, 2),
         "avg_cost_basis_dca": round(dca_avg, 2),
         "strong_buy_signal_count": strong_buy_count,
@@ -968,6 +1038,11 @@ def compile_results(predictions, per_window_cost_improvement,
 def _empty_results(per_window):
     return {
         "cost_basis_improvement_pct": 0.0,
+        "terminal_wealth_improvement_pct": 0.0,
+        "model_terminal_value": 0.0,
+        "dca_terminal_value": 0.0,
+        "model_cash": 0.0,
+        "backtest_objective": "equal_periodic_contribution_terminal_wealth",
         "avg_cost_basis_model": 0.0,
         "avg_cost_basis_dca": 0.0,
         "strong_buy_signal_count": 0,

orchestrator.py

@@ -28,7 +28,7 @@ MAC_MINI_HOST = "bizzle@bizzles-mac-mini-1"
 MAX_ITERATIONS = 50
 CONVERGENCE_WINDOW = 5
 CONVERGENCE_THRESHOLD = 0.01  # 1% improvement
-TARGET_COST_IMPROVEMENT = 20.0  # 20% cost basis improvement = exceptional
+TARGET_COST_IMPROVEMENT = 20.0  # Backward-compatible name: terminal wealth objective
 MIN_SIGNAL_COUNT = 30  # Minimum strong buy signals for valid results
 ML_TIMEOUT = 600  # 10 minutes
 
@@ -49,6 +49,11 @@ def log(msg, color=""):
     print(f"{C.DIM}[{ts}]{C.RESET} {color}{msg}{C.RESET}")
 
 
+def objective_score(results):
+    """Return the equal-capital portfolio objective used for model selection."""
+    return float(results.get("terminal_wealth_improvement_pct", 0.0))
+
+
 def run_cmd(cmd, timeout=120, check=True):
     """Run a shell command and return stdout."""
     result = subprocess.run(
@@ -160,11 +165,12 @@ def print_header():
 
 
 def print_results(results, iteration):
-    cost_imp = results.get("cost_basis_improvement_pct", 0)
-    color = C.GREEN if cost_imp > 15 else C.YELLOW if cost_imp > 10 else C.RED
+    objective = objective_score(results)
+    color = C.GREEN if objective > 15 else C.YELLOW if objective > 10 else C.RED
     print(f"""
 {C.BOLD}--- Iteration {iteration} Results ---{C.RESET}
-  Cost Improvement: {color}{C.BOLD}{cost_imp:.1f}%{C.RESET}
+  Terminal Wealth vs DCA: {color}{C.BOLD}{objective:.1f}%{C.RESET}
+  Legacy Cost Basis Delta: {results.get('cost_basis_improvement_pct', 0):.1f}%
   Avg Cost (Model): ${results.get('avg_cost_basis_model', 0):,.2f}
   Avg Cost (DCA):   ${results.get('avg_cost_basis_dca', 0):,.2f}
   Strong Signals:   {results.get('strong_buy_signal_count', 0)}
@@ -244,7 +250,7 @@ def main():
 
         print_results(results, iteration)
 
-        current_score = results.get("cost_basis_improvement_pct", 0)
+        current_score = objective_score(results)
         signal_count = results.get("strong_buy_signal_count", 0)
         is_best = current_score > best_score and signal_count >= MIN_SIGNAL_COUNT
 
@@ -252,12 +258,14 @@ def main():
             best_score = current_score
             with open(best_config_path, "w") as f:
                 json.dump(config, f, indent=2)
-            log(f"NEW BEST! Cost Improvement: {best_score:.1f}%", f"{C.BOLD}{C.GREEN}")
+            log(f"NEW BEST! Terminal Wealth Improvement: {best_score:.1f}%", f"{C.BOLD}{C.GREEN}")
 
         iter_data = {
             "iteration": iteration,
             "timestamp": datetime.now(timezone.utc).isoformat(),
             "cost_improvement": current_score,
+            "objective_improvement": current_score,
+            "objective": "equal_periodic_contribution_terminal_wealth",
             "avg_30d_return": results.get("avg_quality_score_strong_buy", 0),
             "avg_90d_return": results.get("pct_quality_strong_buy", 0),
             "signal_count": signal_count,
@@ -312,7 +320,7 @@ def main():
 ========================================================{C.RESET}
 
   Total Iterations:     {len(history)}
-  Best Cost Improvement: {C.BOLD}{best_score:.1f}%{C.RESET}
+  Best Terminal Wealth Improvement: {C.BOLD}{best_score:.1f}%{C.RESET}
   Best Config:          {best_config_path}
   Iteration Log:        {ITERATIONS_LOG}
 """)
@@ -405,7 +413,7 @@ def run_optimization_loop(callback=None, config_override=None):
             with open(results_local) as f:
                 results = json.load(f)
 
-            current_score = results.get("cost_basis_improvement_pct", 0)
+            current_score = objective_score(results)
             signal_count = results.get("strong_buy_signal_count", 0)
             is_best = current_score > best_score and signal_count >= MIN_SIGNAL_COUNT
 
@@ -419,6 +427,8 @@ def run_optimization_loop(callback=None, config_override=None):
                 "iteration": iteration,
                 "timestamp": datetime.now(timezone.utc).isoformat(),
                 "cost_improvement": current_score,
+                "objective_improvement": current_score,
+                "objective": "equal_periodic_contribution_terminal_wealth",
                 "signal_count": signal_count,
                 "signal_frequency": results.get("signal_frequency_pct", 0),
                 "r2_score": results.get("model_r2_score", 0),

pyproject.toml

@@ -0,0 +1,34 @@
+[project]
+name = "btc-accumulation-monitor"
+version = "0.1.0"
+description = "Bitcoin accumulation metrics dashboard and historical scoring tools"
+readme = "README.md"
+requires-python = ">=3.11,<3.14"
+
+[dependency-groups]
+runtime = [
+  "fastapi>=0.116,<1",
+  "playwright>=1.54,<2",
+  "requests>=2.32,<3",
+  "uvicorn[standard]>=0.35,<1",
+]
+ml = [
+  "ccxt>=4.4,<5",
+  "numpy>=2.2,<3",
+  "pandas>=2.2,<3",
+  "scikit-learn>=1.6,<2",
+  "ta>=0.11,<1",
+]
+dev = [
+  "pytest>=8.4,<9",
+]
+
+[tool.uv]
+package = false
+
+default-groups = ["runtime", "ml", "dev"]
+
+[tool.pytest.ini_options]
+addopts = "-q"
+testpaths = ["tests"]
+pythonpath = ["."]

scoring/engine.py

@@ -4,6 +4,9 @@ import json
 import os
 import logging
 
+from scoring.policy import SCORE_VERSION, assessment_for_score
+from ml.artifacts import validate_ml_artifact
+
 log = logging.getLogger(__name__)
 
 THRESHOLDS_PATH = os.path.join(
@@ -75,7 +78,8 @@ def score_puell_multiple(value, thresholds=None):
     if value is None:
         return None, "No data"
     t = (thresholds or load_thresholds()).get("puell_multiple", {})
-    ranges = t.get("ranges", [[None, 0.3, 10], [0.3, 0.5, 8], [0.5, 0.8, 5], [0.8, 1.2, 3], [1.2, 2.0, 1], [2.0, None, 0]])
+    # Widened: post-halving Puell floors are rising (2016: 0.15, 2020: 0.3, 2024: 0.5+)
+    ranges = t.get("ranges", [[None, 0.4, 10], [0.4, 0.7, 8], [0.7, 1.0, 5], [1.0, 1.5, 3], [1.5, 2.0, 1], [2.0, None, 0]])
     score = _score_range(value, ranges)
 
     if value < 0.3:
@@ -97,15 +101,17 @@ def score_mvrv_zscore(value, thresholds=None):
     if value is None:
         return None, "No data"
     t = (thresholds or load_thresholds()).get("mvrv_zscore", {})
-    ranges = t.get("ranges", [[None, 0, 10], [0, 0.5, 8], [0.5, 1.5, 5], [1.5, 3, 2], [3, 5, 1], [5, None, 0]])
+    # Widened ranges: BTC cycles compress — Z-Score bottoms are getting shallower
+    # 2015 bottom: -0.6, 2018 bottom: -0.4, 2022 bottom: -0.3, next may be ~0
+    ranges = t.get("ranges", [[None, 0, 10], [0, 1.0, 8], [1.0, 2.0, 5], [2.0, 3, 3], [3, 5, 1], [5, None, 0]])
     score = _score_range(value, ranges)
 
     if value < 0:
         desc = "Below realized value — historically perfect buy zone"
-    elif value < 0.5:
-        desc = "Near realized value — strong accumulation"
-    elif value < 1.5:
-        desc = "Fair value range"
+    elif value < 1.0:
+        desc = "Near realized value — strong accumulation zone"
+    elif value < 2.0:
+        desc = "Fair value — decent entry territory"
     elif value < 3:
         desc = "Above fair value"
     elif value < 5:
@@ -144,15 +150,16 @@ def score_price_vs_200w_sma(price, sma_200w, thresholds=None):
         return None, "No data"
     pct_above = ((price - sma_200w) / sma_200w) * 100
     t = (thresholds or load_thresholds()).get("price_vs_200w_sma", {})
-    ranges = t.get("ranges", [[None, 0, 10], [0, 20, 6], [20, 50, 3], [50, 100, 1], [100, None, 0]])
+    # Widened: BTC increasingly stays above 200W SMA as it matures
+    ranges = t.get("ranges", [[None, 0, 10], [0, 30, 7], [30, 60, 5], [60, 100, 2], [100, None, 0]])
     score = _score_range(pct_above, ranges)
 
     if pct_above < 0:
         desc = f"Below 200W SMA — historically rare buy zone"
-    elif pct_above < 20:
-        desc = f"{pct_above:.0f}% above 200W SMA — good value"
-    elif pct_above < 50:
-        desc = f"{pct_above:.0f}% above 200W SMA — moderate"
+    elif pct_above < 30:
+        desc = f"{pct_above:.0f}% above 200W SMA — strong value"
+    elif pct_above < 60:
+        desc = f"{pct_above:.0f}% above 200W SMA — fair value"
     elif pct_above < 100:
         desc = f"{pct_above:.0f}% above 200W SMA — extended"
     else:
@@ -204,13 +211,15 @@ def score_nupl(value, thresholds=None):
     if value is None:
         return None, "No data"
     t = (thresholds or load_thresholds()).get("nupl", {})
-    ranges = t.get("ranges", [[None, 0, 10], [0, 0.25, 7], [0.25, 0.5, 4], [0.5, 0.75, 1], [0.75, None, 0]])
+    # Widened: NUPL bottoms getting shallower as BTC matures
+    # 2015: -0.3, 2018: -0.28, 2022: -0.28, future may only dip to 0-0.1
+    ranges = t.get("ranges", [[None, 0, 10], [0, 0.3, 8], [0.3, 0.5, 4], [0.5, 0.75, 1], [0.75, None, 0]])
     score = _score_range(value, ranges)
 
     if value < 0:
         desc = "Capitulation — holders underwater"
-    elif value < 0.25:
-        desc = "Hope/Fear — early recovery"
+    elif value < 0.3:
+        desc = "Hope/Fear — early recovery, good accumulation"
     elif value < 0.5:
         desc = "Optimism — moderate profit taking"
     elif value < 0.75:
@@ -226,14 +235,18 @@ def score_lth_realized_price(price, lth_rp, thresholds=None):
         return None, "No data"
     pct_above = ((price - lth_rp) / lth_rp) * 100
     t = (thresholds or load_thresholds()).get("lth_realized_price", {})
-    ranges = t.get("ranges", [[None, 0, 10], [0, 20, 6], [20, 50, 3], [50, None, 1]])
+    # Widened: as BTC matures, price spends more time above LTH RP
+    # In 2024+, even "good" entries are 30-80% above LTH RP
+    ranges = t.get("ranges", [[None, 0, 10], [0, 30, 7], [30, 80, 5], [80, 150, 3], [150, None, 1]])
     score = _score_range(pct_above, ranges)
 
     if pct_above < 0:
         desc = f"Below LTH cost basis — LTHs underwater (extreme value)"
-    elif pct_above < 20:
-        desc = f"{pct_above:.0f}% above LTH cost basis — good value"
-    elif pct_above < 50:
+    elif pct_above < 30:
+        desc = f"{pct_above:.0f}% above LTH cost basis — strong value"
+    elif pct_above < 80:
+        desc = f"{pct_above:.0f}% above LTH cost basis — fair value"
+    elif pct_above < 150:
         desc = f"{pct_above:.0f}% above LTH cost basis — moderate"
     else:
         desc = f"{pct_above:.0f}% above LTH cost basis — extended"
@@ -249,6 +262,65 @@ def score_hash_ribbons(data, thresholds=None):
     return 3, "Normal mining activity"
 
 
+def score_sopr(value, thresholds=None):
+    if value is None:
+        return None, "No data"
+    if value < 0.98:
+        return 10, "Deep loss realization — capitulation, strong accumulation"
+    if value < 1.0:
+        return 8, "Below breakeven — capitulation, good accumulation"
+    if value < 1.02:
+        return 5, "Near breakeven — neutral"
+    if value < 1.05:
+        return 2, "Moderate profit taking"
+    return 0, "Elevated profit taking — caution"
+
+
+def score_sellside_risk(value, thresholds=None):
+    if value is None:
+        return None, "No data"
+    if value < 0.001:
+        return 10, "Very low sell-side risk — strong accumulation"
+    if value < 0.002:
+        return 8, "Low sell-side risk — good accumulation"
+    if value < 0.005:
+        return 5, "Moderate sell-side risk"
+    if value < 0.01:
+        return 2, "Elevated sell-side risk"
+    return 0, "High sell-side risk"
+
+
+def score_momentum_pct(value):
+    if value is None:
+        return None, "No data"
+    pct = value * 100
+    if pct >= 20:
+        return 10, f"Strong positive momentum (+{pct:.0f}%)"
+    if pct >= 0:
+        return 6, f"Mild positive momentum (+{pct:.0f}%)"
+    if pct >= -10:
+        return 4, f"Slightly negative momentum ({pct:.0f}%)"
+    if pct >= -25:
+        return 2, f"Weak momentum ({pct:.0f}%)"
+    return 1, f"Strong negative momentum ({pct:.0f}%)"
+
+
+def score_nvt_price(nvt_price, spot_price):
+    if nvt_price is None or spot_price is None or spot_price <= 0:
+        return None, "No data"
+    premium = (nvt_price - spot_price) / spot_price * 100
+    if premium < -25:
+        return 10, f"NVT price {abs(premium):.0f}% below spot — deep value"
+    if premium < -10:
+        return 8, f"NVT price {abs(premium):.0f}% below spot — undervalued"
+    if premium < 10:
+        relation = "below" if premium < 0 else "above"
+        return 5, f"NVT price {abs(premium):.0f}% {relation} spot — fair value"
+    if premium < 30:
+        return 2, f"NVT price {premium:.0f}% above spot — extended"
+    return 0, f"NVT price {premium:.0f}% above spot — overheated"
+
+
 def score_all(metrics):
     """Score all metrics and return individual + composite scores."""
     thresholds = load_thresholds()
@@ -389,6 +461,85 @@ def score_all(metrics):
         "recent": [],
     })
 
+    # SOPR
+    sopr = metrics.get("sopr", {})
+    sopr_score, sopr_desc = score_sopr(sopr.get("value"), thresholds)
+    results.append({
+        "name": "SOPR",
+        "key": "sopr",
+        "value": sopr.get("value"),
+        "display_value": f"{sopr.get('value', 'N/A'):.4f}" if sopr.get("value") is not None else "N/A",
+        "score": sopr_score,
+        "description": sopr_desc,
+        "recent": sopr.get("recent", []),
+    })
+
+    # Sell-side Risk Ratio
+    ssr = metrics.get("sellside_risk", {})
+    ssr_score, ssr_desc = score_sellside_risk(ssr.get("value"), thresholds)
+    results.append({
+        "name": "Sell-side Risk Ratio",
+        "key": "sellside_risk",
+        "value": ssr.get("value"),
+        "display_value": f"{ssr.get('value', 'N/A'):.6f}" if ssr.get("value") is not None else "N/A",
+        "score": ssr_score,
+        "description": ssr_desc,
+        "recent": ssr.get("recent", []),
+    })
+
+    # Active Address Momentum
+    aam = metrics.get("active_address_momentum", {})
+    aam_score, aam_desc = score_momentum_pct(aam.get("value"))
+    results.append({
+        "name": "Active Address Momentum",
+        "key": "active_address_momentum",
+        "value": aam.get("value"),
+        "display_value": f"{aam.get('value') * 100:.1f}%" if aam.get("value") is not None else "N/A",
+        "score": aam_score,
+        "description": aam_desc,
+        "recent": aam.get("recent", []),
+    })
+
+    # Transaction Count Momentum
+    txm = metrics.get("txcount_momentum", {})
+    txm_score, txm_desc = score_momentum_pct(txm.get("value"))
+    results.append({
+        "name": "Transaction Count Momentum",
+        "key": "txcount_momentum",
+        "value": txm.get("value"),
+        "display_value": f"{txm.get('value') * 100:.1f}%" if txm.get("value") is not None else "N/A",
+        "score": txm_score,
+        "description": txm_desc,
+        "recent": txm.get("recent", []),
+    })
+
+    # NVT Price
+    nvt = metrics.get("nvt_price", {})
+    nvt_score, nvt_desc = score_nvt_price(nvt.get("value"), current_price)
+    results.append({
+        "name": "NVT Price",
+        "key": "nvt_price",
+        "value": nvt.get("value"),
+        "display_value": f"${nvt.get('value'):,.0f}" if nvt.get("value") is not None else "N/A",
+        "score": nvt_score,
+        "description": nvt_desc,
+        "recent": nvt.get("recent", []),
+    })
+
+    # VDD Multiple
+    vdd = metrics.get("vdd_multiple", {})
+    vdd_score, vdd_desc = score_momentum_pct(vdd.get("value"))
+    results.append({
+        "name": "VDD 30-Period Momentum",
+        "key": "vdd_multiple",
+        "transform": "30_period_return",
+        "value": vdd.get("value"),
+        "display_value": f"{vdd.get('value') * 100:.1f}%" if vdd.get("value") is not None else "N/A",
+        "score": vdd_score,
+        "description": vdd_desc,
+        "recent": vdd.get("recent", []),
+    })
+
     # Compute composite
     valid_scores = [r["score"] for r in results if r["score"] is not None]
     if valid_scores:
@@ -397,17 +548,7 @@ def score_all(metrics):
     else:
         composite = 0
 
-    # Assessment text
-    if composite >= 71:
-        assessment = "STRONG ACCUMULATION ZONE"
-    elif composite >= 51:
-        assessment = "MODERATE OPPORTUNITY"
-    elif composite >= 31:
-        assessment = "NEUTRAL"
-    elif composite >= 15:
-        assessment = "CAUTION — OVERHEATED"
-    else:
-        assessment = "EXTREME CAUTION"
+    assessment = assessment_for_score(composite)
 
     return {
         "metrics": results,
@@ -415,4 +556,130 @@ def score_all(metrics):
         "assessment": assessment,
         "scored_count": len(valid_scores),
         "total_count": len(results),
+        "score_version": SCORE_VERSION,
+        "metric_panel": {
+            "id": "live-all-v1",
+            "keys": [result["key"] for result in results],
+            "count": len(results),
+        },
+        "coverage": {
+            "available_count": len(valid_scores),
+            "panel_count": len(results),
+            "ratio": len(valid_scores) / len(results),
+        },
+    }
+
+
+# ── ML-Optimized Scoring ──────────────────────────────────────────────
+
+ML_WEIGHTS_PATH = os.path.join(
+    os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
+    "config",
+    "ml_weights.json",
+)
+
+# Maps scoring engine metric keys to ML weight keys
+_ML_KEY_MAP = {
+    "fear_greed": "fear_greed",
+    "puell_multiple": "puell_multiple",
+    "mvrv_zscore": "mvrv_zscore",
+    "drawdown": "drawdown",
+    "price_vs_200w_sma": "pct_above_200w_sma",
+    "reserve_risk": "reserve_risk",
+    "rhodl_ratio": "rhodl_ratio",
+    "nupl": "nupl",
+    "lth_realized_price": "pct_above_lth_rp",
+}
+
+
+_ml_artifact_status = {"valid": False, "errors": ["not_loaded"]}
+
+
+def load_ml_weights():
+    """Load weights only when their schema and training provenance are valid."""
+    global _ml_artifact_status
+    try:
+        with open(ML_WEIGHTS_PATH) as f:
+            data = json.load(f)
+        _ml_artifact_status = validate_ml_artifact(data)
+        if not _ml_artifact_status["valid"]:
+            log.error("Rejected invalid ML artifact: %s", ", ".join(_ml_artifact_status["errors"]))
+            return {}
+        return data.get("weights", {})
+    except Exception as exc:
+        _ml_artifact_status = {"valid": False, "errors": [f"load_error:{exc}"]}
+        return {}
+
+
+def get_ml_artifact_status():
+    """Return the status from the most recent artifact load attempt."""
+    return dict(_ml_artifact_status)
+
+
+def score_all_ml(metrics):
+    """Score all metrics using ML-optimized weights.
+
+    Same output format as score_all() but uses learned weights
+    instead of equal weighting. Each metric still shows its
+    individual 0-10 score plus the ML weight applied to it.
+    """
+    # Get classic scores first (reuses all individual scoring logic)
+    classic = score_all(metrics)
+    ml_weights = load_ml_weights()
+
+    if not ml_weights:
+        # Fallback to classic if no ML weights available
+        classic["ml_mode"] = False
+        status = get_ml_artifact_status()
+        if status.get("errors") and status["errors"] != ["not_loaded"]:
+            classic["ml_error"] = "ML artifact invalid: " + ", ".join(status["errors"])
+        else:
+            classic["ml_error"] = "ML weights not found — run ml/optimizer.py"
+        classic["ml_artifact"] = status
+        return classic
+
+    results = classic["metrics"]
+
+    # Compute raw ML weights first, then normalize across only the currently
+    # scored metrics. This keeps the dashboard's displayed per-metric weights and
+    # contribution points consistent with the normalized composite score even
+    # when optional metrics are missing or hash ribbons receives its fallback.
+    weighted_metrics = []
+    for m in results:
+        if m["score"] is None:
+            continue
+        ml_key = _ML_KEY_MAP.get(m["key"])
+        if ml_key is None:
+            # Hash ribbons or unknown metric — use small default weight
+            raw_weight = 0.01
+        else:
+            raw_weight = ml_weights.get(ml_key, 0.0)
+        weighted_metrics.append((m, raw_weight))
+
+    weight_total = sum(raw_weight for _, raw_weight in weighted_metrics)
+    if weight_total > 0:
+        composite = sum(m["score"] * raw_weight for m, raw_weight in weighted_metrics) / weight_total * 10
+    else:
+        composite = 0
+
+    for m, raw_weight in weighted_metrics:
+        effective_weight = raw_weight / weight_total if weight_total > 0 else 0.0
+        m["ml_raw_weight"] = round(raw_weight, 4)
+        m["ml_weight"] = round(effective_weight, 4)
+        m["ml_contribution"] = round(m["score"] * effective_weight * 10, 2)
+
+    assessment = assessment_for_score(composite)
+
+    return {
+        "metrics": results,
+        "composite_score": round(composite, 1),
+        "assessment": assessment,
+        "scored_count": classic["scored_count"],
+        "total_count": classic["total_count"],
+        "ml_mode": True,
+        "classic_score": classic["composite_score"],
+        "ml_weight_total": round(weight_total, 4),
+        "score_version": SCORE_VERSION,
+        "metric_panel": classic["metric_panel"],
+        "coverage": classic["coverage"],
     }

scoring/policy.py

@@ -0,0 +1,35 @@
+"""Canonical score version, brackets, and assessment semantics."""
+
+SCORE_VERSION = "accumulation-score-v2"
+
+# Half-open intervals [low, high), except the final bracket includes 100.
+# Keep labels canonical because they are persisted in live and backtest output.
+SCORE_BRACKETS = [
+    (0, 20, "EXTREME CAUTION"),
+    (20, 35, "CAUTION — OVERHEATED"),
+    (35, 50, "NEUTRAL"),
+    (50, 65, "MODERATE OPPORTUNITY"),
+    (65, 80, "STRONG ACCUMULATION ZONE"),
+    (80, 100, "EXTREME ACCUMULATION ZONE"),
+]
+
+
+def score_in_bracket(score, bracket):
+    """Return whether a 0-100 score belongs to a canonical bracket."""
+    low, high, _ = bracket
+    if not 0 <= score <= 100:
+        return False
+    return low <= score < high or (high == 100 and score == 100)
+
+
+def bracket_for_score(score):
+    """Return the one canonical bracket for a 0-100 score."""
+    for bracket in SCORE_BRACKETS:
+        if score_in_bracket(score, bracket):
+            return bracket
+    raise ValueError(f"score must be between 0 and 100, got {score!r}")
+
+
+def assessment_for_score(score):
+    """Return the canonical assessment label for a score."""
+    return bracket_for_score(score)[2]

scrapers/checkonchain.py

@@ -0,0 +1,170 @@
+"""Scraper for static CheckOnChain Plotly chart HTML pages."""
+
+from __future__ import annotations
+
+import array
+import base64
+import json
+import logging
+import re
+from html import unescape
+
+import requests
+
+log = logging.getLogger(__name__)
+
+CHARTS = {
+    "sopr": {
+        "url": "https://charts.checkonchain.com/btconchain/realised/sopr/sopr_light.html",
+        "traces": ["SOPR"],
+    },
+    "sellside_risk": {
+        "url": "https://charts.checkonchain.com/btconchain/realised/sellsideriskratio_all/sellsideriskratio_all_light.html",
+        "traces": ["Sell-side Risk Ratio", "Sellside Risk Ratio", "SSR"],
+    },
+    "active_address_momentum": {
+        "url": "https://charts.checkonchain.com/btconchain/adoption/actaddress_momentum/actaddress_momentum_light.html",
+        "traces": ["30DMA", "30 Day", "Active Address"],
+    },
+    "txcount_momentum": {
+        "url": "https://charts.checkonchain.com/btconchain/adoption/txcount_momentum/txcount_momentum_light.html",
+        "traces": ["30DMA", "30 Day", "Transaction"],
+    },
+    "nvt_price": {
+        "url": "https://charts.checkonchain.com/btconchain/pricing/pricing_nvtprice/pricing_nvtprice_light.html",
+        "traces": ["NVT Price", "NVT"],
+    },
+    "vdd_multiple": {
+        "url": "https://charts.checkonchain.com/btconchain/lifespan/vddmultiple/vddmultiple_light.html",
+        "traces": ["VDD Multiple", "Value Days Destroyed"],
+    },
+}
+
+
+def _extract_plotly_traces(html_text: str):
+    """Extract first Plotly.newPlot trace array from a static Plotly HTML page."""
+    marker = "Plotly.newPlot("
+    start = html_text.find(marker)
+    if start < 0:
+        return []
+    first_array = html_text.find("[", start)
+    if first_array < 0:
+        return []
+
+    depth = 0
+    in_string = False
+    escape = False
+    quote = ""
+    for idx in range(first_array, len(html_text)):
+        ch = html_text[idx]
+        if in_string:
+            if escape:
+                escape = False
+            elif ch == "\\":
+                escape = True
+            elif ch == quote:
+                in_string = False
+            continue
+        if ch in {'"', "'"}:
+            in_string = True
+            quote = ch
+        elif ch == "[":
+            depth += 1
+        elif ch == "]":
+            depth -= 1
+            if depth == 0:
+                raw = html_text[first_array:idx + 1]
+                return json.loads(raw)
+    return []
+
+
+def scrape_chart(url: str, timeout=30):
+    resp = requests.get(url, headers={"User-Agent": "Mozilla/5.0"}, timeout=timeout)
+    resp.raise_for_status()
+    return _extract_plotly_traces(unescape(resp.text))
+
+
+def _find_trace(traces, names):
+    names = [n.lower() for n in names if n]
+    # Prefer non-price traces with the requested terms.
+    for trace in traces:
+        trace_name = str(trace.get("name", "")).lower()
+        if "price" in trace_name and not any("price" in n for n in names):
+            continue
+        if any(n in trace_name for n in names):
+            return trace
+    # Fallback: first numeric non-price trace.
+    for trace in traces:
+        trace_name = str(trace.get("name", "")).lower()
+        if "price" in trace_name:
+            continue
+        y = trace.get("y") or []
+        if any(v is not None for v in y[-30:]):
+            return trace
+    return None
+
+
+def _decode_plotly_array(values):
+    """Decode Plotly typed-array JSON ({dtype, bdata}) or return plain values."""
+    if not isinstance(values, dict) or "bdata" not in values:
+        return values or []
+
+    dtype = values.get("dtype")
+    typecodes = {
+        "f8": "d", "float64": "d",
+        "f4": "f", "float32": "f",
+        "i8": "q", "int64": "q",
+        "i4": "i", "int32": "i",
+        "u8": "Q", "uint64": "Q",
+        "u4": "I", "uint32": "I",
+    }
+    typecode = typecodes.get(dtype)
+    if not typecode:
+        return []
+    decoded = base64.b64decode(values["bdata"])
+    arr = array.array(typecode)
+    arr.frombytes(decoded)
+    if values.get("byteorder") == "big":
+        arr.byteswap()
+    return arr.tolist()
+
+
+def _numeric_values(trace):
+    values = []
+    for value in _decode_plotly_array((trace or {}).get("y", [])):
+        if value is None:
+            continue
+        try:
+            values.append(float(value))
+        except (TypeError, ValueError):
+            pass
+    return values
+
+
+def _latest(values):
+    return values[-1] if values else None
+
+
+def _momentum(values, window=30):
+    if len(values) <= window or values[-window] == 0:
+        return None
+    return (values[-1] - values[-window]) / values[-window]
+
+
+def scrape_all():
+    results = {}
+    for key, cfg in CHARTS.items():
+        log.info("Scraping CheckOnChain %s ...", key)
+        try:
+            traces = scrape_chart(cfg["url"])
+            trace = _find_trace(traces, cfg.get("traces", []))
+            values = _numeric_values(trace)
+            value = _latest(values)
+            if key in {"active_address_momentum", "txcount_momentum", "vdd_multiple"}:
+                # The card value is momentum, while the sparkline shows the raw metric.
+                value = _momentum(values)
+            results[key] = {"value": value, "recent": values[-30:]}
+        except Exception as exc:
+            log.error("CheckOnChain scrape failed for %s: %s", key, exc)
+            results[key] = {"value": None, "error": str(exc)}
+    return results

scrapers/history_collector.py

@@ -114,8 +114,15 @@ def collect_onchain_history(progress_cb=None):
 
             for metric_key, trace_name in cfg["traces"].items():
                 if trace_name is None:
-                    # Grab first trace with numeric data
-                    for candidate in traces:
+                    if metric_key == "lth_supply":
+                        from scrapers.lookintobitcoin import _find_lth_supply_trace
+                        candidates = [_find_lth_supply_trace(traces)]
+                    else:
+                        candidates = traces
+                    # Grab the first validated trace with numeric data.
+                    for candidate in candidates:
+                        if not candidate:
+                            continue
                         y = candidate.get("y", [])
                         if y and any(v is not None for v in y[-10:]):
                             dates, values = _extract_series(candidate)

scrapers/history_updater.py

@@ -0,0 +1,112 @@
+"""Incremental history updater — appends new daily data to history.json from cache."""
+
+import json
+import logging
+import os
+from datetime import datetime, timezone
+
+log = logging.getLogger(__name__)
+
+DATA_DIR = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "data")
+HISTORY_PATH = os.path.join(DATA_DIR, "history.json")
+CACHE_PATH = os.path.join(DATA_DIR, "cache.json")
+
+
+def update_history():
+    """Append today's values from cache to history.json. Only adds NEW dates."""
+    if not os.path.exists(HISTORY_PATH):
+        log.warning("No history.json found — run full collection first")
+        return False
+
+    if not os.path.exists(CACHE_PATH):
+        log.warning("No cache.json found — run a scrape first")
+        return False
+
+    with open(HISTORY_PATH) as f:
+        history = json.load(f)
+    with open(CACHE_PATH) as f:
+        cache = json.load(f)
+
+    today = datetime.now(timezone.utc).strftime("%Y-%m-%d")
+    updated = False
+
+    # Map of cache keys to history keys and how to extract the value
+    mappings = {
+        "puell_multiple": {"history_key": "puell_multiple", "value_key": "value"},
+        "mvrv_zscore": {"history_key": "mvrv_zscore", "value_key": "value"},
+        "reserve_risk": {"history_key": "reserve_risk", "value_key": "value"},
+        "rhodl_ratio": {"history_key": "rhodl_ratio", "value_key": "value"},
+        "nupl": {"history_key": "nupl", "value_key": "value"},
+        "200w_sma": {"history_key": "200w_sma", "value_key": "value"},
+        "lth_realized_price": {"history_key": "lth_realized_price", "value_key": "value"},
+        "lth_supply": {"history_key": "lth_supply", "value_key": "value"},
+    }
+
+    for cache_key, mapping in mappings.items():
+        hkey = mapping["history_key"]
+        if hkey not in history:
+            continue
+
+        h = history[hkey]
+        dates = h.get("dates", [])
+        values = h.get("values", [])
+
+        # Skip if today already in history
+        if dates and dates[-1] >= today:
+            continue
+
+        # Get value from cache
+        cached = cache.get(cache_key, {})
+        val = cached.get(mapping["value_key"])
+        if val is not None:
+            dates.append(today)
+            values.append(val)
+            h["dates"] = dates
+            h["values"] = values
+            updated = True
+            log.info("Appended %s: %s = %s", hkey, today, val)
+
+    # Also update btc_price from cache
+    price_data = cache.get("price", {})
+    btc_price = price_data.get("price")
+    if btc_price and "btc_price" in history:
+        h = history["btc_price"]
+        if h["dates"][-1] < today:
+            h["dates"].append(today)
+            h["values"].append(btc_price)
+            updated = True
+
+    # BTC price for SMA chart
+    if btc_price and "btc_price_sma" in history:
+        h = history["btc_price_sma"]
+        if h["dates"][-1] < today:
+            h["dates"].append(today)
+            h["values"].append(btc_price)
+            updated = True
+
+    # BTC price for LTH chart
+    if btc_price and "btc_price_lth" in history:
+        h = history["btc_price_lth"]
+        if h["dates"][-1] < today:
+            h["dates"].append(today)
+            h["values"].append(btc_price)
+            updated = True
+
+    # Fear & Greed
+    fg = cache.get("fear_greed", {})
+    fg_val = fg.get("value")
+    if fg_val is not None and "fear_greed" in history:
+        h = history["fear_greed"]
+        if h["dates"][-1] < today:
+            h["dates"].append(today)
+            h["values"].append(int(fg_val))
+            updated = True
+
+    if updated:
+        with open(HISTORY_PATH, "w") as f:
+            json.dump(history, f)
+        log.info("History updated with %s data", today)
+    else:
+        log.info("History already up to date (last date >= %s)", today)
+
+    return updated

scrapers/lookintobitcoin.py

@@ -2,6 +2,7 @@
 
 import logging
 import traceback
+from contextlib import contextmanager
 
 log = logging.getLogger(__name__)
 
@@ -51,31 +52,42 @@ CHARTS = {
 }
 
 
-def scrape_chart(chart_path, timeout=25000):
-    """Scrape a single chart from LookIntoBitcoin. Returns list of trace dicts or None."""
+@contextmanager
+def browser_page():
+    """Open one headless browser page for a batch of chart requests."""
     from playwright.sync_api import sync_playwright
 
+    with sync_playwright() as playwright:
+        browser = playwright.chromium.launch(headless=True)
+        try:
+            yield browser.new_page()
+        finally:
+            browser.close()
+
+
+def scrape_chart(chart_path, timeout=25000, page=None):
+    """Scrape one chart, optionally reusing a caller-owned browser page."""
+    if page is None:
+        with browser_page() as owned_page:
+            return scrape_chart(chart_path, timeout=timeout, page=owned_page)
+
     store = {"data": None}
 
-    with sync_playwright() as p:
-        browser = p.chromium.launch(headless=True)
-        page = browser.new_page()
+    def handle_response(response):
+        if "_dash-update-component" in response.url:
+            try:
+                store["data"] = response.json()
+            except Exception:
+                pass
 
-        def handle_response(response):
-            if "_dash-update-component" in response.url:
-                try:
-                    store["data"] = response.json()
-                except Exception:
-                    pass
-
-        page.on("response", handle_response)
-        try:
-            page.goto(f"{BASE_URL}{chart_path}", timeout=timeout)
-            page.wait_for_timeout(6000)
-        except Exception as e:
-            log.warning("Navigation error for %s: %s", chart_path, e)
-        finally:
-            browser.close()
+    page.on("response", handle_response)
+    try:
+        page.goto(f"{BASE_URL}{chart_path}", timeout=timeout)
+        page.wait_for_timeout(6000)
+    except Exception as exc:
+        log.warning("Navigation error for %s: %s", chart_path, exc)
+    finally:
+        page.remove_listener("response", handle_response)
 
     if store["data"]:
         try:
@@ -115,6 +127,30 @@ def _find_trace(traces, name):
     return None
 
 
+def _trace_signal_is_active(trace):
+    """Return true only when the signal trace is active at its latest point."""
+    if not trace:
+        return False
+    values = trace.get("y", [])
+    if not values:
+        return False
+    latest = values[-1]
+    try:
+        return latest is not None and float(latest) != 0
+    except (TypeError, ValueError):
+        return bool(latest)
+
+
+def _find_lth_supply_trace(traces):
+    """Select an explicitly named LTH supply series and never a price fallback."""
+    for trace in traces or []:
+        name = str(trace.get("name", "")).lower()
+        is_lth = "long-term holder" in name or "long term holder" in name or "lth" in name
+        if is_lth and "supply" in name and "price" not in name:
+            return trace
+    return None
+
+
 def _get_latest_value(trace):
     """Get the most recent non-null y value from a trace."""
     if not trace:
@@ -148,13 +184,18 @@ def _get_recent_values(trace, n=30):
 
 
 def scrape_all():
-    """Scrape all charts and return parsed metric values."""
+    """Scrape all charts while reusing one browser process and page."""
+    with browser_page() as page:
+        return _scrape_all_with_page(page)
+
+
+def _scrape_all_with_page(page):
     results = {}
 
     for metric_key, chart_info in CHARTS.items():
         log.info("Scraping %s ...", metric_key)
         try:
-            traces = scrape_chart(chart_info["path"])
+            traces = scrape_chart(chart_info["path"], page=page)
             if not traces:
                 log.warning("No data for %s", metric_key)
                 results[metric_key] = {"value": None, "error": "No data returned"}
@@ -210,21 +251,16 @@ def scrape_all():
                     ],
                     "value": None,
                 }
-                # Try to detect buy signal from trace names/colors
+                # A named signal trace is not itself proof that the signal is active.
                 for t in traces:
                     name = t.get("name", "").lower()
-                    if "buy" in name or "signal" in name:
+                    if ("buy" in name or "signal" in name) and _trace_signal_is_active(t):
                         results[metric_key]["buy_signal"] = True
                         break
 
             elif metric_key == "lth_supply":
-                # Get main supply trace
-                t = traces[0] if traces else None
-                for candidate in traces:
-                    name = candidate.get("name", "").lower()
-                    if "supply" in name or "lth" in name:
-                        t = candidate
-                        break
+                # Require an explicitly named LTH supply trace; price is not supply.
+                t = _find_lth_supply_trace(traces)
                 recent = _get_recent_values(t, 60)
                 # Determine trend: compare recent avg to older avg
                 trend = None

scripts/run.sh

@@ -0,0 +1,11 @@
+#!/bin/sh
+set -eu
+
+ROOT=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
+cd "$ROOT"
+
+export PYTHONPATH="${PYTHONPATH:-.}"
+export PLAYWRIGHT_BROWSERS_PATH="${PLAYWRIGHT_BROWSERS_PATH:-$ROOT/.playwright}"
+
+exec uv run --frozen --no-dev --group runtime --group ml \
+  python -m uvicorn dashboard.server:app --host 0.0.0.0 --port "${PORT:-3088}"

tests/test_backtest_cache.py

@@ -0,0 +1,56 @@
+import json
+import os
+
+from backtesting import engine
+
+
+def test_run_backtest_caches_by_input_file_signature(monkeypatch, tmp_path):
+    history = tmp_path / "history.json"
+    thresholds = tmp_path / "thresholds.json"
+    weights = tmp_path / "weights.json"
+    cache = tmp_path / "cache.json"
+    for path in (history, thresholds, weights, cache):
+        path.write_text("{}")
+
+    monkeypatch.setattr(engine, "HISTORY_PATH", str(history))
+    monkeypatch.setattr(engine, "_THRESH_PATH", str(thresholds))
+    monkeypatch.setattr(engine, "ML_WEIGHTS_PATH", str(weights))
+    monkeypatch.setattr(engine, "CACHE_PATH", str(cache))
+    calls = []
+    monkeypatch.setattr(
+        engine, "_compute_backtest",
+        lambda ml_mode=False: calls.append(ml_mode) or {"ml_mode": ml_mode, "calls": len(calls)},
+    )
+    engine.clear_backtest_cache()
+
+    first = engine.run_backtest()
+    second = engine.run_backtest()
+    ml_first = engine.run_backtest(ml_mode=True)
+    ml_second = engine.run_backtest(ml_mode=True)
+    classic_after_ml = engine.run_backtest()
+
+    assert first == second == {"ml_mode": False, "calls": 1}
+    assert ml_first == ml_second == {"ml_mode": True, "calls": 2}
+    assert classic_after_ml == first
+    assert calls == [False, True]
+
+    history.write_text('{"changed": true}')
+    os.utime(history, None)
+    invalidated = engine.run_backtest()
+    assert invalidated == {"ml_mode": False, "calls": 3}
+
+
+def test_cached_backtest_results_are_isolated_from_caller_mutation(monkeypatch, tmp_path):
+    history = tmp_path / "history.json"
+    history.write_text("{}")
+    monkeypatch.setattr(engine, "HISTORY_PATH", str(history))
+    monkeypatch.setattr(engine, "_THRESH_PATH", str(tmp_path / "missing-thresholds.json"))
+    monkeypatch.setattr(engine, "ML_WEIGHTS_PATH", str(tmp_path / "missing-weights.json"))
+    monkeypatch.setattr(engine, "CACHE_PATH", str(tmp_path / "missing-cache.json"))
+    monkeypatch.setattr(engine, "_compute_backtest", lambda ml_mode=False: {"chart_data": [{"score": 10}]})
+    engine.clear_backtest_cache()
+
+    first = engine.run_backtest()
+    first["chart_data"][0]["score"] = 99
+
+    assert engine.run_backtest()["chart_data"][0]["score"] == 10

tests/test_backtest_data_quality.py

@@ -0,0 +1,61 @@
+from backtesting import engine as backtest
+from scoring import engine as scoring
+
+
+def test_metric_specific_staleness_does_not_apply_generic_30_day_fill():
+    lookup = {"2024-01-01": 42}
+
+    assert backtest._metric_observation(lookup, "2024-01-03", "fear_greed") == (42, "2024-01-01", 2)
+    assert backtest._metric_observation(lookup, "2024-01-04", "fear_greed") == (None, None, None)
+    assert backtest._metric_observation(lookup, "2024-01-08", "200w_sma") == (42, "2024-01-01", 7)
+    assert backtest._metric_observation(lookup, "2024-01-02", "unknown_metric") == (None, None, None)
+
+
+def test_current_context_score_uses_only_the_common_backtest_panel():
+    cached_scored = {
+        "composite_score": 100,
+        "metrics": [
+            {"key": "fear_greed", "score": 10},
+            {"key": "puell_multiple", "score": 0},
+            {"key": "sopr", "score": 10},
+            {"key": "vdd_multiple", "score": 10},
+        ],
+    }
+
+    score, coverage = backtest._common_panel_current_score(cached_scored)
+
+    assert score == 50.0
+    assert coverage == {
+        "available_count": 2,
+        "panel_count": len(backtest.BACKTEST_METRIC_PANEL),
+        "available_keys": ["fear_greed", "puell_multiple"],
+    }
+
+
+def test_live_and_backtest_outputs_publish_panel_and_coverage_metadata():
+    live = scoring.score_all({"fear_greed": {"value": 10}})
+
+    assert live["metric_panel"]["id"] == "live-all-v1"
+    assert live["metric_panel"]["count"] == live["total_count"]
+    assert live["coverage"]["available_count"] == live["scored_count"]
+    assert live["coverage"]["ratio"] == live["scored_count"] / live["total_count"]
+    assert "score_version" in live
+
+    metadata = backtest._backtest_data_quality_metadata([3, 5, 9])
+    assert metadata["metric_panel"]["id"] == "historical-common-v1"
+    assert metadata["metric_panel"]["keys"] == list(backtest.BACKTEST_METRIC_PANEL)
+    assert metadata["coverage"] == {
+        "minimum_metrics": 3,
+        "maximum_metrics": 9,
+        "average_metrics": 5.7,
+        "panel_count": 9,
+    }
+    assert metadata["staleness_days"] == backtest.METRIC_MAX_AGE_DAYS
+
+
+def test_chart_data_exposes_metric_values_under_frontend_contract():
+    result = backtest.run_backtest()
+    entries = [entry for entry in result["chart_data"] if entry.get("metric_values")]
+
+    assert entries
+    assert all("metrics" not in entry for entry in entries)

tests/test_backtest_ml_scope.py

@@ -0,0 +1,64 @@
+from backtesting import engine
+from ml.artifacts import ML_ARTIFACT_SCHEMA_VERSION, REQUIRED_WEIGHT_KEYS
+from scoring.policy import SCORE_VERSION
+
+
+def _weights(focus):
+    weights = {key: 0.0 for key in REQUIRED_WEIGHT_KEYS}
+    weights[focus] = 1.0
+    return weights
+
+
+def _artifact(with_folds=True):
+    artifact = {
+        "artifact_schema_version": ML_ARTIFACT_SCHEMA_VERSION,
+        "score_version": SCORE_VERSION,
+        "weights": _weights("fear_greed"),
+        "provenance": {
+            "validation_method": "purged_expanding_window",
+            "label_horizon_days": 365,
+            "weight_scope": "full_history_fit",
+            "training_date_range": {"start": "2018-01-01", "end": "2024-01-01"},
+            "trained_at": "2026-07-01T00:00:00+00:00",
+        },
+        "cv_results": {"folds": []},
+    }
+    if with_folds:
+        artifact["cv_results"]["folds"] = [
+            {
+                "fold": 1,
+                "weights": _weights("drawdown"),
+                "date_ranges": {"validation": "2020-01-01 to 2020-12-31"},
+            },
+            {
+                "fold": 2,
+                "weights": _weights("nupl"),
+                "date_ranges": {"validation": "2021-01-01 to 2021-12-31"},
+            },
+        ]
+    return artifact
+
+
+def test_ml_backtest_plan_prefers_fold_weights_and_marks_them_oos():
+    plan = engine._build_ml_backtest_plan(_artifact(with_folds=True))
+
+    weights, fold = engine._weights_for_backtest_date("2021-06-01", plan)
+
+    assert weights == _weights("nupl")
+    assert fold == 2
+    assert plan["evaluation_scope"] == "out_of_sample_validation_folds"
+    assert plan["is_out_of_sample"] is True
+    assert plan["weighting_source"] == "fold_specific_weights"
+    assert engine._weights_for_backtest_date("2019-12-31", plan) == (None, None)
+
+
+def test_full_history_weights_are_explicitly_not_oos():
+    plan = engine._build_ml_backtest_plan(_artifact(with_folds=False))
+
+    weights, fold = engine._weights_for_backtest_date("2021-06-01", plan)
+
+    assert weights == _weights("fear_greed")
+    assert fold is None
+    assert plan["evaluation_scope"] == "in_sample_full_history_weights"
+    assert plan["is_out_of_sample"] is False
+    assert plan["weighting_source"] == "final_full_history_weights"

tests/test_backtest_statistics.py

@@ -0,0 +1,61 @@
+from backtesting import engine, statistics
+
+
+def test_moving_block_bootstrap_is_deterministic_and_handles_constant_series():
+    first = statistics.moving_block_bootstrap_ci(
+        [12.5] * 120,
+        block_size=15,
+        n_resamples=200,
+        seed=7,
+    )
+    second = statistics.moving_block_bootstrap_ci(
+        [12.5] * 120,
+        block_size=15,
+        n_resamples=200,
+        seed=7,
+    )
+
+    assert first == second
+    assert first == {"estimate": 12.5, "ci_low": 12.5, "ci_high": 12.5, "n": 120}
+
+
+def test_summarize_returns_reports_observations_and_block_bootstrap_interval():
+    summary = statistics.summarize_returns(
+        [10.0, -5.0, 20.0, -10.0],
+        block_size=2,
+        n_resamples=200,
+        seed=3,
+    )
+
+    assert summary["n"] == 4
+    assert summary["mean"] == 3.75
+    assert summary["median"] == 2.5
+    assert summary["win_rate"] == 50.0
+    assert summary["mean_ci_low"] <= summary["mean"] <= summary["mean_ci_high"]
+
+
+def test_backtest_brackets_publish_bootstrap_confidence_intervals():
+    stats = {}
+
+    engine._add_return_statistics(stats, "90d", [10.0, -5.0, 20.0, -10.0])
+
+    assert stats["avg_90d"] == 3.75
+    assert stats["median_90d"] == 2.5
+    assert stats["win_rate_90d"] == 50.0
+    assert stats["avg_90d_ci_low"] <= stats["avg_90d"] <= stats["avg_90d_ci_high"]
+
+
+def test_long_horizon_returns_use_a_matching_dependence_block(monkeypatch):
+    observed = {}
+
+    def fake_summary(values, *, block_size, n_resamples):
+        observed.update(block_size=block_size, n_resamples=n_resamples)
+        return {
+            "mean": 1.0, "median": 1.0, "win_rate": 100.0,
+            "mean_ci_low": 0.5, "mean_ci_high": 1.5, "n": len(values),
+        }
+
+    monkeypatch.setattr(engine, "summarize_returns", fake_summary)
+    engine._add_return_statistics({}, "365d", [1.0] * 500)
+
+    assert observed == {"block_size": 365, "n_resamples": 400}

tests/test_job_registry.py

@@ -0,0 +1,53 @@
+import threading
+
+from dashboard.jobs import JobRegistry
+
+
+def test_job_registry_atomically_reserves_only_one_job_per_kind(tmp_path):
+    registry = JobRegistry(tmp_path / "jobs.json")
+    barrier = threading.Barrier(10)
+    results = []
+
+    def reserve():
+        barrier.wait()
+        results.append(registry.reserve("refresh", details={"full": False}))
+
+    threads = [threading.Thread(target=reserve) for _ in range(10)]
+    for thread in threads:
+        thread.start()
+    for thread in threads:
+        thread.join()
+
+    reserved = [job for job in results if job is not None]
+    assert len(reserved) == 1
+    assert reserved[0]["id"]
+    assert reserved[0]["status"] == "queued"
+    assert registry.active("refresh")["id"] == reserved[0]["id"]
+
+
+def test_job_registry_tracks_completion_and_result(tmp_path):
+    registry = JobRegistry(tmp_path / "jobs.json")
+    job = registry.reserve("history")
+
+    result = registry.run(job["id"], lambda: {"records": 42})
+
+    assert result == {"records": 42}
+    saved = registry.get(job["id"])
+    assert saved["status"] == "complete"
+    assert saved["result"] == {"records": 42}
+    assert saved["started_at"]
+    assert saved["finished_at"]
+    assert registry.active("history") is None
+
+
+def test_job_registry_marks_abandoned_active_jobs_interrupted_on_restart(tmp_path):
+    path = tmp_path / "jobs.json"
+    first = JobRegistry(path)
+    job = first.reserve("refresh")
+
+    restarted = JobRegistry(path)
+
+    recovered = restarted.get(job["id"])
+    assert recovered["status"] == "interrupted"
+    assert recovered["finished_at"]
+    assert restarted.active("refresh") is None

tests/test_legacy_ml_validation.py

@@ -0,0 +1,103 @@
+import numpy as np
+import pandas as pd
+
+import orchestrator
+from ml_engine import train_and_backtest as legacy
+from ml_engine.train_and_backtest import create_accumulation_target
+
+
+def _frame(prices):
+    return pd.DataFrame({"close": prices})
+
+
+def test_accumulation_target_for_existing_row_is_invariant_to_unrelated_future_rows():
+    config = {
+        "timeframe": "4h",
+        "target": {
+            "forward_periods_4h": [1, 2, 3],
+            "weights": [0.2, 0.3, 0.5],
+            "return_scales_pct": [5, 10, 20],
+        },
+    }
+    base = _frame([100, 102, 104, 106, 108, 110, 112, 114])
+    extended = _frame([100, 102, 104, 106, 108, 110, 112, 114, 1000, 1, 2000])
+
+    base_target = create_accumulation_target(base, config)
+    extended_target = create_accumulation_target(extended, config)
+
+    assert np.isclose(base_target.iloc[0], extended_target.iloc[0])
+    assert 0 <= base_target.iloc[0] <= 100
+
+
+def test_rolling_validation_purges_forward_label_horizon_at_train_boundaries(monkeypatch):
+    rows = 200
+    frame = pd.DataFrame({
+        "feature": np.linspace(0, 1, rows),
+        "target": np.arange(rows, dtype=float) % 100,
+        "close": np.linspace(10_000, 20_000, rows),
+    })
+    observed = []
+
+    def fake_train(X_train, y_train, X_val, y_val, X_test, *args):
+        observed.append((len(X_train), len(X_val), len(X_test)))
+        return np.full(len(X_test), 50.0), np.array([1.0])
+
+    monkeypatch.setattr(legacy, "_train_and_predict_window", fake_train)
+    config = {
+        "model_type": "xgboost",
+        "target": {"forward_periods_4h": [1, 2, 3]},
+        "training": {
+            "rolling_train_size": 120,
+            "rolling_test_size": 40,
+            "validation_pct": 0.25,
+        },
+        "features": {"use_scaler": False, "use_pca": False},
+        "strategy": {},
+    }
+
+    legacy.rolling_window_train_test(frame, ["feature"], config)
+
+    assert observed[0] == (87, 27, 40)
+
+
+def test_periodic_accumulation_compares_equal_contributions_and_retains_cash():
+    result = legacy.simulate_periodic_accumulation(
+        predicted_scores=np.array([90, 10, 90, 10], dtype=float),
+        close_prices=np.array([100, 300, 100, 200], dtype=float),
+        buy_threshold=70,
+        contribution=100,
+    )
+
+    assert np.isclose(result["dca_contributed"], 400)
+    assert np.isclose(result["model_contributed"], 400)
+    assert np.isclose(result["model_cash"], 100)
+    assert np.isclose(result["model_btc"], 3)
+    assert result["model_terminal_value"] > result["dca_terminal_value"]
+
+
+def test_compiled_results_publish_equal_capital_terminal_wealth_metric():
+    predictions = [
+        {"predicted": score, "actual": 50.0, "close": price}
+        for score, price in zip([90, 10, 90, 10], [100, 300, 100, 200])
+    ]
+
+    result = legacy.compile_results(
+        predictions,
+        per_window_cost_improvement=[],
+        fi_sum=np.array([1.0]),
+        fi_count=1,
+        feature_cols=["feature"],
+        config={"model_type": "xgboost", "strategy": {"good_buy_threshold": 70}},
+    )
+
+    assert result["terminal_wealth_improvement_pct"] > 0
+    assert result["backtest_objective"] == "equal_periodic_contribution_terminal_wealth"
+
+
+def test_orchestrator_selects_models_by_terminal_wealth_not_cost_basis():
+    results = {
+        "terminal_wealth_improvement_pct": 4.5,
+        "cost_basis_improvement_pct": 99.0,
+    }
+
+    assert orchestrator.objective_score(results) == 4.5

tests/test_ml_artifacts.py

@@ -0,0 +1,60 @@
+import json
+from pathlib import Path
+
+from ml import artifacts
+from scoring import engine
+
+
+REPO_ROOT = Path(__file__).resolve().parents[1]
+
+
+def _valid_artifact():
+    return {
+        "artifact_schema_version": artifacts.ML_ARTIFACT_SCHEMA_VERSION,
+        "score_version": artifacts.SCORE_VERSION,
+        "weights": {key: 1 / len(artifacts.REQUIRED_WEIGHT_KEYS) for key in artifacts.REQUIRED_WEIGHT_KEYS},
+        "provenance": {
+            "validation_method": "purged_expanding_window",
+            "label_horizon_days": 365,
+            "weight_scope": "full_history_fit",
+            "training_date_range": {"start": "2018-02-01", "end": "2025-03-21"},
+            "trained_at": "2026-07-01T00:00:00+00:00",
+        },
+    }
+
+
+def test_repository_artifact_has_current_schema_and_purged_provenance():
+    artifact_path = REPO_ROOT / "config" / "ml_weights.json"
+    artifact = json.loads(artifact_path.read_text())
+
+    status = artifacts.validate_ml_artifact(artifact)
+
+    assert status["valid"] is True
+    assert status["schema_version"] == artifacts.ML_ARTIFACT_SCHEMA_VERSION
+    assert status["score_version"] == artifacts.SCORE_VERSION
+    assert status["has_oos_fold_weights"] is True
+    assert status["errors"] == []
+
+
+def test_valid_artifact_requires_schema_score_version_and_purged_provenance():
+    artifact = _valid_artifact()
+
+    status = artifacts.validate_ml_artifact(artifact)
+
+    assert status == {
+        "valid": True,
+        "schema_version": artifacts.ML_ARTIFACT_SCHEMA_VERSION,
+        "score_version": artifacts.SCORE_VERSION,
+        "weight_scope": "full_history_fit",
+        "has_oos_fold_weights": False,
+        "errors": [],
+    }
+
+
+def test_live_scoring_refuses_schema_less_weights(tmp_path, monkeypatch):
+    path = tmp_path / "ml_weights.json"
+    path.write_text(json.dumps({"weights": {"fear_greed": 1.0}}))
+    monkeypatch.setattr(engine, "ML_WEIGHTS_PATH", str(path))
+
+    assert engine.load_ml_weights() == {}
+    assert engine.get_ml_artifact_status()["valid"] is False

tests/test_ml_optimizer_validation.py

@@ -0,0 +1,107 @@
+from datetime import datetime, timedelta
+
+import numpy as np
+
+from ml import optimizer
+
+
+def _row(date, returns=10.0, score_200w=10, score_drawdown=0):
+    row = {
+        "date": date,
+        "price": 100.0,
+        "fwd_365d": returns,
+        "score_puell_multiple": 0,
+        "score_mvrv_zscore": 0,
+        "score_reserve_risk": 0,
+        "score_rhodl_ratio": 0,
+        "score_nupl": 0,
+        "score_fear_greed": 0,
+        "score_drawdown": score_drawdown,
+        "score_pct_above_200w_sma": score_200w,
+        "score_pct_above_lth_rp": 0,
+    }
+    return row
+
+
+def test_purged_time_series_splits_remove_overlapping_forward_label_windows():
+    start = datetime(2020, 1, 1)
+    rows = [_row((start + timedelta(days=i)).strftime("%Y-%m-%d")) for i in range(900)]
+
+    splits = list(
+        optimizer.purged_time_series_splits(
+            rows,
+            n_splits=3,
+            label_horizon_days=365,
+            embargo_days=0,
+        )
+    )
+
+    assert splits, "expected at least one viable split"
+    for train_idx, val_idx in splits:
+        val_start = datetime.strptime(rows[val_idx[0]]["date"], "%Y-%m-%d")
+        latest_allowed_train_date = val_start - timedelta(days=365)
+        assert len(train_idx) > 0, "purging should keep non-overlapping expanding-window training rows"
+        for idx in train_idx:
+            train_date = datetime.strptime(rows[idx]["date"], "%Y-%m-%d")
+            assert train_date <= latest_allowed_train_date
+
+
+def test_run_out_of_sample_comparison_scores_only_validation_rows_with_fold_weights():
+    rows = [
+        _row("2020-01-01", returns=-10, score_200w=0, score_drawdown=10),
+        _row("2020-01-02", returns=-5, score_200w=0, score_drawdown=10),
+        _row("2020-01-03", returns=100, score_200w=10, score_drawdown=10),
+        _row("2020-01-04", returns=120, score_200w=10, score_drawdown=10),
+    ]
+    fold_results = [
+        {
+            "fold": 1,
+            "val_idx": [2, 3],
+            "weights": {"pct_above_200w_sma": 1.0, "drawdown": 0.0},
+        }
+    ]
+
+    comparison = optimizer.run_out_of_sample_comparison(rows, fold_results)
+
+    assert comparison["validation_days"] == 2
+    assert comparison["folds"] == 1
+    assert sum(bucket["days"] for bucket in comparison["ml_weighted"]) == 2
+    assert sum(bucket["days"] for bucket in comparison["equal_weight"]) == 2
+
+    extreme_ml = next(bucket for bucket in comparison["ml_weighted"] if bucket["label"] == "EXTREME ACCUMULATION ZONE")
+    assert extreme_ml["days"] == 2
+    assert extreme_ml["avg_365d"] == 110.0
+
+    caution_equal = next(bucket for bucket in comparison["equal_weight"] if bucket["label"] == "CAUTION — OVERHEATED")
+    assert caution_equal["days"] == 2
+
+
+def test_classification_splits_skip_training_windows_with_one_class():
+    y = np.array([1, 1, 1, 0, 1, 0])
+    splits = [
+        (np.array([0, 1]), np.array([2, 3])),
+        (np.array([0, 1, 3, 4]), np.array([5])),
+    ]
+
+    viable = list(optimizer.viable_classification_splits(y, splits))
+
+    assert len(viable) == 1
+    assert viable[0][0].tolist() == [0, 1, 3, 4]
+
+
+def test_artifact_folds_omit_large_internal_index_arrays():
+    folds = [{
+        "fold": 1,
+        "train_idx": [0, 1],
+        "val_idx": [2, 3],
+        "weights": {"fear_greed": 1.0},
+        "date_ranges": {"validation": "2024-01-01 to 2024-01-02"},
+    }]
+
+    saved = optimizer.artifact_fold_results(folds)
+
+    assert saved == [{
+        "fold": 1,
+        "weights": {"fear_greed": 1.0},
+        "date_ranges": {"validation": "2024-01-01 to 2024-01-02"},
+    }]

tests/test_reliability_persistence.py

@@ -0,0 +1,121 @@
+import json
+import threading
+from datetime import datetime, timedelta, timezone
+
+import pytest
+
+from dashboard.persistence import (
+    append_daily_jsonl,
+    atomic_write_json,
+    load_jsonl_tail,
+    merge_observation,
+    onchain_refresh_due,
+)
+
+
+def test_atomic_write_json_remains_readable_under_concurrent_writers(tmp_path):
+    path = tmp_path / "cache.json"
+
+    threads = [
+        threading.Thread(target=atomic_write_json, args=(path, {"writer": i, "values": list(range(100))}))
+        for i in range(12)
+    ]
+    for thread in threads:
+        thread.start()
+    for thread in threads:
+        thread.join()
+
+    saved = json.loads(path.read_text())
+    assert saved["writer"] in range(12)
+    assert saved["values"] == list(range(100))
+    assert not list(tmp_path.glob(".cache.json.*.tmp"))
+
+
+def test_append_daily_jsonl_writes_at_most_one_entry_per_utc_day(tmp_path):
+    path = tmp_path / "scores.jsonl"
+    first = {"timestamp": "2026-07-26T01:00:00+00:00", "score": 10}
+    duplicate_day = {"timestamp": "2026-07-26T23:59:00+00:00", "score": 20}
+    next_day = {"timestamp": "2026-07-27T00:01:00+00:00", "score": 30}
+
+    assert append_daily_jsonl(path, first) is True
+    assert append_daily_jsonl(path, duplicate_day) is False
+    assert append_daily_jsonl(path, next_day) is True
+
+    assert load_jsonl_tail(path, limit=90) == [first, next_day]
+
+
+def test_load_jsonl_tail_is_bounded_and_ignores_malformed_lines(tmp_path):
+    path = tmp_path / "scores.jsonl"
+    path.write_text("".join(json.dumps({"n": i}) + "\n" for i in range(200)) + "partial{")
+
+    assert load_jsonl_tail(path, limit=3, chunk_size=64) == [{"n": 197}, {"n": 198}, {"n": 199}]
+
+
+def test_merge_observation_preserves_last_known_good_with_stale_metadata():
+    old = {
+        "value": 1.25,
+        "observed_at": "2026-07-25T12:00:00+00:00",
+        "source": "lookintobitcoin",
+        "stale": False,
+        "last_error": None,
+    }
+
+    merged = merge_observation(old, None, source="lookintobitcoin", error="timeout")
+
+    assert merged == {
+        "value": 1.25,
+        "observed_at": "2026-07-25T12:00:00+00:00",
+        "source": "lookintobitcoin",
+        "stale": True,
+        "last_error": "timeout",
+    }
+
+
+def test_merge_observation_records_metadata_for_fresh_value():
+    observed_at = "2026-07-26T12:00:00+00:00"
+
+    merged = merge_observation(
+        {"value": 1.0}, {"value": 2.0, "trend": "up"},
+        source="checkonchain", observed_at=observed_at,
+    )
+
+    assert merged["value"] == 2.0
+    assert merged["trend"] == "up"
+    assert merged["observed_at"] == observed_at
+    assert merged["source"] == "checkonchain"
+    assert merged["stale"] is False
+    assert merged["last_error"] is None
+
+
+def test_merge_observation_rejects_error_only_payload_as_fresh_data():
+    old = {
+        "value": 1.25,
+        "observed_at": "2026-07-25T12:00:00+00:00",
+        "source": "lookintobitcoin",
+        "stale": False,
+        "last_error": None,
+    }
+
+    merged = merge_observation(
+        old,
+        {"value": None, "error": "No data returned"},
+        source="lookintobitcoin",
+        error="metric missing from scrape",
+    )
+
+    assert merged["value"] == 1.25
+    assert merged["observed_at"] == old["observed_at"]
+    assert merged["stale"] is True
+    assert merged["last_error"] == "No data returned"
+
+
+@pytest.mark.parametrize("timestamp", [None, "", "not-a-time"])
+def test_onchain_refresh_due_when_timestamp_is_missing_or_invalid(timestamp):
+    assert onchain_refresh_due(timestamp, now=datetime(2026, 7, 26, tzinfo=timezone.utc)) is True
+
+
+def test_onchain_refresh_due_after_ttl():
+    now = datetime(2026, 7, 26, 12, tzinfo=timezone.utc)
+
+    assert onchain_refresh_due((now - timedelta(hours=5)).isoformat(), now=now, ttl_seconds=21600) is False
+    assert onchain_refresh_due((now - timedelta(hours=7)).isoformat(), now=now, ttl_seconds=21600) is True

tests/test_score_policy.py

@@ -0,0 +1,38 @@
+import pytest
+
+from backtesting import engine as backtest_engine
+from ml import optimizer
+from scoring import engine as scoring_engine
+from scoring import policy
+
+
+def test_score_brackets_are_contiguous_and_shared_by_all_scoring_paths():
+    assert backtest_engine.BRACKETS is policy.SCORE_BRACKETS
+    assert optimizer.BRACKETS is policy.SCORE_BRACKETS
+
+    for left, right in zip(policy.SCORE_BRACKETS, policy.SCORE_BRACKETS[1:]):
+        assert left[1] == right[0]
+
+    for tenth in range(0, 1001):
+        score = tenth / 10
+        matches = [bracket for bracket in policy.SCORE_BRACKETS if policy.score_in_bracket(score, bracket)]
+        assert len(matches) == 1, f"score {score} matched {matches}"
+        assert policy.assessment_for_score(score) == matches[0][2]
+
+
+@pytest.mark.parametrize(
+    ("score", "assessment"),
+    [
+        (0, "EXTREME CAUTION"),
+        (19.999, "EXTREME CAUTION"),
+        (20, "CAUTION — OVERHEATED"),
+        (35, "NEUTRAL"),
+        (50, "MODERATE OPPORTUNITY"),
+        (65, "STRONG ACCUMULATION ZONE"),
+        (80, "EXTREME ACCUMULATION ZONE"),
+        (100, "EXTREME ACCUMULATION ZONE"),
+    ],
+)
+def test_assessment_boundaries_match_live_scoring(score, assessment):
+    assert policy.assessment_for_score(score) == assessment
+    assert scoring_engine.assessment_for_score(score) == assessment

tests/test_scoring_engine_ml.py

@@ -0,0 +1,64 @@
+import math
+
+from scoring import engine
+
+
+def _complete_metrics():
+    return {
+        "fear_greed": {"value": 10, "classification": "Extreme Fear"},
+        "puell_multiple": {"value": 0.3},
+        "mvrv_zscore": {"value": -0.1},
+        "drawdown": {"value": 60.0, "ath": 250.0},
+        "price": {"price": 100.0},
+        "200w_sma": {"value": 120.0},
+        "reserve_risk": {"value": 0.001},
+        "rhodl_ratio": {"value": 50.0},
+        "nupl": {"value": -0.1},
+        "lth_realized_price": {"value": 120.0},
+        "hash_ribbons": {"buy_signal": False},
+    }
+
+
+def test_score_all_ml_normalizes_displayed_weights_and_contributions(monkeypatch):
+    monkeypatch.setattr(
+        engine,
+        "load_ml_weights",
+        lambda: {
+            "fear_greed": 0.40,
+            "puell_multiple": 0.20,
+            "mvrv_zscore": 0.15,
+            "drawdown": 0.10,
+            "pct_above_200w_sma": 0.05,
+            "reserve_risk": 0.04,
+            "rhodl_ratio": 0.03,
+            "nupl": 0.02,
+            "pct_above_lth_rp": 0.01,
+        },
+    )
+
+    scored = engine.score_all_ml(_complete_metrics())
+
+    assert scored["ml_mode"] is True
+    valid_metrics = [m for m in scored["metrics"] if m["score"] is not None]
+    assert scored["ml_weight_total"] == 1.01  # trained weights + small hash-ribbons fallback
+    assert math.isclose(sum(m["ml_weight"] for m in valid_metrics), 1.0, abs_tol=0.001)
+    assert math.isclose(
+        sum(m["ml_contribution"] for m in valid_metrics),
+        scored["composite_score"],
+        abs_tol=0.05,
+    )
+
+    hash_ribbons = next(m for m in valid_metrics if m["key"] == "hash_ribbons")
+    assert hash_ribbons["ml_raw_weight"] == 0.01
+    assert hash_ribbons["ml_weight"] == round(0.01 / 1.01, 4)
+
+
+def test_score_all_ml_preserves_classic_fallback_when_weights_missing(monkeypatch):
+    monkeypatch.setattr(engine, "load_ml_weights", lambda: {})
+
+    scored = engine.score_all_ml(_complete_metrics())
+
+    assert scored["ml_mode"] is False
+    assert scored["ml_error"]
+    assert scored["ml_artifact"]["valid"] is False
+    assert "classic_score" not in scored

tests/test_scraper_semantics.py

@@ -0,0 +1,56 @@
+from contextlib import contextmanager
+
+from scrapers import lookintobitcoin
+from scoring import engine
+
+
+def test_hash_ribbon_signal_requires_a_current_truthy_marker():
+    named_but_inactive = {"name": "Buy Signal", "y": [1, None, None]}
+    active = {"name": "Buy Signal", "y": [None, 0, 1]}
+
+    assert lookintobitcoin._trace_signal_is_active(named_but_inactive) is False
+    assert lookintobitcoin._trace_signal_is_active(active) is True
+
+
+def test_lth_supply_trace_selection_never_falls_back_to_price():
+    traces = [
+        {"name": "BTC Price", "y": [60000, 61000]},
+        {"name": "Long-Term Holder Supply", "y": [14_000_000, 14_100_000]},
+    ]
+
+    assert lookintobitcoin._find_lth_supply_trace(traces)["name"] == "Long-Term Holder Supply"
+    assert lookintobitcoin._find_lth_supply_trace(traces[:1]) is None
+
+
+def test_vdd_derived_return_is_labeled_as_momentum_not_raw_multiple():
+    result = engine.score_all({"vdd_multiple": {"value": 0.12}})
+    vdd = next(metric for metric in result["metrics"] if metric["key"] == "vdd_multiple")
+
+    assert vdd["name"] == "VDD 30-Period Momentum"
+    assert vdd["transform"] == "30_period_return"
+
+
+def test_scrape_all_reuses_one_browser_page(monkeypatch):
+    page = object()
+    seen_pages = []
+
+    @contextmanager
+    def fake_browser_page():
+        yield page
+
+    def fake_scrape_chart(_path, timeout=25000, page=None):
+        seen_pages.append(page)
+        return [{"name": "metric", "y": [1.0]}]
+
+    monkeypatch.setattr(lookintobitcoin, "CHARTS", {
+        "first": {"path": "/first", "traces": ["metric"]},
+        "second": {"path": "/second", "traces": ["metric"]},
+    })
+    monkeypatch.setattr(lookintobitcoin, "browser_page", fake_browser_page)
+    monkeypatch.setattr(lookintobitcoin, "scrape_chart", fake_scrape_chart)
+
+    result = lookintobitcoin.scrape_all()
+
+    assert seen_pages == [page, page]
+    assert result["first"]["value"] == 1.0
+    assert result["second"]["value"] == 1.0

tests/test_server_reliability.py

@@ -0,0 +1,194 @@
+import importlib
+import json
+import sys
+import types
+from datetime import datetime, timedelta, timezone
+
+import pytest
+
+
+@pytest.fixture
+def server(monkeypatch):
+    started = []
+    monkeypatch.setattr("threading.Thread.start", lambda self: started.append(self))
+    sys.modules.pop("dashboard.server", None)
+    module = importlib.import_module("dashboard.server")
+    module._threads_started_during_import = started
+    return module
+
+
+def test_server_import_does_not_start_scheduler_threads(server):
+    assert server._threads_started_during_import == []
+
+
+def test_frontend_uses_backtest_metric_values_contract(server):
+    assert ".filter(d => d.metric_values && d.metric_values[metricKey] != null)" in server.DASHBOARD_HTML
+    assert ".map(d => ({ date: d.date, value: d.metric_values[metricKey]" in server.DASHBOARD_HTML
+
+
+def test_dashboard_does_not_issue_duplicate_initial_backtest_request(server):
+    assert "const br = await fetch('/api/backtest');" not in server.DASHBOARD_HTML
+    assert server.DASHBOARD_HTML.count("fetch('/api/backtest?mode=' + currentMode)") == 1
+
+
+def test_health_endpoints_distinguish_process_liveness_from_data_readiness(server, monkeypatch):
+    assert server.health_live() == {"status": "ok"}
+
+    monkeypatch.setattr(server, "load_cache", lambda: {})
+    unavailable = server.health_ready()
+    assert unavailable.status_code == 503
+
+    monkeypatch.setattr(
+        server,
+        "load_cache",
+        lambda: {"_scored": {"composite_score": 72, "scored_count": 8}},
+    )
+    assert server.health_ready() == {
+        "status": "ready",
+        "score": 72,
+        "scored_metrics": 8,
+    }
+
+
+def test_server_cache_and_history_use_reliable_persistence(server, monkeypatch, tmp_path):
+    monkeypatch.setattr(server, "CACHE_PATH", str(tmp_path / "cache.json"))
+    monkeypatch.setattr(server, "HISTORY_PATH", str(tmp_path / "scores.jsonl"))
+
+    server.save_cache({"metric": {"value": 1}})
+    server.append_history({
+        "composite_score": 50,
+        "scored_count": 1,
+        "metrics": [{"key": "metric", "score": 5, "value": 1}],
+    })
+    server.append_history({
+        "composite_score": 60,
+        "scored_count": 1,
+        "metrics": [{"key": "metric", "score": 6, "value": 2}],
+    })
+
+    assert server.load_cache() == {"metric": {"value": 1}}
+    assert len(server.load_history()) == 1
+
+
+def test_partial_fast_scrape_preserves_last_known_good_metric(server, monkeypatch, tmp_path):
+    cache_path = tmp_path / "cache.json"
+    history_path = tmp_path / "scores.jsonl"
+    now = datetime.now(timezone.utc)
+    cache_path.write_text(json.dumps({
+        "price": {
+            "price": 65000,
+            "observed_at": (now - timedelta(minutes=15)).isoformat(),
+            "source": "coingecko",
+            "stale": False,
+            "last_error": None,
+        },
+        "puell_multiple": {"value": 1.2},
+        "_onchain_timestamp": now.isoformat(),
+    }))
+    monkeypatch.setattr(server, "CACHE_PATH", str(cache_path))
+    monkeypatch.setattr(server, "HISTORY_PATH", str(history_path))
+    monkeypatch.setattr(server.fear_greed, "fetch", lambda: {"value": 25})
+    monkeypatch.setattr(server.price, "fetch_current", lambda: (_ for _ in ()).throw(RuntimeError("price timeout")))
+    monkeypatch.setattr(server.price, "fetch_ath", lambda: (_ for _ in ()).throw(RuntimeError("ATH timeout")))
+    monkeypatch.setattr(server.price, "fetch_historical", lambda: (_ for _ in ()).throw(RuntimeError("history timeout")))
+    monkeypatch.setattr(server.engine, "score_all", lambda metrics: {"composite_score": 50, "scored_count": 1, "metrics": []})
+    monkeypatch.setattr(server.engine, "score_all_ml", lambda metrics: {"composite_score": 50, "scored_count": 1, "metrics": []})
+    fake_updater = types.ModuleType("scrapers.history_updater")
+    fake_updater.update_history = lambda: None
+    monkeypatch.setitem(sys.modules, "scrapers.history_updater", fake_updater)
+
+    server.run_scrape()
+
+    saved = json.loads(cache_path.read_text())
+    assert saved["price"]["price"] == 65000
+    assert saved["price"]["stale"] is True
+    assert "price timeout" in saved["price"]["last_error"]
+    assert saved["fear_greed"]["value"] == 25
+    assert saved["fear_greed"]["stale"] is False
+    assert saved["fear_greed"]["source"] == "alternative.me"
+
+
+def test_onchain_sources_fail_independently(server, monkeypatch):
+    fake_lib = types.ModuleType("scrapers.lookintobitcoin")
+    setattr(fake_lib, "scrape_all", lambda: (_ for _ in ()).throw(RuntimeError("LIB down")))
+    fake_coc = types.ModuleType("scrapers.checkonchain")
+    setattr(fake_coc, "scrape_all", lambda: {"sopr": {"value": 0.99}})
+    monkeypatch.setitem(sys.modules, "scrapers.lookintobitcoin", fake_lib)
+    monkeypatch.setitem(sys.modules, "scrapers.checkonchain", fake_coc)
+
+    observations, errors, successful_sources = server._scrape_onchain_sources()
+
+    assert observations["sopr"]["value"] == 0.99
+    assert successful_sources == 1
+    assert any("LookIntoBitcoin" in error for error in errors)
+
+
+def test_expired_onchain_timestamp_triggers_real_refresh(server, monkeypatch, tmp_path):
+    old = datetime.now(timezone.utc) - timedelta(hours=7)
+    cache_path = tmp_path / "cache.json"
+    cache_path.write_text(json.dumps({
+        "puell_multiple": {"value": 1.2},
+        "_onchain_timestamp": old.isoformat(),
+    }))
+    monkeypatch.setattr(server, "CACHE_PATH", str(cache_path))
+    monkeypatch.setattr(server, "HISTORY_PATH", str(tmp_path / "scores.jsonl"))
+    monkeypatch.setattr(server.fear_greed, "fetch", lambda: {"value": 25})
+    monkeypatch.setattr(server.price, "fetch_current", lambda: {"price": 65000})
+    monkeypatch.setattr(server.price, "fetch_ath", lambda: {"ath": 70000})
+    monkeypatch.setattr(server.price, "fetch_historical", lambda: [])
+    monkeypatch.setattr(server.engine, "score_all", lambda metrics: {"composite_score": 50, "scored_count": 1, "metrics": []})
+    monkeypatch.setattr(server.engine, "score_all_ml", lambda metrics: {"composite_score": 50, "scored_count": 1, "metrics": []})
+
+    calls = []
+    fake_lib = types.ModuleType("scrapers.lookintobitcoin")
+    fake_lib.scrape_all = lambda: calls.append("lib") or {"puell_multiple": {"value": 0.8}}
+    fake_coc = types.ModuleType("scrapers.checkonchain")
+    fake_coc.scrape_all = lambda: calls.append("coc") or {"sopr": {"value": 0.99}}
+    fake_updater = types.ModuleType("scrapers.history_updater")
+    fake_updater.update_history = lambda: None
+    import scrapers
+    monkeypatch.setattr(scrapers, "lookintobitcoin", fake_lib, raising=False)
+    monkeypatch.setattr(scrapers, "checkonchain", fake_coc, raising=False)
+    monkeypatch.setitem(sys.modules, "scrapers.lookintobitcoin", fake_lib)
+    monkeypatch.setitem(sys.modules, "scrapers.checkonchain", fake_coc)
+    monkeypatch.setitem(sys.modules, "scrapers.history_updater", fake_updater)
+
+    server.run_scrape()
+
+    assert calls == ["lib", "coc"]
+    saved = json.loads(cache_path.read_text())
+    assert saved["puell_multiple"]["value"] == 0.8
+    assert saved["puell_multiple"]["source"] == "lookintobitcoin"
+    assert saved["sopr"]["source"] == "checkonchain"
+    assert saved["_onchain_timestamp"] != old.isoformat()
+
+
+def test_refresh_job_is_reserved_before_thread_start(server, monkeypatch, tmp_path):
+    from dashboard.jobs import JobRegistry
+
+    registry = JobRegistry(tmp_path / "jobs.json")
+    monkeypatch.setattr(server, "_jobs", registry, raising=False)
+    monkeypatch.setattr(server, "_scraper_running", False)
+
+    started = server.api_refresh(full=False)
+    duplicate = server.api_refresh(full=False)
+
+    assert started["job_id"]
+    assert started["status"] == "queued"
+    assert registry.get(started["job_id"])["status"] == "queued"
+    assert duplicate.status_code == 409
+
+
+def test_history_collection_has_job_id_and_job_scoped_progress(server, monkeypatch, tmp_path):
+    from dashboard.jobs import JobRegistry
+
+    registry = JobRegistry(tmp_path / "jobs.json")
+    monkeypatch.setattr(server, "_jobs", registry, raising=False)
+
+    started = server.api_backtest_collect()
+    job = registry.get(started["job_id"])
+
+    assert started["status"] == "queued"
+    assert job["kind"] == "history"
+    assert job["progress"] == {"status": "starting", "current": "", "step": 0, "total": 0}
+    assert server.api_job_status(started["job_id"])["id"] == started["job_id"]