pivot: rewrite as BTC accumulation signal optimizer

Replace day-trading bot with long-term accumulation signal model.
Predicts optimal BUY times using forward return analysis at 7d/30d/90d
horizons, scoring each candle 0-100. Primary metric is now
cost_basis_improvement_pct (model buy price vs DCA).

- train_and_backtest.py: regression models (XGBoost/LSTM hybrid),
  accumulation-focused features (price position, momentum, volatility,
  volume, cycle), forward return targets, signal quality backtesting
- orchestrator.py: cost improvement scoring, signal count validation
- analyzer.py: accumulation-focused LLM system prompt
- dashboard: cost improvement display, signal metrics table
- config: new accumulation-focused parameters

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

llm_client/analyzer.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python3
 """
-LLM Strategy Analyzer -- Calls Ollama on Mac Mini to analyze results
+LLM Accumulation Signal Analyzer -- Calls Ollama on Mac Mini to analyze results
 and suggest config modifications for the next iteration.
 """
 
@@ -11,121 +11,130 @@ import requests
 OLLAMA_URL = "http://100.100.242.21:11434"
 MODEL = "qwen3.5:27b"
 
-SYSTEM_PROMPT = """You are a quantitative trading strategy optimizer. You analyze ML model backtesting results for a BTC/USDT trading strategy and suggest precise modifications to improve performance.
+SYSTEM_PROMPT = """You are a quantitative analyst optimizing a BTC ACCUMULATION SIGNAL model. The goal is NOT day-trading -- it is finding statistically optimal times to BUY BTC for long-term holding.
 
-## Your Task
-Given the current configuration and results, suggest 1-3 specific, justified changes to the configuration for the next iteration. Be methodical and scientific -- change one thing at a time when possible.
+## Core Question
+"Given current market conditions, is NOW a good time to BUY BTC for long-term holding?"
+
+## What the Model Does
+For each candle, the model predicts an Accumulation Score (0-100):
+- 90-100: STRONG BUY -- historically rare, excellent entry point
+- 70-89: GOOD BUY -- better than average entry
+- 50-69: NEUTRAL -- average time to buy
+- 30-49: WAIT -- price likely to come down
+- 0-29: POOR -- historically bad time to buy (near local tops)
+
+The model is trained on ACTUAL forward returns at 7d, 30d, and 90d horizons, weighted 20/30/50. Times when buying led to the best long-term returns get the highest scores.
+
+## Primary Metric: cost_basis_improvement_pct
+This measures how much better the model's average buy price is vs uniform DCA.
+- 10%+ = good
+- 15%+ = excellent
+- 20%+ = exceptional
+Also require strong_buy_signal_count >= 30 for statistical validity.
 
 ## Config Parameters You Can Modify
 
-**model_type**: "xgboost", "lightgbm", "catboost", "ensemble", "lstm", or "hybrid"
-  - xgboost: Generally best for structured data, fast GPU training
-  - lightgbm: Faster training, good with large feature sets
-  - catboost: Handles feature interactions well, less tuning needed
-  - ensemble: Combines xgboost+lightgbm+catboost, reduces variance but slower
-  - lstm: PyTorch LSTM neural network, captures temporal/sequential patterns in price data
-  - hybrid: Combines LSTM (60% weight) + XGBoost (40% weight). Only enters trades when BOTH models agree on direction. The hybrid model typically outperforms single models -- LSTM captures temporal patterns while XGBoost handles feature interactions. Recommended as default.
+**model_type**: "xgboost", "lightgbm", "catboost", "lstm", or "hybrid"
+  - hybrid: Average of LSTM + XGBoost regression predictions. Recommended default.
+  - xgboost: Fast GPU training, good for structured features.
+  - lstm: Captures temporal patterns in price sequences.
 
 **hyperparameters** (gradient boosting):
-  - learning_rate (0.001-0.3): Lower = more robust but slower. If overfitting, decrease.
-  - max_depth (3-10): Controls model complexity. Deeper = more overfitting risk.
-  - n_estimators (100-2000): More trees = better fit but diminishing returns.
-  - subsample (0.5-1.0): Row sampling. Lower = more regularization.
-  - colsample_bytree (0.5-1.0): Feature sampling per tree. Lower = more diversity.
-  - min_child_weight (1-20): Higher = more conservative splits.
-  - gamma (0-5): Minimum loss reduction for split. Higher = more pruning.
-  - reg_alpha (0-10): L1 regularization. Encourages sparsity.
-  - reg_lambda (0-10): L2 regularization. Prevents large weights.
+  - learning_rate (0.001-0.1): Lower = more robust. Start conservative.
+  - max_depth (3-8): Controls complexity. Deeper risks overfitting.
+  - n_estimators (200-1500): More trees = better fit but diminishing returns.
+  - subsample (0.5-1.0): Row sampling for regularization.
+  - colsample_bytree (0.5-1.0): Feature sampling per tree.
+  - min_child_weight (5-30): Higher = more conservative (important for noisy targets).
+  - gamma (0-5): Minimum loss reduction for split.
+  - reg_alpha (0-10): L1 regularization.
+  - reg_lambda (1-10): L2 regularization. Higher values prevent overfitting.
 
-**hyperparameters** (LSTM-specific, used by lstm and hybrid model_types):
-  - lstm_hidden_size (32-256): LSTM hidden units. Larger = more capacity but overfitting risk. Default 128.
-  - lstm_num_layers (1-4): Stacked LSTM layers. 2 is usually optimal. More layers need more data.
-  - lstm_dropout (0.1-0.5): Dropout between LSTM layers and before output. Higher = more regularization.
-  - lstm_epochs (50-200): Max training epochs. Early stopping usually triggers before this.
-  - lstm_batch_size (32-128): Training batch size. Smaller = noisier gradients but better generalization.
-  - lstm_sequence_length (10-50): How many past candles the LSTM sees per prediction. Longer = more context but more memory. Default 20.
-  - lstm_patience (5-20): Early stopping patience on validation loss. Lower = stop sooner.
+**hyperparameters** (LSTM):
+  - lstm_hidden_size (32-256): Hidden units.
+  - lstm_num_layers (1-4): Stacked layers. 2 is usually optimal.
+  - lstm_dropout (0.1-0.5): Regularization.
+  - lstm_epochs (50-200): Max training epochs (early stopping usually triggers).
+  - lstm_batch_size (32-128): Smaller = noisier but better generalization.
+  - lstm_sequence_length (15-60): Past candles the LSTM sees. Longer = more context.
+  - lstm_patience (5-20): Early stopping patience.
 
 **target**:
-  - direction: "long", "short", or "both"
-  - horizon_candles (1-20): How far ahead to predict. Longer = smoother but lagging.
-  - threshold_pct (0.3-3.0): Minimum move % to label as positive. Higher = fewer but clearer signals.
+  - forward_periods_4h: List of 3 forward periods in 4h candles [short, medium, long].
+    Defaults: [42, 180, 540] = roughly [7d, 30d, 90d]
+  - weights: Weights for each period. Default [0.2, 0.3, 0.5] (emphasize long-term).
+  - score_range: [0, 100] -- do not change.
 
 **strategy**:
-  - entry_threshold (0.5-0.8): Min prediction probability to enter trade. Higher = fewer trades, higher quality.
-  - stop_loss_pct (0.5-5.0): Max loss before exit (used when dynamic_sl_tp is false).
-  - take_profit_pct (1.0-10.0): Target profit (used when dynamic_sl_tp is false). Should be > stop_loss for positive expectancy.
-  - trailing_stop_pct (0.5-3.0): Trailing stop distance. Tighter = locks profit faster but exits early.
-  - min_confidence_to_trade (0.5-0.9): Absolute minimum confidence to consider.
-  - exit_type: "trailing_stop" or "fixed" (just SL/TP)
-  - dynamic_sl_tp (true/false): Use ATR-based dynamic stop-loss and take-profit instead of fixed percentages. Adapts to current volatility. Recommended: true.
-  - atr_sl_multiplier (1.0-3.0): ATR multiplier for stop-loss. E.g., 1.5 means SL = 1.5 * ATR(14). Lower = tighter stops.
-  - atr_tp_multiplier (2.0-5.0): ATR multiplier for take-profit. E.g., 3.0 means TP = 3.0 * ATR(14). Should be > atr_sl_multiplier.
+  - strong_buy_threshold (70-95): Score above which = STRONG BUY signal. Higher = fewer but better signals.
+  - good_buy_threshold (50-80): Score above which = GOOD BUY. Used for cost basis comparison.
+  - poor_threshold (10-40): Score below which = POOR time to buy.
 
 **features**:
-  - use_volume_features (true/false): Volume features can be noisy in crypto.
-  - use_candle_patterns (true/false): Candle patterns may or may not help.
-  - use_lag_features (true/false): Lagged features capture momentum.
-  - lag_periods: List of lag periods [1,2,3,5,10]
-  - lookback_periods: List of lookback windows [3,5,10,20]
-  - use_scaler (true/false): Apply StandardScaler normalization to all features. Critical for LSTM, also helps gradient boosting. Recommended: true.
-  - use_pca (true/false): Apply PCA dimensionality reduction after scaling. Reduces noise and multicollinearity. Recommended with many features.
-  - pca_variance (0.80-0.99): Fraction of variance to retain with PCA. 0.95 keeps 95% of information. Lower = fewer dimensions, more noise removed.
+  - use_price_position (true/false): Distance from ATH, 52w high/low, percentile.
+  - use_momentum (true/false): RSI, MACD, Stochastic, Williams %R, ROC.
+  - use_volatility (true/false): Bollinger Bands, ATR, consecutive red candles, drawdown.
+  - use_volume (true/false): Volume ratio, OBV, red/green volume ratio.
+  - use_cycle (true/false): MA cross regime, candles since major drawdown.
+  - use_pca (true/false): PCA dimensionality reduction.
+  - pca_variance (0.80-0.99): Variance to retain.
+  - use_scaler (true/false): StandardScaler. Critical for LSTM.
 
 **training**:
-  - walk_forward_windows (3-10): More windows = more robust but less data per window. Used when rolling_window is false.
-  - rolling_window (true/false): Use rolling window instead of static walk-forward splits. Trains on last N candles, tests on next M, slides forward. More realistic for time series. Recommended: true.
-  - rolling_train_size (1000-5000): Number of candles in the rolling training window. Larger = more data but older patterns.
-  - rolling_test_size (100-500): Number of candles in the rolling test window. Smaller = more retraining, better adaptation.
+  - rolling_window (true/false): Rolling vs static walk-forward.
+  - rolling_train_size (1500-5000): Training window candles.
+  - rolling_test_size (100-500): Test window candles.
 
-## Key Metrics to Optimize (in priority order)
-1. **Sharpe Ratio** (target: > 2.0): Risk-adjusted return. Most important metric.
-2. **Profit Factor** (target: > 1.5): Gross profit / gross loss.
-3. **Max Drawdown** (target: > -15%): Worst peak-to-trough decline.
-4. **Win Rate** (target: > 55%): Percentage of winning trades.
-5. **Trade Count**: Need enough trades for statistical significance (>50).
+## Key Metrics to Analyze
+1. **cost_basis_improvement_pct**: PRIMARY metric. How much better is model buy price vs DCA.
+2. **strong_buy_signal_count**: Must be >= 30 for validity. Too few = raise threshold. Too many = lower it.
+3. **signal_frequency_pct**: Should be 5-15%. If outside, adjust thresholds.
+4. **avg_score_at_actual_bottoms**: Should be high (>70). Model should recognize bottoms.
+5. **avg_score_at_actual_tops**: Should be low (<30). Model should avoid tops.
+6. **model_r2_score**: Regression fit quality. > 0.2 is decent for financial data.
+7. **per_window_cost_improvement**: Consistency across windows. Low variance = robust.
 
 ## Decision Guidelines
-- If Sharpe < 1.0: The strategy is not working well. Consider larger changes (switch to hybrid, enable PCA/scaler, adjust target).
-- If Sharpe 1.0-1.5: Decent. Fine-tune hyperparameters and thresholds.
-- If Sharpe 1.5-2.0: Good. Make small, targeted improvements.
-- If Sharpe > 2.0: Very good. Be careful not to overfit.
-- If win_rate < 0.50 but profit_factor > 1.5: Strategy relies on big wins -- ok, tighten SL.
-- If win_rate > 0.60 but profit_factor < 1.2: Many small wins but losses are too big -- widen TP or tighten SL.
-- If trade_count < 30: Not enough trades. Lower entry_threshold or min_confidence.
-- If max_drawdown < -20%: Too risky. Increase regularization, tighten stop loss, enable dynamic_sl_tp.
-- If per_window_sharpe has high variance: Model is not stable. More regularization, enable PCA, or try hybrid.
-- Check feature_importances: If top features make financial sense, good. If random features dominate, possible overfitting -- enable PCA or reduce features.
-- For LSTM/hybrid: if underfitting, increase lstm_hidden_size or lstm_num_layers. If overfitting, increase lstm_dropout or decrease lstm_sequence_length.
-- The hybrid model combining LSTM + XGBoost typically outperforms single models. LSTM captures temporal patterns while XGBoost handles feature interactions. Use hybrid as the default unless you have a specific reason not to.
+- If cost_improvement < 5%: Strategy is barely working. Try: switch model type, enable all features, increase training window, lower good_buy_threshold.
+- If cost_improvement 5-10%: Decent. Fine-tune thresholds and hyperparameters.
+- If cost_improvement 10-15%: Good. Make targeted improvements -- focus on signal consistency.
+- If cost_improvement > 15%: Very good. Be careful not to overfit. Check per_window variance.
+- If signal_count < 30: Not statistically valid. Lower strong_buy_threshold, increase training data.
+- If signal_frequency > 20%: Too many signals = not selective enough. Raise threshold.
+- If signal_frequency < 3%: Too few signals. Lower threshold.
+- If score_at_bottoms < 60: Model is missing bottoms. More features, different model type.
+- If score_at_tops > 40: Model is not avoiding tops. More regularization.
+- If per_window has high variance: Model is unstable. Increase regularization, try hybrid.
+- Check feature_importances: price position features should dominate (distance from ATH, percentile).
 
 ## Response Format
 You MUST respond with ONLY a JSON object (no markdown, no explanation outside the JSON):
 ```
 {
-  "reasoning": "Explanation of what you observed and why you're making these changes",
+  "reasoning": "Explanation of observations and why you are making these changes",
   "changes": ["Change 1 description", "Change 2 description"],
   "config": { <complete modified config JSON> }
 }
 ```
-The "config" field must contain the COMPLETE config (not just changes) so it can be used directly."""
+The "config" field must contain the COMPLETE config so it can be used directly."""
 
 
-def analyze_and_suggest(current_config: dict, results: dict,
-                        iteration_history: list = None) -> tuple[dict, str]:
+def analyze_and_suggest(current_config, results, iteration_history=None):
     """
     Send current results to LLM and get suggested config modifications.
     Returns (new_config, reasoning).
     """
-    # Build the user prompt with context
     history_text = ""
     if iteration_history:
         history_text = "\n## Previous Iterations (most recent last)\n"
         for h in iteration_history[-5:]:
             history_text += (
-                f"- Iteration {h['iteration']}: Sharpe={h['sharpe']}, "
-                f"Return={h['return']}%, WinRate={h['win_rate']}, "
-                f"Trades={h['trades']}, Model={h['model_type']}\n"
+                f"- Iteration {h.get('iteration', '?')}: "
+                f"CostImprovement={h.get('cost_improvement', 0):.1f}%, "
+                f"Signals={h.get('signal_count', 0)}, "
+                f"R2={h.get('r2_score', 0):.4f}, "
+                f"Model={h.get('model_type', '?')}\n"
             )
 
     user_prompt = f"""## Current Configuration
@@ -134,21 +143,24 @@ def analyze_and_suggest(current_config: dict, results: dict,
 ```
 
 ## Current Results
-- Sharpe Ratio: {results.get('sharpe_ratio', 0)}
-- Total Return: {results.get('total_return_pct', 0)}%
-- Max Drawdown: {results.get('max_drawdown_pct', 0)}%
-- Win Rate: {results.get('win_rate', 0)}
-- Trade Count: {results.get('trade_count', 0)}
-- Profit Factor: {results.get('profit_factor', 0)}
-- Avg Trade Duration: {results.get('avg_trade_duration_candles', 0)} candles
-- Per-Window Sharpe: {results.get('per_window_sharpe', [])}
+- Cost Basis Improvement: {results.get('cost_basis_improvement_pct', 0):.1f}%
+- Avg Cost (Model): ${results.get('avg_cost_basis_model', 0):,.2f}
+- Avg Cost (DCA): ${results.get('avg_cost_basis_dca', 0):,.2f}
+- Strong Buy Signals: {results.get('strong_buy_signal_count', 0)}
+- Good Buy Signals: {results.get('good_buy_signal_count', 0)}
+- Signal Frequency: {results.get('signal_frequency_pct', 0):.1f}%
+- Quality of Strong Buys: {results.get('pct_quality_strong_buy', 0):.1%}
+- Model R2: {results.get('model_r2_score', 0):.4f}
+- Score at Actual Bottoms: {results.get('avg_score_at_actual_bottoms', 0):.1f}
+- Score at Actual Tops: {results.get('avg_score_at_actual_tops', 0):.1f}
+- Per-Window Improvement: {results.get('per_window_cost_improvement', [])}
+- Score Distribution: {results.get('score_distribution', {})}
 
 ## Top Feature Importances
 {json.dumps(dict(list(results.get('feature_importances', {}).items())[:15]), indent=2)}
 {history_text}
 Analyze these results and suggest 1-3 specific modifications to the config. Return ONLY valid JSON."""
 
-    # Call Ollama
     payload = {
         "model": MODEL,
         "messages": [
@@ -168,7 +180,6 @@ Analyze these results and suggest 1-3 specific modifications to the config. Retu
     resp.raise_for_status()
     content = resp.json()["message"]["content"]
 
-    # Parse JSON from response (handle markdown code blocks)
     # Strip thinking tags if present
     content = re.sub(r"<think>.*?</think>", "", content, flags=re.DOTALL).strip()
 
@@ -176,8 +187,6 @@ Analyze these results and suggest 1-3 specific modifications to the config. Retu
     if json_match:
         parsed = json.loads(json_match.group(1))
     else:
-        # Try parsing the whole response as JSON
-        # Find the outermost JSON object
         brace_start = content.find("{")
         if brace_start >= 0:
             depth = 0
@@ -198,7 +207,6 @@ Analyze these results and suggest 1-3 specific modifications to the config. Retu
     changes = parsed.get("changes", [])
     new_config = parsed.get("config", current_config)
 
-    # Validate that config has required fields
     required_keys = ["model_type", "features", "target", "hyperparameters", "strategy", "training"]
     for key in required_keys:
         if key not in new_config:
@@ -209,22 +217,25 @@ Analyze these results and suggest 1-3 specific modifications to the config. Retu
 
 
 if __name__ == "__main__":
-    # Test with dummy data
     import sys
     config_path = sys.argv[1] if len(sys.argv) > 1 else "config/initial_config.json"
     with open(config_path) as f:
         config = json.load(f)
 
     dummy_results = {
-        "sharpe_ratio": 1.2,
-        "total_return_pct": 15.3,
-        "max_drawdown_pct": -12.5,
-        "win_rate": 0.55,
-        "trade_count": 120,
-        "profit_factor": 1.4,
-        "avg_trade_duration_candles": 7.2,
-        "feature_importances": {"RSI_14": 0.15, "MACD_hist": 0.12, "BB_width": 0.10},
-        "per_window_sharpe": [1.0, 1.3, 1.5, 0.9, 1.1],
+        "cost_basis_improvement_pct": 8.5,
+        "avg_cost_basis_model": 65000,
+        "avg_cost_basis_dca": 71000,
+        "strong_buy_signal_count": 45,
+        "good_buy_signal_count": 120,
+        "signal_frequency_pct": 7.2,
+        "pct_quality_strong_buy": 0.72,
+        "model_r2_score": 0.22,
+        "avg_score_at_actual_bottoms": 68.5,
+        "avg_score_at_actual_tops": 35.2,
+        "per_window_cost_improvement": [7.1, 9.3, 8.8, 10.2, 7.0],
+        "score_distribution": {"0-20": 80, "20-40": 150, "40-60": 200, "60-80": 130, "80-100": 40},
+        "feature_importances": {"dist_from_ath_pct": 0.18, "RSI_14": 0.12, "price_percentile_365": 0.10},
     }
 
     new_config, reasoning = analyze_and_suggest(config, dummy_results)