Skip to content

Reports should include data windows and sources used by each analyst #1197

Description

@kagura-agent

Problem

When a run completes, the generated report (complete_report.md and per-section markdowns) contains only the ticker and a generation timestamp. There is no record of:

  • What date range each analyst actually queried for stock data (OHLCV)
  • How many articles the news/sentiment analysts retrieved, and from what date range
  • Which technical indicators were computed and over what lookback period
  • What fundamental data (quarterly/annual, which periods) was used

This makes it impossible to audit or reproduce a decision. Two runs for the same ticker + date can silently use different data windows (since the LLM agents decide the start_date/end_date parameters dynamically), yet the reports look identical in structure.

Why it matters

  • Reproducibility: Without knowing the exact data window, you cannot reproduce the analysis
  • Trust: A "Hold" recommendation means very different things if based on 30 days vs 1 year of price history
  • Debugging: When a decision looks wrong, the first question is "what data did it see?" — currently unanswerable from the report alone
  • Backtesting: The planned backtesting framework (issue Enhancing TradingAgents with RL for Event-Driven Stock Price Prediction #1) will need to track which data windows led to which decisions to evaluate strategy quality

Proposed solution

Add a "Data Sources" section to each analyst report (and the consolidated report) that records:

### Data Sources
- **Price data**: AAPL, 2024-07-15 to 2025-01-15 (6 months, 126 trading days)
- **Technical indicators**: SMA(20, 50, 200), RSI(14), MACD(12,26,9)
- **News articles**: 18 articles, 2025-01-08 to 2025-01-15
- **Global news**: 10 articles, lookback 7 days
- **Fundamentals**: Q3 2024 earnings, annual 2023 balance sheet

Implementation options

  1. Tool-call logging: Capture the arguments passed to each @tool function during a run and attach them to the state
  2. Post-hoc extraction: Have each analyst include a structured "sources used" block in their output (prompt-level change)
  3. Hybrid: Log tool calls at the framework level AND ask agents to summarize what they used

Option 1 (tool-call logging) is the most reliable since it does not depend on the LLM accurately self-reporting.

Current behavior

reporting.py writes final_state["market_report"] etc. as raw text with no metadata. The complete_report.md header is just:

header = f"# Trading Analysis Report: {ticker}\n\nGenerated: {datetime.now()...}\n\n"

No data provenance information is preserved.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions