Skip to content

fix(reward): use normalized ToolBench ROUGE thresholds - #10148

Merged
hjh0119 merged 1 commit into
modelscope:mainfrom
MrCapricornLiu:lch/fix-toolbench-rouge-scale
Sep 17, 2026
Merged

hjh0119 merged 1 commit into
modelscope:mainfrom
MrCapricornLiu:lch/fix-toolbench-rouge-scale

Conversation

@MrCapricornLiu

Copy link
Copy Markdown
Contributor

PR type

  • Bug Fix
  • New Feature
  • Document Updates
  • More Models or Datasets Support

PR information

The ToolBench reward returns zero even for identical plain-text action inputs. rouge supplies an F1 score in [0, 1], but ReactORM compares it against 10 and 20, so this branch always assigns zero.

Use 0.1 and 0.2 for the existing thresholds. The binary reward policy, JSON argument comparison, and tool-name checks remain unchanged.

For example, two identical Action: search\nAction Input: New York weather completions now receive 1 instead of 0; unrelated text and a different tool still receive 0.

Experiment results

  • Three unittest methods cover real ROUGE scoring, string and message-dict inputs, threshold boundaries, JSON arguments, mismatched tools, and invalid inputs. The original code has four failing assertions; all tests pass with the fix.
  • The real GRPO score_completions path produces [1, 1, 0, 0] for exact/overlapping/unrelated/wrong-tool completions, versus [0, 0, 0, 0] before the fix (rouge 1.0.1).
  • All pre-commit hooks pass. This validates reward computation, not a full training run.
python -m unittest discover -s tests -p test_toolbench_reward.py -v

Signed-off-by: Chenghao Liu <chliu@stu.pku.edu.cn>
@hjh0119
hjh0119 merged commit 0bdfcd8 into modelscope:main Sep 17, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants