Skip to content

fix(train): disable persistent workers when no workers are used - #10123

Merged
hjh0119 merged 1 commit into
modelscope:mainfrom
Excelius-Wang:fix/zero-worker-persistence
Sep 17, 2026
Merged

hjh0119 merged 1 commit into
modelscope:mainfrom
Excelius-Wang:fix/zero-worker-persistence

Conversation

@Excelius-Wang

Copy link
Copy Markdown
Contributor

Since #10036 enabled persistent DataLoader workers by default, setting --dataloader_num_workers 0 raises ValueError: persistent_workers option needs num_workers > 0 when the training loader is created. This also affects the Windows default, which selects zero workers.

Disable persistence during training-argument initialization when the resolved worker count is zero. Positive worker counts retain the new default and the existing explicit opt-out. Update the argument documentation in both languages to describe the effective default.

Validation:

  • Six CPU unit tests cover real HF map/iterable datasets, training/eval/prediction loaders, both training-argument classes, simulated Windows defaults, explicit persistence settings, and existing invalid-prefetch validation. Nine subcases fail on unpatched main and pass with the fix.
  • Ran the actual SwiftSft pipeline with a local tiny Qwen2 model on public SQuAD, CMRC2018 and HumanEval samples, using both regular and streaming loading. All six zero-worker configurations reproduce the startup failure on main and complete training/evaluation with the fix. Losses and final model weights exactly match main with persistence explicitly disabled.
  • Compared main and the patch with 1/2 workers and persistence enabled/disabled; losses and final weights match. Pre-commit passed for all four changed files.

Testing used Linux CPU and randomly initialized model weights; native Windows, GPU/distributed training and full RLHF runs were not tested.

Signed-off-by: zhao.wang <57819425+Excelius-Wang@users.noreply.github.com>
@hjh0119
hjh0119 merged commit 55d6157 into modelscope:main Sep 17, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants