🍭[Roadmap] ms-swift 4.2-4.4

[Jintao-Huang]

中文版

重构

1. dataset模块：✅样本级别控制 enable_thinking, max_pixels等；streaming训练速度优化。
2. 模型模块：transformers 分类/embedding/reranker任务lm_head部分逻辑统一，方便维护。
3. infer_engine模块，数据预处理异步处理；异常处理优化。
4. 序列并行 (ulysses/ring attention) 模块重构。
5. Web-UI模块重构。

Megatron-SWIFT

1. 新模型：✅deepseek-v4, ✅gemma4, mimo-v2.5, OLMoE2, ✅ring2.5, ✅qwen3_asr
2. 多模态部分的并行支持
3. ✅GLM-5 共享MTP支持
4. ✅Qwen3.5 FP8训练
5. ✅自定义Megatron模型文档
6. FP4支持
7. megatron-fsdp 支持
8. ✅镜像使用cuda13，flash-attn-4

RL

1. ✅RAY & Megatron (GRPO/GKD)
2. ✅Megatron GRPO/GKD 权重同步 支持仅同步LoRA
3. ✅Megatron 多轮RL
4. ✅GKD/OPSD teacher_server_api 兼容多模态训练
5. ✅GKD/OPSD buffer与采样速度优化
6. vLLM原生权重同步接口支持
7. ✅新算法支持：FIPO
8. ✅distillation RL (KL advantages)
9. fully async rollout

训练

1. ✅新模型 day0支持
2. ✅Qwen3-TTS支持训练；Omni talker的训练
3. ✅Qwen3-Embedding MRL 支持
4. ✅ulysses/ring attention支持Qwen3.5
5. 训练中 验证/评测 支持vllm推理

全链路

1. sglang 多模态模型
2. vllm-omni

English Version

Refactoring

1. Dataset module: ✅Sample-level control for enable_thinking, max_pixels, etc.; streaming training speed optimization.
2. Model module: Unified lm_head logic for transformers classification/embedding/reranker tasks, improving maintainability.
3. infer_engine module: Asynchronous data preprocessing; optimized exception handling.
4. Sequence Parallelism (Ulysses / Ring Attention) module refactored.
5. Web-UI module refactored.

Megatron-SWIFT

1. New models: ✅deepseek-v4, ✅gemma4, mimo-v2.5, OLMoE2, ✅ring2.5, ✅qwen3_asr
2. Parallel support for multimodal components.
3. ✅GLM-5 shared MTP support.
4. ✅Qwen3.5 FP8 training.
5. ✅Custom Megatron model documentation.
6. FP4 support.
7. megatron-fsdp support.
8. ✅Docker image upgraded to CUDA 13 with flash-attn-4.

RL

1. ✅RAY & Megatron (GRPO/GKD).
2. ✅Megatron GRPO/GKD weight synchronization now supports syncing LoRA weights only.
3. ✅Megatron multi-turn RL.
4. ✅GKD/OPSD teacher_server_api is now compatible with multimodal training.
5. ✅GKD/OPSD buffer and sampling speed optimization.
6. vLLM native weight synchronization interface support.
7. ✅New algorithm support: FIPO.
8. ✅distillation RL (KL advantages)
9. fully async rollout

Training

1. ✅Day-0 support for new models.
2. ✅Qwen3-TTS training support; Omni Talker training support.
3. ✅Qwen3-Embedding MRL support.
4. ✅Ulysses/Ring Attention support for Qwen3.5.
5. Validation/evaluation during training with vLLM inference support.

Full Pipeline

1. SGLang multimodal model support.
2. vllm-omni support.

[li624120638]

会支持eagle3的训练吗？

[Cglasses]

Upcoming talker training support for Omni is highly anticipated. When will this feature be officially launched?

[randydl]

可以考虑使用Megatron-Energon改善大规模数据加载效率

[ilc123]

您好，请问一下后续大概什么时候能支持DeepSeek v4的FP4

[towanma]

Qwen3-TTS支持训练还在计划中吗？

[gusario]

SGLang as engine for RL

[hjh0119]

@gusario

Before we evaluate the priority of SGLang support,
could you help us with a few questions?

1. What is your use case for using SGLang as the rollout engine?

2. Do you have any experimental results comparing SGLang and vLLM?

This will help us better assess the priority of supporting SGLang, thanks!

[gusario]

Hello, @hjh0119!
Thanks for the questions, digging into them changed my mind on the request.

1. My use case was faster weight sync/refit for MoE GRPO (Qwen3-32B-A3B, Megatron). I'd assumed SGLang was the reason slime gets fast refits https://hebiao064.github.io/rl-weight-sync (they contribute heavily to SGLang, and ms-swift already supports SGLang but only as an inference engine, so it seemed like an obvious gap). But my assumption that slime with SGLang is the most efficient RL pipeline turned out to be wrong. NeMo-RL gets similar numbers on vLLM NeMo-RL: Journey of Optimizing Weight Transfer in Large MoE Models by 10x NVIDIA-NeMo/RL#1189 (850s -> 51s on DeepSeek-V3)

2. No comparative results and given the above, I don't think either SGLang or vLLM has a significant edge over the other

So honestly, SGLang isn't important to me anymore. The things I've ended up more interested in are both on the vLLM path anyway: whether the refit path can get the slime/NeMo treatment (dropping intermediate-format conversions in the sync), and whether speculative decoding could go into rollout https://research.nvidia.com/labs/nemotron/rl-speculative-decoding/

[hjh0119]

@gusario
Thanks for raising this, I want to clarify that ms-swift already covers most of the performance-oriented optimizations.

1. Streaming weight export (no full-model materialization)

• Megatron: bridge.export_weights() yields an iterator; the Ray rollout path streams tensors directly into bucketed IPC buffers (RolloutAdapter → BucketedWeightSender).
• HF / colocate: --move_model_batches splits layer-wise transfers instead of loading the full model at once — [grpo] support move_model_batches for external mode #4453

2. Bucketed / packed weight transfer (NeMo "tensor packing" equivalent)

• Multiple parameters are packed into a single uint8 bucket buffer; one CUDA IPC handle per sync round, with per-bucket ZMQ metadata + ack — introduced with Megatron-Ray in [1/N] Initial support for Megatron-Ray #9381, building on earlier GRPO work in [grpo] Optimize vLLM weight synchronization & update buitin accuracy reward #5773

3. CUDA IPC zero-copy (colocate / hybrid)

• Trainer → vLLM weight path via WeightSyncWorkerExtension.update_weights_from_ipc + ZMQ REQ/REP, with correct handshake ordering (RPC kicked before handshake to avoid deadlock) — [1/N] Initial support for Megatron-Ray #9381.

4. Cross-step IPC buffer reuse

• Persistent sender buffer (RolloutAdapter._ipc_buffer) + receiver-side IPC mapping cache (_swift_ipc_buffer / handle signature) to skip repeated rebuild_cuda_tensor and reduce IPC mapping leaks — [1/N] Initial support for Megatron-Ray #9381, [2/N] support megatron-ray gkd #9471.

5. LoRA-only sync

• Adapter-only weight sync via peft_format=True + TensorLoRARequest — Optimize weight synchronization for LoRA adapter weights #9077.

We are also tracking ongoing improvements in this area from other frameworks. If you have any specific feature requests or would like to contribute, we'd love to hear from you!