fix(train): initialize GaLore through split optimizer hooks

[MrCapricornLiu]

PR type

• Bug Fix
• New Feature
• Document Updates
• More Models or Datasets Support

PR information

With Transformers 5.3, --use_galore true constructs GaloreOptimizerCallback, but the trainer calls the separate optimizer and scheduler hooks. GaLore only overrides the combined hook, so training silently uses ordinary AdamW without any GaLore parameter groups.

Implement the separate hooks and have the combined hook reuse them. Optimizer construction retains the existing parameter groups and projection settings. Scheduler construction uses the trainer-provided step count and get_warmup_steps, including ratio-based warmup. Per-parameter schedulers now advance on the same training-step scale as their wrapper, rather than a doubled schedule.

Experiment results

• python -m unittest discover -s tests -p test_galore_schedule.py -v: both tests pass, covering optimizer selection and eight learning-rate scenarios. The original code fails the optimizer-type check and seven schedule scenarios. The schedule tests perform actual forward/backward and GaLore optimizer steps.
• A real Swift SFT run with a tiny Qwen3 initialized from config completed five FP32 steps on one CUDA device. The probe verifies the underlying optimizer is GaLoreAdamW and every parameter group's learning rate follows the expected warmup/decay. The same probe on the original code detects ordinary AdamW.
• Pre-commit checks passed.

Validated with PyTorch 2.12.0, Transformers 5.3.0, and Accelerate 1.15.0. Quantized GaLore and distributed training were not tested.