Skip to content

core: support conditional layers in parallelism. - #15246

Open
sayakpaul wants to merge 3 commits into
mainfrom
conditional-layers-parallelism
Open

sayakpaul wants to merge 3 commits into
mainfrom
conditional-layers-parallelism

Conversation

@sayakpaul

@sayakpaul sayakpaul commented Nov 11, 2026 •

Copy link
Copy Markdown
Member

Fixes #15218.

Instead of Optional, I opted for Conditional to avoid conflicts with Python typing.

Decided to test it in a standalone manner rather than generalizing through tests/models/testing_utils/parallelism.py to keep things simple.

I have run the entire TP and CP test suite and they pass as expected.

@sayakpaul sayakpaul added this to the Release 0.44.0 milestone Nov 11, 2026
@sayakpaul sayakpaul changed the title core: support conditional layers in l layers in parallelism. core: support conditional layers in parallelism. Nov 11, 2026
parallel_config
if isinstance(parallel_config, TensorParallelConfig)
else parallel_config.tensor_parallel_config
else getattr(parallel_config, "tensor_parallel_config", None)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To support passing bare ContextParallelConfig to from_pretrained() just like TensorParallelConfig.

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@sayakpaul

Copy link
Copy Markdown
Member Author

@yzhautouskay could you test the Cosmos PR with this one instead of the _get_tp_plan() approach?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

_tp_plan / _cp_plan: allow entries for modules that are not always present

2 participants