Skip to content

feat: skip a provider whose instance can't hold the Pod's ephemeral-storage - #147

Merged
InftyAI-Agent merged 3 commits into
InftyAI:mainfrom
kerthcet:feat/provider-max-disk
Nov 18, 2026
Merged

InftyAI-Agent merged 3 commits into
InftyAI:mainfrom
kerthcet:feat/provider-max-disk

Conversation

@kerthcet

@kerthcet kerthcet commented Nov 18, 2026 •

Copy link
Copy Markdown
Member

Problem

A Modal sandbox can't request disk. It gets a fixed per-container quota of 512 GiB: Modal's Python client sets ephemeral_disk = None for sandboxes, and the Go SDK's SandboxCreateParams has no disk field, though the backend protocol has ephemeral_disk_mb.

A Pod asking for more ephemeral-storage reached Modal and failed at Provision with modal: N GiB disk exceeds the 512 GiB Modal provides. That error matches no failover category, so it blocked nothing. A controller's replacement Pod went back to Modal and failed again, and never tried a provider that could serve it, such as AWS, which sizes gp3 to the request up to 16354 GiB. AWS had the same gap: it refuses disk past awsMaxDiskGiB (gp3's 16 TiB less the 30 GiB AMI root) at launch, with the same churn.

Change

  • provider.Capabilities.MaxDiskGiBPerPod is the most local disk the provider can give one Pod; each Pod gets its own, and 0 means no known limit. ServesDisk sits next to ServesCapacityTier and ServesEgress. Modal sets 512 and AWS sets awsMaxDiskGiB (16354), each matching the line its launch already refuses at. RunPod and Kubernetes keep 0: neither refuses disk today.
  • Placement reads the Pod's ephemeral-storage (first container's limit, else its request, rounded up to GiB, as before) and skips a provider past its limit. The skip is recorded as disk_unsupported. The Pod lands on the pool's next provider, or stays gated as no_candidate if none fits.
  • The virtual node advertises MaxDiskGiBPerPod per Pod, summed over its 1k Pods: 500 TiB on nebula-modal, 16Pi elsewhere as before. Node capacity is a total the scheduler checks across all bound Pods, so a node can't cap one Pod's disk; a flat 512Gi would admit a single 512 GiB Pod at a time. Placement enforces the per-Pod limit.
  • Modal's refusal in sandboxSpecFromPod stays as a backstop for a Pod placed by an older manager.
  • Docs: disk_unsupported is added to the skip reasons in docs/metrics.md.

Units: the limit is 512 GiB. 512Gi stays on Modal; 513Gi, and decimal 550G (~512.2 GiB, rounded up to 513), skip it.

Testing

  • TestPlacement_DiskPastAProvidersMaxSkipsIt: 512Gi stays on Modal; 513Gi and 550G fall through to the next provider; 513Gi with no other provider stays gated. With ServesDisk forced to true, the three over-limit cases fail.
  • TestPlacement_GPUPodPastAWSDiskSkipsIt: a GPU Pod asking 16Ti skips AWS for the next provider, or stays gated.
  • TestServesDisk, TestVirtualCapacity_DiskFollowsMaxDiskGiBPerPod, and TestDiskLimit_PlacementAndRefusalAgree for Modal and AWS (each one's advertised limit equals the line its launch refuses at).
  • go test ./... (non-e2e) and make lint pass.

🤖 Generated with Claude Code

…torage

A Modal sandbox can't request disk: it gets a fixed 512 GiB quota (the Python
client drops ephemeral_disk for sandboxes, and the Go SDK has no field). A Pod
asking more used to reach Modal and fail at Provision, and as that error blocks
nothing, its controller's replacement went back to Modal and failed again,
never trying a provider that could serve it.

Capabilities.MaxDiskGiB (0 = no known limit) declares the most disk one
instance can have, and placement skips a provider past it, recording
disk_unsupported, so the Pod lands on the pool's next provider or stays gated
as no_candidate. Modal sets 512; the others keep 0. Modal's own refusal stays,
for a Pod placed by an older manager.

The virtual node advertises MaxDiskGiB per Pod, summed over its 1k Pods
(500 TiB on nebula-modal): the scheduler checks the node's total, so a node
cannot cap one Pod's disk, and a flat 512Gi would admit a single such Pod.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 18, 2026 12:03
@InftyAI-Agent InftyAI-Agent added needs-triage Indicates an issue or PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. do-not-merge/needs-kind Indicates a PR lacks a label and requires one. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Nov 18, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the change to filter providers by Pod ephemeral-storage requirements.

No open issues 🎉

What changed in this PR

This PR makes placement skip providers that cannot satisfy a Pod’s ephemeral-storage needs, preventing repeated provisioning failures on Modal and allowing fallback to another provider.

Changes:

  • Adds MaxDiskGiB and ServesDisk, with Modal advertising its fixed 512 GiB limit.
  • Filters placement candidates by disk capacity and adjusts virtual-node storage totals.
  • Adds boundary and fallback tests, plus a documented disk_unsupported metric reason.
File Description
pkg/​vnode/​node.go Derives advertised storage from the provider’s per-instance limit and virtual Pod capacity.
pkg/​vnode/​node_test.go Tests limited and unlimited virtual-node storage totals.
pkg/​provider/​provider.go Defines the disk capability and eligibility check.
pkg/​provider/​modal/​modal.go Advertises Modal’s 512 GiB limit and explains the existing provisioning backstop.
pkg/​provider/​modal/​modal_test.go Checks agreement between the advertised limit and provisioning refusal.
pkg/​provider/​capabilities_test.go Tests disk eligibility boundaries and unknown limits.
pkg/​metrics/​placement.go Defines the disk_unsupported skip reason.
internal/​controller/​pod_placement_helpers.go Skips providers whose disk limit is below the Pod’s needs.
internal/​controller/​pod_placement_controller_test.go Tests boundary values, decimal-unit rounding, fallback, and retained gating.
internal/​controller/​nodeclaim_controller_test.go Adds configurable disk limits to the shared fake provider.
docs/​metrics.md Documents the new disk-related skip reason.

Review effort: Balanced

It is checked against one Pod's ephemeral-storage, and each Pod gets its own
instance, so two Pods at the limit both fit; the old name read as a total.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 18, 2026 12:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

AWS refuses ephemeral-storage past awsMaxDiskGiB (gp3's 16 TiB less the 30 GiB
AMI root, 16354 GiB) at launch, but left MaxDiskGiBPerPod at zero, so placement
read it as unlimited: a GPU Pod asking 16Ti was ungated onto AWS and failed
there, and as that error blocks nothing, its replacement did the same. AWS now
declares the limit, so such a Pod skips AWS for the next provider or stays
gated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 18, 2026 14:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

@kerthcet

Copy link
Copy Markdown
Member Author

/lgtm
/approve
/kind feature

@InftyAI-Agent InftyAI-Agent added lgtm Looks good to me, indicates that a PR is ready to be merged. feature Categorizes issue or PR as related to a new feature. and removed do-not-merge/needs-kind Indicates a PR lacks a label and requires one. labels Nov 18, 2026

@InftyAI-Agent InftyAI-Agent left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved: PR has both lgtm and approved labels

@InftyAI-Agent InftyAI-Agent left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved: PR has both lgtm and approved labels

@InftyAI-Agent
InftyAI-Agent merged commit 1996d99 into InftyAI:main Nov 18, 2026
23 of 25 checks passed
@kerthcet
kerthcet deleted the feat/provider-max-disk branch November 18, 2026 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. feature Categorizes issue or PR as related to a new feature. lgtm Looks good to me, indicates that a PR is ready to be merged. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants