Skip to content

feat: add Lambda Cloud GPU provider - #153

Open
googs1025 wants to merge 4 commits into
InftyAI:mainfrom
googs1025:feat/lambda-cloud-provider
Open

googs1025 wants to merge 4 commits into
InftyAI:mainfrom
googs1025:feat/lambda-cloud-provider

Conversation

@googs1025

Copy link
Copy Markdown
Member

What this PR does / why we need it

Adds an opt-in lambda provider for Lambda Cloud On-Demand GPU instances. Nebula launches one VM per workload, starts a public container with cloud-init, and reports readiness after the container responds to /healthz. The initial catalog includes one A10 SKU.

Records launch attempts on NodeClaims so a lost API response can be recovered without starting a second instance. The adapter confirms termination before the NodeClaim finalizer releases.

Which issue(s) this PR fixes

Fixes #152

Special notes for your reviewer

  • make test, race tests for the Lambda adapter and virtual kubelet, and lint passed.
  • On a K3s cluster, a real Lambda A10 instance in us-west-1 ran the sample container and returned its GPU model from /healthz. The manager adopted the same VM after a restart. Deleting the smoke-test Pod removed its NodeClaim, and the Lambda API confirmed termination.
  • A separate one-replica Deployment reached 1/1 Ready on a real A10 instance.
  • kubectl logs and kubectl exec are deferred for the Lambda provider.

Does this PR introduce a user-facing change?

Nebula can run single-container On-Demand GPU workloads on Lambda Cloud using the opt-in `lambda` provider. Configure the Lambda credentials and workspace settings in `docs/deploy.md` to enable it.

@InftyAI-Agent InftyAI-Agent added needs-triage Indicates an issue or PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. do-not-merge/needs-kind Indicates a PR lacks a label and requires one. labels Nov 20, 2026
@googs1025
googs1025 marked this pull request as ready for review November 20, 2026 16:07
Copilot AI balanced review requested due to automatic review settings November 20, 2026 16:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Lambda workload restrictions prevent failover to compatible providers, and filtering terminated VM records can leave Pods Pending indefinitely after a lost launch response.

2 open issues
What changed in this PR

Adds an opt-in Lambda Cloud provider so Nebula can schedule single-container GPU workloads on On-Demand VMs, initially supporting one A10 SKU.

Changes:

  • Implements VM provisioning, cloud-init container startup, /healthz readiness checks, and confirmed termination.
  • Records launch attempts on NodeClaims and handles unresolved launches without immediately failing the Pod.
  • Adds provider registration, deployment configuration, samples, documentation, and tests.
File Description
pkg/​vnode/​handler.go Tracks unresolved launches and adopts recovered instances.
pkg/​vnode/​handler_test.go Tests unresolved launch recovery and teardown.
pkg/​provider/​lambda/​lambda.go Implements the Lambda provider lifecycle and readiness checks.
pkg/​provider/​lambda/​lambda_test.go Tests recovery, ownership, readiness, and concurrent provisioning.
pkg/​provider/​lambda/​client.go Adds the authenticated, rate-limited Lambda API client.
pkg/​provider/​lambda/​client_test.go Tests API responses, pagination failures, and error handling.
pkg/​provider/​lambda/​bootstrap.go Validates workloads and generates container startup scripts.
pkg/​provider/​lambda/​attempt.go Persists launch attempts and verifies instance ownership.
pkg/​provider/​errors.go Defines the unresolved-provision error.
pkg/​provider/​catalog/​data/​lambda.csv Adds the initial A10 offering.
hack/​deploy.sh Adds Lambda credential Secret creation.
docs/​deploy.md Documents configuration, supported workloads, and recovery.
config/​samples/​lambda-nodepool.yaml Provides an On-Demand Lambda NodePool example.
config/​samples/​lambda-deployment.yaml Provides a GPU workload with a health endpoint.
config/​rbac/​role.yaml Permits Lambda virtual-node impersonation.
config/​manager/​manager.yaml Loads optional Lambda credentials and configuration.
config/​catalog/​kustomization.yaml Includes the Lambda catalog in deployment configuration.
cmd/​main.go Registers Lambda with a direct API reader.
cmd/​main_test.go Tests explicit Lambda provider selection.
.env.example Adds the Lambda API key placeholder.

Review effort: Balanced

Comment thread pkg/provider/lambda/lambda.go Resolved
Comment thread pkg/provider/lambda/lambda.go Resolved
Copilot AI balanced review requested due to automatic review settings November 20, 2026 17:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

@googs1025

Copy link
Copy Markdown
Member Author

@codex review this pr

@googs1025

Copy link
Copy Markdown
Member Author

/kind feature

@InftyAI-Agent InftyAI-Agent added feature Categorizes issue or PR as related to a new feature. and removed do-not-merge/needs-kind Indicates a PR lacks a label and requires one. labels Nov 21, 2026
@kerthcet
kerthcet requested a balanced review from Copilot November 21, 2026 08:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Cold Pod adoption repeatedly probes the entire VM fleet, causing N² health requests after a manager restart and allowing slow probes to delay unrelated Pod adoption.

No open issues 🎉

1 previously missed issue

In code that hasn't changed since last review

Medium severity Avoid fleet-wide health probes on each cold pod adoption

pkg/​provider/​lambda/​lambda.go:508

Each cold GetPod calls instanceByClaim in pkg/vnode/handler.go, which invokes List for the entire provider. This loop then probes every active VM, so adopting N Pods after a manager restart performs N² /healthz requests rather than N; slow probes also delay adoption of unrelated Pods. Please use a claim-scoped, non-probing lookup for cold adoption and reserve fleet-wide health checks for the poll loop, or share a safely coalesced observation snapshot across these calls.

2 resolved since last review

Review effort: Balanced

@kerthcet

Copy link
Copy Markdown
Member

@googs1025 you should be able to ask copilot to review now, if not, let me know.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Active polling races can apply stale observations to newer Pod entries or drop previously observed VMs, causing Pods to fail incorrectly or remain Pending indefinitely.

2 open issues

Review effort: Balanced

Comment thread pkg/vnode/handler.go Resolved
Comment thread pkg/provider/lambda/lambda.go Resolved
Copilot AI balanced review requested due to automatic review settings November 22, 2026 08:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because omitted findings identify risks of paid VM bootstrap failures and silently weakened workload isolation.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support Lambda Cloud as a GPU provider

4 participants