Skip to content

Classify SDK tests for Cloud execution - #3033

Draft
THardy98 wants to merge 4 commits into
feat/cloud-test-namespacefrom
feat/cloud-test-filtering
Draft

Classify SDK tests for Cloud execution#3033
THardy98 wants to merge 4 commits into
feat/cloud-test-namespacefrom
feat/cloud-test-filtering

Conversation

@THardy98

@THardy98 THardy98 commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

What was changed?

Temporal SDK tests are now eligible for Cloud execution by default. Tests that cannot run in the isolated Cloud namespace use JUnit 4 categories with one of three explicit reasons: they require a local server, require Cloud resources that CI does not provision, or still need Cloud-specific adaptation. Every exclusion also carries a required CloudTestExclusionNote explaining its concrete limitation.

The new :temporal-sdk:testCloud task excludes that category hierarchy while leaving the normal test task unchanged. Cloud CI now runs the eligible task through the envconfig harness and per-run mTLS namespace introduced by #2998 and #3032 instead of selecting one workflow method by name.

Straightforward compatibility issues were fixed rather than excluded: secondary clients use the configured namespace and connection, direct external clients preserve envconfig TLS and authentication, and assertions use the active namespace. Tests that need to combine envconfig with test-specific service or client options are explicitly classified as NeedsCloudAdaptation; option composition intentionally remains outside the minimal harness. Nexus endpoint and custom search-attribute suites remain excluded until their required resources can be provisioned.

This PR is stacked on #3032 and should be retargeted to main after the two lower PRs merge.

Review guide

The commits separate filtering infrastructure, annotation-only classifications, compatibility adaptations, and the CI switch. Reviewers can inspect the large mechanical classification commit independently from behavioral changes.

Why?

A hand-picked Cloud smoke test does not show which parts of the SDK suite are compatible with Temporal Cloud. Runner-native categories keep eligibility beside each test, make exclusions reviewable, and allow new compatible tests to enter the Cloud lane automatically without maintaining a separate allowlist. Machine-enforced notes preserve reviewer context while keeping reason filtering idiomatic to JUnit.

Breaking changes?

None. The category types and Gradle task are internal test infrastructure, and normal local test execution is unchanged.

Server PR

None.

How was this tested?

  • CloudTestExclusionTest and the namespace manager tests pass together.
  • After rebasing onto current main and classifying its new incompatibilities, testCloud --test-dry-run discovers 1,428 eligible tests.
  • The previous CI Cloud run passed the full filtered suite against an ephemeral namespace and deleted the namespace successfully. The first current-main run identified five newly added classification gaps and also deleted its namespace successfully; the refreshed run validates those exclusions.
  • ./gradlew :temporal-sdk:test (one unrelated MetricsTest#testStickyCacheSize timing failure passed on isolated rerun)
  • actionlint .github/workflows/ci.yml
  • ./gradlew spotlessApply
  • git diff --check

git range-diff confirms that the four filtering patches are unchanged apart from import context required by current Nexus APIs.

Documentation

CONTRIBUTING.md documents the Cloud task, dry-run inventory, default eligibility policy, exclusion reasons, and required note metadata.

@THardy98
THardy98 force-pushed the feat/cloud-test-namespace branch from 80d1c35 to e951a65 Compare August 25, 2026 12:06
@THardy98
THardy98 force-pushed the feat/cloud-test-filtering branch 4 times, most recently from 63b9ab2 to ed1103c Compare August 25, 2026 14:51
@THardy98
THardy98 force-pushed the feat/cloud-test-namespace branch from e951a65 to 21ae1f4 Compare August 25, 2026 17:00
@THardy98
THardy98 force-pushed the feat/cloud-test-filtering branch from ed1103c to 5e01735 Compare August 25, 2026 20:17
@THardy98
THardy98 force-pushed the feat/cloud-test-filtering branch 5 times, most recently from 1e47aec to a33de7e Compare September 2, 2026 18:48
@THardy98
THardy98 force-pushed the feat/cloud-test-filtering branch from a33de7e to 602d1ec Compare September 3, 2026 12:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant