Repository navigation
Feature-support validation: fail fast on missing Antalya features on antalya-26.6 (swarms, oauth, s3 export, iceberg) #155
Description
This issue records work done while running regression suites against the new antalya-26.6 base. Antalya features are forward-ported to a new base at different times, so several suites failed en masse because the features they test are not present yet. The work was consolidated into a single commit, plus two follow-up commits (see below). Assisted by AI (Cursor / Claude Opus 4.8).
Summary
On antalya-26.6, suites that exercise not-yet-forward-ported Antalya features produced large numbers of unrelated failures (e.g. swarms had ~3000+ failures, all rooted in one missing setting), which makes triage hard and hides real regressions.
Added a reusable feature-support validation so each affected suite/sub-feature emits one clear failure (a feature support validation subtest) instead of many, and does not run the rest once the feature is known to be missing. Gates are antalya-only where applicable, so non-antalya behavior is unchanged, and they auto-recover once the feature is forward-ported (the check passes and the tests run normally).
Fix commit: fb33c1f4c
Generic component
helpers/feature_support.py:
feature_support_validation— a@TestScenarionamedfeature support validationthat fails with a clear message when the feature is absent.validate_feature_support(self, feature, check)— runs the detection on the module context and the named subtest; returns the support bool so a suite canreturn/skip the rest.setting_supported(name)— check that a setting exists (queries thenamecolumn ofsystem.settings, not itsvalue— several settings default to an empty string, so a value-based check gives false negatives).get_clickhouse_binary/run_clickhouse_local— pre-cluster host probe viaclickhouse local, for features that stop the server from starting.
Per-suite gates
| Suite / sub-feature | Detection | Notes |
|---|---|---|
swarms |
object_storage_cluster setting |
collapsed ~3000+ failures into one |
oauth |
CREATE USER ... IDENTIFIED WITH jwt via clickhouse local (pre-cluster) |
server can't even start without JWT (static jwt_user in users.xml is rejected at boot) |
s3 export part |
allow_experimental_export_merge_tree_part |
|
s3 export partition |
export_merge_tree_partition_force_export |
partition-specific, not the general export-part setting |
iceberg swarm |
object_storage_cluster |
|
iceberg iterator race condition |
object_storage_cluster_join_mode |
|
iceberg sort key timezone |
iceberg_partition_timezone |
|
iceberg export partition |
export_merge_tree_partition_force_export |
server crashes on config restart without it; gate runs before applying the config |
iceberg system.tables partition/sorting keys |
capability probe (partition_key populated in system.tables) |
PR Altinity/ClickHouse#1432 — no setting exists, so we probe the actual capability |
Detection notes
- Most features are gated by a setting existence check (post-cluster query, or pre-cluster
clickhouse localfor oauth). system.tables partition/sorting keyshas no setting (compiled-in behavior from PR #1432, "Antalya 26.1"), so it uses a capability probe; the test's own version gate (>26.3.10.20001) wrongly assumed presence on 26.6.- Confirmed detection empirically against a build that has the features (
26.3.10.20001.altinityantalya, passes) vsantalya-26.6(fails fast).
Follow-up commits
6933926130— all setting-based gates now callsetting_supported()directly. Removes three duplicate wrappers (export_merge_tree_part_supported,export_merge_tree_partition_supported,object_storage_cluster_supported) andswarms/tests/feature_support.py. From Julian's review: the same check forexport_merge_tree_partition_force_exporthad been written two different ways across the s3 and iceberg suites.867b4caf2f— fixes aTypeErrorthis commit introduced inswarms/regression.py.validate_feature_support()ends with aScenario, so thewith And(...)below it had noStepsibling to inherit its subtype from. It only triggers when the gate passes, so the swarms suite did not actually run on a swarm-capable build until this fix — the "tests run normally" part of the summary above only holds from867b4caf2fon.
Follow-ups (not addressed here)
- None
Activity
CarlosFelipeOR commentedon Aug 13, 2026
The gate added here is a false negative on Antalya < 26.3: run 31281037311 fails on 26.1 with
OAuth/JWT authentication is not supported by this build, on a build that does support it.The probe used
CREATE USER ... IDENTIFIED WITH jwt, which detects SQL-created JWT users, not JWT support — that statement is blocked before 26.3 even on builds that have JWT, and upstream returns the same error. 26.1 does OAuth viatoken_processors+ thetokenuser directory, andsanitypassed there before the gate existed.Fixed in 06c6b83: the check is now the presence of the
enable_token_authserver setting, which is what actually gates the feature (without it:token_processorsnot parsed,tokenuser directory not added,Authorization: Bearerrejected). It runs pre-cluster withclickhouse local, since a build lacking the feature rejects the static<jwt>user inusers.xmlat boot.CarlosFelipeOR commentedon Aug 14, 2026
Second failure from the same gate, on 25.8 Antalya: run 31280950789 errored all of
/icebergwithAttributeError: feature_supported.An ffails
/*pattern over a feature also matches theGiventhatvalidate_feature_supportcreates inside it, so the skip left the flag unset and the return raised. 26.1 passed because the skip condition (<26.1) was false there. Fixed in cbc43d5 by defaulting the flag toFalse.