FEAT: Generated transport conformance tests should cover new reject/dlq/requeue paths

.agent_instructions/generated_tests.md

@@ -16,12 +16,13 @@ tools/Paramore.Brighter.Test.Generator/
 │   ├── IAmAMessageBuilder.cs.liquid        ← Message builder interface
 │   ├── IAmAMessageAssertion.cs.liquid      ← Message assertion interface
 │   ├── Outbox/
-│   │   ├── Sync/       ← 13 Liquid templates for sync outbox tests (12 tests + 1 interface)
-│   │   ├── Async/      ← 13 Liquid templates for async outbox tests (12 tests + 1 interface)
+│   │   ├── Sync/       ← 14 Liquid templates for sync outbox tests (13 tests + 1 interface)
+│   │   ├── Async/      ← 14 Liquid templates for async outbox tests (13 tests + 1 interface)
 │   │   └── Causation/  ← Liquid template for causation-tracking outbox tests (subclass of CausationTrackingOutboxBaseTests)
 │   └── MessagingGateway/
-│       ├── Reactor/    ← 13 Liquid templates for sync messaging gateway tests (12 tests + 1 interface)
-│       └── Proactor/   ← 13 Liquid templates for async messaging gateway tests (12 tests + 1 interface)
+│       ├── Reactor/    ← 21 Liquid templates for sync messaging gateway tests (20 tests + 1 interface)
+│       ├── Proactor/   ← 21 Liquid templates for async messaging gateway tests (20 tests + 1 interface)
+│       └── Shared/     ← 2 Liquid templates shared by both variants (ConformanceDeferredPump, RejectionMetadataKeys)
 ├── Generators/
 │   ├── BaseGenerator.cs
 │   ├── OutboxGenerator.cs
@@ -50,20 +51,28 @@ tests/Paramore.Brighter.*.Tests/
 
 | Project | Outbox | Messaging Gateway | Variants |
 |---|---|---|---|
-| `Paramore.Brighter.MySQL.Tests` | ✅ | | Text, Binary |
-| `Paramore.Brighter.PostgresSQL.Tests` | ✅ | ✅ | Text, Binary (outbox); single (gateway) |
-| `Paramore.Brighter.MSSQL.Tests` | ✅ | ✅ | Text, Binary (outbox); single (gateway) |
-| `Paramore.Brighter.Sqlite.Tests` | ✅ | | Text, Binary |
-| `Paramore.Brighter.DynamoDB.Tests` | ✅ | | single |
-| `Paramore.Brighter.DynamoDB.V4.Tests` | ✅ | | single |
-| `Paramore.Brighter.MongoDb.Tests` | ✅ | | single |
+| `Paramore.Brighter.AWS.Tests` |  | ✅ | SnsStandard, SnsFifo, SqsStandard, SqsFifo |
+| `Paramore.Brighter.AWS.V4.Tests` |  | ✅ | SnsStandard, SnsFifo, SqsStandard, SqsFifo |
+| `Paramore.Brighter.AzureServiceBus.Tests` |  | ✅ | single |
+| `Paramore.Brighter.DynamoDB.Tests` | ✅ |  | single |
+| `Paramore.Brighter.DynamoDB.V4.Tests` | ✅ |  | single |
 | `Paramore.Brighter.Gcp.Tests` | ✅ | ✅ | Firestore, SpannerBinary, SpannerText (outbox); Pull, PullOrdering, Stream, StreamOrdering (gateway) |
-| `Paramore.Brighter.RMQ.Async.Tests` | | ✅ | Classic, Quorum |
-| `Paramore.Brighter.AWS.Tests` | | ✅ | SnsStandard, SnsFifo, SqsStandard, SqsFifo |
-| `Paramore.Brighter.AWS.V4.Tests` | | ✅ | SnsStandard, SnsFifo, SqsStandard, SqsFifo |
-| `Paramore.Brighter.Kafka.Tests` | | ✅ | Standard, PartitionKey |
-| `Paramore.Brighter.Redis.Tests` | | ✅ | single |
-| `Paramore.Brighter.RocketMQ.Tests` | | ✅ | single |
+| `Paramore.Brighter.Kafka.Tests` |  | ✅ | Classic, Consumer, PartitionKey |
+| `Paramore.Brighter.MQTT.Tests` |  | ✅ | single |
+| `Paramore.Brighter.MSSQL.Tests` | ✅ | ✅ | Text, Binary (outbox); single (gateway) |
+| `Paramore.Brighter.MongoDb.Tests` | ✅ |  | single |
+| `Paramore.Brighter.MySQL.Tests` | ✅ |  | Text, Binary |
+| `Paramore.Brighter.PostgresSQL.Tests` | ✅ | ✅ | Text, Binary (outbox); single (gateway) |
+| `Paramore.Brighter.RMQ.Async.Tests` |  | ✅ | Classic, Quorum |
+| `Paramore.Brighter.RMQ.Sync.Tests` |  | ✅ | single |
+| `Paramore.Brighter.Redis.Tests` |  | ✅ | single |
+| `Paramore.Brighter.RocketMQ.Tests` |  | ✅ | single |
+| `Paramore.Brighter.Sqlite.Tests` | ✅ |  | Text, Binary |
+
+> **All 17 projects carrying a `test-configuration.json` are listed above**, derived from those
+> files. If you add a project or change its variants, rebuild the whole row set from
+> `tests/*/test-configuration.json` rather than editing one line — that is how this table came to
+> be missing three projects and to describe Kafka's variants as they were two renames ago.
 
 ## Templates
 
@@ -80,7 +89,7 @@ Generated into the test project root. These provide common helpers used by both
 
 ### Outbox Templates
 
-**Sync** (13 templates) and **Async** (13 templates) — each contains 1 provider interface + 12 test scenarios:
+**Sync** (14 templates) and **Async** (14 templates) — each contains 1 provider interface + 13 test scenarios:
 
 | Template | Description |
 |---|---|
@@ -97,6 +106,7 @@ Generated into the test project root. These provide common helpers used by both
 | `When_Retrieving_All_Messages_They_Should_Include_Dispatched_And_Undispatched` | Full message listing |
 | `When_Retrieving_Outstanding_Messages_It_Should_Filter_By_Age` | Outstanding message age filtering |
 | `When_Retrieving_Dispatched_Messages_It_Should_Filter_By_Age` | Dispatched message age filtering |
+| `When_Storing_A_Message_With_A_Relative_DataSchema_It_Should_Be_Read_Back` | Relative `DataSchema` URI survives round-trip |
 
 Templates containing `Transaction` in the filename are skipped when `SupportsTransactions` is `false`.
 
@@ -114,7 +124,7 @@ Templates containing `Transaction` in the filename are skipped when `SupportsTra
 
 ### Messaging Gateway Templates
 
-**Reactor** (13 templates) and **Proactor** (13 templates) — each contains 1 provider interface + 12 test scenarios:
+**Reactor** (21 templates) and **Proactor** (21 templates) — each contains 1 provider interface + 20 test scenarios:
 
 | Template | Description |
 |---|---|
@@ -125,12 +135,20 @@ Templates containing `Transaction` in the filename are skipped when `SupportsTra
 | `When_a_message_consumer_reads_multiple_messages_should_receive_all_messages` | Batch receive |
 | `When_sending_a_message_should_propagate_activity_context` | OpenTelemetry context propagation |
 | `When_confirming_posting_a_message_should_receive_publish_confirmation` | Publisher confirms |
-| `When_requeuing_a_failed_message_should_receive_message_again` | Basic requeue |
-| `When_requeuing_a_failed_message_with_delay_should_receive_message_again` | Delayed requeue |
-| `When_requeuing_a_message_too_many_times_should_move_to_dead_letter_queue` | DLQ redrive |
 | `When_infrastructure_missing_and_assume_channel_should_throw_exception` | Assume mode error |
 | `When_infrastructure_missing_and_validate_channel_should_throw_exception` | Validate mode error |
-| `When_reading_a_delayed_message_via_the_messaging_gateway_should_delay_delivery` | Scheduled delivery |
+| `When_nacking_a_message_it_should_be_redelivered` | Nack redelivery (FR-16) |
+| `When_requeuing_a_failed_message_should_be_redelivered` | Basic requeue |
+| `When_requeuing_a_failed_message_with_delay_should_redeliver_after_delay` | Delayed requeue (FR-2) |
+| `When_requeuing_a_failed_message_with_zero_delay_should_redeliver_immediately` | Zero-delay requeue |
+| `When_requeuing_a_message_too_many_times_should_move_to_dead_letter_queue` | Requeue budget exhausted to DLQ (FR-23) |
+| `When_sending_a_delayed_message_should_deliver_after_delay` | Delayed send (FR-9) |
+| `When_rejecting_message_should_include_metadata` | Rejection metadata stamped on the rejected message (FR-8) |
+| `When_rejecting_message_with_delivery_error_should_send_to_dlq` | Delivery-error reject routes to the DLQ |
+| `When_rejecting_message_with_unknown_reason_should_send_to_dlq` | Unknown reason defaults to the DLQ |
+| `When_rejecting_message_with_unacceptable_reason_should_send_to_invalid_channel` | Unacceptable routes to the invalid channel |
+| `When_rejecting_message_with_unacceptable_and_no_invalid_channel_should_fallback_to_dlq` | Unacceptable falls back to the DLQ when no invalid channel exists |
+| `When_rejecting_message_with_no_channels_configured_should_acknowledge_and_log` | No DLQ and no invalid channel: acknowledge and continue |
 
 Templates are skipped based on feature support flags (see [Feature Flags](#messaging-gateway-feature-flags) below).
 
@@ -213,8 +231,6 @@ To disable transaction tests for providers that don't support them (e.g. MongoDB
     "MessageGatewayProvider": "Paramore.Brighter.Redis.Tests.MessagingGateway.RedisMessageGatewayProvider",
     "Category": "Redis",
     "CollectionName": "RedisMessagingGateway",
-    "HasSupportToDeadLetterQueue": true,
-    "HasSupportToRequeue": true,
     "HasSupportToValidateInfrastructure": false
   }
 }
@@ -240,10 +256,7 @@ This generates tests into:
       "CollectionName": "Classic",
       "ReceiveMessageTimeoutInMilliseconds": 4000,
       "HasSupportToPublishConfirmation": true,
-      "HasSupportToDeadLetterQueue": true,
-      "HasSupportToDelayedMessages": false,
-      "HasSupportToValidateBrokerExistence": true,
-      "HasSupportToRequeue": true
+      "HasSupportToValidateBrokerExistence": true
     },
     "Quorum": {
       "Publication": "Paramore.Brighter.MessagingGateway.RMQ.Async.RmqPublication",
@@ -253,10 +266,7 @@ This generates tests into:
       "CollectionName": "Quorum",
       "ReceiveMessageTimeoutInMilliseconds": 4000,
       "HasSupportToPublishConfirmation": true,
-      "HasSupportToDeadLetterQueue": true,
-      "HasSupportToDelayedMessages": false,
-      "HasSupportToValidateBrokerExistence": true,
-      "HasSupportToRequeue": true
+      "HasSupportToValidateBrokerExistence": true
     }
   }
 }
@@ -280,11 +290,9 @@ Feature flags control which test templates are generated. When a flag is `false`
 | Flag | Default | Skips templates containing | Use case |
 |---|---|---|---|
 | `HasSupportToPublishConfirmation` | `false` | `confirming_posting` | Transport doesn't support publisher confirms |
-| `HasSupportToDelayedMessages` | `false` | `delayed_message`, `with_delay` | No delayed/scheduled message support |
-| `HasSupportToDeadLetterQueue` | `false` | `dead_letter_queue` | No dead letter queue support |
 | `HasSupportToValidateBrokerExistence` | `false` | `no_broker_created` | Transport can't validate broker existence |
-| `HasSupportToRequeue` | `false` | `requeuing` | Transport doesn't support message requeue |
 | `HasSupportToValidateInfrastructure` | **`true`** | `assume_channel`, `validate_channel` | Transport can't validate infrastructure existence |
+| `HasSupportToDetectMissingInfrastructureOnAssume` | **`true`** | `assume_channel` | Transport honours an explicit `Validate` but completes silently against infrastructure that is absent when `OnMissingChannel.Assume` told it not to look (Kafka's KIP-848 consumer — gateway defect, tracked by #4299; drop the flag when fixed) |
 
 ### Messaging Gateway Provider Pattern
 
@@ -430,7 +438,7 @@ If you change a template pattern that also exists in these base classes, update
 
 ### Important: Generator Does Not Delete Stale Files
 
-The generator only creates or overwrites files — it **never deletes** existing generated files. If you change a feature flag from `true` to `false` (e.g. disabling `HasSupportToDelayedMessages`), you must **manually delete** the previously-generated test files that are no longer wanted. Otherwise stale tests will remain and may fail.
+The generator only creates or overwrites files — it **never deletes** existing generated files. If you change a feature flag from `true` to `false` (e.g. disabling `HasSupportToPublishConfirmation`), you must **manually delete** the previously-generated test files that are no longer wanted. Otherwise stale tests will remain and may fail.
 
 Similarly, if you rename or remove a template, the old generated files remain on disk.
 

.agent_instructions/testing.md

@@ -155,4 +155,24 @@ A run in `review-after` that drops any of these is defective — it is not "a di
   - Write the integration test that exercises the behavior
   - Get approval for the test
   - Implement the code to make it pass
-  - This ensures you're building the right behavior from the start
+  - This ensures you're building the right behavior from the start
+
+### RabbitMQ: run each suite against the broker version it targets
+
+The two RabbitMQ suites do not target the same broker, and pointing one at the other's version
+produces a wall of red that is not a regression.
+
+- **`RMQ.Async` runs on 4.2 and 4.3+.** `RmqSubscription.isDurable` defaults to `true` there (#4355),
+  so a default subscription declares a queue 4.3 accepts.
+- **`RMQ.Sync` runs on 4.2 only.** It targets the RabbitMQ 3.x line through `RabbitMQ.Client` 6.x, so
+  its `isDurable` default is deliberately still `false`. RabbitMQ **4.3 removed transient
+  non-exclusive queues**, so on a 4.3+ broker every queue declaration is rejected with
+  `INTERNAL_ERROR - Feature 'transient_nonexcl_queues' is deprecated`. Measured against 4.3.5:
+  **46 failed / 35 passed / 3 skipped**. This applies to the package, not just its tests — a default
+  subscription from `Paramore.Brighter.MessagingGateway.RMQ.Sync` is rejected by 4.3+ too.
+
+`docker-compose-rmq.yaml` is pinned to 4.2 for this reason and CI uses it, so `rabbitmq-sync-ci` is
+green. **Check the broker version before reading a local RabbitMQ failure** — `docker ps` will show
+an unrelated 4.3 container holding 5672 on a developer machine. Do not "fix" the `isDurable: false`
+sites in `Paramore.Brighter.RMQ.Sync.Tests` to make a 4.3 run pass; they match the product default
+the package deliberately keeps.

.github/workflows/ci.yml

@@ -34,9 +34,8 @@ env:
 
 jobs:
   build:
-    # Increased from 30 to 40 minutes to prevent cancellations during test execution.
-    # The build includes compilation + running 3 test suites (Core.Tests, Extensions.Tests,
-    # Transforms.Adaptors.Tests) which can take 20-30 minutes in slower CI environments.
+    # Compilation plus four test suites: Core.Tests, Extensions.Tests, Transforms.Adaptors.Tests
+    # and Test.Generator.Tests. 60 minutes leaves room for a cold NuGet cache on a slow runner.
     timeout-minutes: 60
     runs-on: ubuntu-latest
     steps:
@@ -68,6 +67,13 @@ jobs:
         # question already answered. net10.0 matches what generate-test.sh runs.
         run: dotnet test tests/Paramore.Brighter.Test.Generator.Tests/Paramore.Brighter.Test.Generator.Tests.csproj -c Release --no-restore --logger GitHubActions --no-build -f net10.0
 
+      # The RocketMQ gateway's broker-free tests. rocketmq-ci is commented out below, so without
+      # this step the whole RocketMQ suite - and with it the builder-block extraction in
+      # RocketMqMessagePublisher - is executed by nothing in CI. The filter is what keeps the step
+      # honest: the other 47 tests in that project need a live broker and would fail here.
+      - name: RocketMQ Broker-Free Tests
+        run: dotnet test tests/Paramore.Brighter.RocketMQ.Tests/Paramore.Brighter.RocketMQ.Tests.csproj --filter "Category=RocketMQBrokerFree" -c Release --logger GitHubActions --no-restore --no-build
+
       - name: Core Tests
         run: |
           dotnet test tests/Paramore.Brighter.Core.Tests/Paramore.Brighter.Core.Tests.csproj --filter "Fragile!=CI" -c Release --logger GitHubActions --no-restore --no-build
@@ -172,10 +178,9 @@ jobs:
 
   redis-ci:
     runs-on: ubuntu-latest
-    # Increased from 5 to 8 minutes to accommodate slower CI environments
-    # and service initialization time. This provides adequate buffer without
-    # being excessive for typical test execution (P95 < 5 min observed).
-    timeout-minutes: 8
+    # Raised from 8 to 15 when the generated conformance suite landed. Run 34214484655 measured
+    # this job at 7.0 minutes against the old 8-minute cap - the closest of any job to timing out.
+    timeout-minutes: 15
     needs: [build]
 
     services:
@@ -204,7 +209,7 @@ jobs:
 
   mqtt-ci:
     runs-on: ubuntu-latest
-    timeout-minutes: 8
+    timeout-minutes: 15
     needs: [build]
 
     services:
@@ -229,12 +234,12 @@ jobs:
 
   rabbitmq-async-ci:
     runs-on: ubuntu-latest
-    timeout-minutes: 8
+    timeout-minutes: 15
     needs: [build]
 
     services:
       rabbitmq:
-        image: brightercommand/rabbitmq:4.2-management-delay
+        image: rabbitmq:4.2-management
         ports:
         - 5672:5672 
         # needed because the rabbitmq container does not provide a healthcheck
@@ -257,12 +262,12 @@ jobs:
 
   rabbitmq-sync-ci:
     runs-on: ubuntu-latest
-    timeout-minutes: 8
+    timeout-minutes: 15
     needs: [build]
 
     services:
       rabbitmq:
-        image: brightercommand/rabbitmq:3.13-management-delay
+        image: rabbitmq:3.13-management
         ports:
         - 5672:5672 
         # needed because the rabbitmq container does not provide a healthcheck
@@ -362,7 +367,7 @@ jobs:
 
   postgres-ci:
     runs-on: ubuntu-latest
-    timeout-minutes: 8
+    timeout-minutes: 15
     needs: [build]
 
     services:
@@ -393,7 +398,7 @@ jobs:
 
   sqlserver-ci:
     runs-on: ubuntu-latest
-    timeout-minutes: 8
+    timeout-minutes: 15
     needs: [build]
 
     services:
@@ -479,7 +484,7 @@ jobs:
           dotnet test ./tests/Paramore.Brighter.DynamoDB.V4.Tests/Paramore.Brighter.DynamoDB.V4.Tests.csproj --filter "Fragile!=CI" --configuration Release --logger "console;verbosity=normal" --logger GitHubActions --blame -v n
   aws-mock-ci:
     runs-on: ubuntu-latest
-    timeout-minutes: 8
+    timeout-minutes: 20
     needs: build
     env:
       AWS_SERVICE_URL: http://localhost:4566
@@ -527,8 +532,9 @@ jobs:
     runs-on: ubuntu-latest
     # Two projects, run one after the other, and each carries live teardown tests that poll a
     # deletion to a deadline — SQS documents a deleted queue as visible for up to sixty seconds,
-    # so the waiting is the test rather than slack in it. 8 minutes no longer leaves room.
-    timeout-minutes: 12
+    # so the waiting is the test rather than slack in it. 8 minutes no longer leaves room, and
+    # the conformance suite takes the AWS generated files from 168 to 312, so 12 does not either.
+    timeout-minutes: 20
     if: |
       (github.event_name == 'push' && (github.ref_name == 'main' || github.ref_name == 'master')) ||
       (github.event_name == 'pull_request' && !github.event.pull_request.head.repo.fork)
@@ -616,7 +622,7 @@ jobs:
 
   azure-ci:
     runs-on: ubuntu-latest
-    timeout-minutes: 8
+    timeout-minutes: 15
     if: |
       (github.event_name == 'push' && (github.ref_name == 'main' || github.ref_name == 'master')) ||
       (github.event_name == 'pull_request' && !github.event.pull_request.head.repo.fork)