undici transport parity: header rejections degrade, fileBody short-write detection, SOCKS refusal (#81) - #96
Merged
Conversation
…adapters `undiciTransport().send()` failed outright for four model-valid headers, where TRANSPORT-12 requires the header to give way and the request to dispatch. undici 6.28.0 rejects `expect` (`NotSupportedError`), `keep-alive` and `upgrade` (`InvalidArgumentError`) unconditionally, `connection` with any value but `close`/`keep-alive`, and any name outside RFC 9110 `token` -- while `@dexpace/core` admits every printable ASCII byte in a name, so `X Custom` is model-valid and unsendable. `toDispatchError` mapped all five to a bare non-retryable `TypeError`; nothing reached the wire. The fetch twin was no better on the runtime that ships. Node's global `fetch` is undici-backed and undici's `Headers` deliberately does not implement the WHATWG forbidden-name list, so `expect`/`keep-alive`/`upgrade` reach the same validation and `fetch()` rejects with a bare `TypeError: fetch failed` -- which this transport can only classify as the RETRYABLE `TransportFailureError`, so a permanent misconfiguration spends the caller's whole retry budget re-proving itself. Bun 1.3.14 diverges a third way: it forwards `expect` and `keep-alive` to the wire and hangs indefinitely on `upgrade`. Measured on both, 2026-09-05. - `UNDICI_FORBIDDEN_HEADERS` gains `expect`, `keep-alive`, `upgrade`. - `FETCH_FORBIDDEN_HEADERS` gains the same three (`connection` was already in it), which is the widening D19 left conditional on a row proving it. - `toUndiciHeaders` validates each name against RFC 9110 `token` and each `connection` value against undici's own two, dropping and logging what fails -- the degrade transport-fetch gets for free from `try`/`catch` around `Headers.append`. Value grammar is not re-checked: undici's admits obs-text, which is strictly wider than the outbound rule `mapOutboundHeaders` applied. Five conformance rows both transports run (`Expect`, `Keep-Alive`, `Upgrade`, a non-token name, `Connection: upgrade`); all five were red on undici and three on fetch beforehand. Plus two unit rows against a recording dispatcher, and one Node-conformance case -- the Bun rows prove a weaker claim than Node's, because only on Node does an undropped name reject the send. Refs #81, #67. Deviation ledger: none -- TRANSPORT-11's drop set is transport-specific by its own text, and TRANSPORT-12 is now satisfied rather than departed from.
…Y-13)
The file branch of `prepareBody` handed `createReadStream(path, {start, end})`
straight to undici. That is one fewer userspace copy, and it skipped the
descriptor's `writeTo` entirely, so `@dexpace/body-file`'s
`transferred === count` invariant never ran. `content-length` is dropped
outbound, so undici framed the body chunked and the wire could not detect a
short write either: a file truncated between `stat` and `send` POSTed its
surviving bytes and resolved 200, where `@dexpace/transport-fetch` raised
`TransportFailureError`. The Phase 8a checklist marked BODY-13 done while
recording the bypass.
The branch is gone rather than rerouted, so `prepareBody` is now the same
function on both transports: buffered at or below 1,000,000 declared bytes,
streamed above. D19 said "takes the same `pumpBody` path the streamed case
uses"; deleting the branch is that for a large file and the buffered sibling
for a small one, and it is the only version in which the two adapters cannot
drift again. Framing changes for a small file body -- `content-length` where it
used to be chunked, matching the fetch twin.
TRANSPORT-28's zero-copy clause is a SHOULD no user-space path in either client
can honour (`docs/deviations.md` item 13, recorded since Phase 8a). Its MUSTs --
a file body is replayable, and exactly its declared byte range reaches the wire
-- are honoured by the descriptor, on both transports, by one code path. The
zero-count case needs no branch of its own: `isMaterializable` admits
`contentLength === 0` and `materializeBody` never opens a read stream.
Rows: a shared conformance row on the buffered path (intact ranged body through
`writeTo`, and truncate-after-stat), red on undici and green on fetch before
this; the streamed leg in `tests/node-conformance/transport.test.mjs` with a
real `fileBody()`, likewise red on undici only. The streamed leg cannot live in
the Bun suite -- Bun 1.3.14's `Readable.fromWeb` leaks the abort reason as
unhandled rejections when the web readable behind it is aborted mid-pull, on
both transports and with no SDK code involved (isolated; `node --test` is
clean). `run-suite.ts`'s constant says so, so nobody raises it back.
Refs #81, #67. Deviation ledger: none -- BODY-13 is now satisfied rather than
departed from, and TRANSPORT-28's SHOULD already has item 13.
…the factory
`toProxyAgentOptions` used `proxy.type` as the URI scheme with no check, so
`undiciTransport({proxy: createProxyOptions({type: 'socks5', …})})` reached
`new ProxyAgent({uri: 'socks5://…'})` and threw undici's
`InvalidArgumentError('Invalid URL protocol: socks5:')` straight out of a public
factory -- untyped, undocumented, and not in the SDK's error vocabulary. The
configuration can express it perfectly legitimately: core maps `ALL_PROXY`'s
`socks:`, `socks4:`, `socks4a:`, `socks5:` and `socks5h:` schemes onto
`ProxyType` (CFG-22, `config/proxy.ts:372-380`).
`selectDispatchers` now refuses anything but `http` with a `TypeError` naming
the type, matching its existing dispatcher-plus-proxy refusal and deliberately
outside the `IoError` tree, so `retry/classify.ts`'s allow-list makes it
non-retryable for free (RETRY-2). The check runs before `new undici.Agent(...)`,
so a refused construction allocates nothing -- there would be no transport left
to close it through. `@throws` on `undiciTransport` and on
`UndiciTransportOptions.proxy` say so.
`ProxyType` keeps `socks4`/`socks5`: narrowing a `@public` union is a
release-pass decision (D1/D19), and the gap is recorded in `docs/deviations.md`
instead. `fetchTransport()` needs no equivalent -- it ships no `proxy` option at
all, deliberately (design doc §6).
The conformance row runs on both transports. `TransportCapabilities` gains an
optional `unsupportedProxy: {type, build()}`, because only the adapter knows
which of `ProxyType`'s values its client refuses; where it is absent the row
asserts `supportsProxy === false` rather than skipping, so "no proxy surface"
and "an unasserted gap" cannot look the same. Red before this change: undici's
raw error is neither a `TypeError` nor names `socks5`. Plus a unit row covering
both SOCKS values and asserting no `Agent` was constructed on the way out.
Refs #81, #67.
`docs/deviations.md` gains one row at the end of "Deviations recorded outside a phase" (D0): `CFG-22`'s proxy model carries SOCKS4/SOCKS5 in full, core resolves both from `ALL_PROXY`'s five schemes, and neither shipped transport can send over one -- undici's `ProxyAgent` is an HTTP CONNECT tunnel and `@dexpace/transport-fetch` has no proxy option at all. The row records why `ProxyType` keeps the two values (narrowing a `@public` union is a release-pass decision, and `CFG-22`'s MUST is about the model, which would then stop satisfying it) and where the refusal now happens instead. `write-a-transport.md` grows from nine rules to eleven, both of them things #81 found the shipped transports getting wrong: - Rule 3, split out of rule 2: whatever your native client refuses, drop that header, never the request -- and find out *where* it decides, because WHATWG `Headers.append` throws at construction while undici validates inside `dispatch`. Getting it wrong does not look like a transport bug; it looks like a retryable network failure. - Rule 9, rewritten: recognise a file body structurally and still write it through `writeTo`, because reading `path` yourself skips BODY-13's `transferred === count` and `Content-Length` is dropped by rule 2, so nothing else can see a short write. - Rule 10, new: refuse a proxy you cannot honour at construction, typed, named, and outside the `IoError` tree. Plus the new optional `unsupportedProxy` capability in the "Prove it" example. Fence check passes. Refs #81, #67.
`fetch-transport.ts:146-152` for the `Headers.append` degrade became `:159-166` when this branch widened the fetch drop set above it. And `prepareBody` is not "identical in shape" to the fetch twin's -- the two return different structures; what is identical is the two decisions it makes and their order, which is what the comment meant and now says. Plus the undici version behind the `lib/core/request.js` line numbers, in the one of the three places that omitted it. Refs #81, #67.
`README.md:53` still listed `Content-Length`, `Host`, `Transfer-Encoding` and `Connection`. `FETCH_FORBIDDEN_HEADERS` gained `Expect`, `Keep-Alive` and `Upgrade` earlier on this branch, so the sentence has been false since f02dd8e. Rewritten as three bullets rather than one: which names the client computes and which the layer underneath refuses; why the refused three are dropped rather than forwarded (WHATWG names all four forbidden, the implementations enforce none of it, and they disagree about what happens instead -- Node's undici-backed `fetch` fails the send with the RETRYABLE `TransportFailureError`, Bun 1.3.14 forwards two to the wire and hangs on `Upgrade`); and that a non-token header name degrades to the same drop, which the README had never said at all. `packages/transport-undici/README.md` needed nothing: 6df9645 already widened its enumeration, including the `Connection`-value split undici alone has. Refs #81, #67.
This was referenced Sep 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #81. Part of #67, milestone 5. Wave 6a; #82 lands on top of this.
Decision followed: D19, with two documented departures from its letter (below).
What changed
1. Headers the native client refuses degrade to a logged drop (TRANSPORT-11/12/13)
undiciTransport().send()failed outright for four model-valid headers. undici 6.28.0 rejectsexpect(NotSupportedError),keep-aliveandupgrade(InvalidArgumentError) unconditionally(
lib/core/request.js:398,409),connectionwith any value butclose/keep-alive(:400-404),and any name outside RFC 9110
token(lib/core/util.js:547-587) — while@dexpace/coreadmitsevery printable ASCII byte in a name (
http/ascii-validation.ts:29-33), soX Customismodel-valid and unsendable. All five reached
toDispatchErroras a bare non-retryableTypeError;nothing was sent.
UNDICI_FORBIDDEN_HEADERSgainsexpect,keep-alive,upgrade.toUndiciHeadersvalidates each name against RFC 9110tokenand eachconnectionvalue againstundici's own two (case-insensitively, as undici compares), dropping and logging what fails. Values
are deliberately not re-checked: undici's value grammar admits obs-text 0x80–0xFF, strictly
wider than the outbound rule
mapOutboundHeadersalready applied (HTTP-18).connectionstays out of the by-name drop set — §17's own note — soclose/keep-alivestillreach the wire and
dropsConnectionHeader: falseis still true.FETCH_FORBIDDEN_HEADERSwidened too, and a row proves it. D19 made that conditional. Measured2026-09-05: on Node,
fetchTransport()rejects all three withTransportFailureError—retryable, so a permanent misconfiguration spends the caller's whole retry budget re-proving
itself — because Node's global
fetchis undici-backed and undici'sHeadersdeliberately does notimplement the WHATWG forbidden-name list. On Bun 1.3.14 it diverges a third way:
expectandkeep-alivego to the wire, andupgradehangs indefinitely. Neither is TRANSPORT-12's outcome.2. A file body goes through its own
writeTo(BODY-13, TRANSPORT-28)The file branch of
prepareBodyhandedcreateReadStream(path, {start, end})to undici, skippingthe descriptor's
writeToand so@dexpace/body-file'stransferred === countinvariant.content-lengthis dropped outbound, so undici framed the body chunked and the wire could not see ashort write either: a 1 MB
fileBodytruncated to 10 bytes beforesend()POSTed 10 bytes andresolved 200, where
@dexpace/transport-fetchraisedTransportFailureError.Departure from D19's letter, accepted: D19 said the file branch "takes the same
pumpBodypaththe streamed case uses". The branch is deleted instead, so
prepareBodyis now the same twodecisions on both transports — buffered at or below 1,000,000 declared bytes, streamed above. That
is
pumpBodyfor a large file and the buffered sibling for a small one, and it is the only versionin which the two adapters cannot drift again. The zero-count branch goes with it:
isMaterializableadmitscontentLength === 0andmaterializeBodynever opens a read stream.Framing changed, as D19 asked to be told. A file body at or below 1,000,000 bytes is now sent
with
content-lengthwhere it used to be chunked. Above that it stays chunked. Both match the fetchtwin exactly.
TRANSPORT-28's zero-copy clause is a SHOULD no user-space path in either client can honour
(
docs/deviations.mditem 13, recorded since Phase 8a — thecreateReadStreampath was neversendfile(2)either). Its MUSTs — replayable, and exactly the declared range on the wire — are thedescriptor's, honoured on both transports by one code path, and asserted by a row.
3. A SOCKS proxy is refused at the factory (TRANSPORT-30)
toProxyAgentOptionsusedproxy.typeas the URI scheme unchecked, soundiciTransport({proxy: createProxyOptions({type: 'socks5', …})})threw undici'sInvalidArgumentError('Invalid URL protocol: socks5:')straight out of a public factory. Coreresolves
ALL_PROXY'ssocks:,socks4:,socks4a:,socks5:andsocks5h:ontoProxyTypequite legitimately (CFG-22,
config/proxy.ts:372-380).selectDispatchersnow refuses anything buthttpwith aTypeErrornaming the type, matching itsexisting dispatcher-plus-proxy refusal and deliberately outside the
IoErrortree soretry/classify.ts's allow-list makes it non-retryable (RETRY-2). The check runs beforenew undici.Agent(...), so a refused construction allocates nothing.@throwsonundiciTransportand on
UndiciTransportOptions.proxy.ProxyTypekeepssocks4/socks5(D19; narrowing a@publicunion is a release-pass call).fetchTransport()needs no equivalent — it has noproxyoption at all.
Test rows added
Shared suite (
packages/transport-conformance/src/{run-suite,fixtures}.ts), both transports runevery one:
Expect: 100-continuedropped, logged, request still dispatchesKeep-Alive: timeout=5— sameUpgrade: websocket— sameX Custom(non-token name) — sameConnection: upgradeis dropped on bothwriteTowrites.count === 0)InvalidArgumentError)TransportCapabilitiesgains an optionalunsupportedProxy: {type, build()}— only the adapterknows which
ProxyTypevalues its client refuses. Where it is absent (fetch) the row assertssupportsProxy === falserather than skipping, so "no proxy surface" and "an unasserted gap" cannotlook alike.
New fixture:
fileBodyFixture(path, {start?, count, writes?})infixtures.ts— akind: 'file'descriptor carrying BODY-13's check itself. Deliberately a stand-in, not the real
fileBody():@dexpace/transport-conformanceisprivate, resolves unbuilt and depends on@dexpace/corealone, and taking
@dexpace/body-filewould add a fifth entry to the rootbuild:depschain plus alockfile change for one row — all three files outside this task's partition.
Unit rows in
packages/transport-undici/src/undici-transport.test.ts: every refused header absentfrom the dispatched argument array with a sibling surviving;
Connectioncarried forclose/Keep-Aliveand dropped forupgrade; both SOCKS values refused with noAgentconstructed. The two existing file-body rows now assert
writeTowas called (they asserted itmust not be).
Node conformance (
tests/node-conformance/transport.test.mjs, both transports):weaker claim;
fileBody()over a 1.1 MB file.Findings
Bun 1.3.14's
Readable.fromWebleaks unhandled rejections when the web readable behind it isaborted mid-pull — two or three per abort. That is exactly what a producer failure on the streamed
request-body path does, and
bun:testscores them against whichever row is running. Isolated tosixteen lines with no SDK code in them;
node --testis clean on both transports. It is why thestreamed leg of the truncate row lives in the Node tree, and
TRUNCATED_FILE_BYTES's TSDoc says soin case anyone raises it past 1,000,000. Reachable in production only under Bun, only on a >1 MB
streaming request body whose producer fails mid-flight — not this task's to fix, and not this SDK's
bug, but #82 should know, because D20's producer-failure-race item is in the same code.
packages/transport-fetch/README.md:53is now stale — it lists the drop set as"
Content-Length,Host,Transfer-Encoding, andConnection". Outside this task's partition(which names
src/fetch-transport.tsonly), so left alone per contract item 4. One-line fix for#82 or the supervisor: add
Expect,Keep-AliveandUpgradeto that list.tests/node-conformance/transport.test.mjswas edited outside the partition, as #72/#76 didbefore, for the reason CLAUDE.md's membership rule gives: the divergence is real and measured (Bun
forwards
expect/keep-aliveand hangs onupgrade; Node rejects all three), and that file is theonly layer where a real
fileBody()meets a real transport — its own header comment says so.For #82
run-suite.tsgainedregisterNativeRejectionRowsandregisterFileBodyRows(registered betweenregisterHeaderRowsandregisterInboundHeaderRows) andregisterProxyRefusalRows(afterregisterDropSetRows);fixtures.tsgainedfileBodyFixture. Merge seams for D20's rows.TransportCapabilitiesis now four fields, the fourth optional.prepareBodyinundici-transport.tsno longer importsnode:fsorFileBodyDescriptor.UNDICI_FORBIDDEN_HEADERSandFETCH_FORBIDDEN_HEADERSboth changed; D20'sCONTROL_BYTEandbody === nullwork sits beside them.defaultTimeoutMsis still unvalidated on both transports (carried from Domain-model input validation: prototype keys, invalidDate, fractionaltimeoutMs, lone surrogates, frozengetAll#76 to fetch transport: permanent-error classification; both transports: 204/304/HEAD rows,CONTROL_BYTEregex, producer-failure race #82, untouchedhere).
ftp://URL still reachesdispatcher.request({origin: …})unchecked — D20's classificationitem.
Gates
node .claude/skills/ci-preflight/run-ci.mjs --clean— all 20 steps passed (install,verify:knowledge-structure,typecheck,lint,build,test,test:scripts,api,lint:publish,verify:dual-consumption,verify:consumer-types,verify:seam-1,verify:sse-37,verify:runtime-floor,verify:test-partition,test:examples,verify:import-cycles,verify:reproducible-build,audit,test:node). 2466 Bun tests across166 files; 178 Node tests. Run without
--node-floor, so CI's Node 20.3.0 leg is untested locally.housekeeping/probe.mjs: no drift.check-fences.mjs: pass.api:localregenerated fortransport-undiciandtransport-fetch; both reports arebyte-identical — the changed TSDoc is prose on existing signatures, which api-extractor's report
does not carry. Nothing to commit.
bun run apiverifies all nine.Deviations recorded
One row appended at the end of
docs/deviations.md's "Deviations recorded outside a phase" (D0):CFG-22's SOCKS proxy types are resolved by the configuration layer and supported by neithershipped transport; the refusal is at the transport factory, and
ProxyTypekeeps them.No phase-ledger edits, no §10 edits. Nothing else is a deviation: TRANSPORT-12 and BODY-13 are now
satisfied rather than departed from, and TRANSPORT-11's drop set is transport-specific by its own
text.
Deferred — release machinery
Suspended under D1; nothing below was done.
@dexpace/transport-undici:send()no longer fails forExpect,Keep-Alive,Upgrade, a non-token header name, or aConnectionvalue other thanclose/keep-alive— each is dropped and logged instead (behaviour change: a send that used tothrow now succeeds without that header). A file body is written through its own
writeTo, so ashort write now fails the send, and a file body at or below 1 MB is framed with
content-lengthrather than chunked.
undiciTransport()throwsTypeErrorfor a non-httpproxy.typethatpreviously escaped as undici's
InvalidArgumentError. Shipped.d.tsprose changed forundiciTransport(@throws) andUndiciTransportOptions.proxy.@dexpace/transport-fetch:Expect,Keep-AliveandUpgradeare nowdropped outbound. On Node these previously failed the send with a retryable
TransportFailureError; on BunExpect/Keep-Alivereached the wire andUpgradehung. No.d.tschange.ProxyTypenarrowing, and the question behind it.@dexpace/core'sProxyTypestill admitssocks4andsocks5, andresolveProxyOptionsstill resolves them from the environment, whileno shipped transport can send over one. Dropping them from the union is free only before the first
version bump — but it would also put
@dexpace/corein breach ofCFG-22, whose MUST enumerates"HTTP, SOCKS4, SOCKS5" for the proxy model. Recommendation: keep the union and leave the
docs/deviations.mdrow standing. The model is complete and the transport layer is not, whichis an honest state and one a future SOCKS transport closes without a public-shape change. If the
release pass disagrees, it is a minor changeset for
@dexpace/core(breaking for anyonenaming the type) plus a matching one for
@dexpace/transport-undici.docs/first-release.mdedits.