Skip to content

perf(windows): stream Cloud Files hydration with one range request - #519

Merged
veryCrunchy merged 5 commits into
mainfrom
perf/windows-cloud-files-hydration
Oct 19, 2026
Merged

veryCrunchy merged 5 commits into
mainfrom
perf/windows-cloud-files-hydration

Conversation

@veryCrunchy

Copy link
Copy Markdown
Member

Outcome

Opening a cloud-only file from the Windows Cloud Files root now downloads it with one streamed range request, so hydration runs at roughly the speed of an ordinary download. Before this change it was much slower: Windows showed "Photos is downloading" for a long time on a single JPEG.

Cause. Measured on a real sync root on this machine: when an application reads a placeholder sequentially, Windows issues one FETCH_DATA whose required range is the whole file. The optional range was empty. For example, a 24 MiB file arrived as required=0+25165824. The provider split that range into 4 MiB chunks and made a separate request for each one. Every request was ETag-pinned and fully buffered before CfExecute received any of it. That means:

  • each chunk paid a full server round trip, including Nextcloud request and auth overhead;
  • servers that have to seek to serve a range paid progressively more for each later chunk;
  • network transfer and CfExecute never overlapped.

Change.

  • hydrateWindowsCloudRange (new WindowsCloudFilesHydration.kt) fetches the aligned span of each FETCH_DATA with one streamed range. It transfers 1 MiB CfExecute chunks as bytes arrive, checks cancellation between chunks, and fails if any chunk does not match the plan. Hydration planning and the read-handle contract move into the same file.
  • DesktopFileRangeTransport (new, extracted from DesktopNextcloudServices) serves both the existing bounded read and the new stream. Both keep the existing guarantees: If-Match on the pinned ETag, HTTP 206 required, an exact Content-Range match, and failure when the body is shorter or longer than the range.
  • NextcloudFileRangeSession.stream is new. Platforms without streaming fall back to one bounded read per chunk, so Android behavior is unchanged.
  • DesktopNextcloudServices.kt shrinks by 46 lines and WindowsCloudFilesProvider.kt by 44. Both size baselines are lowered.

Verification

  • Every check relevant to the changed scope passes, or each unrun check is listed below with a reason
  • bash tools/check-repository.sh passes (after the fragment received this PR number)
  • A new changes/unreleased/ fragment records the change
  • No credentials, private server data, machine-local paths, or generated output are included

Desktop tests on Windows 11 (10.0.26200), JDK 21, all with nothing skipped:

Suite Tests
WindowsCloudFilesHydrationTest (new) 5
DesktopFileRangeTransportTest (new, MockWebServer) 4
WindowsCloudFilesProviderTest 75
WindowsCloudFilesDehydrationTest (real sync root, hydrates real placeholders) 6
WindowsCloudFilesStartupTest 7

The new tests check:

  • WindowsCloudFilesHydrationTest: a whole-file request makes exactly one backend request and delivers aligned chunks with exact content; an interior request covers only its aligned span; the non-streaming fallback still delivers the same bytes; cancellation stops at a chunk boundary; a short chunk fails before it is transferred.
  • DesktopFileRangeTransportTest: one request carries the Range and If-Match headers and delivers chunks in order; bounded reads still work; HTTP 412, a mismatched Content-Range, a truncated body and an oversized body each fail before any chunk is delivered, and each failure is recorded; a closed transport makes no request.

Also run: ./gradlew :androidApp:compileDebugKotlin succeeds, and bash tools/check-kotlin-architecture.sh passes.

The FETCH_DATA pattern above came from a temporary probe on a real sync root. The probe is not included in this PR.

Not run:

  • Hydration against a real Nextcloud server, and a packaged build.
  • Android unit or instrumented tests. Android only gains an unused default for stream.

Compatibility and risk

  • Windows Cloud Files hydration only. Bounded range reads keep the same validation.
  • A failure in the middle of a stream fails the FETCH_DATA request with the bytes delivered so far. This is the same as a failure on a later chunk before this change. Windows can retry the read.
  • One open range response per in-flight FETCH_DATA, as before. The provider executor still runs at most 4 at a time.
  • Not yet validated end to end against a real server. The expected speedup comes from removing per-chunk round trips; it has not been measured on a real server.

Visual changes

Not applicable.

Opening a cloud-only file makes Windows issue one FETCH_DATA for the
whole file. The provider split it into 4 MiB chunks and made a separate
ETag-pinned range request per chunk, buffering each response before
handing it to CfExecute. Every chunk paid a full server round trip, and
servers that must seek to serve a range paid more for each later chunk,
so hydration ran far slower than an ordinary download.

The aligned span of each request is now fetched with one streamed
range response and transferred in 1 MiB chunks as bytes arrive.
Cancellation is still checked between chunks. The range transport moves
out of DesktopNextcloudServices into DesktopFileRangeTransport, which
serves both the existing bounded reads and the new stream with the same
If-Match, exact Content-Range and length checks. Hydration planning and
the read-handle contract move into WindowsCloudFilesHydration.kt.
NextcloudFileRangeSession gains a stream call that falls back to bounded
reads on platforms without streaming.
@obiente-cloud
obiente-cloud Bot temporarily deployed to Obiente Preview / PR #519 / NC Native October 19, 2026 04:40 Destroyed
@obiente-cloud

obiente-cloud Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Obiente preview

NC Native · 3133473443da · Removed

View preview status

View in Obiente

Obiente updates this comment as the preview changes.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-19T04:52:55.701538Z 3133473 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@obiente-cloud
obiente-cloud Bot temporarily deployed to Obiente Preview / PR #519 / NC Native October 19, 2026 04:42 Destroyed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 71e748dd88

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@obiente-cloud
obiente-cloud Bot temporarily deployed to Obiente Preview / PR #519 / NC Native October 19, 2026 04:50 Destroyed
CANCEL_FETCH_DATA only set a flag that was checked after the next
1 MiB chunk arrived, so a slow or stalled response kept a cancelled
request's connection and provider worker busy until another chunk or
a network timeout.

Each callback request now has a WindowsCloudRequestCancellation. Its
cancel() also closes the read handle the request has open, which
cancels the in-flight HTTP call; a request cancelled before it starts
never opens a stream. Provider shutdown cancels in-flight hydrations
the same way.
@obiente-cloud
obiente-cloud Bot temporarily deployed to Obiente Preview / PR #519 / NC Native October 19, 2026 04:50 Destroyed
@veryCrunchy
veryCrunchy merged commit 6f7c2f9 into main Oct 19, 2026
5 of 6 checks passed
@veryCrunchy
veryCrunchy deleted the perf/windows-cloud-files-hydration branch October 19, 2026 05:01

This branch was successfully deployed

No deployments
Obiente Preview / PR #519 / NC Native — 31334734 Deployed Oct 19, 2026 by obiente-cloud[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant