Skip to content

perf: speed up storage and cleanup scans in Swift - #469

Open
xcv58 wants to merge 1 commit into
mactools-app:mainfrom
xcv58:codex/swift-filesystem-scan-performance
Open

xcv58 wants to merge 1 commit into
mactools-app:mainfrom
xcv58:codex/swift-filesystem-scan-performance

Conversation

@xcv58

@xcv58 xcv58 commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

Problem and result

Folders with many small files make storage and cleanup scans spend substantial time on per-entry metadata queries and temporary objects. Stream bounded metadata batches in Storage Explorer, reuse bulk metadata sizing in Xcode Clean, and use bulk discovery in Disk Clean to reduce scan time and wide-folder memory use.

Changes

  • Add FileSystemDirectoryReader.readBatches while retaining the collecting API. Unsupported bulk reads fall back before delivery; later errors propagate without restarting enumeration or duplicating earlier batches.
  • Keep compact scan metadata in Storage Explorer and construct URLs and public rows when publishing or retaining them. Preserve deterministic hard-link accounting, snapshot serialization, package handling and manual refresh behavior; cap directory-cache retention at 10,000 entries.
  • Use the shared reader for Xcode Clean sizing, preserving native package boundaries and root and child symlink exclusions. Skip dataless directories and check cancellation after sizing. Bundle the static filesystem module through the plugin's project configuration.
  • Reuse Disk Clean's existing bulk entry source for installer and developer-artifact discovery. Anchor unsupported-volume fallback to the original open directory, report metadata failures, and reject installer entries that are no longer regular files when revalidated. Keep project markers, depth and mount checks, allowlists and pre-removal validation.

User-visible behavior and scope

No UI layout, copy, settings, or scan-trigger changes. Filesystem scans remain off the main thread, with progress and results delivered through the existing controllers. Cleanup categories, default selections, permission requirements, allowlists, and pre-removal validation are unchanged.

Users may see faster completion and earlier progress updates. This also includes limited changes to scan results and error handling:

  • Disk Clean reports unresolved metadata errors as unavailable or permission-denied results instead of silently skipping them, and excludes installer candidates that become symlinks or other non-regular files during discovery.
  • Xcode Clean skips dataless cloud-placeholder directories when calculating size and propagates cancellation after sizing.
  • Storage Explorer retains readable entries and marks results incomplete when a directory read fails after delivering a batch.

Verification

  • make ci: 228 script tests and 2,706 XCTest tests passed (four opt-in Window Switcher UI tests skipped); PluginKit v7 binary compatibility client passed.
  • All nine Storage Explorer progress tests passed after integrating current main, including skipped-folder warning accounting and retained snapshot updates.
  • Changelog validation and git diff --check passed.
  • Focused coverage verifies bounded batches, cancellation, snapshot round trips, package and symlink behavior, permission errors, unsupported-volume fallback, depth limits and mount protection.

Recorded Release Swift 6 benchmarks on local APFS against baseline d914bfbe, with five measured samples per variant after warmup:

Operation Fixture Before After Speedup
Storage Explorer 500k files across 1,000 folders 2,367 ms 713 ms 3.3x
Storage Explorer 100k files in one folder 658 ms 229 ms 2.9x
Xcode Clean 500k files across 1,000 folders 27,982 ms 1,207 ms 23.2x
Xcode Clean 100k files in one folder 4,920 ms 101 ms 48.8x
Disk Clean installer discovery 100k files 1,225 ms 301 ms 4.1x
Disk Clean developer discovery 100 projects / 100k ordinary files 476 ms 153 ms 3.1x

Storage Explorer's peak RSS for the wide fixture fell from 161.0 to 32.9 MiB. Its benchmark used six workers, progress-only snapshots and a 10,000-file retention limit. File totals, retention bounds and per-directory largest-file sizes matched; Disk Clean candidate fields matched. These are synthetic metadata scans of empty or sparse files with warm caches; they do not measure payload throughput or production UI latency. Raw samples and source snapshots remain in ignored local benchmark artifacts.

UI evidence

No UI layout or copy changes. A manual UI smoke test was not performed. Cleanup execution was not exercised against user files; validation used temporary directories and fake stores.

Checklist

  • Core behavior and relevant risks are verified using existing tests, focused additions, or manual checks; results or limitations are recorded.
  • UI evidence is attached where applicable; shared components, themes, localization, and accessibility are respected.
  • User-facing documentation and changelog fragments are updated when required.
  • Plugin metadata, runtime contracts, action policies, and minimum-host compatibility agree.
  • Licensing and third-party notices are complete; no credentials, local configuration, or unrelated generated files are included.

@xcv58
xcv58 marked this pull request as ready for review October 7, 2026 02:24

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant