Use App Management API instead of browser for E2E teardown checks - #8316
Draft
isaacroldan wants to merge 5 commits into
Draft
Use App Management API instead of browser for E2E teardown checks#8316isaacroldan wants to merge 5 commits into
isaacroldan wants to merge 5 commits into
Conversation
The E2E teardown drove the Dev Dashboard and store admin UI for work the App Management API can answer directly: - App lookup: replace the browser pagination in findAppOnDevDashboard with appByKey (client_id) and an appsConnection title search fallback. - Install gating: replace the isStoreAppsEmpty page scrape and the disabled-Delete-button probing with installCount polling. The same check gates both store deletion and app deletion. - Store uninstall: drop the browser click-through fallback; the Admin API path (from #8309) is now the only one in teardown. The browser is only used for the final delete-app click, which has no API mutation. The cleanup scripts keep their browser paths: org-wide sweeps have no local app dir to mint Admin API tokens from. API calls reuse the worker's CLI session via cli-kit, following the cleanup-stores.ts pattern. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
github-actions
Bot
added
the
no-changelog
Importing cli-kit's dist ESM inside Playwright's transpiled harness
crashes Node's require(esm) path on CI's Node version ("Unexpected
module status 3"). tsx's loader handles the interop — the same reason
the cleanup scripts import cli-kit under tsx without issues.
This also removes the process.env XDG mutation: the session dirs are
passed to the subprocess environment directly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The App Management API resolves the request's organization from specifically-named GraphQL variables (organizationId, apiKey, appId). The lookup variable was named "key", so every teardown lookup failed with 404 "Cannot find a valid organization" and app/store deletion was skipped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- appByKey now sends organizationId: without it a deleted app cannot
resolve an organization and 404s instead of returning null.
- Client IDs are only taken from /apps/{segment} URL parts that are
non-numeric; deploy output yields /apps/{numericAppId} URLs, which
now fall back to a name search.
- The settings-page navigation clicks through the accounts.shopify.com
account picker, which cold browser contexts bounce to — the main
reason direct-URL app deletion has been failing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
isaacroldan
force-pushed
the
isaac/e2e-teardown-via-api
branch
from
August 12, 2026 14:21
c6d8bce to
69bb1f6
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| await page.waitForTimeout(BROWSER_TIMEOUT.medium) | ||
|
|
||
| await navigateToDashboard({browserPage: page, email, orgId: org}) | ||
| if (!page.url().startsWith('https://accounts.shopify.com')) return |
Settings pages under client-key URLs usually render without the Delete button; the numeric-id form (what the dashboard links to and the CLI's appDeepLink builds) works reliably — the run-level cleanup deletes 11/11 apps with it while key-form teardown deletes went 2/13. The app GID from the API lookup provides the numeric id. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Removes most Playwright usage from the E2E per-test teardown. The browser was driving the Dev Dashboard and store admin UI for work the App Management API answers directly:
findAppOnDevDashboardpaginated the dashboard UI by clickingnext_cursorlinks. Replaced withappByKey(client_id from the app TOML or the app URL) plus anappsConnectiontitle-search fallback. An API "not found" now authoritatively means "already deleted".isStoreAppsEmptypage scrape (before store deletion) and the disabled-Delete-button probing (before app deletion, the source of the constantscrollIntoViewIfNeeded: Timeout 5000msnoise in CI logs) are both replaced by oneinstallCountpoll after uninstall.appDiris missing the test failed beforecreateApp, so there is nothing to uninstall.Implementation notes:
setup/app-management-api.tsmakes minimal raw GraphQL requests (same style assetup/admin-api.ts). The API resolves the request's organization only from specifically-named variables (organizationId,apiKey,appId) — verified againstGraphQLOrganizationResolutionin core.setup/get-app-management-token.ts): importing cli-kit's dist inside Playwright's transpiled harness crashes Node'srequire(esm)path on CI's Node version.Out of scope, intentionally:
cleanup-apps.ts/cleanup-stores.tskeep their browser paths — org-wide sweeps have no local app dir to mint Admin API tokens from.Testing
pnpm tsc --noEmitand eslint pass inpackages/e2e.installCountgate →store deletion confirmed by CLI.apiKey/organizationIdvariable fixes; zeroCannot find a valid organizationerrors.🤖 Generated with Claude Code