Repository navigation
feat(ai): graduate video generation with start, status, result and cancel - #63657
Conversation
There was a problem hiding this comment.
The new video job API looks sound to me: start returns a token, then status, result and cancel each read the job from that token, and queued routes get a new attach. Unit tests and type checks pass. There are no xAI credentials, so I couldn't run the flow against the real provider.
One regression, in streamed file writes (inline comment): when a download fails, the target file has already been truncated. On the base, a 404 left an existing file untouched. With this PR the file ends up empty. I checked Media.write against both branches. The Promise ai.write opens the file with "w" before downloading, so it should behave the same, though I didn't run it.
Public API impact (@opencode/ai). Breaking for anyone using ai.experimental.video; it was experimental:
- Removed:
ai.experimental.videoin both the Effect and Promise clients, along withgenerate,stream,resumeand the live handle for video. Removed from@opencode/ai/experimental:Video,VideoClient,VideoAspectRatio,VideoModel,VideoModelSchema,VideoResponseand the video types.VideoEventandVideoFramesare gone entirely. - Added:
ai.video.{request,start,status,result,cancel}in both clients. New exports:VideoClient(Effect entry),VideoDuration,VideoJob,VideoStatus, plus the video exports above moved to the main entry. AlsoMedia.Asset.stream()and a requiredattachonMediaRoute.QueuedRoute. - Changed request fields:
frames.first/frames.last→start/enddurationSeconds: number→duration: "N seconds"audio→soundaspectnow also accepts"auto"referencesaccepts any media kindn,seed,negativePromptandvideoare removed (useproviderOptions)
- xAI edit and extend now come from
providerOptions.modeplusproviderOptions.video, instead of the request'svideofield. Media.writeand Promiseai.writenow stream URL downloads to disk instead of buffering them; see the regression above.
| .writeFile(path, data) | ||
| .pipe(Effect.mapError((cause) => invalid(`Failed to write media file ${path}`, cause))) | ||
| yield* asset.stream().pipe( | ||
| Stream.run(fs.sink(path)), |
There was a problem hiding this comment.
fs.sink(path) opens the target with "w" before the download starts. A failed download (for example a 404 from an expired video URL) or a mid-stream error now truncates an existing file. I checked this: on the base a 404 left the file's bytes alone, and with this PR it ends up empty. The Promise ai.write in src/index.ts calls open(path, "w") before streaming too, so it should behave the same. Streaming into a temporary file next to path and renaming it into place on success would keep the old behaviour without buffering.
There was a problem hiding this comment.
Fixed in fc6528a: both Media.write and the Promise ai.write now stream into a temporary file next to the target and rename it into place only after the download finishes, removing it on failure. A 404 leaves the existing file untouched; tests cover both writers.
There was a problem hiding this comment.
I checked fc6528a and the fix is correct.
Media.write(media.ts~376–392): it writes to${path}.<uuid>.tmp, then renames that file ontopath.Effect.onErrordeletes the temp file on any failure, including interruption, and ignores errors from that delete. The originalAIError(for example the 404) is still passed through.- Promise
ai.write(index.ts~207–224): it works the same way. The file handle is closed in the innerfinallybefore the rename or the delete runs.rm(..., { force: true })is safe even whenopenfailed. Abort reasons andAIErrorare rethrown as they are, and anything else is wrapped as a file error. - Temp file location: it sits next to the target, so the rename stays on one filesystem and replaces the old file in a single step.
What I ran: the shared checkout is still on 3f25c4b, so I ran test/media.test.ts and test/promise.test.ts in a temporary worktree of pr/head. All 28 tests passed, including the new 404 cases. I didn't run the full suite or typecheck.
Left alone: if the process is killed during a download (for example SIGKILL), a stray *.tmp file can stay behind. That's normal for this pattern and I don't think it's worth handling.
I didn't change anything. The temporary worktree is removed, and the bun.lock modification in repo/ was already there, so I left it.
There was a problem hiding this comment.
Your new commits fix the file-write problem from my last review. Media.write and the Promise ai.write now stream into a temporary file next to the target, rename it into place only once the download finishes, and delete it on failure. A failed download leaves an existing file as it was, and the two new tests cover that. The touched test files and the type checks pass. I found nothing new.
The public API impact is the same as in my earlier review. These commits only change how the two write functions save the file.
Issue for this PR
No issue. Follows #63634, which removed the dead Gemini Veo and Runway video protocols.
Type of change
What does this PR do?
Video generation in
@opencode/aimoves out ofexperimentaland becomes a job you start and then check on yourself, instead of a call that blocks and polls for you. That's what opencode core needs: it decides when to poll, and it can pick a job back up after a restart from a stored token.The Promise client has the same calls under
ai.video. Every call afterstarttakes(model, token), so recovering a job is the same as reading it normally.What changed in the request:
frames: { first, last }is nowstartandend.referencestakes any media kind; protocols route images, video and audio to their own fields.durationSeconds: 10is nowduration: "10 seconds"(an Effect duration string, seconds only for now).aspectalso accepts"auto"(follow the start image),resolutiondocuments480p | 720p | 1080p | 4kand passes anything else through, andaudiois nowsound.n,seed,negativePromptand the sourcevideofield are gone from the shared request. They're on too few providers; useproviderOptions.Removed:
generate,stream,resumeand the live handle (await,events,refresh) for video. There's no built-in polling loop and no queue position, since no video provider reports one. Other modalities keepGenerationas is.Enforcement: the package only rejects locally what a protocol has no way to send, such as a reference video on xAI. Values a provider may reject (durations, resolutions, reference counts, mixing frames and references) go to the provider and its error comes back typed. Spelling and routing differences are handled silently by each protocol.
How it works:
MediaRoute.queuedgetsattach(model, token, execute), which decodes a token into the route's status, result and cancel operations without a status read first.resumenow uses it too.VideoClientis its own small service (start,status,result,cancel) on top of that, andVideoModelonly accepts queued routes.starttoimage,endtolast_frame, image references toreference_images, audio references toreference_audios,soundtogenerate_audio, and leavesaspect_ratioout for"auto"when a start frame sets the shape. Edit and extend stay reachable withproviderOptions: { mode, video: { url } }.Media.Assetgetsstream(), andMedia.writeand the Promiseai.writenow stream URL downloads to disk instead of buffering whole files.Trade-offs:
headers/httpoverlay only applies tostart. Status and result reads are rebuilt from the token, so they use the model's deployment settings, which is what a resumed job in another process gets anyway.ai.experimental.videois removed without aliases, since it was experimental. The release notes should mention thatai.videograduated with this shape.No feature flag: this is a library API with no opencode runtime behavior yet.
How did you verify your code works?
bun typecheckandbun run typecheck:tscinpackages/aibun testinpackages/ai: 2073 pass, 0 failbun run checkat the rootstart, thenstatusuntil terminal, thenresult, and still replays the existing cassette: the request body is unchanged."auto"aspect, edit and extend via provider options, rejecting reference videos and non-second durations before sending, token round trips and foreign tokens, status mapping and progress, result on pending, failed, expired and moderated jobs, unsupported cancel, read retries (503, 429retry-after, no retry on 400) andMedia.Asset.stream.Checklist
Added by the opencode bot (AI-generated).
Public API: graduates video to
ai.video(start/status/result/cancel), removingai.experimental.videoand renaming the request fields (breaking for experimental callers)