[skycap] Follow-ups after the Harbor integration #2318

Follow-ups after the SkyRL Harbor integration (#2307). Plan: #2306.

Training

• R3 with step-wise output ([skycap] R3 through skycap: step-wise rows carry their own routes #2343, validated on Qwen3-30B-A3B: logprob gap 0.0113 vs 0.0232). _validate_per_token_side_channels refuses routed experts with step-wise output, so HarborSkycapGenerator refuses enable_return_routed_experts. skycap already records the experts; the trainer check is what needs to change.
• React to unexpected forks in the generator. generate/num_unbridged_trajectories and generate/num_unbridged_calls report them per step. A harness that should be append-only could get a policy: zero the reward, or mask the trajectory.
• One row per trial (final_path), as an alternative to one row per path.
• Response truncation to max_seq_len.
• A threshold for timeout masking.
• Less harsh masking. Today any error masks the whole instance, which costs a lot at small prompt counts.

Validation

• A smoke run that writes bridged. Every existing record predates [skycap] Record whether each token-mode call extended the previous tokens #2315, so the per-call bridged field and the unbridged metrics haven't been checked on real records.
• Decoding parity for use_raw_content. The raw-content text should match vLLM's with and without skip_special_tokens.

Tooling and docs

• [ ] A linearity check (skycap check). Given a harness, report whether a history that should be linear stays one path, and the cause of each fork. Candidates for the docs as the reference validation. belongs to skycap-viewer (see [skycap] skycap check: report whether a record stays on one path, and why each fork happened #2440 (comment))
• One programmatic constructor for embedders ([skycap] CaptureService: embed a server from options, with the backend built inside #2321: skycap.CaptureService). Something like skycap.tokens.build_server(upstream_url, tokenizer, engine=..., **the serve flags), shared with skycap serve. An embedder such as SkyRL would name only its engine instead of building TokensBackend and RenderersRenderer itself, and the CLI and embedders couldn't drift. Backend would then be explicitly private. See [skycap] CaptureService: embed a server from options, with the backend built inside #2321.
• Push skycap-viewer to NovaSky-AI. It's a Node project, built and tested locally against the records of the parity runs, and its path, target and token counts match skycap's Python reader on 4,316 records. Its token view isn't virtualized yet.
• skycap/docs/format.md fixes, found while writing the viewer (the spec's second implementation):
  • shadowed_by is described backwards for token mode: later history continues from the shadowed node. graph.py's docstring has the same wording.
  • Path order (leaves in id order) and training targets (a model node trains in the first path containing it) are only in samples.py.
  • An all-empty sampling_mask sidecar is dropped while nodes keep mask_offset: 0, mask_rows: 0. That contradicts "a document that exists always has its sidecars".
  • Require exactly one zstd frame per file. Node's zstd decoder silently drops frames after the first.
  • The JS example ignores a Node Buffer's byteOffset and alignment.
  • Smaller gaps:
    • logprobs are NaN on scaffold when has_logprobs is false;
    • sidecars is {} for empty token-mode trajectories;
    • capture.raw_content isn't listed.

Scale and infrastructure

• Size the skycap server pool (num_servers, CPUs per server) at full scale.
• A fully async entrypoint.

On hold

• Unclosed thinking in renderers. When a Qwen3 turn's thinking never closes, re-rendering it drops the thinking, and the history forks. Measured on the 4B parity run: 27 of 1,280 trajectories (143 turns), all reward 0, 0.55% of trained tokens. This belongs in the renderer, not in skycap.

Cleanup

• Retire the old copies: decide what happens to the standalone NovaSky-AI/skycap repo and the older skyrl-capture/ prototype.

Planned: a central skycap integration in SkyRL, shared by every harness

About 85% of examples/train_integrations/harbor_skycap/ isn't Harbor-specific: the server pool, SkyRLEngine, the composer (step-wise rows, R3 routes, masking and overlong policy), the retry and missing-routes checks, the record index (#2351) and the config. Harbor appears only in _trial_config and the ~15 lines of _attempt that run a Trial and map its result to a reward. Move the shared part into SkyRL so any OpenAI-client harness gets skycap capture by writing one method.

Layout (needs SkyRL to depend on skycap through an optional skycap extra):

skyrl/train/integrations/skycap/
  config.py      SkycapConfig under cfg.skycap (servers, placement, record_dir, record_mirror(_config),
                 use_raw_content, renderer_pool_size, ttl, wandb{enabled, phases})
  servers.py     the Ray actor pool + SkyRLEngine
  compose.py     FinishResult samples -> step-wise GeneratorOutput, metrics, R3, masking/overlong policy
  generator.py   SkycapGenerator: the shared rollout loop
  index.py       SkycapRecordIndex callback + RecordLog

The interface a harness implements:

@dataclass
class Rollout:
    reward: float
    stop_reason: str = "complete"   # agent_timeout / error mask the instance; context_length trains with 0
    annotations: dict = field(default_factory=dict)

class SkycapGenerator(GeneratorInterface):
    async def run_rollout(self, prompt, *, base_url: str, api_key: str, sampling: dict) -> Rollout: ...
    # shared: CapturePool, rate limit, retries on a fresh trajectory, cache_salt, finish,
    # captured-nothing / missing-routes checks, RecordLog per phase, compose with R3 and sampler support.

Harbor becomes a subclass whose run_rollout runs a Trial and maps its result (~40 lines). verifiers (already OpenAI-compatible), openreward and openenv would follow the same shape.

Lifecycle in the trainer: with cfg.skycap.enabled, BasePPOExp:

• starts the pool before get_generator and passes the URLs to the generator;
• stops it at the end, which drains the records and the mirror;
• registers SkycapRecordIndex when trainer.logger=wandb.

Entry points stop managing skycap. Config checks live in one place:

• step_wise_trajectories=true and merge_stepwise_output=false are required;
• R3 is allowed ([skycap] R3 through skycap: step-wise rows carry their own routes #2343);
• record_mirror_config needs record_mirror;
• the index is on by default only with the W&B logger.

Layers:

Plan:

• [skycap] Mirror records to remote storage, and say where each is at finish #2350 (record mirror) and [skycap] Index each step's skycap records in W&B #2351 (W&B index, written with no Harbor dependency so it moves as is) land first.
• Move PR: move the shared files into skyrl/train/integrations/skycap/, add the skycap extra, and extract SkycapGenerator from HarborSkycapGenerator. No behaviour change: the Harbor tests run unchanged against the subclass.
• Trainer hook PR: cfg.skycap.enabled starts and stops the pool and registers the index; Harbor's HarborSkycapExp and start_skycap go away.
• A second harness on SkycapGenerator (likely verifiers), to prove the interface is shared.
• Decide: cfg.skycap top-level (preferred: the pool and the index are trainer-lifecycle concerns) or cfg.generator.skycap.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Jpzm4Ap4fvWY7NtJWsFDgb