[Test] Baseline benchmarks for async collectors and DreamerV3

[vmoens]

No description provided.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/4312

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[vmoens]

Branch validation run with the current head (skip-upload): https://github.com/pytorch/rl/actions/runs/34331912977. Reporting validation, CPU and GPU benchmark jobs and the summary step all passed; 31 series per repeat on GPU, three fresh-process repeats.

New series on GPU (median of the three run medians, run range, execution recorded in the JSON):

Series	frames/s	Run range	Execution
test_async_collection_pixels[async-process-slots-integrated-uniform]	991.7	989.5-998.5	graph; 1 envs/worker; process acting
test_async_collection_pixels[async-process-slots-integrated-slow-reset]	995.2	991.9-996.1	graph; 1 envs/worker; process acting
test_dreamer_v3_async_training[thread-ratio2]	85.2	84.8-85.6	eager learner on cuda:0; thread inference; train ratio 2; 8 envs; inference batch <= 1; 0.33 learner updates/s
test_dreamer_v3_async_training[thread-ratio16]	53.1	53.1-53.7	same with train ratio 16; 1.66 learner updates/s

async-process-slots-uniform (eager) measured 995.9 frames/s in the same run, so the integrated series matches the fixed one on this revision as intended. Run-to-run spread of the new series is about 1 percent. The process DreamerV3 series skipped as designed (no collector.inference_backend on this revision).

Cost: each pytest repeat took 17.3 minutes on the GPU runner; the two DreamerV3 series account for 105 s (ratio2) and 156 s (ratio16) of that, about 13 minutes per job over the three repeats. Startup and shutdown of the example are about 20 s per test; the rest is the seven 1,024-step rounds. With inference limited to one request per batch, the ratio2 series moves about 85 frames/s across 8 environments, that is roughly 90 ms per environment step, so its level is set by per-request policy latency rather than by the batched pipeline.