Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/.vuepress/sidebars/en.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ export const enSidebar: ThemeSidebarMulti = {
items: [
{ text: 'MinerU Scale — Real workload', link: 'mineru-scale' },
{ text: 'MinerU PDF', link: 'mineru' },
{ text: 'Flash-MinerU application', link: 'flash-mineru' },
{ text: 'Panda-70M Video Caption', link: 'panda70m' },
{ text: 'YOLO → SAM', link: 'yolo-sam' },
{ text: 'Dual vLLM', link: 'dual-vllm' },
Expand Down
1 change: 1 addition & 0 deletions docs/.vuepress/sidebars/zh.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ export const zhSidebar: ThemeSidebarMulti = {
items: [
{ text: 'MinerU Scale(真实负载)', link: 'mineru-scale' },
{ text: 'MinerU PDF', link: 'mineru' },
{ text: 'Flash-MinerU 应用', link: 'flash-mineru' },
{ text: 'Panda-70M 视频描述', link: 'panda70m' },
{ text: 'YOLO → SAM', link: 'yolo-sam' },
{ text: '双 vLLM', link: 'dual-vllm' },
Expand Down
31 changes: 30 additions & 1 deletion docs/en/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,17 @@ config:
background: tint-plate
hero:
name: RayOrch
tagline: Run every stage as soon as its data is ready.
tagline: Write multimodal data pipelines like programs. Scale them across Ray.
text: A completion-driven dataflow runtime for multi-stage, multi-model AI workloads on Ray.
actions:
-
theme: brand
text: Get Started
link: /en/guide/
-
theme: alt
text: See 1 → M → 1
link: /en/guide/fan-out-and-reduce.html
-
theme: alt
text: API Reference
Expand All @@ -24,3 +28,28 @@ config:
text: GitHub →
link: https://github.com/OpenDCAI/RayOrch
---

## RayOrch in one minute

Many multimodal workloads repeat the same shape: one PDF becomes pages, one video becomes frames, or one image becomes regions; a model processes those children; the children are assembled back into one result. This is the `1 → M → 1` pattern.

RayOrch makes that relationship explicit in Python. You write batched UDFs, declare `F.expand` and `F.reduce`, and keep parent ownership, child order, readiness, and failure scope while Ray handles actors, resources, placement, and RPCs.

```text
PDF ──► pages ──► OCR / VLM ──► document
video ──► frames ──► vision model ──► summary
```

## Follow the path that matches your question

- **I want to run code:** start with [Installation](guide/installation.md) and [Your First Pipeline](guide/first-pipeline.md).
- **I need variable-cardinality data:** read [Fan-out and Ordered Reduction](guide/fan-out-and-reduce.md) and [Cardinality and Lineage](concepts/cardinality.md).
- **I need to understand the runtime:** read [Framework Design](architecture/), then [Runtime Architecture](architecture/runtime.md).
- **I want a complete application:** run the [MinerU benchmark](benchmarks/mineru.md), then read the [Flash-MinerU application guide](benchmarks/flash-mineru.md) and its [upstream repository](https://github.com/OpenDCAI/Flash-MinerU).
- **I want reproducible numbers:** use [Benchmarks](benchmarks/) and read [Interpret Performance Results](benchmarks/performance.md) before comparing runs.

## What stays stable

A physical batch can mix ready children from different parents. It is an execution detail, not the source of truth. RayOrch keeps the logical lineage stable so batching, placement, and retry timing can change without changing which output belongs to which input.

Read the [boundaries](guide/boundaries.md) before adopting RayOrch for a single function, an unbounded stream, or an operator that depends on global cross-row state.
3 changes: 3 additions & 0 deletions docs/en/benchmarks/built-ins.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ Built-in Benchmarks now have dedicated pages. Every page includes a workload exp
- [Dual vLLM](dual-vllm.md): generation and refinement through two persistent LLM engines.
- [SGLang → vLLM](sglang-vllm.md): one Pipeline whose inference backends run in separate Conda environments.

## Application integrations

- [Flash-MinerU](flash-mineru.md): a maintained PDF → pages → GPU VLM/OCR → ordered document application, with the current pipeline-parallel benchmark context.
## Dependency-free topology references

- [Nested Document](document-topology.md): two levels of Document → Page → TableJob expansion and reduction.
Expand Down
73 changes: 73 additions & 0 deletions docs/en/benchmarks/flash-mineru.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# Flash-MinerU application

[Flash-MinerU](https://github.com/OpenDCAI/Flash-MinerU) is the clearest end-to-end application of the RayOrch programming model today. It keeps MinerU's parsing and output logic, while using a RayOrch pipeline to overlap PDF rendering, VLM/OCR inference, and document assembly.

## The shape of the workload

A PDF is not one model request. It has the same variable-cardinality shape used throughout RayOrch:

```text
PDF ──► pages ──► GPU VLM/OCR ──► ordered Markdown / JSON / images
1 M 1
```

The page count depends on the input. Ready pages from different PDFs can share a model batch, while each PDF keeps its own page order and output directory.

## Minimal API

Install Flash-MinerU and its inference backend in the workload environment:

```bash
pip install flash-mineru
# or: pip install "flash-mineru[vllm]"
```

The maintained pipeline is exposed through `MineruEngine`:

```python
from flash_mineru import MineruEngine

engine = MineruEngine(
model="/shared/models/MinerU2.5-2509-1.2B",
batch_size=16,
replicas=8,
num_gpus_per_replica=0.9,
save_dir="outputs_mineru",
inflight=4,
)

outputs = engine.run(["paper-a.pdf", "paper-b.pdf"])
```

`batch_size` controls the logical PDF batch accepted by the application. `replicas` controls the number of persistent model instances, and `inflight` controls how many pipeline batches can overlap. The exact settings depend on model memory, GPU count, and storage throughput; start with the defaults in the [Flash-MinerU README](https://github.com/OpenDCAI/Flash-MinerU#-quickstart). `inflight` is Flash-MinerU's application-level pipeline depth; RayOrch's generic `max_active_input_batches` controls overlapping input lifecycles in the lower-level Benchmark API.

## What RayOrch contributes

| Concern | RayOrch / Flash-MinerU behavior |
| --- | --- |
| Variable cardinality | one PDF expands into its actual page list with `F.expand` |
| Model execution | ready pages are sent to persistent GPU actor pools |
| Cross-input batching | pages from different PDFs can share a physical model batch |
| Reconstruction | page results return to their owning PDF and original index with `F.reduce` |
| Stage overlap | render, VLM/OCR, and assembly can progress on different in-flight batches |
| Output contract | MinerU's Markdown, layout JSON, and image layout remain the application output |

RayOrch owns the logical relation and readiness rules. Ray still owns actor placement, resources, object transport, and cluster scheduling. Flash-MinerU owns the MinerU-specific model and artifact logic.

## Reported result

The current Flash-MinerU README reports this single-host comparison:

| Configuration | Corpus | Result |
| --- | --- | --- |
| Flash-MinerU pipeline-parallel path | 368 PDFs, 8 × NVIDIA A100 | about **8.5 min** |
| Eight-process MinerU baseline | same corpus and host | about **14 min** |

That is roughly **1.7× faster** for this configuration. It is a workload measurement rather than a guarantee for every model, PDF collection, storage system, or GPU. Re-run the [Flash-MinerU benchmark scripts](https://github.com/OpenDCAI/Flash-MinerU/tree/main/docs) when comparing versions or hardware.

## How to use this page

- Learn the generic pattern first: [Fan-out and Ordered Reduction](../guide/fan-out-and-reduce.md).
- Understand the runtime boundary: [Completion-driven Runtime](../architecture/runtime.md).
- Run the dependency-free topology case before installing model dependencies: [Nested Document Benchmark](document-topology.md).
- Then follow Flash-MinerU's own [quickstart](https://github.com/OpenDCAI/Flash-MinerU#-quickstart) and [benchmark guide](https://github.com/OpenDCAI/Flash-MinerU/blob/main/docs/BENCHMARK.md).
4 changes: 4 additions & 0 deletions docs/en/benchmarks/mineru.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ For local/HDFS multi-root input, fractional GPU actor layouts, atomic document
commits, resume, and the validated large-scale configuration, use
[`MinerUScaleBench`](mineru-scale.md).

## Flash-MinerU: the maintained application path

[Flash-MinerU](https://github.com/OpenDCAI/Flash-MinerU) applies the same shape to a maintained MinerU integration: PDF → pages → GPU VLM/OCR → ordered document assembly. Its current pipeline-parallel path reports about **8.5 minutes for 368 PDFs on one 8×A100 host**, compared with about **14 minutes** for its eight-process MinerU baseline (roughly **1.7×** in that setup). Read the [Flash-MinerU benchmark notes](https://github.com/OpenDCAI/Flash-MinerU#-benchmark) for the dataset and commands. These figures are workload measurements, not universal guarantees.

## Topology

```mermaid
Expand Down
12 changes: 12 additions & 0 deletions docs/en/concepts/cardinality.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,18 @@

Cardinality describes how many logical items exist and how they relate to their parents. RayOrch makes those relationships explicit with `rayorch.F`.

## The recurring `1 → M → 1` shape

This pattern appears whenever one record expands into an input-dependent number of work items and later returns to one parent result:

| Workload | Expansion | Model stage | Reduction |
| --- | --- | --- | --- |
| PDF understanding | document → pages or regions | OCR / VLM | pages → document |
| Video understanding | video → frames or clips | vision / caption model | frames → summary |
| Image segmentation | image → detected regions | segmenter | regions → annotated image |

The number of children is data-dependent. The runtime therefore keeps two facts separate: **logical membership** (which parent owns a child and where it belongs) and **physical batching** (which ready children happen to share one UDF call). `expand` and `reduce` declare the first; the scheduler can optimize the second.

## `expand`: one parent becomes many children

The UDF returns one list of children for each input row. `expand` moves from the parent domain into the child domain.
Expand Down
6 changes: 6 additions & 0 deletions docs/en/concepts/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,3 +53,9 @@ my_workload/
```

Start with these three files. Add configuration machinery only when the experiment becomes reusable.
## Continue from here

- Need a variable number of children? Read [Cardinality and Lineage](cardinality.md).
- Need to understand dropped, failed, and suppressed outputs? Read [Results, Failures, and Recovery](results-failures.md).
- Need to tune actors and batches? Read [Resources and Batching](../distributed/resources.md).
- Need to run across machines or environments? Read [Distributed Execution](../distributed/) and [Cross-environment Stages](../distributed/cross-environment.md).
31 changes: 30 additions & 1 deletion docs/zh/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,17 @@ config:
background: tint-plate
hero:
name: RayOrch
tagline: 数据一旦就绪,立即运行下一阶段。
tagline: 像写程序一样构建多模态数据管线,随 Ray 集群扩展。
text: 构建在 Ray 之上的完成驱动数据流运行时,面向多阶段、多模型 AI 工作负载。
actions:
-
theme: brand
text: 快速上手
link: /zh/guide/
-
theme: alt
text: 先看 1 → M → 1
link: /zh/guide/fan-out-and-reduce.html
-
theme: alt
text: API Reference
Expand All @@ -24,3 +28,28 @@ config:
text: GitHub →
link: https://github.com/OpenDCAI/RayOrch
---

## 一分钟了解 RayOrch

很多多模态负载都会重复同一个形状:一个 PDF 变成很多页面,一个视频变成很多帧,或一张图片变成多个区域;模型逐项处理这些子项;最后再把子项组装回一个结果。这就是 `1 → M → 1`。

RayOrch 把这层关系写进 Python。你编写批处理 UDF,声明 `F.expand` 和 `F.reduce`,同时保留父项归属、子项顺序、就绪条件和失败范围;Actor、资源、放置和 RPC 交给 Ray。

```text
PDF ──► 页面 ──► OCR / VLM ──► 文档
视频 ──► 帧 ──► 视觉模型 ──► 摘要
```

## 按你的问题选择阅读路径

- **我想先跑起来:** 从[安装](guide/installation.md)和[第一个 Pipeline](guide/first-pipeline.md)开始。
- **我的数据会改变数量:** 阅读[一对多与有序归并](guide/fan-out-and-reduce.md)和[基数与血缘](concepts/cardinality.md)。
- **我想理解运行时:** 先看[框架设计](architecture/),再看[运行时架构](architecture/runtime.md)。
- **我想看完整应用:** 先运行[MinerU Benchmark](benchmarks/mineru.md),再阅读 [Flash-MinerU 应用指南](benchmarks/flash-mineru.md) 和它的[上游仓库](https://github.com/OpenDCAI/Flash-MinerU)。
- **我想得到可复现实验数字:** 使用[Benchmark 总览](benchmarks/)并先阅读[如何解读性能结果](benchmarks/performance.md)。

## 什么保持不变

一个物理 batch 可以混合不同父项的就绪子项,但它只是执行细节,不是事实来源。RayOrch 保存稳定的逻辑血缘,因此 batch 组成、Actor 放置和重试时机变化时,输出仍然属于正确的输入。

如果任务只是一个简单函数、无界流式处理,或依赖全局跨行状态,请先阅读[适用边界](guide/boundaries.md),再决定是否使用 RayOrch。
3 changes: 3 additions & 0 deletions docs/zh/benchmarks/built-ins.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@
- [双 vLLM](dual-vllm.md):两个常驻 LLM 引擎串联的生成与改写流程。
- [SGLang → vLLM](sglang-vllm.md):两个推理后端位于不同 Conda 环境的跨环境 Pipeline。

## 应用集成

- [Flash-MinerU](flash-mineru.md):持续维护的 PDF → 页面 → GPU VLM/OCR → 按序组装文档应用,并包含当前流水线并行 Benchmark 背景。
## 无依赖拓扑参考

- [嵌套文档](document-topology.md):Document → Page → TableJob 的两层展开与归并。
Expand Down
73 changes: 73 additions & 0 deletions docs/zh/benchmarks/flash-mineru.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# Flash-MinerU 应用

[Flash-MinerU](https://github.com/OpenDCAI/Flash-MinerU) 是当前最完整、最典型的 RayOrch 应用场景。它保留 MinerU 的解析和产物逻辑,同时用 RayOrch 管理 PDF 渲染、VLM/OCR 推理和文档组装之间的流水线重叠。

## 负载形状

PDF 不是一次模型调用,而是 RayOrch 中反复出现的动态基数负载:

```text
PDF ──► 页面 ──► GPU VLM/OCR ──► 有序 Markdown / JSON / 图片
1 M 1
```

页面数量由输入决定。不同 PDF 中已经就绪的页面可以共享模型 batch,但每个 PDF 仍保留自己的页序和输出目录。

## 最小调用方式

在负载环境中安装 Flash-MinerU 和推理后端:

```bash
pip install flash-mineru
# 或:pip install "flash-mineru[vllm]"
```

当前维护的流水线通过 `MineruEngine` 暴露:

```python
from flash_mineru import MineruEngine

engine = MineruEngine(
model="/shared/models/MinerU2.5-2509-1.2B",
batch_size=16,
replicas=8,
num_gpus_per_replica=0.9,
save_dir="outputs_mineru",
inflight=4,
)

outputs = engine.run(["paper-a.pdf", "paper-b.pdf"])
```

`batch_size` 控制应用一次接收多少个逻辑 PDF,`replicas` 控制常驻模型实例数,`inflight` 控制可以重叠的流水线批次数量。具体值取决于模型显存、GPU 数量和存储吞吐,建议先参考 [Flash-MinerU README](https://github.com/OpenDCAI/Flash-MinerU#-quickstart) 的默认配置。 `inflight` 是 Flash-MinerU 应用层的流水线深度;更底层的 RayOrch Benchmark API 使用 `max_active_input_batches` 控制重叠的输入生命周期。

## RayOrch 负责什么

| 关注点 | RayOrch / Flash-MinerU 的行为 |
| --- | --- |
| 动态基数 | 一个 PDF 通过 `F.expand` 展开成实际页面列表 |
| 模型执行 | 已就绪页面进入常驻 GPU Actor 池 |
| 跨输入组批 | 不同 PDF 的页面可以共享物理模型 batch |
| 结果重建 | 页面结果通过 `F.reduce` 回到所属 PDF 和原始页码 |
| 阶段重叠 | 渲染、VLM/OCR 和组装可以在不同在途批次上并行推进 |
| 产物合同 | 继续输出 MinerU 的 Markdown、版面 JSON 和图片目录 |

RayOrch 负责逻辑关系和就绪规则;Ray 仍负责 Actor 放置、资源、对象传输和集群调度;Flash-MinerU 负责 MinerU 特有的模型与产物逻辑。

## 已报告的结果

Flash-MinerU 当前 README 给出了下面这组单机对比:

| 配置 | 语料 | 结果 |
| --- | --- | --- |
| Flash-MinerU 流水线并行路径 | 368 个 PDF,8 × NVIDIA A100 | 约 **8.5 分钟** |
| 八进程 MinerU 基线 | 相同语料和机器 | 约 **14 分钟** |

在这组配置下约为 **1.7×**。这是特定模型、PDF 集合、存储系统和 GPU 上的实测结果,不是对所有环境的保证。比较版本或硬件时,请重新运行 [Flash-MinerU Benchmark 脚本](https://github.com/OpenDCAI/Flash-MinerU/tree/main/docs)。

## 推荐阅读路径

- 先理解通用模式:[一对多与有序归并](../guide/fan-out-and-reduce.md)。
- 再理解运行时边界:[完成驱动运行时](../architecture/runtime.md)。
- 安装模型依赖前,先运行无依赖拓扑案例:[嵌套文档 Benchmark](document-topology.md)。
- 最后按照 Flash-MinerU 的[快速上手](https://github.com/OpenDCAI/Flash-MinerU#-quickstart)和 [Benchmark 指南](https://github.com/OpenDCAI/Flash-MinerU/blob/main/docs/BENCHMARK.zh.md)运行真实负载。
4 changes: 4 additions & 0 deletions docs/zh/benchmarks/mineru.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@
如果需要本地/HDFS 多输入、分数 GPU Actor、原子文档提交、断点复用和已验证的大规模
配置,请使用 [`MinerUScaleBench`](mineru-scale.md)。

## Flash-MinerU:持续维护的应用路径

[Flash-MinerU](https://github.com/OpenDCAI/Flash-MinerU) 将同样的形状用于持续维护的 MinerU 集成:PDF → 页面 → GPU VLM/OCR → 按序组装文档。它当前的流水线并行路径在单机 8×A100 上处理 368 个 PDF 约 **8.5 分钟**,八进程 MinerU 基线约 **14 分钟**(该配置下约 **1.7×**)。数据集和命令见 [Flash-MinerU Benchmark](https://github.com/OpenDCAI/Flash-MinerU#-benchmark)。这些数字是特定负载的实测结果,不是对所有环境的通用承诺。

## 拓扑

```mermaid
Expand Down
12 changes: 12 additions & 0 deletions docs/zh/concepts/cardinality.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,18 @@

基数描述逻辑数据有多少条、它们与父级输入是什么关系。RayOrch 通过 `rayorch.F` 显式表达这些关系。

## 反复出现的 `1 → M → 1` 形状

只要一个输入会展开成数量依赖数据的工作项,最后又要回到一个父项结果,就会遇到这个模式:

| 负载 | 展开 | 模型阶段 | 聚合 |
| --- | --- | --- | --- |
| PDF 理解 | 文档 → 页面或区域 | OCR / VLM | 页面 → 文档 |
| 视频理解 | 视频 → 帧或片段 | 视觉 / 描述模型 | 帧 → 摘要 |
| 图像分割 | 图像 → 检测区域 | 分割模型 | 区域 → 标注图像 |

子项数量由数据决定,因此运行时必须分开保存两件事:**逻辑成员关系**(子项属于哪个父项、处在什么位置)和**物理组批**(哪些已经就绪的子项碰巧进入同一个 UDF 调用)。`expand` 与 `reduce` 声明前者,调度器可以自由优化后者。

## `expand`:一个父项展开成多个子项

UDF 为每个输入行返回一个子项列表,`expand` 从父级 Domain 进入子级 Domain。
Expand Down
6 changes: 6 additions & 0 deletions docs/zh/concepts/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,3 +53,9 @@ my_workload/
```

先从这三个文件开始。只有实验需要被复用时,才增加配置和 Benchmark 包装。
## 接下来阅读

- 需要处理数量不固定的子项?阅读[基数与血缘](cardinality.md)。
- 想理解 DROPPED、FAILED 和 SUPPRESSED?阅读[结果、失败与恢复](results-failures.md)。
- 需要调整 Actor 和 batch?阅读[资源、副本与批处理](../distributed/resources.md)。
- 需要跨机器或跨环境运行?阅读[分布式运行契约](../distributed/)和[跨环境阶段](../distributed/cross-environment.md)。
Loading