Skip to content

perf(lcao): batch full RI radial tables on CPU - #8279

Merged
mohanchen merged 1 commit into
deepmodeling:developfrom
Stardust0831:perf/ri-cpu-radial-tables
Nov 5, 2026
Merged

mohanchen merged 1 commit into
deepmodeling:developfrom
Stardust0831:perf/ri-cpu-radial-tables

Conversation

@Stardust0831

Copy link
Copy Markdown
Collaborator

Follow-up to #8257. That PR enabled batched radial-table construction on GPU; CPU execution still initializes full tables one orbital pair at a time. This change routes the CPU callers through the existing batched builder, enabling BLAS-based quadrature and shared product-orbital preparation.

What's changed?

Matrix_Orbs11::init_radial_table and Matrix_Orbs21::init_radial_table collect their orbitals and pass the existing use_gpu argument to the batch overload. CPU execution uses the existing double-precision BLAS implementation; GPU execution keeps the existing CUDA path. The quadrature rule and accuracy settings are unchanged.

The diff contains two source files: 4 insertions and 24 deletions.

Performance

Measurements on SAI used an AMD Ryzen Threadripper PRO 5995WX, one MPI rank bound to four physical CPU cores, four OpenMP threads, and one V100 for GPU runs. Current comparisons used exclusive nodes and two runs in reversed order. OpenBLAS 0.3.32 uses OpenMP and reports four configured threads; OPENBLAS_NUM_THREADS=1 does not select serial BLAS for this build. OMP_THREAD_LIMIT=4 and CPU affinity bounded execution to four cores.

The displaced Si32 (32 atoms) and CsPbI3 (5 atoms) cases are the same as those used for the earlier GPU radial-table benchmarks: STRU, KPT, pseudopotential and orbital hashes match. GPU INPUT files differ only in data-directory paths. CPU runs select device cpu and ks_solver scalapack_gvx; GPU runs select device gpu and ks_solver cusolver. LibRI contractions run on CPU, without DDLA.

All times are seconds. Initialization is Exx_LRI.init; full-run time includes process launch, HSE SCF, forces and stress. Values are means of two runs unless marked otherwise.

Version and execution path Si32 initialization Si32 full run CsPbI3 initialization CsPbI3 full run
Earlier baseline (d20090cc137f), GPU mode with CPU radial tables 6.83 192.97 343.81 425.31
Earlier baseline with the GPU radial-table change alone 1.37 184.36 15.05 95.87
Earlier combined Libxc, integral-reuse and GPU radial-table optimizations (one run) 2.17 126.41 14.80 50.61
Current baseline (f5315651166e), GPU mode 1.22 104.47 13.80 47.07
Current baseline (f5315651166e), CPU mode 4.57 133.43 250.25 300.61
This change (b28ed06649cc), CPU mode 1.31 130.93 20.41 71.26

The controlled CPU comparison gives 3.49x / 12.26x faster initialization and 1.019x / 4.219x full-run speedups for Si32 / CsPbI3. GPU full-run control timings differ by less than 1%. Sampled peak host RSS decreases from 4.605 to 4.464 GiB for Si32 and from 14.040 to 6.557 GiB for CsPbI3 (maximum of two runs, sampled approximately every 0.5 seconds).

The historical rows ran on shared nodes. Current develop also includes the Libxc scheduling change and integral reuse, so comparisons across historical and current rows include other code and allocation differences. These two cases do not establish system-size or multi-node scaling. Historical inputs and results are available in the benchmark record.

Unit Tests and/or Case Tests for my changes

CPU-only and CUDA Release builds passed with LibRI and Libxc enabled. Build-directory paths below are abbreviated. Each CTest command passed with both OMP_NUM_THREADS=1 and OMP_NUM_THREADS=4, without skips.

cmake --build build-cpu --target abacus_std_para MODULE_LCAO_radial_tables MODULE_LCAO_integral_reuse --parallel 24
cmake --build build-gpu --target abacus_std_gpu MODULE_LCAO_radial_tables MODULE_LCAO_integral_reuse MODULE_LCAO_radial_tables_gpu --parallel 24

ctest --test-dir build-cpu -V --no-tests=error --timeout 180 -R '^MODULE_LCAO_radial_tables$'
ctest --test-dir build-cpu -V --no-tests=error --timeout 180 -R '^MODULE_LCAO_integral_reuse$'
ctest --test-dir build-gpu -V --no-tests=error --timeout 180 -R '^MODULE_LCAO_radial_tables$'
ctest --test-dir build-gpu -V --no-tests=error --timeout 180 -R '^MODULE_LCAO_integral_reuse$'
ctest --test-dir build-gpu -V --no-tests=error --timeout 180 -R '^MODULE_LCAO_radial_tables_gpu$'

Existing tests cover scalar/batched overlap and gradients, tiling, selective initialization, shared ownership lifetime and finite-difference gradients. They are reused because this change connects the already-tested batch implementation at its callers.

All 20 final HSE calculations converged, with matching SCF/EXX iteration counts. Controls include CPU execution in a CUDA build and two MPI ranks with a regular 2x1x1 k-point mesh and complex tensors. Maximum candidate-versus-baseline differences were 7.640e-12 eV/atom in energy, 2.100e-9 eV/Angstrom in force components, and 1.020e-8 kbar in stress components. Source and executable hashes were audited against the tested revisions. Both versions used identical timer JSON instrumentation in benchmark builds.

git diff f5315651166ea265ba673996e3c9efeb5e4cd9b0...HEAD --check passed. The full repository test suite was not run; validation focused on RI initialization and complete HSE calculations.

Linked Issue

Follow-up to GPU radial-table batching. No separate issue is needed for enabling the existing CPU implementation at the same callers.

Governance Notes

  • INPUT/docs changes: no documentation update required; no INPUT keyword, public API, dependency or accuracy setting changes.
  • Core module impact: only full radial-table initialization in source_lcao/module_ri; the existing use_gpu argument selects the backend.
  • Exceptions requested: none.

Reminder

  • I have read AGENTS.md and docs/developers_guide/agent_governance.md.
  • I have linked an issue or explained why this PR does not need one.
  • I have added adequate unit tests and/or case tests, or explained why not.
  • I have listed the exact verification commands run and their results.
  • I have described user-visible behavior changes, including INPUT parameter changes.
  • I have explained core-module impact for the source changes.
  • I have requested any needed governance exception above.

Route full two- and three-center radial table construction through the existing batched builders. The use_gpu argument selects CPU BLAS or CUDA, and three-center tables share product-orbital preparation.
@Stardust0831
Stardust0831 marked this pull request as ready for review November 3, 2026 14:54

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static review found one non-blocking input-domain limitation below. No builds or runtime checks were run.

@mohanchen mohanchen added GPU & DCU & HPC GPU and DCU and HPC related any issues EXX/lr-TDDFT/RPA/BSE/GW Related to beyond DFT methods Performance Issues related to fail running ABACUS labels Nov 4, 2026
@mohanchen
mohanchen merged commit 60a351b into deepmodeling:develop Nov 5, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

EXX/lr-TDDFT/RPA/BSE/GW Related to beyond DFT methods GPU & DCU & HPC GPU and DCU and HPC related any issues Performance Issues related to fail running ABACUS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants