Describe the bug
Setting timer_enable_nvtx 1 in INPUT produces no NVTX ranges in any CUDA build.The parameter is parsed and documented, but the code that consumes it is removedby the preprocessor, so profiling timelines contain no ABACUS stage labels.
Root cause: __USE_NVTX is defined only on the final executable target(source/CMakeLists.txt:520), whose single translation unit source_main/main.cppnever includes an NVTX header and never calls NVTX. The two targets that actually need the macro — source_base/timer.cpp (the base OBJECT library) and source_main/driver.cpp (the driver OBJECT library) — guard their NVTX calls
with #if defined(__CUDA) && defined(__USE_NVTX), so those blocks are deleted in every CUDA build and timer::enable_nvtx_ stays at its initial value false.
Expected behavior
timer_enable_nvtx 1 should emit NVTX ranges (e.g. :Operator:hPsi, :HSolverPW:solve, :ElecStatePW:psiToRho) visible in Nsight Systems, matching the documentation in docs/parameters.yaml and docs/advanced/input_files/input-main.md.
To Reproduce
- Configure with CUDA:
cmake -B build -S . -DUSE_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=89
- Build: cmake --build build -j
- Add
timer_enable_nvtx 1 to a PW INPUT (e.g. tests/11_PW_GPU/scf_cg)
- Run under Nsight Systems and check for NVTX ranges:
nsys profile --trace=cuda,nvtx -o prof ./build/abacus_basic_gpu
nsys stats --report nvtx_pushpop_sum prof.nsys-rep
→ no NVTX ranges are reported.
Compile-level evidence (no GPU needed):
nm -C build/source/source_base/CMakeFiles/base.dir/timer.cpp.o | grep -i nvtx
returns only ModuleBase::timer::enable_nvtx_ — no nvtxRangePushA/nvtxRangePop.
driver.cpp.o contains no reference to enable_nvtx_, i.e. the enable call
at driver.cpp:129 was also removed by the preprocessor.
Environment
- OS: WSL2 (Ubuntu)
- Compiler: g++ / nvcc 13.1
- CUDA: 13.1 (NVTX v2 removed in 12.9; nvtx3/nvToolsExt.h is used via cuda_compat.h)
- GPU: NVIDIA RTX 4060 Laptop
Additional Context
No response
Task list for Issue attackers (only for developers)
Describe the bug
Setting
timer_enable_nvtx 1in INPUT produces no NVTX ranges in any CUDA build.The parameter is parsed and documented, but the code that consumes it is removedby the preprocessor, so profiling timelines contain no ABACUS stage labels.Root cause:
__USE_NVTXis defined only on the final executable target(source/CMakeLists.txt:520), whose single translation unitsource_main/main.cppnever includes an NVTX header and never calls NVTX. The two targets that actually need the macro —source_base/timer.cpp(thebaseOBJECT library) andsource_main/driver.cpp(thedriverOBJECT library) — guard their NVTX callswith
#if defined(__CUDA) && defined(__USE_NVTX), so those blocks are deleted in every CUDA build andtimer::enable_nvtx_stays at its initial valuefalse.Expected behavior
timer_enable_nvtx 1should emit NVTX ranges (e.g.:Operator:hPsi,:HSolverPW:solve,:ElecStatePW:psiToRho) visible in Nsight Systems, matching the documentation indocs/parameters.yamlanddocs/advanced/input_files/input-main.md.To Reproduce
cmake -B build -S . -DUSE_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=89
timer_enable_nvtx 1to a PW INPUT (e.g. tests/11_PW_GPU/scf_cg)nsys profile --trace=cuda,nvtx -o prof ./build/abacus_basic_gpu
nsys stats --report nvtx_pushpop_sum prof.nsys-rep
→ no NVTX ranges are reported.
Compile-level evidence (no GPU needed):
nm -C build/source/source_base/CMakeFiles/base.dir/timer.cpp.o | grep -i nvtxreturns only
ModuleBase::timer::enable_nvtx_— nonvtxRangePushA/nvtxRangePop.driver.cpp.ocontains no reference toenable_nvtx_, i.e. the enable callat driver.cpp:129 was also removed by the preprocessor.
Environment
Additional Context
No response
Task list for Issue attackers (only for developers)