[ondrejmirtes]

PHP often loads Xdebug, PCOV, and other profilers enabled, and that multiplies the counts. In a loop of method calls, Xdebug in its default develop mode counts 6.5× and PCOV with pcov.enabled=1 3.4×. The workers can't switch them off: they get neither the -d options of tally run nor XDEBUG_MODE, and a setting can't unload an extension. Loaded but switched off (xdebug.mode=off, pcov.enabled=0), they counted exactly what a PHP without them counts.

Stripping

• When PHP's ini files load one of Configuration::strippedExtensions(), tally run builds a php.ini from them with those lines commented out. The approach follows phpstan-src's restart (IniComposer): each file is cut at its first [PATH=]/[HOST=] section, which CLI never applies, and only the file name counts (tideways-php-8.5.so is Tideways, tideways_xhprof.so is not).
• Both runs of a case then start with -c <php.ini> and a PHP_INI_SCAN_DIR that doesn't exist, because proc_open leaves out a variable whose value is empty.
• Without such a line the command line stays the same. The Benchmarks job loads no extension, so the expected numbers don't move.
• A worker that still loads one, because it's built in or loaded with -d, fails its case instead of counting with it.
• Default list: xdebug, pcov, uopz, blackfire, tideways, tideways_xhprof, xhprof, spx, excimer, memprof, opentelemetry. withStrippedExtensions() changes it.

E2E tests

The new e2e job runs e2e/stripped-extensions/stripped-extensions_test.sh with bashunit, set up like phpstan-src's e2e tests, in three rows: Xdebug, PCOV, both. With CodSpeed's Valgrind, each test loads the extensions another way: setup-php's scan directory, another scan directory, the php.ini in PHPRC, or -c. It checks that the case doesn't see them and counts within 0.1 % of a PHP without them. Two controls keep the checks from passing for the wrong reason: with withStrippedExtensions([]) the case sees the extensions, and with them loaded by -d the worker refuses to count.

Run locally with Xdebug 3.5.3 and PCOV on PHP 8.5, all three combinations pass. They fail when stripping is switched off, and when the composed ini changes another setting (opcache.enable_cli=1: −27 %).