Skip to content

loop-induced output: fix standalone, refuse the formats with no MadLoop backend - #81

Open
oliviermattelaer wants to merge 6 commits into
mainfrom
claude/fix-loop-induced-standalone-flavor
Open

loop-induced output: fix standalone, refuse the formats with no MadLoop backend#81
oliviermattelaer wants to merge 6 commits into
mainfrom
claude/fix-loop-induced-standalone-flavor

Conversation

@oliviermattelaer

Copy link
Copy Markdown
Contributor

output standalone crashed for any loop-induced ([noborn=…]) process. Two
separate bugs, plus a decision that has been revisited now that the first one is
understood.

The reported crash was not the real bug

The symptom was an IndexError in write_check_sa's flavour-grouping block, which
looked like a flavour-grouping problem. It was not: legs_with_decays was correct all
along (4 entries for g g > z z); the flavour tuples had 6.

_flavor_enumeration_context (madgraph/core/helas_objects.py) collects every
motherless wavefunction, and in a LoopHelasMatrixElement the L-cut wavefunctions are
motherless too, carrying number_external = nexternal+1/+2. Instrumented on base:
pdgs=[21, 21, 23, 23, 1, -1]. LoopHelasMatrixElement.get_nexternal_ninitial already
carries exactly the not wf.get('is_loop') filter for this reason; the flavour
enumeration never got it.

Fixing that exposes the second bug: ExportV4Factory's format.startswith('standalone')
branch returned the tree-level ProcessExporterFortranSA unconditionally, unlike the
two madevent branches below it. The output then died in the tree-level
write_matrix_element_v4 with "wavefunction_rank has not been computed".

Allowed and refused

A previously-shelved decision (refuse loop-induced output everywhere it has no backend)
is folded in here, minus its standalone entry — because standalone can now work.

Allowed: madevent, standalone, plugin.
Refused with an InvalidCmd naming [sqrvirt=]: matrix, standalone_msP,
standalone_msF, standalone_rw, standalone_cpp, matchbox, matchbox_cpp, mg7,
mg7_v5, standalone_mg7. aloha is unaffected — it returns before the check.

standalone is a prefix shared by formats that do not work, so every membership test
is exact (format not in LOOP_INDUCED_FORMATS), never startswith. The one surviving
startswith('standalone') — the factory branch selector — has the exact test inside it.
test_loop_induced_formats_are_matched_exactly fails if a future refactor "simplifies"
that back to a prefix.

A refusal must never destroy data, so the check also sits in do_output above the
shutil.rmtree that wipes an existing output directory; the factories run after it. That
covers --me_exporter= too, which writes into the same directory and would otherwise
reach only the factory backstop — verified by test that the directory survives both ways.

validate_model

generate g g > h [noborn=QCD] as the very first command died with
AttributeError: 'NoneType' object has no attribute 'merged_particles'. A guard on that
line alone would have moved the crash three lines down — the rest of the function
dereferences _curr_model five times. The real defect is ordering: master_interface
calls validate_model directly for noborn, before create_loop_induced runs
check_add, whereas [virt=]/[real=] arrive only after check_generate has imported
the default model. Fixed by importing the model at the top of validate_model exactly as
check_generate does; the sm → loop_sm upgrade below still runs.

Validation

  • Builds and runs, not merely generates: g g > z z [noborn=QCD]
    2.5927822006924310E-004 (MadLoop rc 217, stable), bit-identical to the same process
    generated as [sqrvirt=] on base — which reaches the MadLoop interface and was never
    broken. g g > h h3.2829343688358318E-005, matching the recorded reference;
    g g > h9.3702613225405892E-003.
  • Nothing else moved: PYTHONHASHSEED=0, base vs branch, check_sa.f and matrix.f
    byte-identical for g g > t t~ (plain) and p p > e+ e- (merged particles), with the
    merged flavour table keeping its sign transfer (FLAVOR(2,2)=2 → PDG -2). The only
    generated-code change anywhere in a loop-induced madevent tree is a comment,
    C FLAVOR = [1, 1, 1, 1, 1]C FLAVOR = [1, 1, 1] — the bug becoming visibly fixed.
  • Tests: tests/unit_tests/loop/test_loop_induced_output.py, 16 tests. 12 fail on
    base
    ; the 4 that pass are exactly the controls (madevent, sqrvirt, virt, the
    tree exporter). 16/16 on branch. Plus testIO_loop_induced_standalone_output, which
    fails on base with the original IndexError.

One pre-existing bug found, worked around rather than fixed

Building a loop matrix element with compute_loop_nc=False poisons a process-global
colour memo that a later loop-induced output madevent reuses, dying in
get_icolamp_lines on v[4]-v[5]. Root cause: ColorBasis._canonical_dict
(madgraph/core/color_amp.py:51) is a class attribute; ColorBasis.__init__ shadows
it with an instance dict but LoopColorBasis.__init__ never calls its parent's
__init__, and the cached loop_Nc_power is neither part of the cache key nor
recomputed on a hit. Reproducible on clean main with a plain user script, and this
branch does not make it worse — the route opened here sets compute_loop_nc=True and so
cannot poison the cache. The tests build with compute_loop_nc=True to stay clear of it,
matching what group_subprocs does in production.

oliviermattelaer and others added 6 commits August 21, 2026 23:06
`output standalone` on a loop-induced process died with an IndexError in
export_v4.write_check_sa, on `all_pdgs[j-1]`, because the flavor tuples from
`get_external_flavors()` were longer than the process' leg list.  The reported
symptom pointed at the flavor-grouping block, but that block is innocent:
`legs_with_decays` was correct all along (4 entries for g g > z z), and the
flavor tuples were wrong (6 entries).

Root cause is in `HelasMatrixElement._flavor_enumeration_context`, the single
place that decides what the external legs of a matrix element are.  It collects
every motherless wavefunction and walks `number_external` upwards.  In a
LoopHelasMatrixElement the L-cut wavefunctions are motherless too and carry
number_external = nexternal+1 / nexternal+2, so two fictitious "external legs"
(the cut quark and antiquark of the first loop) were appended to `pdgs`.  Every
consumer of `allowed_flavors` then saw nexternal+2 slots.

`LoopHelasMatrixElement.get_nexternal_ninitial` already overrides the base
method with exactly the `not wf.get('is_loop')` filter for this reason; the
flavor enumeration simply never got the same treatment.  Apply it there too.

No effect on tree-level matrix elements: no tree wavefunction has is_loop set,
so `pdgs` and hence the emitted FLAVOR / PDG_FOR_FLAVOR tables are unchanged
(verified byte-identical for a plain and for a merged-particle process).  For a
loop ME the only shipped consumer today is the madevent loop-induced exporter,
where the fix shows up as the `C FLAVOR = [...]` comment in auto_dsig1.f
finally having nexternal entries.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Fixing the flavor tuples above lets write_check_sa run, and `output standalone`
then dies one step further on, in the tree-level write_matrix_element_v4, with
"wavefunction_rank has not been computed".  Same underlying cause: the exporter
itself is wrong.

For `[noborn=]`, master_interface borrows the MadLoop interface only to validate
the model, then switches back to 'MadGraph' and calls create_loop_induced.  So
the export goes through ExportV4Factory with output_type='default', whose
`format.startswith('standalone')` branch returns the tree-level
ProcessExporterFortranSA unconditionally -- unlike the two madevent branches
right below it, which do check for a LoopAmplitude.  ProcessExporterFortranSA
cannot write a LoopHelasMatrixElement.

Route that branch to the MadLoop standalone exporter when the amplitude is a
LoopAmplitude, mirroring what the madevent branches do.  The one obstacle was
LoopProcessExporterFortranSA.generate_subprocess_directory naming its third
positional parameter `second_exporter` and asserting it is None: the base-class
convention for that slot is the subprocess number, which is what
madgraph_interface.export passes (loop_interface.ML5export omits it).  Renamed;
`second_exporter` stays available as a keyword and keeps its assert.

The result is the ordinary MadLoop standalone layout.  For g g > z z it is
identical, modulo the process comment line, to what `[sqrvirt=QCD]` -- which
reaches the MadLoop interface and so was never broken -- produces on main, and
`./check` in the generated P0 dir returns bit-identical values from both.

Other formats with no MadLoop backend (matrix, standalone_cpp, mg7) still fail
on a loop-induced process; they are out of scope here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
'standalone' now has a MadLoop backend to route to, but the other formats a
loop-induced ([noborn=]) process can reach through the tree-level do_output
still have none, and still die deep inside a tree-level exporter:

  output matrix         -> "wavefunction_rank has not been computed"
  output mg7 / mg7_v5   -> KeyError on the first loop leg, which the mg7
                           exporter's edge-name map does not contain
  output standalone_cpp -> same shape; no C++ exporter has a loop backend

All pre-existing. Refuse them up front and point at [sqrvirt=], which stays in
the MadLoop interface and yields the very same matrix element: g g > h h
[sqrvirt=QCD] + output standalone gives a directory whose ./check returns
exactly the 3.2829343688358318E-005 of the [noborn=] output.

The allow-list is LOOP_INDUCED_FORMATS = ['madevent', 'plugin', 'standalone'],
and membership is tested EXACTLY, never with startswith: 'standalone' is a
prefix of standalone_cpp / _mg7 / _msP / _msF / _rw, none of which has a loop
backend, and a startswith test would silently re-open all five.

Checked in three places: MadGraphCmd.do_output, ahead of the rmtree that cleans
an existing output directory so a guaranteed refusal never deletes one first;
plus ExportV4Factory and ExportCPPFactory as backstops for direct callers.
[virt=]/[sqrvirt=] are untouched -- they go through loop_interface.do_output
(output_type='madloop') and reach none of the three sites -- and 'output aloha'
returns before the check.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`generate g g > h [noborn=QCD]` as the first command of a session died with
AttributeError: 'NoneType' object has no attribute 'merged_particles'.

master_interface calls validate_model directly for noborn, before
create_loop_induced runs check_add. The [virt=]/[real=] paths go through
check_add -> check_generate first, which imports the Standard Model when none
is active; noborn never gets that far. A guard on the merged_particles line
alone would not do: the rest of validate_model dereferences _curr_model too
(perturbation_couplings, get('name'), get('gauge')). So import the default
model at the top instead, exactly as check_generate does -- the existing
sm -> loop_sm upgrade below then runs unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nothing covered this before: the only test of non-optimized loop-induced output
was the IOTest short_ML_SMQCD_LoopInduced/gg_hh, which drives the exporter
directly and never reaches ExportV4Factory.

tests/unit_tests/loop/test_loop_induced_output.py, 15 tests, ~22 s:
  - the flavor tuples of a loop ME have nexternal entries with the right PDGs
    (3 for g g > h, 4 for g g > z z);
  - 'output standalone' goes through and writes a MadLoop directory, and the
    factory hands it a MadLoop exporter while a tree process keeps the
    tree-level one;
  - matrix / mg7 / standalone_cpp / standalone_msP refuse it and name
    [sqrvirt=], the refusal does not wipe an existing output directory, and
    LOOP_INDUCED_FORMATS is asserted not to contain the standalone_* formats
    so nobody can turn the membership test into a startswith;
  - [noborn=] works with no model imported;
  - madevent on [noborn=], and standalone on [sqrvirt=] and [virt=], still
    work -- these are the controls: the refusal keys off the amplitude being a
    LoopAmplitude, which those are too, and they are spared only because they
    never reach the two factories.
11 of the 15 fail on a clean tree; the 4 that pass are exactly those controls.

testIO_loop_induced_standalone_output joins the existing
MadLoop_output_from_the_interface group: runs the real `output standalone` on
g g > h [noborn=QCD] through MasterCmd -- with no `import model`, so it covers
the validate_model bootstrap too -- and compares check_sa.f and loop_matrix.f.
On a clean tree it fails with the original IndexError. ~4 s.

One trap worth knowing: building a loop ME with compute_loop_nc=False leaves
colour data that a later loop-induced madevent output reuses, and it then dies
in get_icolamp_lines on a None Nc power. That is pre-existing and reproducible
on a clean tree; the tests build their matrix elements the way group_subprocs
does (compute_loop_nc=True) to stay clear of it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The guard in do_output tested only self._export_format, so a refusable format
asked for as the *second* matrix-element exporter fell through to the factory
backstop -- which runs after the shutil.rmtree that cleans an existing output
directory:

  output madevent <existing-dir> --me_exporter=mg7 -f   # g g > h [noborn=QCD]
  -> InvalidCmd from ExportCPPFactory, and <existing-dir> already deleted

Not a regression (a clean tree wipes it too, then crashes deeper), but holding
that directory is the entire reason the check sits above the rmtree. Check both
names in the same loop. A plugin me_exporter is named by its plugin, so it is
refused like any other unlisted name -- which is what the backstops already do
with it.

Also trim the comments back to the house rule (they outnumbered the product
code): drop the block comment on LOOP_INDUCED_FORMATS restated by the docstring
under it, keeping the exact-membership warning, which is the load-bearing part.
The bare `except Exception` around perturbation_couplings is gone with it: both
callers pass a Process or ProcessDefinition, which always has that key. And note
why the 'matchbox' factory branch carries no backstop, unlike its neighbours.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant