loop-induced output: fix standalone, refuse the formats with no MadLoop backend - #81
Open
oliviermattelaer wants to merge 6 commits into
Open
loop-induced output: fix standalone, refuse the formats with no MadLoop backend#81oliviermattelaer wants to merge 6 commits into
oliviermattelaer wants to merge 6 commits into
Conversation
`output standalone` on a loop-induced process died with an IndexError in
export_v4.write_check_sa, on `all_pdgs[j-1]`, because the flavor tuples from
`get_external_flavors()` were longer than the process' leg list. The reported
symptom pointed at the flavor-grouping block, but that block is innocent:
`legs_with_decays` was correct all along (4 entries for g g > z z), and the
flavor tuples were wrong (6 entries).
Root cause is in `HelasMatrixElement._flavor_enumeration_context`, the single
place that decides what the external legs of a matrix element are. It collects
every motherless wavefunction and walks `number_external` upwards. In a
LoopHelasMatrixElement the L-cut wavefunctions are motherless too and carry
number_external = nexternal+1 / nexternal+2, so two fictitious "external legs"
(the cut quark and antiquark of the first loop) were appended to `pdgs`. Every
consumer of `allowed_flavors` then saw nexternal+2 slots.
`LoopHelasMatrixElement.get_nexternal_ninitial` already overrides the base
method with exactly the `not wf.get('is_loop')` filter for this reason; the
flavor enumeration simply never got the same treatment. Apply it there too.
No effect on tree-level matrix elements: no tree wavefunction has is_loop set,
so `pdgs` and hence the emitted FLAVOR / PDG_FOR_FLAVOR tables are unchanged
(verified byte-identical for a plain and for a merged-particle process). For a
loop ME the only shipped consumer today is the madevent loop-induced exporter,
where the fix shows up as the `C FLAVOR = [...]` comment in auto_dsig1.f
finally having nexternal entries.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Fixing the flavor tuples above lets write_check_sa run, and `output standalone`
then dies one step further on, in the tree-level write_matrix_element_v4, with
"wavefunction_rank has not been computed". Same underlying cause: the exporter
itself is wrong.
For `[noborn=]`, master_interface borrows the MadLoop interface only to validate
the model, then switches back to 'MadGraph' and calls create_loop_induced. So
the export goes through ExportV4Factory with output_type='default', whose
`format.startswith('standalone')` branch returns the tree-level
ProcessExporterFortranSA unconditionally -- unlike the two madevent branches
right below it, which do check for a LoopAmplitude. ProcessExporterFortranSA
cannot write a LoopHelasMatrixElement.
Route that branch to the MadLoop standalone exporter when the amplitude is a
LoopAmplitude, mirroring what the madevent branches do. The one obstacle was
LoopProcessExporterFortranSA.generate_subprocess_directory naming its third
positional parameter `second_exporter` and asserting it is None: the base-class
convention for that slot is the subprocess number, which is what
madgraph_interface.export passes (loop_interface.ML5export omits it). Renamed;
`second_exporter` stays available as a keyword and keeps its assert.
The result is the ordinary MadLoop standalone layout. For g g > z z it is
identical, modulo the process comment line, to what `[sqrvirt=QCD]` -- which
reaches the MadLoop interface and so was never broken -- produces on main, and
`./check` in the generated P0 dir returns bit-identical values from both.
Other formats with no MadLoop backend (matrix, standalone_cpp, mg7) still fail
on a loop-induced process; they are out of scope here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
'standalone' now has a MadLoop backend to route to, but the other formats a
loop-induced ([noborn=]) process can reach through the tree-level do_output
still have none, and still die deep inside a tree-level exporter:
output matrix -> "wavefunction_rank has not been computed"
output mg7 / mg7_v5 -> KeyError on the first loop leg, which the mg7
exporter's edge-name map does not contain
output standalone_cpp -> same shape; no C++ exporter has a loop backend
All pre-existing. Refuse them up front and point at [sqrvirt=], which stays in
the MadLoop interface and yields the very same matrix element: g g > h h
[sqrvirt=QCD] + output standalone gives a directory whose ./check returns
exactly the 3.2829343688358318E-005 of the [noborn=] output.
The allow-list is LOOP_INDUCED_FORMATS = ['madevent', 'plugin', 'standalone'],
and membership is tested EXACTLY, never with startswith: 'standalone' is a
prefix of standalone_cpp / _mg7 / _msP / _msF / _rw, none of which has a loop
backend, and a startswith test would silently re-open all five.
Checked in three places: MadGraphCmd.do_output, ahead of the rmtree that cleans
an existing output directory so a guaranteed refusal never deletes one first;
plus ExportV4Factory and ExportCPPFactory as backstops for direct callers.
[virt=]/[sqrvirt=] are untouched -- they go through loop_interface.do_output
(output_type='madloop') and reach none of the three sites -- and 'output aloha'
returns before the check.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`generate g g > h [noborn=QCD]` as the first command of a session died with
AttributeError: 'NoneType' object has no attribute 'merged_particles'.
master_interface calls validate_model directly for noborn, before
create_loop_induced runs check_add. The [virt=]/[real=] paths go through
check_add -> check_generate first, which imports the Standard Model when none
is active; noborn never gets that far. A guard on the merged_particles line
alone would not do: the rest of validate_model dereferences _curr_model too
(perturbation_couplings, get('name'), get('gauge')). So import the default
model at the top instead, exactly as check_generate does -- the existing
sm -> loop_sm upgrade below then runs unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nothing covered this before: the only test of non-optimized loop-induced output
was the IOTest short_ML_SMQCD_LoopInduced/gg_hh, which drives the exporter
directly and never reaches ExportV4Factory.
tests/unit_tests/loop/test_loop_induced_output.py, 15 tests, ~22 s:
- the flavor tuples of a loop ME have nexternal entries with the right PDGs
(3 for g g > h, 4 for g g > z z);
- 'output standalone' goes through and writes a MadLoop directory, and the
factory hands it a MadLoop exporter while a tree process keeps the
tree-level one;
- matrix / mg7 / standalone_cpp / standalone_msP refuse it and name
[sqrvirt=], the refusal does not wipe an existing output directory, and
LOOP_INDUCED_FORMATS is asserted not to contain the standalone_* formats
so nobody can turn the membership test into a startswith;
- [noborn=] works with no model imported;
- madevent on [noborn=], and standalone on [sqrvirt=] and [virt=], still
work -- these are the controls: the refusal keys off the amplitude being a
LoopAmplitude, which those are too, and they are spared only because they
never reach the two factories.
11 of the 15 fail on a clean tree; the 4 that pass are exactly those controls.
testIO_loop_induced_standalone_output joins the existing
MadLoop_output_from_the_interface group: runs the real `output standalone` on
g g > h [noborn=QCD] through MasterCmd -- with no `import model`, so it covers
the validate_model bootstrap too -- and compares check_sa.f and loop_matrix.f.
On a clean tree it fails with the original IndexError. ~4 s.
One trap worth knowing: building a loop ME with compute_loop_nc=False leaves
colour data that a later loop-induced madevent output reuses, and it then dies
in get_icolamp_lines on a None Nc power. That is pre-existing and reproducible
on a clean tree; the tests build their matrix elements the way group_subprocs
does (compute_loop_nc=True) to stay clear of it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The guard in do_output tested only self._export_format, so a refusable format asked for as the *second* matrix-element exporter fell through to the factory backstop -- which runs after the shutil.rmtree that cleans an existing output directory: output madevent <existing-dir> --me_exporter=mg7 -f # g g > h [noborn=QCD] -> InvalidCmd from ExportCPPFactory, and <existing-dir> already deleted Not a regression (a clean tree wipes it too, then crashes deeper), but holding that directory is the entire reason the check sits above the rmtree. Check both names in the same loop. A plugin me_exporter is named by its plugin, so it is refused like any other unlisted name -- which is what the backstops already do with it. Also trim the comments back to the house rule (they outnumbered the product code): drop the block comment on LOOP_INDUCED_FORMATS restated by the docstring under it, keeping the exact-membership warning, which is the load-bearing part. The bare `except Exception` around perturbation_couplings is gone with it: both callers pass a Process or ProcessDefinition, which always has that key. And note why the 'matchbox' factory branch carries no backstop, unlike its neighbours. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
output standalonecrashed for any loop-induced ([noborn=…]) process. Twoseparate bugs, plus a decision that has been revisited now that the first one is
understood.
The reported crash was not the real bug
The symptom was an
IndexErrorinwrite_check_sa's flavour-grouping block, whichlooked like a flavour-grouping problem. It was not:
legs_with_decayswas correct allalong (4 entries for
g g > z z); the flavour tuples had 6._flavor_enumeration_context(madgraph/core/helas_objects.py) collects everymotherless wavefunction, and in a
LoopHelasMatrixElementthe L-cut wavefunctions aremotherless too, carrying
number_external = nexternal+1/+2. Instrumented on base:pdgs=[21, 21, 23, 23, 1, -1].LoopHelasMatrixElement.get_nexternal_ninitialalreadycarries exactly the
not wf.get('is_loop')filter for this reason; the flavourenumeration never got it.
Fixing that exposes the second bug:
ExportV4Factory'sformat.startswith('standalone')branch returned the tree-level
ProcessExporterFortranSAunconditionally, unlike thetwo
madeventbranches below it. The output then died in the tree-levelwrite_matrix_element_v4with "wavefunction_rank has not been computed".Allowed and refused
A previously-shelved decision (refuse loop-induced output everywhere it has no backend)
is folded in here, minus its
standaloneentry — becausestandalonecan now work.Allowed:
madevent,standalone,plugin.Refused with an
InvalidCmdnaming[sqrvirt=]:matrix,standalone_msP,standalone_msF,standalone_rw,standalone_cpp,matchbox,matchbox_cpp,mg7,mg7_v5,standalone_mg7.alohais unaffected — it returns before the check.standaloneis a prefix shared by formats that do not work, so every membership testis exact (
format not in LOOP_INDUCED_FORMATS), neverstartswith. The one survivingstartswith('standalone')— the factory branch selector — has the exact test inside it.test_loop_induced_formats_are_matched_exactlyfails if a future refactor "simplifies"that back to a prefix.
A refusal must never destroy data, so the check also sits in
do_outputabove theshutil.rmtreethat wipes an existing output directory; the factories run after it. Thatcovers
--me_exporter=too, which writes into the same directory and would otherwisereach only the factory backstop — verified by test that the directory survives both ways.
validate_modelgenerate g g > h [noborn=QCD]as the very first command died withAttributeError: 'NoneType' object has no attribute 'merged_particles'. A guard on thatline alone would have moved the crash three lines down — the rest of the function
dereferences
_curr_modelfive times. The real defect is ordering:master_interfacecalls
validate_modeldirectly fornoborn, beforecreate_loop_inducedrunscheck_add, whereas[virt=]/[real=]arrive only aftercheck_generatehas importedthe default model. Fixed by importing the model at the top of
validate_modelexactly ascheck_generatedoes; the sm → loop_sm upgrade below still runs.Validation
g g > z z [noborn=QCD]→2.5927822006924310E-004(MadLoop rc 217, stable), bit-identical to the same processgenerated as
[sqrvirt=]on base — which reaches the MadLoop interface and was neverbroken.
g g > h h→3.2829343688358318E-005, matching the recorded reference;g g > h→9.3702613225405892E-003.PYTHONHASHSEED=0, base vs branch,check_sa.fandmatrix.fbyte-identical for
g g > t t~(plain) andp p > e+ e-(merged particles), with themerged flavour table keeping its sign transfer (
FLAVOR(2,2)=2 → PDG -2). The onlygenerated-code change anywhere in a loop-induced madevent tree is a comment,
C FLAVOR = [1, 1, 1, 1, 1]→C FLAVOR = [1, 1, 1]— the bug becoming visibly fixed.tests/unit_tests/loop/test_loop_induced_output.py, 16 tests. 12 fail onbase; the 4 that pass are exactly the controls (
madevent,sqrvirt,virt, thetree exporter). 16/16 on branch. Plus
testIO_loop_induced_standalone_output, whichfails on base with the original
IndexError.One pre-existing bug found, worked around rather than fixed
Building a loop matrix element with
compute_loop_nc=Falsepoisons a process-globalcolour memo that a later loop-induced
output madeventreuses, dying inget_icolamp_linesonv[4]-v[5]. Root cause:ColorBasis._canonical_dict(
madgraph/core/color_amp.py:51) is a class attribute;ColorBasis.__init__shadowsit with an instance dict but
LoopColorBasis.__init__never calls its parent's__init__, and the cachedloop_Nc_poweris neither part of the cache key norrecomputed on a hit. Reproducible on clean
mainwith a plain user script, and thisbranch does not make it worse — the route opened here sets
compute_loop_nc=Trueand socannot poison the cache. The tests build with
compute_loop_nc=Trueto stay clear of it,matching what
group_subprocsdoes in production.