JDTLS startup waits are unbounded and CI cannot identify the stalled phase

[TyceHerrman]

Problem

Windows JVM CI can spend the remainder of the job inside JDTLS startup without
identifying which readiness phase stalled or preserving live Python/JVM state.

The concrete occurrence is
run 30496140622, job 90725317302:
the Test with pytest step ran from 22:28:00 to 23:26:43 UTC and was then
cancelled by the 60-minute job limit. The job produced no Python thread dump,
JVM thread dump, LSP tail, or last-startup-phase record.

The last pytest output was the start of
TestSerenaAgent::test_find_symbol_references[java_model_refs]. The preceding
test_find_symbol[java_model_class] had passed. This narrows the affected test
boundary, but it still does not show whether the new JDTLS instance stalled in
IntelliCode registration, ServiceReady, an upstream initialization job, or
shutdown/order handling.

Why this is a Serena correctness issue

Serena uses real JDTLS readiness notifications, but currently performs two
startup waits without client-side deadlines:

• IntelliCode command registration
• JDTLS ServiceReady

Only ProjectStatus has a 20-second bound:
current Serena implementation.

The official vscode-java client also treats ServiceReady as its readiness
signal:
upstream client.
JDTLS can wait up to one hour for initialization jobs before sending that signal:
JDTLS implementation.
That is suspiciously close to Serena's CI cap, but it does not prove the
observed stall occurred inside that upstream wait.

Even if a particular JDTLS/m2e defect is later found and fixed, Serena should
not wait forever or lose the evidence needed to diagnose the next startup
failure.

Investigation results

Draft PR #1786 added independent startup phase markers, a bounded payload-free
LSP metadata tail, Python thread capture, JVM Thread.print/jstack, process
tree capture, and Eclipse workspace logs.

A deliberate Windows JDTLS pause validated the complete capture path in
run 30602092217, job 91066841484:

• the Python dump showed the JDTLS startup thread at the deliberate phase
• a live JDTLS JVM was identified and jcmd Thread.print -l succeeded
• the phase tail ended at canary_stall_started
• the bounded LSP tail showed ProjectStatus, Ready, and ServiceReady
• the Eclipse log and process tree were preserved

The canary's JVM had initialized normally before the deliberate Python-side
pause. It did not show a JDTLS, m2e, Buildship, MavenSettings, or Plexus
deadlock.

The original Java test boundary was then exercised 53 times on Windows:

• three full JVM-batch attempts:
  the target reference test passed three times; each batch later failed in the
  unrelated safe-delete test after the repeated mutable suite produced
  pupackage test_repo;. The watchdog explicitly reported no hang capture.
• 25 fresh-workspace attempts:
  all passed.
• 25 reused-workspace attempts:
  all passed.

No natural stall occurred, so there is still no dump identifying an actionable
upstream defect.

Proposed fix

Draft PR #1790 is the smaller production change:

• put a configurable deadline around each required IntelliCode and
  ServiceReady signal (600 seconds by default)
• track the current startup phase and latest language/status value
• report both in the timeout error
• terminate JDTLS instead of leaving startup blocked
• directly test success, invalid configuration, timeout, status reporting, and
  shutdown-error behavior

Draft PR #1788 separately excludes mutable
EclipseJDTLS/workspaces state from the shared language-server cache. That is a
cache-correctness improvement, not a claimed fix for the original hang.

The larger diagnostic PR #1786 validated the watchdog and completed the fixed
reproduction budget, but is superseded by the focused production fix.

Upstream boundary

Do not file a JDTLS, m2e, or Buildship issue from the one-hour timing
correlation alone.

• If a naturally stalled JVM dump shows the MavenSettings/PlexusContainerManager
  monitor cycle or another blocked upstream initialization job, file the
  corresponding upstream issue with that dump and exact bundled versions.
• If the JVM is idle while Python waits for IntelliCode or ServiceReady, or
  the trace shows a missed notification/order race, treat that as primarily
  Serena/SolidLSP.

Acceptance criteria

• Startup cannot block until the outer 60-minute CI limit.
• A timeout error names the phase and last status observed.
• JDTLS is shut down or killed after the timeout.
• Tests cover success and timeout/error behavior without timing-dependent
  multi-threaded orchestration.

[TyceHerrman]

Follow-up from the Windows JVM check on #1790: this cancellation was not a JDTLS stall.

The finalized log shows the original boundary test, test_find_symbol_references[java_model_refs], passing at 04:38:13 UTC. All later Java basic and diagnostics tests also passed. The job subsequently stopped in TestKotlinDiagnostics::test_file_diagnostics[kotlin] until GitHub cancelled it at the 60-minute cap.

Kotlin is intentionally disabled on CI in test/conftest.py, but that diagnostics class is missing the skipif used by the adjacent Kotlin basic tests. This is a separate Serena test-quarantine gap and supplies no actionable evidence for an upstream JDTLS/m2e report.