fix(build): probe positive form of -Wdeprecated-enum-enum-conversion

[MarkAtwood]

Fixes jenkins issue #559 — https://jenkins-supervisor.wolfssl.com/#/open-issues/559
(root-cause task #556: https://jenkins-supervisor.wolfssl.com/#/open-issues/556)

---

Problem

The probe added in 627f516 intends to add -Wno-deprecated-enum-enum-conversion only "if applicable". It does not achieve that: the flag is accepted on compilers that have never heard of it.

ax_save_c_werror_flag=$ac_c_werror_flag
ac_c_werror_flag=yes
AX_CHECK_COMPILE_FLAG([-Wno-deprecated-enum-enum-conversion],
  [AX_APPEND_FLAG([-Wno-deprecated-enum-enum-conversion], [AM_CFLAGS])])
ac_c_werror_flag=$ax_save_c_werror_flag

GCC reports an unrecognized -Wno-foo only if some other diagnostic is emitted in the same translation unit. AX_CHECK_COMPILE_FLAG's conftest is trivial and emits nothing, so there is no stderr for ac_c_werror_flag=yes to catch, and the probe returns yes. The flag is then appended to AM_CFLAGS, reaching every C translation unit in the tree.

It causes no trouble at all until an unrelated warning shows up. Then the deferred complaint fires and -Werror converts a single warning into two hard errors:

tests/api/test_dh.c:569:19: error: comparison of unsigned expression < 0 is always false [-Werror=type-limits]
tests/api/test_dh.c: At top level:
cc1: error: unrecognized command line option '-Wno-deprecated-enum-enum-conversion' [-Werror]
cc1: all warnings being treated as errors

The second error is pure noise that sends you looking in the wrong place — it appears and disappears purely as a function of whether anything else in the file warned.

Fix

Probe the positive -Wdeprecated-enum-enum-conversion, which GCC rejects immediately and unconditionally when unknown, then append the -Wno- form on success. The ac_c_werror_flag juggling is no longer needed.

Verification

Measured on gcc 9.4.0 (Ubuntu 20.04), the compiler where this actually bites:

probe	result
clean conftest + -Werror -Wno-deprecated-enum-enum-conversion (current)	exit 0, no output — flag wrongly accepted
clean conftest + -Wdeprecated-enum-enum-conversion (this PR)	exit 1, gcc: error: unrecognized command line option — correctly skipped

Behaviour after this change:

• pre-GCC-11 C compiler — unknown option, probe fails, flag not added
• GCC 11+ compiling C — "valid for C++/ObjC++ but not for C", probe fails, flag not added (C never needed it)
• C++20 compiler — accepted, -Wno- form added, original intent preserved

Also confirmed the flag really does land in C flags today: AM_CFLAGS in a generated Makefile contains it, and config.log shows it baked in at configure time.

Question for the original author

configure.ac has no AC_PROG_CXX, no AM_CXXFLAGS, and compiles no C++, so it isn't obvious which build path was meant to receive this. .github/workflows/multi-compiler.yml passes CXX=g++-9..12 / clang++-14,19 to configure. If the intended target is genuinely C++ compilation, AM_CXXFLAGS under AC_LANG_PUSH([C++]) would be the more precise home and I'm glad to redo it that way.

Independent of #11050, which fixes the -Wtype-limits warning that exposed this. Either PR alone makes the observed failure go away; only this one disarms it for the next warning.

[Copilot]

Pull request overview

This PR adjusts the Autoconf probe that conditionally adds -Wno-deprecated-enum-enum-conversion so the check no longer “succeeds” on compilers that don’t actually support the warning option (notably due to GCC’s behavior with unknown -Wno-* flags in trivial conftests).

Changes:

• Switch the probe from -Wno-deprecated-enum-enum-conversion to the positive -Wdeprecated-enum-enum-conversion form.
• Remove the previous ac_c_werror_flag save/restore logic around the probe.
• Update the surrounding comments to explain why probing -Wno-* is unreliable.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

MemBrowse Memory Report

No memory changes detected for:

• gcc-arm-cortex-m0plus
• gcc-arm-cortex-m3
• gcc-arm-cortex-m4
• gcc-arm-cortex-m4-baremetal
• gcc-arm-cortex-m4-crypto-only
• gcc-arm-cortex-m4-dtls13
• gcc-arm-cortex-m4-min-ecc
• gcc-arm-cortex-m4-openssl-compat
• gcc-arm-cortex-m4-pkcs7
• gcc-arm-cortex-m4-pq
• gcc-arm-cortex-m4-rsa-only
• gcc-arm-cortex-m4-sp-math
• gcc-arm-cortex-m4-tls12
• gcc-arm-cortex-m4-tls13
• gcc-arm-cortex-m7
• gcc-arm-cortex-m7-pq
• gcc-arm-cortex-m7-tls13
• linuxkm-pie
• linuxkm-standard
• stm32-sim-stm32h753

[MarkAtwood]

@douzzer — you added this probe in 627f516 ("add -Wno-deprecated-enum-enum-conversion to CFLAGS to suppress C++20 default if applicable"), so you're the one who knows the answer here. One question, because it decides whether this PR is the whole fix or half of it.

Which build path was meant to receive the flag?

-Wdeprecated-enum-enum-conversion is C++-only, and it's appended to AM_CFLAGS — the C flags. For that to ever be useful, something has to be feeding AM_CFLAGS to a C++ compiler. I couldn't find that path in the tree:

• no AC_PROG_CXX
• no AM_CXXFLAGS
• no .cpp/.cc sources in Makefile.am
• .github/workflows/multi-compiler.yml varies CXX=g++-9..12 / clang++-14,19 while CC stays ccache gcc

Two readings, with different consequences:

(a) It's for CC=g++-style builds — wolfSSL's C sources compiled by a C++ compiler for header-compat checking. Then AM_CFLAGS is the right home, this PR preserves your intent exactly, and nothing further is needed.

(b) It was meant for C++ compilation and landed in AM_CFLAGS incidentally. Then this PR stops the breakage but leaves the flag as dead config, and the real home is AM_CXXFLAGS under AC_LANG_PUSH([C++]). Happy to redo it that way.

I deliberately didn't guess: moving it under (a) would silently drop C++20 suppression someone depends on, and leaving it under (b) enshrines config that does nothing.

Worth noting this PR is safe under either reading — it never adds the flag to a compiler that rejects it, so the deferred-diagnostic failure can't recur regardless. It's just that (b) would want a follow-up.

Context on why this surfaced: the flag has been silently riding in AM_CFLAGS on every pre-GCC-11 builder. It costs nothing until some other warning appears in the same TU, at which point gcc reports the deferred "unrecognized -Wno-" complaint and -Werror turns one warning into two errors. That's been breaking a nightly small-footprint build (nightly-memusage-v2) for 11 consecutive runs on a gcc 9.4.0 node. #11050 fixes the warning that triggered it; this PR disarms the flag for the next one.