Repository navigation
build: amalgamate.py --with-log and --include-prefix, so more embedders can use the generated files - #169
Conversation
…eyond the core Redis and Valkey vendor only the core, but memtier_benchmark also uses the log codec (hdr_log_*) and the time helpers, and Node.js keeps the upstream src/ + include/hdr/ layout. The release assets could not replace either: - the log files include private headers (hdr_tests.h, hdr_endian.h, hdr_encoding.h, hdr_histogram_internal.h) that the assets do not ship, and <hdr/...> includes that a flat directory does not have; - the generated .c included "hdr_histogram.h", which does not resolve when the header lives in include/hdr/. --with-log also emits hdr_histogram_log.c, hdr_encoding.c and hdr_time.c with their private headers inlined, plus their public headers rewritten for a flat layout. --include-prefix hdr/ makes the generated files include "hdr/hdr_histogram.h" and leaves the header byte-identical to the upstream one. Without the new flags the output is byte-identical to before. script/check-amalgamation.sh verifies both variants (per-file preprocessor equivalence with the sources, -Werror builds, a log encode/decode round trip, and the hdr/ layout) and is run by the amalgamation CI job and by the release job before anything is attached. The release now also attaches a hdr_histogram-amalgamation-with-log-<version>.zip. docs/AMALGAMATION.md gains a section on migrating an existing vendored copy. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…c use the caller's allocator) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
🤖 Automated first-pass review — a human maintainer's review is still required before merge. The generator changes look fine, and the default/ |
|
Node.js result, as promised in the description: a full build of What changed in The histogram-related tests all pass: 21 run, 21 ok, 0 failed, 0 skipped ( I ran only the histogram-related tests, not Node's whole suite. |
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
What
Two opt-in options for
script/amalgamate.py, plus a check script, so that the projects that vendor a flattened copy of the library can switch to the generated files:--with-logalso emits the log codec:hdr_histogram_log.c,hdr_encoding.c,hdr_time.cand their public headershdr_histogram_log.h,hdr_time.h. Each.chas its private headers inlined.--include-prefix hdr/makes the generated.cinclude"hdr/hdr_histogram.h", for trees that keep the public headers in ahdr/directory.script/check-amalgamation.shverifies both, and is run by the amalgamation CI job and by the release job before anything is attached.The release now also attaches
hdr_histogram-amalgamation-with-log-<version>.zip.docs/AMALGAMATION.mdgains a section on migrating an existing vendored copy.No library source changes. Without the new flags the generator's output is byte-identical to what is on
main(checked for the default and for--malloc-include).Why
I tried replacing the vendored copies of Redis, Valkey, memtier_benchmark and Node.js with the generated files, now that #162, #164, #165, #166 and #167 are in. Two of the four did not work as is:
hdr_log_*) and the time helpers. The log files include private headers (hdr_tests.h,hdr_histogram_internal.h,hdr_encoding.h,hdr_endian.h) that the release assets do not ship, plus<hdr/...>includes, so compiling them next to the core amalgamation failed on all three files.src/plusinclude/hdr/. The generatedhdr_histogram.cincludes"hdr_histogram.h", which does not resolve there:fatal error: 'hdr_histogram.h' file not found.Redis and Valkey work with the existing options (Valkey's
CMakeLists.txtalso needs one line removed, since it listshdr_atomic.h; that is in the new doc section).Validation
check-amalgamation.shpasses with gcc and clang underdash: each of the four amalgamated files preprocesses to the same translation unit as its source and builds with-Wall -Wextra -Wmissing-prototypes -Werror; a log encode/decode round trip works using only the generated files;--with-log --malloc-includeroutes the core and the codec through the caller's allocator; and with--include-prefix hdr/the header is byte-identical toinclude/hdr/hdr_histogram.hand the file compiles with-Isrc -Iinclude. It fails (exit 1) when the generator is deliberately broken.run:step of both workflows was executed locally, including the new two-zip packaging and checksums.9af6e958d), replaced with no Makefile change: builds with 0 compiler warnings; the latency-monitor, info, redis-cli and redis-benchmark suites pass (122 tests).redis-benchmarkhistogram output steps by 0.1 ms up to 2.1 ms and by 1.0 ms after, as the iterator setter should.4ebfa71): replaced, builds from a freshautoreconf/configure/make. The same workload (2 threads x 5 clients x 20,000 requests with--hdr-file-prefixand--json-out-file) through the old and the new build records 200,000 requests in both, and the compressed histograms written byhdr_log_encodedecode correctly with the independent Pythonhdrhlibrary. memtier's own integration tests were not run.463711aa): the generated file, in Node's layout and withhistogram.gypunchanged, compiles inside Node's own build with Node's flags, and Node'ssrc/histogram.cccompiles against the replaced header. Node's code never stores a negative count (subtract refuses or clamps to 0, the CBOR reader accepts unsigned only), so it fits the non-negative counts contract. The full Node link and its test suite had not finished when this was opened; I will comment with the result.Not tested
gh release uploadand therelease: publishedtrigger need an actual release. A manual "Run workflow" exercises everything except the upload.🤖 Generated with Claude Code