[LauYeeYu]

What does this PR do?

cache_skip_pinned walks from a queue's victim end past the pinned objects. PrefixSim pins the blocks of the request it is serving, then calls evict once for each block it has to free. Each of those calls walks the same pinned run again from the tail, which costs O(pinned × needed) per request. On the FreeInference trace at 16k blocks, LRU takes 1,031 s with pins and 79 s without, with the same results.

This PR makes the scan resume where the last scan from the same starting object stopped.

• A small thread-local memo maps a starting object to the last pinned object found behind it. It has 8 entries, enough for the sub-queues of S3-FIFO, SLRU, W-TinyLFU and the like.
• An entry is dropped when either of its objects is unlinked, moved or freed: in remove_obj_from_list, move_obj_to_head, move_obj_to_tail and cache_remove_obj_base. cache_set_pin_predicate clears the whole memo.
• This is safe for two reasons:
  • Objects enter a queue only at its ends, so a run of pinned objects stays contiguous while it stays in place.
  • The pin set must not change while a predicate is installed. The cache_set_pin_predicate doc now says so.

Every algorithm that calls cache_skip_pinned gets the speedup without changes. With no predicate installed, the list helpers pay one thread-local NULL check.

Suggested by @haochengxia in cacheMon/PrefixSim#4.

Type of change

• Bug fix
• New eviction / admission / prefetch algorithm
• New trace reader or trace format
• Performance improvement
• Documentation
• Build / CI
• Other:

How was it tested?

• ctest --test-dir _build --output-on-failure passes, also with -fsanitize=leak

• Build is warning-free (CI uses -Wall -Wextra -Werror)

• New testPinScan. A randomized differential test: 5 seeds of 500k operations each, on 3 lists.

  • It compares about 1.2M resumed scans against a scan from scratch.
  • Meanwhile, objects are inserted at both ends, evicted, moved and removed, and the pins change.
  • With the invalidation disabled, the test crashes.

• PrefixSim, built against develop and against this branch. Every RESULT line is identical across 217 runs:

  • 44 algorithms;
  • qwen traceA at 2k and 8k blocks, plus raw ids at 8k;
  • the first 100k FreeInference requests at 16k and 64k blocks.

  The full FreeInference trace gives identical results at all 11 points in the table below.

• cachesim with no pin predicate, on twitter_cluster52_10m with LRU, S3-FIFO, Sieve and Clock at 2 sizes. Miss ratios are identical, and there is no measurable time difference (medians of 5 runs on a dedicated core):

  	develop	this PR
  LRU	2.49 s	2.49 s
  S3-FIFO	4.44 s	4.45 s
  Sieve	2.46 s	2.46 s
  Clock	2.47 s	2.46 s

Results

These are PrefixSim runs on the full FreeInference trace, each on a dedicated core:

prefixsim --trace <freeinference trace> --cache-size <size> --algo <algo>

Results are identical to develop at every point.

Algorithm, cache size	develop	this PR	Speedup
lru 16k	1030.5 s	78.8 s	13.1x
fifo 16k	869.7 s	76.0 s	11.4x
s3fifo 16k	2307.6 s	591.9 s	3.9x
clock 64k	237.0 s	95.7 s	2.5x
fifo 64k	166.7 s	71.8 s	2.3x
s4fifo 64k	685.2 s	306.4 s	2.2x
s3fifo 64k	719.8 s	435.3 s	1.7x
lru 64k	118.5 s	87.0 s	1.4x
wtinylfu 64k	386.7 s	330.0 s	1.2x
sieve 64k	96.1 s	80.5 s	1.2x (noise; Sieve does not use the scan)
random_compute 16k	1136.0 s	1136.6 s	1.0x (does not use the scan)

Checklist

• Formatted with clang-format (or the pre-commit hook from scripts/setup_hooks.sh)
• Added or updated tests
• Added or updated documentation
• New algorithms are registered in the CLI and listed in the README

[1a1a11a]

What is this about?

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

[LauYeeYu]

What is this about?

This is a comment that optimize the performance for pinning. @haochengxia mentioned in in cacheMon/PrefixSim#4:

I also tested evict_n lol. It indeed works with clock, lru, and fifo. It would be nice if we can resume the pin scan rather than from the sractch every time.

This speeds up cache_skip_pinned, which only PrefixSim uses. Do you want to remove all pinning from libCacheSim?