Repository navigation
allocator: Implement support for allocation cache - #1107
Merged
Merged
Conversation
This is the initial implementation of allocation cache; it was initially developed for clusterlod.h but this change adds it to the library proper. The cache is a fixed number of fixed-size blocks; each thread can claim ownership of a block for the duration of its temporary allocations. Each block acts as an arena, using the invariant that all meshopt_ allocations are performed in stack order. When the last allocation is freed, the block is returned to the pool; however we keep the index of the block for the next allocation. If the block is still available, we will keep using the same block which improves cache locality unless threads migrate across CPUs. Note that this ignores the previously setup allocation callbacks as this is using the exact same implementation that clusterlod.h was tested against; future commits will refine it.
MSVC does not define __cplusplus as 201103 unless explicitly opted into, but it does support both thread_local and std::atomic; so we explicitly check _MSC_VER to avoid disabling the cache. Also fix gCacheLocal linkage, and avoid false sharing for blocks.
Allocating a zero-sized object would previously be problematic because freeing the object allocating after that would return the block, so we'd free the same pointer again after that. Passing more than 64 blocks is not allowed and asserts, but we now set the mask to all 1s which effectively just uses the first 64 blocks. Setting up the cache is now checking for integer overflow, and the deallocate path is a bit more defensive if it ever gets a rogue pointer; this should never happen, but the new code is a bit safer.
If meshopt_setAllocator is used to override the callbacks, setting the cache after that now uses the callbacks to allocate the cache itself and handle the fallback allocations too. Once the cache is setup, using meshopt_setAllocator is not allowed; this would disable the cache and resetting the cache later would restore the old cache, which is problematic too. The cache needs to be disabled first to change the allocation callbacks again.
At /W4, MSVC complains that GlobalCache struct was padded due to alignas specifier. That... makes sense, but also does not seem fixable short of using the padding of an exact length which varies depending on the target pointer size. Instead, put the atomic first, add a manual padding after it to ensure false sharing isn't a problem, and align the global itself. Also fix the warning due to negation of an unsigned value.
We get the atomic initialization either way through cache setup; having an explicit initializer here but not for other fields result in a dynamic initializer, but if this is omitted then compilers remove it as long as the gCache itself has an initializer. Also flip static and thread_local for readability.
compare_exchange_weak already updates blocks, so we do not need to load() it inside the loop; also use relaxed memory order although it does not really matter for performance since this only runs for the outermost allocation in a given meshopt_ call. For all other accesses, we now do not rely on atomic operators and use explicit functions. Also outside of setup, the memory order is explicit: block allocation uses acquire, which synchronizes with block release that uses release.
Also assert that block size is not redundant; while you could certainly call setAllocatorCache with 64 blocks of 0 bytes, that seems like a mistake we should catch.
We do not test interaction with multiple threads for simplicity; the cache we set up is also tiny, normally you'd allocate several megabytes per thread of course. This is only necessary to provide basic code coverage and exercise assertions etc. Also use a 32MB cache in nanite.cpp as a more practical demonstration.
Add documentation for meshopt_setAllocatorCache
This does not matter that much because it's just the first allocation in a given allocation stack from one meshopt_ call, but switch to a different loop structure for computing index from the mask, which both Clang and GCC recognize as an idiom and lower to bsr (with some extra checks). Also move this to a separate function in case we decide to use intrinsics in the future, and tighten the assertions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For applications that end up using many allocations, such as when
processing many small meshes or individual clusters as part of
clusterlod.h, it may be beneficial to set up the allocator cache. I'vwritten about this before, in Billons of triangles in minutes and
Billions of triangles redux; this change exposes the cache as an
opt-in library function,
meshopt_setAllocatorCache.The cache is a fixed number of fixed-size blocks; each thread can claim
ownership of a block for the duration of its temporary allocations. Each
block acts as an arena, using the invariant that all meshopt_
allocations are performed in stack order.
When the last allocation is freed, the block is returned to the pool;
however we keep the index of the block for the next allocation. If the
block is still available, we will keep using the same block which
improves cache locality unless threads migrate across CPUs.
This requires C++11 and
thread_localsupport; while this should begenerally available, a define
MESHOPTIMIZER_ALLOC_NOCACHEis providedin case this needs to be disabled during the build, and C++98/03 builds
disable this too.
On Zorah on Windows, without a cache set up (and disabling the custom
arena that
vk_lod_clustersimplements), the processing takes ~247 secondswith NT heap and ~243 seconds with segment heap. Setting up a cache with
16 blocks of 32 MB each changes the processing time to ~220 seconds at no
significant memory cost.
It's of course up to the users to configure this however it makes sense, or
ignore this addition if the allocation pressure is not a bottleneck. On Linux,
this change helps too but a bit less drastically (~230 seconds without, ~222
seconds with).
This contribution is sponsored by Valve.