Skip to content

Reconcile remote registry snapshots with a map instead of an ordset - #90

Open
vheathen wants to merge 1 commit into
ostinelli:masterfrom
vheathen:reconcile-snapshot-with-map
Open

vheathen wants to merge 1 commit into
ostinelli:masterfrom
vheathen:reconcile-snapshot-with-map

Conversation

@vheathen

Copy link
Copy Markdown

reconcile_remote_registry_snapshot/3 builds the snapshot names with ordsets:from_list/1 and calls ordsets:is_element/2 once per entry the node already holds for that remote node, so the reconciliation is quadratic.

After an ordinary join or reconnection the receiver holds no entries for that node, because they were purged when the remote scope went down, so the cost stays hidden. It shows when an ack_sync arrives from a node whose entries are already in the table. One way to get there: init/1 calls net_kernel:monitor_nodes(true) before handle_continue(after_init) sends discover to nodes(), so a node that connects in between gets discover twice and answers with two full snapshots. The second one is compared against everything the first one inserted, inside the scope process, and every register on that node waits behind it.

Membership test alone, N snapshot names against N existing entries, OTP 27.3.4, median of three runs:

entries ordsets map
10 000 748 ms 1 ms
30 000 7 185 ms 5 ms
100 000 96 424 ms 20 ms

The fix uses a map and is_map_key/2. The map is built with maps:from_keys/2 from OTP 24 and with maps:from_list/1 below it. The suite cannot cover that fallback, because it needs peer and so OTP 25, so I checked it by hand on OTP 23: src compiles and the fallback is the branch compiled in.

The new test case fails on the current code with 50 308 359 reductions for a 10 000 entry snapshot and passes with the fix at about 281 000. Tests pass on OTP 25 through 29 and dialyzer is clean.

reconcile_remote_registry_snapshot/3 tested every existing entry of the
remote node against the snapshot names with ordsets:is_element/2, which
is a linear scan, so the reconciliation was quadratic. It shows when an
ack_sync arrives from a node whose entries are already in the table,
for example when a node is discovered twice at scope start.

The names now go into a map and membership is tested with is_map_key/2.
The map is built with maps:from_keys/2 from OTP 24 and with
maps:from_list/1 below it.

The new test case asserts the cost of reconciling a 10 000 entry
snapshot in reductions.
@vheathen

Copy link
Copy Markdown
Author

@ostinelli may I ask you to look if this PR makes sense for you? We are switching to syn, and in some scenarios it helps quite much.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant