Previously, priority was not randomly generated and was instead derived
from `GSequenceNode*` pointer value.
As a result, when a `GSequence` was freed and another was created, the
nodes were returned to memory allocator in such order that allocating
them again caused various performance problems in treap.
To my understanding, the problem develops like this :
1) Initially, memory allocator makes some nodes
2) For each node, priority is derived from pointer alone.
Due to the hash function, initially the priorities are reasonably
randomly distributed.
3) `GSequence` moves inserted nodes around to satisfy treap property.
The priority for node must be >= than priorities of its children
4) When `GSequence` is freed, it frees nodes in a new order.
It finds root node and then recursively frees left/right children.
Due to (3), hashes of freed nodes become partially ordered.
Note that this doesn't depend on choice of hash function.
5) Memory allocator will typically add freed chunks to free list.
This means that it will reallocate nodes in same or inverse order.
6) This results in order of hashes being more and more non-random.
7) This order happens to be increasingly anti-optimal.
That is, `GSequence` needs more `node_rotate` to maintain treap.
This also causes the tree to become more and more unbalanced.
The problem becomes worse with each iteration.
The solution is to use additional noise to maintain reasonable
randomness. This prevents "poisoning" the memory allocator.
On top of that, this patch somehow decreases average tree's height,
which is good because it speeds up various operations. I can't quite
explain why the height decreases with new code, probably the properties
of old hash function didn't quite match the needs of treap?
My averaged results for tree height with different sequence lengths:
Items | before| after |
--------+-------+---------------+
2 | 2,69 | 2,67 -00,74% |
4 | 3,71 | 3,80 +02,43% |
8 | 5,30 | 5,34 +00,75% |
16 | 7,45 | 7,22 -03,09% |
32 | 10,05 | 9,38 -06,67% |
64 | 12,97 | 11,72 -09,64% |
128 | 16,01 | 14,20 -11,31% |
256 | 19,11 | 16,77 -12,24% |
512 | 22,03 | 19,39 -11,98% |
1024 | 25,29 | 22,03 -12,89% |
2048 | 28,43 | 24,82 -12,70% |
4096 | 31,11 | 27,52 -11,54% |
8192 | 34,31 | 30,30 -11,69% |
16384 | 37,40 | 32,81 -12,27% |
32768 | 40,40 | 35,84 -11,29% |
65536 | 43,00 | 38,24 -11,07% |
131072 | 45,50 | 40,83 -10,26% |
262144 | 48,40 | 43,00 -11,16% |
524288 | 52,40 | 46,80 -10,69% |
The memory cost of the patch is zero on 64-bit, because the new field
uses the alignment hole between two other fields.
Note: priorities can sometimes have collisions. This is fine, because
treap allows equal priorities, but these will gradually decrease
performance. The hash function that was used previously has just one
collision on 0xbfff7fff in 32-bit space, but such pointer will not
occur because `g_slice_alloc()` always aligns to sizeof(void*).
However, in 64-bit space the old hash function had collisions anyway,
because it only uses lower 32 bits of pointer.
Closes#2468
The previous test was racy: it assumed that not all queued thread pool
jobs would start to be executed before `g_thread_pool_free()` was
called, whereas actually on some systems (particularly BSD ones), they
would all start (or even finish) being executed, and hence the free
function might never be called.
Rewrite the test to:
• Synchronise the test function and worker thread functions more
closely.
• Not bother about ordering the shared and exclusive variants of the
test differently. That seems to be a hangover from another test
above.
• Limit the number of worker threads to 1, rather than 2, since this
makes the test easier to control.
This has been tested with `--repeat 10000` on Linux, and it succeeds all
of those runs whereas previously it failed quite reliably.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Fixes: #2456
Now that `g_thread_pool_new_full()` can be used to set a user-provided
free function for queue elements, ensure that the internal dummy item
used to wake up the worker threads is removed from the queue before it’s
called.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Helps: #2456
This allows a pattern like
g_test_message ("cannot reticulate splines: %s", error->message);
g_test_fail ();
to be replaced by the simpler
g_test_fail_printf ("cannot reticulate splines: %s", error->message);
with the secondary benefit of making the message available to TAP
consumers as part of the "not ok" message.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Forming the g_test_skip() message from printf-style arguments seems
common enough to deserve a convenience function.
g_test_incomplete() is mechanically almost equivalent to g_test_skip()
(the semantics are different but the implementation is very similar),
so give it a similar mechanism for symmetry.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Include the correct system headers for each test that meson performs.
This allows system capabilities to be detected correctly even if
implicit declaration of functions is considered an error.
We had two compensating bugs here. We didn't correctly clear the
error indicator after testing a test-case that calls g_test_fail(),
which meant we were leaving the error set to exit status 1 when
falling through to the next test; and then we didn't check the exit
status of the next test, but instead assumed that g_spawn_sync()
would fail (it does not).
Signed-off-by: Simon McVittie <smcv@collabora.com>
Let's explain the advantages of relying on GTlsConnection to perform
certificate verification.
Also, document that the issuer property is a little tricky, because the
issuer certificate might not be the certificate that actually gets used
in final certification path building. This is very unexpected to anybody
who is not an expert.
Because TLS certificate verification is extremely complex, the lookup
issuer function may be tempting to misuse even by experienced
developers. There is a notion that the issuer certificate will always be
used in the final certification path, but it's just not always true.
Trying to make security decisions based on the results of this function is
a trap, so let's document that.
It turns out that old versions of glib-networking actually reordered the
certificate chain to match the final verification path. This no longer
happens since a long time ago, because it was a buggy mess. Instead, we
rely on the TLS library to build the final verification path. Their path
building is not very good, but at least it's consistent. The point of
these doc updates is to clarify that only the TLS library can make
security decisions.
Document that HTTP requests may be performed to look up missing
certificates.
Finally, let's document that certificate verification using GTlsDatabase
cannot be as smart as certificate verification performed directly by
GTlsConnection.
g_win32_package_parser_enum_packages() reads beyond the end of a buffer
when doing a memcpy. With app verifier enabled on Windows, it causes
the application to crash on startup.
This change limits the memcpy to the size of the source string.
Fixes: #2454
This reverts commit 7aa0580cc5.
As stated in #2316, that commit was a workaround to allow gnome-keyring
and msmtp to continue to get their session bus address from
`DBUS_SESSION_BUS_ADDRESS`, even though they’re `AT_SECURE`. The timeout
on that workaround has expired so that commit is now being reverted.
Fixes: #2316
You need to separate the first entry in the list from the preceding
paragraph, and you should add a space before the enumerating symbol.
GTK-Doc accepts a very lax Markdown syntax, but any other tool parsing
our documentation will likely fail.
