glib/win_iconv.c: In function 'iso2022jp_mbtowc':
glib/win_iconv.c:1818:18: warning: comparison of integer expressions of different signedness: 'DWORD' {aka 'long unsigned int'} and 'int'
if (cv->mode != ISO2022_MODE(cs, shift))
^~
glib/win_iconv.c: In function 'iso2022jp_wctomb':
glib/win_iconv.c:1889:18: warning: comparison of integer expressions of different signedness: 'DWORD' {aka 'long unsigned int'} and 'int'
if (cv->mode == ISO2022_MODE(cs, shift))
^~
glib/gfileutils.c: In function 'g_file_test':
glib/gfileutils.c:375:18: warning: comparison of integer expressions of different signedness: 'int' and 'long unsigned int'
if (attributes == INVALID_FILE_ATTRIBUTES)
^~
glib/gfileutils.c: In function 'g_get_current_dir':
glib/gfileutils.c:2882:40: warning: comparison of integer expressions of different signedness: 'DWORD' {aka 'long unsigned int'} and 'int'
if (GetCurrentDirectoryW (len, wdir) == len - 1)
^~
glib/gdate.c: In function 'win32_strftime_helper':
glib/gdate.c:2582:12: warning: comparison of integer expressions of different signedness: 'gsize' {aka 'long long unsigned int'} and 'glong' {aka 'long int'}
if (slen <= convlen)
^~
Despite their type, the values returned will always be ≥ 0. It’s
unfortunate they weren’t declared with an unsigned type, but we can’t
change that now without breaking API.
Spotted in !2294.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
glib/gspawn-win32.c: In function 'read_helper_report':
glib/gspawn-win32.c:329:16: warning: comparison of integer expressions of different signedness: 'gint' {aka 'int'} and 'long long unsigned int'
while (bytes < sizeof(gintptr)*2)
^
glib/gspawn-win32.c:366:13: warning: comparison of integer expressions of different signedness: 'gint' {aka 'int'} and 'long long unsigned int'
if (bytes < sizeof(gintptr)*2)
^
MSVC supports AddressSanitizer as well via "/fsanitize=address" option,
but __lsan_ignore_object() equivalent feature is not supported.
Note that there's __declspec(no_sanitize_address) specifier which
provides a similar feature but that's not runtime behavior
so it's not directly applicable to g_ignore_leak() family.
See also https://docs.microsoft.com/en-us/cpp/sanitizers/asan-building?view=msvc-160
glib/gslice.c: In function 'slice_config_init':
glib/gslice.c:394:16: warning: comparison of integer expressions of different signedness: 'int' and 'long long unsigned int'
else if (len >= G_N_ELEMENTS (wvalue))
^~
glib/genviron.c: In function 'g_getenv':
glib/genviron.c:490:52: warning: comparison of integer expressions of different signedness: 'DWORD' {aka 'long unsigned int'} and 'int'
if (GetEnvironmentVariableW (wname, wvalue, len) != len - 1)
^~
glib/genviron.c:507:60: warning: comparison of integer expressions of different signedness: 'DWORD' {aka 'long unsigned int'} and 'int'
if (ExpandEnvironmentStringsW (tem, wvalue, len) != len)
^~
glib/gcharset.c:380:35: warning: comparison of integer expressions of different signedness: 'long long int' and 'long long unsigned int'
else if (modifier - dot < sizeof (buf))
^
This comment was added in a919be3d in 2013. The function basically looked the same
as now at that point. g_main_context_find_source_by_id() can clearly return NULL if
the source id is invalid, so unclear where this comes from? The function in
question are approximately the same since e2fd4e2b in 2000.
However back then g_main_context_find_source_by_id() would actually always return
the last source if there was none with the given source id (wat, that's clearly
unintended?). This was changed in 393503ba in 2014, arguably an API change of that
function but more arguably a bugfix :)
So for a short time between 2013 and 2014, that comment was correct. Now it is not
anymore and can be removed.
These just wrap the `__lsan_enable()` and `__lsan_disable()` functions
from the AddressSanitizer client API. They’re useful in situations where
the intended-to-be-leaked memory is being allocated in third-party code,
such as xdgmime. We can’t patch that code to call `g_ignore_leak()`.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Helps: #2310
g_get_user_database_entry() capitalises the first letter of pw_name
with g_ascii_toupper (pw->pw_name[0]).
However, the manpage for getpwnam() and getpwuid() says the result of
those calls "may point to a static area". GLib is then trying to edit
static memory which belongs to a shared library, so segfaults.
The reentrant variants of the above calls are supposed to fill the user
buffer supplied to them, however Michael Catanzaro also found a bug in
systemd where the data is not copied to the user buffer and still points
to static memory, resulting in the same sort of segfault. See:
https://github.com/systemd/systemd/issues/20679
Solve both these cases in GLib by copying pw_name off to a temporary
variable, set uppercase on that variable, and use the variable to join
into the desired string. Free the variable after it is no longer needed.
Signed-off-by: Jamie Bainbridge <jamie.bainbridge@gmail.com>
It fixes a race. g_source_attach() had the following check to ensure a
loop blocked on poll() would wakeup.
if (do_wakeup && context->owner && context->owner != G_THREAD_SELF)
g_wakeup_signal (context->wakeup);
However it doesn't contemplate an implementation where poll()ing is a
non-blocking operation that will be scheduled while the thread is
released to perform other tasks. This scenario opens up several
different possibilities where the condition would fail to hold true. I
experienced two of such races.
The first race pertains to a mono-threaded application. Do keep in mind
that integrating GLib to a foreign event loop will make GLib act as a
slave in the new event loop. When you post a new work unit to execute in
the thread managed by the foreign event loop, you don't use
g_main_context_invoke(). In fact the only reason to integrate
GMainContext in a foreign event loop is to make the two of them
communicate. So from time to time, the foreign event loop will execute
callbacks that manipulate the GMainContext loop. An illustration
follows.
// in this callback we translate an event from the foreign event loop
// to an event in the GMainContext event loop (that runs in the same
// thread)
static void my_event_loop_callback(void* data)
{
GMainContext* ctx = /* ... */;
// ...
g_source_attach(source, ctx);
}
int main()
{
// ...
my_event_loop_invoke(my_event_loop_callback, data);
// ...
// this function has all mechanisms in place to run the foreign
// event loop and the hooks to call
// g_main_context_{prepare,query,check,dispatch}
my_event_loop_run();
}
In this case, you would have the following series of calls:
1. g_main_context_prepare()
2. g_main_context_query()
3. A callback to my_event_loop is registered when any fd on the set is
ready or the timeout is reached.
4. The thread is released to perform other tasks.
5. One of the tasks executed wishes to communicate with my_event_loop
and enters my_event_loop_callback.
6. g_source_attach() is called.
7. g_source_attach() detects do_wakeup=TRUE, context->owner != NULL, and
context->owner == G_THREAD_SELF so g_wakeup_signal() is skipped.
8. None of the fds on the GLib poll() set becomes ready nor the GLib
timeout expires. The my_event_loop callback that would call
g_main_context_check() is never executed. Deadlock.
A shallow analysis will fail to detect the race here. The explanation
seems to showcase a scenario that will deterministically fail with a
deadlock every time. However do keep in mind that my_event_loop_callback
could be invoked before or after g_main_context_prepare(). There is an
_event_ race here. Furthermore, some GLib libraries such as GDBus will
initialize objects from extra threads (GAsyncInitable interface) and
invoke the result on the original thread when ready (g_source_attach()
will eventually be called). Now you have scenarios closer to standard
race examples.
The other scenario where a race would manifest happens in a
multi-threaded application that has a concurrency design similar to the
actor model. No actor executes in two threads simultaneously, but it's
not guaranteed that it'll always wake-up in the same thread. It'd
perform steps 1-4 just as in the previous example, but before thread
control is returned to the pool, it'd call g_main_context_release(). Now
g_source_attach() would skip g_wakeup_signal() for a different reason:
7. g_source_attach() detects do_wakeup=TRUE, context->owner == NULL so
g_wakeup_signal() is skipped.
8. Same as before.
Certainly there are other concurrency designs where this optimization
would cause a deadlock, but all of them have origin in the same place:
the optimization assumes the poll() implementation is a blocking
operation and the thread will never be released to perform other tasks
(possibly involving GLib calls) while result is not ready. They share
not only the same problem, but also the same solution: do not make
assumptions and just call g_wakeup_signal().
This patch implements this solution by introducing the
G_MAIN_CONTEXT_FLAGS_OWNERLESS_POLLING flag. This flag will force a call
to g_wakeup_signal() and fix the race on foreign event loops. The reason
to prevent changing this option after creation is to avoid other races
that would lead to event loss. Construction is the only proper time to
set this option.
The implementation design means we do not change **any** semantics for
current working code. If you don't set the new flag, the code won't
enter in different branches and current behavior won't be affected. The
patch is small and easy to follow too.
Previously, priority was not randomly generated and was instead derived
from `GSequenceNode*` pointer value.
As a result, when a `GSequence` was freed and another was created, the
nodes were returned to memory allocator in such order that allocating
them again caused various performance problems in treap.
To my understanding, the problem develops like this :
1) Initially, memory allocator makes some nodes
2) For each node, priority is derived from pointer alone.
Due to the hash function, initially the priorities are reasonably
randomly distributed.
3) `GSequence` moves inserted nodes around to satisfy treap property.
The priority for node must be >= than priorities of its children
4) When `GSequence` is freed, it frees nodes in a new order.
It finds root node and then recursively frees left/right children.
Due to (3), hashes of freed nodes become partially ordered.
Note that this doesn't depend on choice of hash function.
5) Memory allocator will typically add freed chunks to free list.
This means that it will reallocate nodes in same or inverse order.
6) This results in order of hashes being more and more non-random.
7) This order happens to be increasingly anti-optimal.
That is, `GSequence` needs more `node_rotate` to maintain treap.
This also causes the tree to become more and more unbalanced.
The problem becomes worse with each iteration.
The solution is to use additional noise to maintain reasonable
randomness. This prevents "poisoning" the memory allocator.
On top of that, this patch somehow decreases average tree's height,
which is good because it speeds up various operations. I can't quite
explain why the height decreases with new code, probably the properties
of old hash function didn't quite match the needs of treap?
My averaged results for tree height with different sequence lengths:
Items | before| after |
--------+-------+---------------+
2 | 2,69 | 2,67 -00,74% |
4 | 3,71 | 3,80 +02,43% |
8 | 5,30 | 5,34 +00,75% |
16 | 7,45 | 7,22 -03,09% |
32 | 10,05 | 9,38 -06,67% |
64 | 12,97 | 11,72 -09,64% |
128 | 16,01 | 14,20 -11,31% |
256 | 19,11 | 16,77 -12,24% |
512 | 22,03 | 19,39 -11,98% |
1024 | 25,29 | 22,03 -12,89% |
2048 | 28,43 | 24,82 -12,70% |
4096 | 31,11 | 27,52 -11,54% |
8192 | 34,31 | 30,30 -11,69% |
16384 | 37,40 | 32,81 -12,27% |
32768 | 40,40 | 35,84 -11,29% |
65536 | 43,00 | 38,24 -11,07% |
131072 | 45,50 | 40,83 -10,26% |
262144 | 48,40 | 43,00 -11,16% |
524288 | 52,40 | 46,80 -10,69% |
The memory cost of the patch is zero on 64-bit, because the new field
uses the alignment hole between two other fields.
Note: priorities can sometimes have collisions. This is fine, because
treap allows equal priorities, but these will gradually decrease
performance. The hash function that was used previously has just one
collision on 0xbfff7fff in 32-bit space, but such pointer will not
occur because `g_slice_alloc()` always aligns to sizeof(void*).
However, in 64-bit space the old hash function had collisions anyway,
because it only uses lower 32 bits of pointer.
Closes#2468
The previous test was racy: it assumed that not all queued thread pool
jobs would start to be executed before `g_thread_pool_free()` was
called, whereas actually on some systems (particularly BSD ones), they
would all start (or even finish) being executed, and hence the free
function might never be called.
Rewrite the test to:
• Synchronise the test function and worker thread functions more
closely.
• Not bother about ordering the shared and exclusive variants of the
test differently. That seems to be a hangover from another test
above.
• Limit the number of worker threads to 1, rather than 2, since this
makes the test easier to control.
This has been tested with `--repeat 10000` on Linux, and it succeeds all
of those runs whereas previously it failed quite reliably.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Fixes: #2456
Now that `g_thread_pool_new_full()` can be used to set a user-provided
free function for queue elements, ensure that the internal dummy item
used to wake up the worker threads is removed from the queue before it’s
called.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Helps: #2456
This allows a pattern like
g_test_message ("cannot reticulate splines: %s", error->message);
g_test_fail ();
to be replaced by the simpler
g_test_fail_printf ("cannot reticulate splines: %s", error->message);
with the secondary benefit of making the message available to TAP
consumers as part of the "not ok" message.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Forming the g_test_skip() message from printf-style arguments seems
common enough to deserve a convenience function.
g_test_incomplete() is mechanically almost equivalent to g_test_skip()
(the semantics are different but the implementation is very similar),
so give it a similar mechanism for symmetry.
Signed-off-by: Simon McVittie <smcv@collabora.com>
We had two compensating bugs here. We didn't correctly clear the
error indicator after testing a test-case that calls g_test_fail(),
which meant we were leaving the error set to exit status 1 when
falling through to the next test; and then we didn't check the exit
status of the next test, but instead assumed that g_spawn_sync()
would fail (it does not).
Signed-off-by: Simon McVittie <smcv@collabora.com>
