Make `G_URI_FLAGS_PARSE_RELAXED` available instead, for the
implementations which need to handle user-provided or incorrect URIs.
The default should nudge people towards being compliant with RFC 3986.
This required also adding a new `G_URI_PARAMS_PARSE_RELAXED` flag, as
previously parsing param strings *always* used relaxed mode and there
was no way to control it. Now it defaults to using strict mode, and the
new flag allows for relaxed mode to be enabled if needed.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #2149
According to my reading of
https://tools.ietf.org/html/rfc3986#section-4, the only requirement for
a URI to be ‘absolute’ (actually, not a relative reference) is for the
scheme to be specified. A hostname doesn’t have to be specified: see any
of the options in the `hier-part` production in
https://tools.ietf.org/html/rfc3986#appendix-A which don’t include
`authority`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This reduces the possibility for overflow, and makes the code a little
more conventional to read.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
`guint8` is the conventional way in modern GLib APIs to represent ‘a byte
which could contain arbitrary binary’. `guchar` is not advised for that
(even though it’s equivalent) because it could be misread as `gchar`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This reduces the chance of the caller accidentally double-freeing or
use-after-free-ing something.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Sometimes there are sensitive details in URI query components, so we
should provide the option for hiding them too.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This brings its naming in line with the ‘generic’ error codes in other
error domains.
This is not an API break since `GUriError` hasn’t been in a release yet.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
RFC 3986, section 3 says:
> The scheme and path components are required, though the path may be
> empty (no characters). When authority is present, the path must
> either be empty or begin with a slash ("/") character. When
> authority is not present, the path cannot begin with two slash
> characters ("//"). These restrictions result in five different ABNF
> rules for a path (Section 3.3), only one of which will match any
> given URI reference.
(See https://tools.ietf.org/html/rfc3986#section-3.)
Given that those conditions are almost always going to be true, and that
typically the number and form of arguments passed to g_uri_join() will
be known at compile time, it would be unnecessarily awkward to add a
`GError` argument to g_uri_join() to detect these situations.
Instead, add precondition checks and document the restrictions.
Developers are responsible for ensuring their paths are in the right
format themselves.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
The test was failing since commit 20ae4b46d ("uri: do not add ipv6
brackets on non-ip host").
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
The g_uri_join_internal() function was making a simplification that
userinfo can be encoded with the same restricted character set as the
user field alone, fix this by allowing the correct character set.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Otherwise, the to_string() encoding will not be reversible. Furthermore,
if no distinction is needed in the first place, g_uri_build() with
userinfo should be used instead.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
g_uri_build_with_user() builds a userinfo, but it shouldn't encode it
itself, but let the user flags declare what's there. Otherwise,
to_string() code paths may encode a second time.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Add encoded flags, similar to what was done in commit 7bee36b4 ("uri:
add G_FLAGS_ENCODED_QUERY").
SoupURI has manual handling of encoded path & fragment, but it can rely
on GUri decoding for the rest.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
GTK ci rightly complains about this when ti builds
GLib as a subproject with -Werror=maybe-uninitialized.
subkey_dynamic_w will be freed without being initialized
when the first goto is taken.
The heuristic is a bit too agressive, as we may have hostname with
%-encoded ':' (as shown in GVfs URI tests).
Add an extra test to check :-decoding as well.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Add a few ipv6 scope parsing corner test cases.
- checking incorrect scoped IPv6 ending with only %25 isn't decoded.
- checking valid scoped IPv6 is passing g_uri_is_valid()
As discussed in
https://gitlab.gnome.org/GNOME/glib/-/merge_requests/1567#note_860499,
for historical reasons, GUri accepts the % preceding the zone-id in the
unescaped form as well.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
g_uri_is_valid() should check the given URI is valid following RFC-3986,
and reject relative references.
Fixes: https://gitlab.gnome.org/GNOME/glib/-/issues/2169
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Just to make it a little more obvious that a thread pool can be
initialised with one thread per logical CPU.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #272
Improve formatting, and clarify that the same *type* of test fixture can
be reused, not the same specific instance of a test fixture.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #250
It may be defined by the environment (we document that as being allowed)
— if so, individual files should not try to redefine it, as that causes
a preprocessor warning.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Apparently, the function calls added in 11729cdc use Foundation, not
Cocoa. Cocoa is a massive superset of Foundation, and is not available
on iOS.
Patch contributed by Jay Freeman, but without providing an e-mail
address. So the git commit cannot be attributed to him.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #1869
Various different BSD systems use a different errno from `E_LOOP` (as
defined by POSIX and used on Linux) to indicate that a file is a symlink
when you try to `open()` it with `O_NOFOLLOW`.
Fix the code which detects this. This is a follow-up to #1302.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
These exercise all the code paths I can manage without adding a load of
machinery to inject faults into `write()`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1302
Where applicable. Where the current use of `g_file_set_contents()` seems
the most appropriate, leave that in place.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1302
This is used when creating the temporary file, or new file from scratch.
I wondered about also allowing the file owner and group to be set, but
that’s not as generally applicable — if your process is operating across
multiple user IDs then it likely has some fairly OS-specific
requirements and will need tighter control of its syscalls anyway.
(Eventually, support for setting the file owner and group atomically
could be added by writing out a file using `O_TMPFILE` so it’s not
addressable, and then linking it into the file system in place of the
old file using something like `renameat2(AT_EMPTY_PATH)` or `linkat()`.
That’s currently not possible without patching the kernel with
https://marc.info/?l=linux-fsdevel&m=152472898003523&w=2, as far as I
know at the moment.)
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #1203
This moves `write_to_temp_file()` into `g_file_set_contents_full()` and
coalesces its handling of `do_fsync` with the `rename_file()` call. It
adds support for `G_FILE_SET_CONTENTS_DURABLE` and
`G_FILE_SET_CONTENTS_NONE` — previously only
`G_FILE_SET_CONTENTS_CONSISTENT | G_FILE_SET_CONTENTS_ONLY_EXISTING` was
supported.
In the case that `G_FILE_SET_CONTENTS_CONSISTENT |
G_FILE_SET_CONTENTS_DURABLE` is set, an additional `fsync()` is now done
on the directory after renaming the temporary file.
In the case that `G_FILE_SET_CONTENTS_ONLY_EXISTING` isn’t set, the
`fsync()` after writing the temporary file will always be done (unless
the file system guarantees it never needs to be done).
In the case that only `G_FILE_SET_CONTENTS_DURABLE` is set, the
destination file will be written to directly (using this mode is not
really advised).
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #1302
This introduces no functional changes, just makes the code a bit more
modular and reusable.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1302
This is a new version of the g_file_set_contents() API which will allow
its safety to be controlled by some flags, allowing the user to choose
their preferred tradeoff between safety (`fsync()` calls) and speed.
Currently, the flags do nothing and the new API behaves like the old
API. This will change in the following commits.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1302
The previous implementation of g_uri_unescape_segment() allowed non-utf8
decoded characters. uri_decoder() allows it too with FLAGS_ENCODED (I
think it's abusing a bit the user-facing flags for some internal
decoding behaviour)
However, it didn't allow \0 in the decoded string. Let's have an extra
check for that, outside of uri_decoder().
Fixes: d83d68d64c
Reported-by: Matthias Clasen <mclasen@redhat.com>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
`getenv()` doesn't work well on Windows, f.ex., it can't fetch env
vars set with `SetEnvironmentVariable()`. This also means that it
doesn't work at all when targeting UWP since that's the only way to
set env vars in that case.
This adds really basic validation that `GTimeZone` can successfully
parse a ‘slim’ format timezone file.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #2129
Since tzcode95f (1995), TZif files have had a trailing
TZ string, used for timestamps after the last transition.
This string is specified in Internet RFC 8536 section 3.3.
init_zone_from_iana_info has ignored this string, causing it
to mishandle timestamps past the year 2038. With zic's new -b
slim flag, init_zone_from_iana_info would even mishandle current
timestamps. Fix this by parsing the trailing TZ string and adding
its transitions.
Closes#2129
Time zone transition times can range from -167:59:59 through
+167:59:59, according to Internet RFC 8536 section 3.3.1;
this is an extension to POSIX. It is needed for proper
support of TZif version 3 files.
It's not clear to me why this argument was excluded in the first place,
and Dan doesn't remember either. At least for consistency with
unescape_string, add it.
See also:
https://gitlab.gnome.org/GNOME/glib/-/merge_requests/1574#note_867283
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
The illegal character set used to be applied only to the decoded
characters.
Fixes: https://gitlab.gnome.org/GNOME/glib/-/issues/2160
Fixes: d83d68d64c ("guri: new URI parsing and generating functions")
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
g_assert_true(), g_assert_cmpint(), and friends, can return to the
caller if test_nonfatal_assertions is set, but this is normally not the
case. In particular, for the purposes of static analysis, we
specifically don't want to assume that they might return. Clang has an
analyzer_noreturn attribute for this exact purpose, which conveniently
already has a macro in gmacros.h.
Fixes: #1288Fixes: #1200
Creating 1000 threads with the default stack size of 8 MiB will fail on
architectures with a 32-bit address space. Move up the existing THREADS
macro and use that instead, but change its definition to 1000 if
pointers are larger than 32 bits.
Signed-off-by: Harald van Dijk <harald@gigawatt.nl>
A query string may have some '=' characters '%'-encoded that could be
split by g_uri_parse_params() incorrectly. Instead, callers should leave
the query part encoded, and let g_uri_parse_params() do the decoding.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
This is a minor convenience, to avoid caller to do further '+' decoding.
According to the W3C HTML specification, space characters are replaced
by '+': https://url.spec.whatwg.org/#urlencoded-parsing
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
This will allow to further enhance the parsing, without breaking API,
and also makes argument on call side a bit clearer than just TRUE/FALSE.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
This should give a bit more flexibility, without drawbacks.
Many URI encoding accept either '&' or ';' as separators.
Change the documentation to reflect that '&' is probably more
common (http query string).
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
gboolean is secretly actually typedef gint gboolean, so the delim_table
is going to take 1KB of stack all by itself. That’s fine, but it could
be smaller.
This strnpbrk()-like block could do with a comment to make it a bit
clearer what it’s doing.
Suggested-by: Philip Withnall <philip@tecnocode.co.uk>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Use this to replace the much-hated `g_debug()` which told people that
`posix_spawn()` (the fast path) wasn’t being used for various reasons.
If people want to make their process spawning faster now, they’ll have
to use a profiling tool like sysprof to check their program’s
performance. Shocking.
I think I was wrong to put this `g_debug()` in there in the first place
— it hasn’t served its purpose of making people speed up their spawn
paths to use `posix_spawn()`, it’s only cluttered up logs and frustrated
people.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This allows you to see how long each `GMainContext` iteration and each
`GSource` `check`/`prepare`/`dispatch` takes. It provides more detail
than sysprof’s speedtrack plugin can provide, since it has access to
more internal GLib data.
Use it with `sysprof-cli`, for example:
```
sysprof-cli --use-trace-fd -- my-test-program
```
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Add some internal wrappers around sysprof tracing, so that it can be
used throughout GLib without exposing all the details of sysprof
internally.
This adds an optional dependency on `libsysprof-capture-4`. sysprof
support is disabled without it.
This depends on the GLib dependency of `libsysprof-capture` being
dropped in https://gitlab.gnome.org/GNOME/sysprof/-/merge_requests/30,
which has bumped the soname of `libsysprof-capture` and added subproject
support.
The next few commits will add marks that trace out each `GMainContext`
iteration and each `GSource` `check`/`prepare`/`dispatch` call.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
There is a limited (1 or 2 byte) read off the end of the buffer if its
final or penultimate byte is `%` and it’s not nul-terminated after that.
If the buffer *is* nul-terminated then the first `g_ascii_isxdigit()`
call safely returns `FALSE` and the code moves on.
Fix it by adding an additional check, and some unit tests to catch the
behaviour.
This bug is present in libsoup, which `GUri` is based on, but not
exploitable due to how the external API only exposes nul-terminated
strings. See https://gitlab.gnome.org/GNOME/libsoup/-/merge_requests/126
for the fix there.
oss-fuzz#23815
oss-fuzz#23818
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Modify the existing test function to run each test twice: once
nul-terminated and once with a length specified.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This introduces no functional changes, but will make it easier to add
more tests in future.
It splits the unescaping tests out so the different types of unescaping
(string, bytes, segment) are tested separately, since they have
different limitations.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Modify the existing test function to run each test twice: once
nul-terminated and once with a length specified.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
It seems that `sig_atomic_t` is not the same width as `int` on FreeBSD,
which is causing CI failures:
```
../glib/gmain.c:5206:3: error: '_GStaticAssertCompileTimeAssertion_73' declared as an array with a negative size
g_atomic_int_set (&any_unix_signal_pending, 0);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../glib/gatomic.h💯5: note: expanded from macro 'g_atomic_int_set'
G_STATIC_ASSERT (sizeof *(atomic) == sizeof (gint)); \
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
Fix that by only using `sig_atomic_t` if the code is *not* using atomic
primitives (i.e. in the fallback case). `sig_atomic_t` is only a typedef
around an integer type and is not magic. Its typedef is chosen by the
platform to be async-signal-safe (i.e. read or written in one instruction),
but not necessarily thread-safe.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Add a set of new URI parsing and generating functions, including a new
parsed-URI type GUri. Move all the code from gurifuncs.c into guri.c,
reimplementing some of those functions (and
g_string_append_uri_encoded()) in terms of the new code.
Fixes:
https://gitlab.gnome.org/GNOME/glib/issues/110
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Allocate a working buffer before calling `fork()` to avoid calling
`malloc()` in the async-signal-safe context between `fork()` and
`exec()`, where it’s not safe to use.
In this case, the buffer is used to assemble a wrapper around `argv` so
it can be run under `/bin/sh`.
See `man 7 signal-safety`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #2140
Allocate a working buffer before calling `fork()` to avoid calling
`malloc()` in the async-signal-safe context between `fork()` and
`exec()`, where it’s not safe to use.
In this case, the buffer is used to assemble elements from `PATH` with
the binary from `argv[0]` to try executing them.
See `man 7 signal-safety`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #2140
Query the environment before calling `fork()` so that it doesn’t have to
be called in the async-signal-safe context between `fork()` and
`exec()`.
See `man 7 signal-safety`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #2140
They’re not safe to call in an async-signal-safe context on Linux.
`sysconf()` is safe to call on FreeBSD and OpenBSD (at least), so
continue doing that.
This will reduce performance in the (already low performance) fallback
case where `/proc` is inaccessible to a forked process on Linux, while
spawning a subprocess.
See `man 7 signal-safety`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #2140
Use the error handling infrastructure which already exists for other
failures in the async-signal-safe context.
`g_assert()` is unlikely to have caused problems in practice because it
is only async-signal-unsafe when the assertion condition fails.
See `man 7 signal-safety`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #2140
While `g_ascii_isdigit()` *is* currently async-signal-safe, it’s going
to be hard to remember to keep it that way if the implementation changes
in future.
It seems more robust to just reimplement it here, given that it’s not
much code.
See `man 7 signal-safety`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #2140
Use normal `close()` instead, which is guaranteed to be
async-signal-safe.
See `man 7 signal-safety`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #2140
Functions called between `fork()` and `exec()` have to be
async-signal-safe.
Add a comment to each function which is called in that context, and
`FIXME` comments to the non-async-signal-safe functions which end up
being called as leaves of the call graph.
The following commits will fix those `FIXME`s.
See `man 7 signal-safety`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #2140
There are two variables which are used to pass state from the Unix
signal handler interrupt function to the rest of `gmain.c`. They are
currently defined as `sig_atomic_t`, which means that they are
guaranteed to be interrupt safe. However, it does not guarantee they are
thread-safe, and GLib attaches its signal handler interrupt function to
a worker thread.
Make them thread-safe using atomics. It’s not possible to use locks, as
pthread mutex functions are not signal-handler-safe. In particular, this
means we have to be careful not to end up using GLib’s fallback atomics
implementation, as that secretly uses a mutex. Better to be unsafe than
have a re-entrant call into `pthread_mutex_lock()` from a nested signal
handler.
This commit solves two problems:
1. Writes to `any_unix_signal_pending` and `unix_signal_pending` could
be delivered out of order to the worker thread which calls
`dispatch_unix_signals()`, resulting in signals not being handled
until the next iteration of that worker thread. This is a
performance problem but not a correctness problem.
2. Setting an element of `unix_signal_pending` from
`g_unix_signal_handler()` and clearing it from
`dispatch_unix_signals_unlocked()` (in the worker thread) could
race, resulting in a signal emission being cleared without being
handled. That’s a correctness problem.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #1670
We need to include the isnan*.c sources as necessary, if any of the
isnan*() functions cannot be found, so that builds on compilers that
lack these functions could be fixed.
Also, if we do have the isnan*() functions, improve the build by not
unnecessarily including the isnan*.c sources in the build.
If the isnan*() functions are found, make sure that the
HAVE_ISNAN*_IN_LIBC macros are defined in the CFLags, so that we do not
accidently require the gnulib implementations for these functions.
The implementation didn’t match the documentation. The implementation
has the right behaviour (wrt not allowing embedded nuls, validating
UTF-8, and returning a default value if an invalid string is detected),
so keep that and fix the documentation to match.
The [`GVariant`
specification](https://people.gnome.org/~desrt/gvariant-serialisation.pdf)
is incorrect on this point, and the implementation of GLib was
purposefully changed after the specification was published (but before
`GVariant` became API-stable in GLib). The behaviour in GLib
(specifically concerning all strings being in UTF-8) is consistent with
D-Bus.
Spotted by William Manley.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This was mostly machine generated with the following command:
```
codespell \
--builtin clear,rare,usage \
--skip './po/*' --skip './.git/*' --skip './NEWS*' \
--write-changes .
```
using the latest git version of `codespell` as per [these
instructions](https://github.com/codespell-project/codespell#user-content-updating).
Then I manually checked each change using `git add -p`, made a few
manual fixups and dropped a load of incorrect changes.
There are still some outdated or loaded terms used in GLib, mostly to do
with git branch terminology. They will need to be changed later as part
of a wider migration of git terminology.
If I’ve missed anything, please file an issue!
Signed-off-by: Philip Withnall <withnall@endlessm.com>
`base` can be `-1` in some situations, which would lead to pointing
outside an allocation area if the sums were evaluated as `(file_name +
base) + 1` rather than `file_name + (base + 1)`.
I don’t see how this can practically cause an issue, as the arithmetic
is all finished before anything’s dereferenced, but let’s keep to the
letter of the C standard to avoid this coming up in code audits in
future.
Fix suggested by fablhx.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Closes: #2077
Some editors automatically remove trailing blank lines, or
automatically add a trailing newline to avoid having a trailing
non-blank line that is not terminated by a newline. To avoid unrelated
whitespace changes when users of such editors contribute to GLib,
let's pre-emptively normalize all files.
Unlike more intrusive whitespace normalization like removing trailing
whitespace from each line, this seems unlikely to cause significant
issues with cherry-picking changes to stable branches.
Implemented by:
find . -name '*.[ch]' -print0 | \
xargs -0 perl -0777 -p -i -e 's/\n+\z//g; s/\z/\n/g'
Signed-off-by: Simon McVittie <smcv@collabora.com>
This is more efficient and also much easier since we already have the
memory allocated that we're going to return from the function. No need
to do that ourselves or reverse a list.
We're roundtripping from a valid file, but we should also roundtrip from
a newly created GBookmarkFile, to ensure that we set all the necessary
fields.
Just like we do for the other fields. Otherwise, when we serialise the
item, we're going to hit a segmentation fault when trying to format a
NULL GDateTime.
They use the `time_t` type, which is not year 2038 safe on 32-bit
systems, so have to be deprecated.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #1931
In preparation for deprecating the old APIs. This shouldn’t functionally
affect the tests.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1931
These are alternatives to the existing `time_t`-based APIs, which will
soon be deprecated due to `time_t` only being Y2038-safe on 64-bit
systems.
The new APIs take a GDateTime instead.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1931
When targeting mingw on architectures other than x86, the earlier cases
don't apply, and the final fallback, raise(SIGTRAP) isn't usable there.
GCC and Clang both support __builtin_trap(), so in case we have no
other alternatives, and are on windows (where raise() isn't available),
we can resort to this.
When timeout grater than 0 in g_poll function, the WaitForMultipleObjects
call will wait for all the threads to return, but when only one thread
got an event the others will sleep until the timeout elapses, and causes
a stall. Triggering the stop event in g_poll in this case is useless as
it is triggered when all the threads where already signaled or timed-out.
Closes: https://gitlab.gnome.org/GNOME/glib/issues/2107
The g_once() function exists to call a callback function exactly once,
and to block multiple contending threads on its completion, then to
return its return value to all of them (so they all see the same value).
The full implementation of g_once() (in g_once_impl()) uses a mutex and
condition variable to achieve this, and is needed in the contended case,
where multiple threads need to be blocked on completion of the callback.
However, most of the times that g_once() is called, the callback will
already have been called, and it just needs to establish that it has
been called and to return the stored return value.
Previously, a fast path was used if we knew that memory barriers were
not needed on the current architecture to safely access two dependent
global variables in the presence of multi-threaded access. This is true
of all sequentially consistent architectures.
Checking whether we could use this fast path (if
`G_ATOMIC_OP_MEMORY_BARRIER_NEEDED` was *not* defined) was a bit of a
pain, though, as it required GLib to know the memory consistency model
of every architecture. This kind of knowledge is traditionally a
compiler’s domain.
So, simplify the fast path by using the compiler-provided atomic
intrinsics, and acquire-release memory consistency semantics, if they
are available. If they’re not available, fall back to always locking as
before.
We definitely need to use `__ATOMIC_ACQUIRE` in the macro implementation
of g_once(). We don’t actually need to make the `__ATOMIC_RELEASE`
changes in `gthread.c` though, since locking and unlocking a mutex
guarantees to insert a full compiler and hardware memory barrier
(enforcing sequential consistency). So the `__ATOMIC_RELEASE` changes
are only in there to make it obvious what stores are logically meant to
match up with the `__ATOMIC_ACQUIRE` loads in `gthread.h`.
Notably, only the second store (and the first load) has to be atomic.
i.e. When storing `once->retval` and `once->status`, the first store is
normal and the second is atomic. This is because the writes have a
happens-before relationship, and all (atomic or non-atomic) writes
which happen-before an atomic store/release are visible in the thread
doing an atomic load/acquire on the same atomic variable, once that load
is complete.
References:
* https://preshing.com/20120913/acquire-and-release-semantics/
* https://gcc.gnu.org/onlinedocs/gcc-9.2.0/gcc/_005f_005fatomic-Builtins.html
* https://gcc.gnu.org/wiki/Atomic/GCCMM/AtomicSync
* https://en.cppreference.com/w/cpp/atomic/memory_order#Release-Acquire_ordering
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #1323
There were multi-threaded tests for g_once_init_{enter,leave}(), but not
for g_once(). Add one which tests multi-threaded contention for entering
and retrieving the value of the `GOnce`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1323
It’s not expected that bindings will use `GThread` over their own
threading APIs (in fact that would generally be a bad idea, since
threads benefit from being integrated into language control flow
structures), but it can’t hurt to have the annotations right for
documentation purposes if nothing else.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #602
Unify the creation of GPtrArray.
Maybe we will add yet another constructor for creating %NULL terminated
arrays. Unify the constructors by adding an internal helper method.
The alternative instead of adding a ptr_array_new() helper, would be to
let everybody call g_ptr_array_full(). For no strong reasons, choose
this approach because the compiler is more eager to inline the static
helper as it would inlining g_ptr_array_full().
This reverts commit c0146be3a4.
The revert was originally added because the original change broke
gnome-build-meta. Now that the problem has been diagnosed, the original
commit can be fixed — see the commit which follows this one.
See: !1487
The public functions exposed as static inlines currently don't have
annotations to describe when they were introduced. This means that
compiling this file:
#include <glib.h>
void foo (void)
{
g_rec_mutex_locker_new (NULL);
}
with:
gcc -c test.c \
-I/tmp/glib/include/glib-2.0 \
-I/tmp/glib/lib/x86_64-linux-gnu/glib-2.0/include \
-Werror \
-DGLIB_VERSION_MAX_ALLOWED=GLIB_VERSION_2_28 \
-DGLIB_VERSION_MIN_REQUIRED=GLIB_VERSION_2_28
will not produce any error message, despite using
`g_rec_mutex_locker_new`, a function that was introduced after 2.28.
This patch adds some annotations to all the publicly exposed static
inline functions I could find.
I could not use the existing G_AVAILABLE* macros, because they may
expand to `extern`. This would then clash with the `static` keyword and
produce:
../glib/gthread.h:397:1: error: multiple storage classes in declaration specifiers
397 | static inline GRecMutexLocker *
| ^~~~~~
So I opted for adding a new set of macros,
GLIB_AVAILABLE_STATIC_INLINE_IN_2_XY.
With this patch applied, the example from above produces the expected
warning:
test.c: In function ‘foo’:
test.c:5:3: error: ‘g_rec_mutex_locker_new’ is deprecated: Not available before 2.60 [-Werror=deprecated-declarations]
5 | g_rec_mutex_locker_new (NULL);
| ^~~~~~~~~~~~~~~~~~~~~~
In file included from /tmp/glib/include/glib-2.0/glib/gasyncqueue.h:32,
from /tmp/glib/include/glib-2.0/glib.h:32,
from test.c:1:
/tmp/glib/include/glib-2.0/glib/gthread.h:398:1: note: declared here
398 | g_rec_mutex_locker_new (GRecMutex *rec_mutex)
| ^~~~~~~~~~~~~~~~~~~~~~
This seems to have existed since the code was written and I can’t see
a need for it.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #2106
g_ptr_array_extend_and_steal() leaves the GPtrArray in an invalid state,
so if you would try to append another pointer, it leads to a crash.
Also adjust the test case so that it would result in the crash (without
the fix).
Fixes: 0675703af0 ('Adding g_ptr_array_extend_and_steal() function to glib/garray.c')
With a trivial file that just includes glib.h:
#include <glib.h>
Compiled with:
gcc -c test.c \
-I /tmp/glib/include/glib-2.0/ \
-I /tmp/glib/lib/x86_64-linux-gnu/glib-2.0/include \
-DGLIB_VERSION_MAX_ALLOWED=GLIB_VERSION_2_28 \
-DGLIB_VERSION_MIN_REQUIRED=GLIB_VERSION_2_28 \
-fmax-errors=1 \
-Werror
We get:
In file included from /tmp/glib/include/glib-2.0/glib/gasyncqueue.h:32,
from /tmp/glib/include/glib-2.0/glib.h:32,
from test.c:1:
/tmp/glib/include/glib-2.0/glib/gthread.h: In function ‘g_rec_mutex_locker_new’:
/tmp/glib/include/glib-2.0/glib/gthread.h:396:3: error: ‘g_rec_mutex_lock’ is deprecated: Not available before 2.32 [-Werror=deprecated-declarations]
396 | g_rec_mutex_lock (rec_mutex);
| ^~~~~~~~~~~~~~~~
/tmp/glib/include/glib-2.0/glib/gthread.h:196:17: note: declared here
196 | void g_rec_mutex_lock (GRecMutex *rec_mutex);
| ^~~~~~~~~~~~~~~~
compilation terminated due to -fmax-errors=1.
The problem is that the code in the static inline functions uses
g_rec_mutex_lock, introduced after 2.28. This code is compiled
regardless of if it's actually used or not.
Suppress the warning by using G_GNUC_BEGIN_IGNORE_DEPRECATIONS /
G_GNUC_END_IGNORE_DEPRECATIONS. There are precedents for doing that,
for example g_main_context_pusher_new in gmain.h.
Tested by building with all variations of GLIB_VERSION_MIN_REQUIRED /
GLIB_VERSION_MAX_ALLOWED:
for i in $(seq 26 2 64); do
gcc -c test.c \
-I/tmp/glib/include/glib-2.0 \
-I/tmp/glib/lib/x86_64-linux-gnu/glib-2.0/include \
-DGLIB_VERSION_MAX_ALLOWED=GLIB_VERSION_2_$i \
-DGLIB_VERSION_MIN_REQUIRED=GLIB_VERSION_2_$i \
-fmax-errors=1 \
-Werror
done
Fixes: #2094
Otherwise the code ends up calling `gettext ("")`, which returns the
current translation’s header. The caller should really have provided a
NULL, parameter string, but the empty string is clear enough.
See
https://gitlab.freedesktop.org/accountsservice/accountsservice/-/merge_requests/56.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
When g_variant_get_child() is called on a variant which has not been
serialized, it serializes it which includes a call to
g_variant_release_children() and therefore means that any children
previously retrieved from the variant are no longer valid (unless
another reference is held on them) and consequently values borrowed from
those children are no longer safe to access. Add a note to the
g_variant_get_child_value() documentation to explain this.
Alternatively, we could say that after the child is freed, values
borrowed from it are no longer valid. But we already have an
implementation which hasn't changed in years which lets them stay valid
if the variant was serialized before the first
g_variant_get_child_value() call.
Here's a demonstration of the memory error:
static const char *get_first_child (GVariant *v) {
g_autoptr(GVariant) child_v = g_variant_get_child_value (v, 0);
return g_variant_get_string (child_v, NULL);
}
int main(int argc, char **argv) {
g_autoptr(GVariant) v = g_variant_new("(@ss)", g_variant_new_string ("hello"), "world");
const char *child1 = get_first_child (v);
const char *child2;
g_variant_get_child (v, 1, "&s", &child2);
printf ("%s\n", child1); // this is a memory error
return 0;
}
The local change from af0e0cb995 in glib ended upstreamed
as 7359c5fd9f312cddd62146896558d8c9bd2bd4cf in valgrind, with
a few minor adjustments requested from there.
Sync this local modification to what ended up upstreamed, to avoid
any doubt regarding it for future syncs of the whole header.
The gnulib math code uses __builtin_isnanf and __builtin_isnanl
within the __GNUC__ >= 4 ifdef, and clang doesn't provide those
builtins, only the one without a suffix. Make the meson check
match the code it controls, using the exactly right builtins.
Set REPLACE_ISNAN to 1 if either of the have_isnan* functions failed,
this matches how gnulib's m4 routines does it (in gnulib/m4/isnan.m4).
This fixes the isnan functions in the gnulib math header replacement
work on Clang.
Many locales have no concept of AM/PM notation, and it’s confusing to
them. It’s an Anglo-centric concept which doesn’t belong in the API at
this level — instead, programs should use more generic format specifiers
which leave the details of how to format a date/time to the locale.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #2082
Meson 0.54.0 added a new method meson.override_dependency() that must be
used to ensure dependency consistency. This patch ensures a project that
depends on glib will never link to a mix of system and subproject
libraries. It would happen in such cases:
The system has glib 2.40 installed, and a project does:
dependency('glib-2.0', version: '>=2.60',
fallback: ['glib', 'glib_dep'])
dependency('gobject-2.0')
The first call will configure glib subproject because the system libglib
is too old, but the 2nd call will return system libgobject.
By overriding 'gobject-2.0' dependency while configuring glib subproject
during the first call, meson knows that on the 2nd call it must return
the subproject dependency instead of system dependency.
This also has the nice side effect that with Meson >0.54.0 an
application depending on glib can declare the fallback without knowing
the dependency variable name: dependency('glib-2.0', fallback: 'glib').
"\t" is not escaped by g_markup_escape_text(), as per its documentation:
"Note that this function doesn't protect whitespace and line endings
from being processed according to the XML rules for normalization of
line endings and attribute values."
The relevant portion of the XML specification
https://www.w3.org/TR/xml/#AVNormalize
"For a character reference, append the referenced character to the
normalized value."
"For a white space character (#x20, #xD, #xA, #x9), append a space
character (#x20) to the normalized value."
So the unescape code in GMarkup does the right thing as can be verified
by the added valid-17.* data files for the markup-parse unit test.
(Note that the valid-13.* data files already tested a plain tab
character in an attribute value, among other white space characters).
Note that the libxml2's xmlSetProp() function escapes "\t" into the
character reference "	".
See https://gitlab.gnome.org/GNOME/glib/-/issues/2080
Update our copy of valgrind.h from the Valgrind 3.15 release tarball,
and then re-apply our downstream change af0e0cb995. No other
changes made.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
malloc() was given too small value (size of pointer,
rather than struct it points to), and subsequent call
getpwnam_r() wrote past the end of allocated block - easily
seen with valgrind.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Don't assume that __MINGW32__ implies x86; Windows runs on ARM/ARM64
as well, and there are mingw toolchains that target those architectures.
This mirrors how the MSVC part of the same expressions are written,
as (defined(_WIN32) && defined(_M_IX86)) and
(defined(_WIN64) && defined(_M_X64)) - not relying on _WIN32/_WIN64
or __MINGW32__/__MINGW64__ alone to indicate architecture.
The __declspec(dllexport) attribute in itself doesn't imply
'extern' - thus any intended variable declaration with
GLIB_VAR/GOBJECT_VAR would actually be a variable definition. With
C compilers defaulting to -fcommon, this isn't an issue, but
upcoming compilers (GCC 10 and Clang 11) will default to -fno-common,
ending up with duplicate definitions of these variables.
Using commands:
```
glib/gen-unicode-tables.pl -both 13.0.0 path/to/UCD
tests/gen-casefold-txt.py 13.0.0 path/to/UCD/CaseFolding.txt \
> tests/casefold.txt
tests/gen-casemap-txt.py 13.0.0 path/to/UCD/UnicodeData.txt \
path/to/UCD/SpecialCasing.txt > tests/casemap.txt
```
Using UCD release https://www.unicode.org/Public/zipped/13.0.0/UCD.zip
With some manual additions to `GUnicodeScript` for the 4 new scripts
added in 13.0, using the first assigned character in each block in
`glib/tests/unicode.c`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
The test checks `g_str_match_string()` function, which performs matches
based on user's locale. For this reason, some tests may fail, e.g., see
issue #868.
Now we explicitly set locale for each test, with C locale as a fallback
when the locale is not available.
clang complains about this in the form of
<source>:6:9: warning: result of comparison against a string literal is
unspecified (use an explicit string comparison function instead)
if (f == (void *)"a") {
^ ~~~~~~~~~~~
Use variables for the strings instead, which should have the same
address.
It seems a bit unhelpful to list every single `g_assert_*()` macro, to
rephrase the documentation to use globs instead.
Add a missing word below.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This is for use in testing POSIX-style functions like `rmdir()`, which
return an integer < 0 on failure, and return their error information in
`errno`.
The new macro prints `errno` and `g_strerror (errno)` on failure.
Includes a unit test.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
`mem-overflow` test disables GCC warning `alloc-size-larger-than` via
diagnostic pragma, but it's still emitted in the linkage stage when
LTO is enabled.
This changes explicitly set `link_args` for the test to disable the
warning.
Some compilers have __GNUC__ defined as a value less than 6, but do
support implicit-fallthrough via __attribute__((fallthrough)). Define
G_GNUC_FALLTHROUGH for those that support __has_attribute.
Spotted by Mohammed Sadiq. `g_array_copy()` was doing a `memcpy()` of
the data from the old array to the new one, based on the reserved
elements in the old array (`array->alloc`). However, the new array was
allocated based on the *assigned* elements in the old array
(`array->len`).
So if the old array had fewer assigned elements than allocated elements,
`memcpy()` would fall off the end of the newly allocated data block.
This was particularly obvious when the old array had no assigned
elements, as the new array’s data pointer would be `NULL`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #2049
Most of these scripts can probably just be deleted (see issue #2045),
but for now it was easier to just mechanically fix the shellcheck
warnings in them, rather than think about whether we actually needed the
script.
Fixes done using shellcheck 0.7.0 with default options. I haven’t tested
any of the changes.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Move the separate function inline using g_once_init_{enter,leave}(),
rather than g_once().
This marginally improves performance, taking 0.39s to create 10000000
UTC timezones, rather than 0.43s previously. (Who cares?)
Follow-up to !1105.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
It’s quite surprising that this wasn’t documented already. Hash tables
are unordered, and any recognisable iteration ordering is not guaranteed
and might change in future releases.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Instead only do a g_critical(). This is something that has to be fixed
one way or another, but a g_critical() is less disruptive and makes sure
that code that worked in previous GLib versions still works as bad as
before.
Fixes https://gitlab.gnome.org/GNOME/glib/issues/2039
Some CI platforms invoke these tests with euid != 0 but with
capabilities. Detect whether we have Linux CAP_DAC_OVERRIDE or other
OSs' equivalents, and skip tests that rely on DAC permissions being
denied if we do have that privilege.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Fixes: https://gitlab.gnome.org/GNOME/glib/issues/2027
Fixes: https://gitlab.gnome.org/GNOME/glib/issues/2028
Some CI platforms invoke tests as euid != 0, but with capabilities that
include CAP_SYS_RESOURCE and/or CAP_SYS_ADMIN. If we detect this,
we can't test what happens if our RLIMIT_NPROC is too low to create a
thread, because RLIMIT_NPROC is bypassed in these cases.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Fixes: https://gitlab.gnome.org/GNOME/glib/issues/2029
This does not have any behaviour changes but is cleaner. The mutex is
only unlocked now after all operations on the context are done and right
before freeing the mutex and the context itself.
Instead of destroying sources directly while freeing the context, and
potentially freeing them if this was the last reference to them, collect
new references of all sources in a separate list before and at the same
time invalidate their context so that they can't access it anymore. Only
once all sources have their context invalidated, destroy them while
still keeping a reference to them. Once all sources are destroyed we get
rid of the additional references and free them if nothing else keeps a
reference to them anymore.
This fixes a regression introduced by 26056558be in 2012.
The previous code that invalidated the context of each source and then
destroyed it before going to the next source without keeping an
additional reference caused memory leaks or memory corruption depending
on the order of the sources in the sources lists.
If a source was destroyed it might happen that this was the last
reference to this source, and it would then be freed. This would cause
the finalize function to be called, which might destroy and unref
another source and potentially free it. This other source would then
either
- go through the normal free logic and change the intern linked list
between the sources, while other sources that are unreffed as part of
the main context freeing would not. As such the list would be in an
inconsistent state and we might dereference freed memory.
- go through the normal destroy and free logic but because the context
pointer was already invalidated it would simply mark the source as
destroyed without actually removing it from the context. This would
then cause a memory leak because the reference owned by the context is
not freed.
Fixes https://github.com/gtk-rs/glib/issues/583 while still keeping
https://bugzilla.gnome.org/show_bug.cgi?id=661767 fixes.
We first have to ref the next source and then unref the previous one.
This might be the last reference to the previous source, and freeing the
previous source might unref and free the next one which would then leave
use with a dangling pointer here.
Fixes https://gitlab.gnome.org/GNOME/glib/issues/2031
It was used for running tests when we built with autotools, but is no
longer used in the Meson build system. If we need something similar in
future, it should be done by adding internal API to override the
directory on a per-call basis, rather than loading a path from a shared
global table every time.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1919
There were a couple of custom paths which could end up being relative,
rather than absolute, due to not properly prefixing them with
`get_option('prefix')`.
The use of `join_paths()` here correctly drops all path components
before the final absolute path in the list of arguments. So if someone
configures GLib with an absolute path for `gio_module_dir`, that will be
used unprefixed; but if someone configures with a relative path, it will
be prefixed by `get_option('prefix)`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #1919
Even if the key already exists in the table, `g_hash_table_add()` will
call the hash table’s key free func on the old key and will then replace
the old key with the newly-passed-in key. So `key` is always `(transfer
full)`.
In particular, `key` should never need to be freed by the caller if
`g_hash_table_add()` returns `FALSE`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Since we (optionally) require nanosecond precision for this
(utimes() is used on *nix), use SetFileTime(), which nominally
has 100ns granularity (actual filesystem might be coarser), instead of
g_utime (), which only has 1-second granularity.
Expand our private statbuf structure with st_mtim, st_atim and st_ctim
fields, which are structs that contain tv_sec and tv_nsec fields,
representing a timestamp with 1-second precision (same value as st_mtime, st_atime
and st_ctime) and a fraction of a second (in nanoseconds) that adds nanosecond
precision to the timestamp.
Because FILEETIME only has 100ns precision, this won't be very precise,
but it's better than nothing.
The private _g_win32_filetime_to_unix_time() function is modified
to also return the nanoseconds-remainder along with the seconds timestamp.
The timestamp struct that we're using is named gtimespec to ensure that
it doesn't clash with any existing timespec structs (MinGW-w64 has one,
MSVC doesn't).
794c1a30bc "macro wrappers for
g_once_init_enter/leave" added this line (whose intent is unclear to me).
<http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1152r4.html>
"Deprecating volatile" (scheduled for inclusion in C++20) will make the
assignment expression
*(location) = (result)
deprecated when the LHS is of (non-class) volatile type, which is the case when
g_once_init_leave is expanded as part of e.g. G_DEFINE_TYPE_WITH_CODE (in
gobject/gtype.h), where location is a pointer to some
static volatile gsize g_define_type_id__volatile = 0;
Recent Clang trunk emits -Wdeprecated-volatile for it under -std=c++2a since
<https://github.com/llvm/llvm-project/commit/
4a6861a7e5b59be24a09b8b9782255d028e7aade> "[c++20] P1152R4: warn on any
simple-assignment to a volatile lvalue".
The fix is to make the assignment expression a discared-value expression by
casting it to void (which in turn requires casting the second branch of the
surrounding conditional expression to void, too; not sure what the top-level
cast to void was intended for, and whether it would still be needed under
certain circumstances).
Previously we would only ever run the initialization code once, but part
of it only if a non-exclusive pool was created. This caused the shared
state to be inconsistent if the first pool to be created was exclusive
and the second non-exclusive.
Fixes https://gitlab.gnome.org/GNOME/glib/issues/2012
g_setenv() and g_unsetenv() can never be thread-safe, so emit a message if
they are used after any threads have been spawned.
This can’t catch interactions between setenv() and g_thread_new(), or
between g_setenv() and pthread_create(), but it’ll catch most
misbehaviour in GLib-centric code.
Currently, the message is a `g_debug()` call. Eventually, I’d like to
upgrade it to a `g_warning()`, but there are a number of GLib tests
which call g_setenv() after threads have been created, and they need to
be fixed first. Emitting a `g_debug()` message gives people an
opportunity to start fixing their code.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #715
This will be used in a following commit to warn if setenv() is used
after another thread has been created.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #715
It’s now used unconditionally for `shared_thread_scheduler_settings` in
`gthreadpool.c`, so it actually needs to have a storage definition.
Fixes a build failure on FreeBSD.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
On Linux the sched_getattr syscall might be available at compile-time
but not actually work at runtime (e.g. because an older kernel is
running or valgrind is used). Instead of killing the process, return
FALSE and handle this gracefully at runtime with some fallback code.
Fixes https://gitlab.gnome.org/GNOME/glib/issues/2007
By default (on POSIX) we would be inheriting thread priorities from the
thread that pushed a new task on non-exclusive thread pools and causes a
new thread to be created. This can cause any non-exclusive thread pool
to accidentally contain threads of different priorities, or e.g. threads
with real-time priority.
To prevent this, custom handling for setting the scheduler settings for
Linux and Windows is added and as a fallback for other platforms a new
thread is added that is responsible for spawning threads for
non-exclusive thread pools.
Fixes https://gitlab.gnome.org/GNOME/glib/issues/1834
This means that GLib will generate appropriate deprecation warnings if
`GLIB_VERSION_MAX_ALLOWED` has been set to target a GLib version older
than 2.58.
The timezone(3) man page on Fedora 31 describes the start/end
field in the POSIX TZ format as follows:
[quote]
The start field specifies when daylight saving time goes
into effect and the end field specifies when the change is
made back to standard time. These fields may have the fol‐
lowing formats:
Jn This specifies the Julian day with n between 1 and
365. Leap days are not counted. In this format,
February 29 can't be represented; February 28 is day
59, and March 1 is always day 60.
n This specifies the zero-based Julian day with n
between 0 and 365. February 29 is counted in leap
years.
Mm.w.d This specifies day d (0 <= d <= 6) of week w (1 <= w
<= 5) of month m (1 <= m <= 12). Week 1 is the
first week in which day d occurs and week 5 is the
last week in which day d occurs. Day 0 is a Sunday.
[/quote]
The GTimeZone code does not correctly parse the 'n' syntax,
treating it as having the range 1-365, the same as the 'Jn'
syntax. This is semantically broken as it makes it impossible
to represent the 366th day, which is the purpose of the 'n'
syntax.
There is a code comment saying this was done because the Linux
semantics are different from zOS and BSD. This is not correct,
as GLibC does indeed use the same 0-365 range as other operating
systems. It is believed that the original author was mislead by
a bug in old versions of the Linux libc timezone(3) man pages
which was fixed in
commit 5a554f8e525faa98354c1b95bfe4aca7125a3657
Author: Peter Schiffer <pschiffe@redhat.com>
Date: Sat Mar 24 16:08:10 2012 +1300
tzset.3: Correct description for Julian 'n' date format
The Julian 'n' date format counts atrting from 0, not 1.
Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>
Fixes: #1999
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
The callers of parse_tz_boundary inverted the value passed
for the ignore_leap parameter. Fortunately the method impl
also had an inverted test cancelling out the first bug.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
The implementation has always wrapped at 76 characters, rather than 72,
ever since it was introduced in commit 5cf8f1d4a8 in 2006. At this
stage, it’s probably best to fix the documentation rather than the
implementation.
The likely bug in the implementation is the comparison
```
(++already) >= 19
```
19 × 4 = 76, so it seems like an off-by-one error in the comparison.
What was actually wanted was 18 × 4 = 72.
Thanks to Simon McVittie for the investigation and diagnosis.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #1997
Similar to 3837b83f, glibc memcpy is declared with the first two
arguments annotated as non-null via an attribute, which results in the
undefined behaviour sanitizer considering it to be UB to pass a null
pointer there (even if we are copying no bytes, and hence not actually
dereferencing the pointer).
Signed-off-by: Simon McVittie <smcv@collabora.com>
g_atomic_pointer_compare_and_exchange() should work with const pointers.
Add a test for that.
It seems clang 9.0.0-2.fc32 does not like this:
../glib/tests/atomic.c:93:9: warning: incompatible pointer types passing 'typeof ((((void *)0))) *' (aka 'void **') to parameter of type 'const char **' [-Wincompatible-pointer-types]
res = g_atomic_pointer_compare_and_exchange (&vp_str, NULL, str);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../glib/gatomic.h:192:44: note: expanded from macro 'g_atomic_pointer_compare_and_exchange'
__atomic_compare_exchange_n ((atomic), &gapcae_oldval, (newval), FALSE, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST) ? TRUE : FALSE; \
^~~~~~~~~~~~~~
../glib/tests/atomic.c:96:9: warning: incompatible pointer types passing 'typeof ((((void *)0))) *' (aka 'void **') to parameter of type 'const char **' [-Wincompatible-pointer-types]
res = g_atomic_pointer_compare_and_exchange (&vp_str_vol, NULL, str);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../glib/gatomic.h:192:44: note: expanded from macro 'g_atomic_pointer_compare_and_exchange'
__atomic_compare_exchange_n ((atomic), &gapcae_oldval, (newval), FALSE, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST) ? TRUE : FALSE; \
^~~~~~~~~~~~~~
Note that this clang version already issues various compiler warnings for
this test. This merely adds another case to check.
Eventually g_atomic_pointer_compare_and_exchange() should be fixed to
avoid compiler warnings.
Actually there is a problem. When you try to use g_atomic_pointer_compare_and_exchange()
with const pointers, it is also not working, because the implementation
as a function expects "void *" arguments. As the test also shows. As such,
it's probably not portable to use g_atomic_pointer_compare_and_exchange()
with const pointers at all. However, the macro implementation is (with the right
compiler) fine with that, so it's an easy "mistake" to make.
Even if g_variant_get_va(), which we eventually call, checks on the
validity of the value argument, check it early to avoid surprises, such
as this intermittent FreeBSD test failure:
(/var/tmp/gitlab_runner/builds/3fe11159/0/GNOME/glib/_build/gio/tests/gdbus-connection:65788): GLib-CRITICAL **: 15:13:25.670: g_variant_get_va: assertion 'value != NULL' failed
Especially check for a valid reference count. This is possible now in
all cases because of the addition of the dispose function and makes
usage of already finalized/finalizing GSources more obvious than the
use-after-free that would otherwise happen.
This allows GSource implementors to safely clear any other references to
the GSource while the GSource is still valid, unlike when doing the same
from the finalize function.
After the dispose function has run, it is valid for the reference count
of the GSource to be > 0 again to allow the case where another thread in
the mean-time got access to the GSource reference before the dispose
function was called.
This allows fixing a thread-safety issue in the GCancellable, GstBus and
various other GSource implementations.
When compiling a program using glib with -Wzero-as-null-pointer-constant
warnings enabled, the compiler warns about this type check in the
g_once_init_enter macro. Fix by replacing "0" with "NULL".
This is a convenience wrapper around getpwnam_r() which handles all the
memory allocation faff.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1687
Although not quite as often-occurring, this should help with constructs
like this:
if (list)
{
g_list_free_full (list, foo);
list = NULL;
}
Closes https://gitlab.gnome.org/GNOME/glib/issues/1943
In C, the proper type for a heap allocate structure is size_t/gsize.
That means, no valid (heap allocated) pointer will ever contain more
bytes than size_t can represent.
Hence, this integer type should also be used when operating on
data like a strv array. Adjust some internal uses to use gsize
instead of gint/guint.
Note that g_strv_length() returns a value of type guint. So this
API cannot be used on string arrays longer of arbitrary size. But
that is not fixable.
This reverts commit fd3ed5e31b.
C11-style atomics have been fixed (on FreeBSD and other platforms) in
the previous commit, “gatomic: Check argument width in
g_atomic_pointer_compare_and_exchange()”.
See !1229 and #1940.
Check that the new value is actually pointer-width, rather than (for
example) `sizeof (int)` which could happen if someone used `0` rather
than `NULL`.
Changes suggested by Simon McVittie.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1940
Don’t pass integers; it’s not type-safe. The macro version of
`g_atomic_pointer_compare_and_exchange()` used to erroneously accept
integers, but they would have the wrong width on some platforms.
Changes originally investigated and suggested by Ting-Wei Lan.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #1940
They’re causing the CI to fail. While someone familiar with FreeBSD
investigates the failure, it’s easiest to disable all C11-style atomics
than add more preprocessor checks to only disable the atomics added in
!1123.
If nobody can fix the new C11-style atomics before the 2.64.0 release,
this commit should be reverted and a more comprehensive set of preprocessor
checks put in place to essentially revert !1123 for BSD only.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1940
In glib/gutf8.c there was an UB in function g_utf8_find_prev_char when
p == str. In this case we substract one from p and now p points to a
location outside of the boundary of str. It's a UB by the standard.
Since this function are meant to be fast, we don't check the boundary
conditions.
Fix glib/tests/utf8-pointer test. It failed due to the UB described
above and aggressive optimisation when -O2 and LTO are enabled. Some
compilers (e.g. GCC with major version >= 8) create an optimised version
of g_utf8_find_prev_char with the first argument fixed and stored
somewhere else (with a different pointer). It can be solved with either
marking str as volatile or creating a copy of str in memory. We choose
the second approach since it's more explicit solution.
Add additional checks to glib/tests/utf8-pointer test.
Closes#1917
Commit 7678b107 seems to have left the GHashTable pretty printer with an
off-by-one error, skipping the first key it encounters and printing an
extra garbage key/value pair instead. This fixes that by moving an
increment to the end of a loop rather than the beginning.
This ensures that when running many instances of the test in parallel,
they don’t collide in the same current directory, and hence spuriously
fail. This can happen when writing `out.xbel`, for example.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #1930
In general, we should aim to always check a `GError` before checking a
boolean, since the error message from the `GError` gives us a lot more
information about failure, which helps with debugging.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
The time handling was assuming that the test would complete in the same
second as it started, which was not always true.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #1930
`g_assert_*()` gives more useful messages on failure, and isn’t compiled
out by `G_DISABLE_ASSERT`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Similar to 3837b83f, glibc memcmp is declared with the first two
arguments annotated as non-null via an attribute, which results in the
undefined behaviour sanitizer considering it to be UB to pass a null
pointer there (even if we are comparing 0 bytes, and hence not actually
dereferencing the pointer).
This shows up in /gvariant/serialiser/children when run with the
undefined behaviour sanitizer.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Similar to 3837b83f, glibc memcmp is declared with the first two
arguments annotated as non-null via an attribute, which results in the
undefined behaviour sanitizer considering it to be UB to pass a null
pointer there (even if we are comparing 0 bytes, and hence not actually
dereferencing the pointer).
This shows up in /gvariant/serialiser/children when run with the
undefined behaviour sanitizer.
Signed-off-by: Simon McVittie <smcv@collabora.com>
kdeinit5 overwrites argv, which in turn results in /proc/self/cmdline
being overwritten. It seems that this is done in a way that does not
necessarily guarantee that /proc/self/cmdline will end up NUL-terminated.
However, g_file_get_contents() is documented to fill a buffer of size
len + 1, where buffer[len] == '\0', even if the file's actual contents
(from buffer[0] to buffer[len-1] inclusive) did not include a NUL;
so we can safely relax this assertion slightly.
Resolves: https://gitlab.gnome.org/GNOME/glib/issues/1923
Signed-off-by: Simon McVittie <smcv@collabora.com>
Change a condition from one to an equivalent one to shut up a
`scan-build` warning about potentially dereferencing a `NULL` value.
This introduces no functional changes, as it’s not actually possible to
dereference a `NULL` value here (but `scan-build` can’t link the
nullability of `error` to the nullability of `result`).
Signed-off-by: Philip Withnall <withnall@endlessm.com>
The compiler can’t work out from the combination of other conditions
that it’s not possible for (m2 == NULL) to hold true when memcmp() is
called, so add an explicit condition.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #1897
Previously, these GTimeZone objects were being cached in the `time_zones` cache, but dropped from it when their final ref was dropped (which was frequently). That meant additional reads of `/etc/localtime` next time they were created, which was noticeable on profiles. Keep a permanent ref to the UTC and local timezones.
This removes the limitation of select() that only FDs with values lower
than FD_SETSIZE can be used. Previously, if the out/err pipe FDs had
high values (which could happen if a large process, like Firefox, was
spawning subprocesses while having a lot of FDs open), GLib would abort
due to an assertion failure in libc.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #954
glibc declares memcpy() with the first two arguments (the pointers)
annotated as non-null via an attribute, which results in the undefined
behaviour sanitizer considering it to be UB to pass a null pointer
in the second argument, even if we are copying 0 bytes (and hence not
actually dereferencing the pointer).
This shows up in array-test when run with the undefined behaviour
sanitizer.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Note that I deliberately haven't used g_autoptr here, because while we
encourage GLib users to use g_autoptr in their own code, GLib itself
still supports being compiled in environments like MSVC that can't
support g_autoptr.
Signed-off-by: Simon McVittie <smcv@collabora.com>
The user_data for g_ptr_array_sort_with_data is passed directly, not
with an extra layer of pointer like the data pointers.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Fixes: 52c130f8
Let's not encourage library users to sprinkle casts through their code
when they don't need to.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Fixes: 52c130f8
This is like `GMutexLocker`, in that if you are able to use
`g_autoptr()`, it makes popping a `GMainContext` off the thread-default
main context stack easier when exiting a function.
A few uses of `G_GNUC_{BEGIN,END}_IGNORE_DEPRECATIONS` are needed to
avoid warnings when building apps against GLib with
`GLIB_VERSION_MAX_ALLOWED < GLIB_VERSION_2_64`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
We may need to declare autocleanups for new types, which will be marked
as ‘deprecated’ if the code which includes GLib doesn’t declare a high
enough `GLIB_VERSION_MAX_ALLOWED`. Despite that, we still need to
declare the autocleanups.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
The token parsing done by g_variant_parse() uses recursive function
calls, so at some point it will hit the stack limit. As with previous
changes to `GVariantType` parsing (commit 7c4e6e9fbe), limit the level
of nesting of containers parsed by g_variant_parse() to something
reasonable. We guarantee 64 levels of nesting, which should be enough
for anyone, and is the same as what we guarantee for types.
oss-fuzz#10286
Signed-off-by: Philip Withnall <withnall@endlessm.com>
On closer reading of `man 3 timezone`, it’s actually permissible for
`TZ` to contain an absolute path which points to a tzfile file outside
the system time zone database. This is indeed what happens when building
GLib under Fedora’s toolbox, so relax that check in the tests.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
There are a lot of Unix-like systems which have not implemented the
os-release spec. On such system, we can use POSIX uname function as a
fallback to get basic information of the system.
/etc/os-release is a spec designed for Linux. While other OSes can
implement it, it doesn't make sense to use Linux as the default value
on systems which don't use Linux.
Most of the info returned is static, the only thing that changes
is the OS version.
This code relies on g_win32_check_windows_version() providing
accurate information (hopefully, MS won't nix RtlGetVersion() on
which we use for that) and supplements it with information from the
registry for Windows >= 8.1.
In many places the pattern
static gboolean warned_once = FALSE;
if (!warned_once)
{
g_warning ("This and that");
warned_once = TRUE;
}
is used to not spam the same warning message over and over again. Add a
helper in glib for this, allowing the above statement to be changed to
g_warning_once ("This and that");
os-release(5) is widely implemented on Linux, but not necessarily
ubiquitous: unusual or minimal Linux distributions might not have it.
It could in principle be implemented by any other Unix OS, but in
practice this has not yet happened.
Closes: https://gitlab.gnome.org/GNOME/glib/issues/1906
Fixes: 349318e8 "gutils: Add g_get_os_info()"
Signed-off-by: Simon McVittie <smcv@collabora.com>
As an unsigned integer, this variable is always greater than or equal to
zero. Fixes a compiler warning on Android.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
There were tests for the situation where it does the exchange and
returns true, but no tests for the situation where it returns false.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Document that g_vasprintf and g_strdup_printf are guaranteed to return a
non-NULL string, unless the format string contains the locale sensitive
conversions %lc or %ls.
Further annotate that the output parameter for g_vasprintf and the
format string for all functions must be non-NULL.
Fixes#1622
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The g_vasprintf method is called by g_strdup_vprintf, g_strdup_printf,
g_string_append_vprintf and more. It has three different implementations
depending on what the build target platform supports:
1. The gnulib impl appears to use the system malloc, but a
'#define malloc g_malloc' causes it to use GLib's wrapper
and thus abort on OOM. This mostly gets used on Windows
platforms or UNIX platforms with broken printf formatting.
2. The main impl mostly used on modern Linux/UNIX calls the
system vasprintf which uses the system malloc and does not
abort on OOM.
3. The final impl used on remaining platforms calls system
vsprintf on a buffer allocated by g_new, and thus always
aborts on OOM.
Of note is that impl 2 (using vasprintf) historically could abort on
OOM, if the application had installed a non-system malloc impl with
GLib. This was because the code would g_strndup the result from
vasprintf() in that scenario. This was removed in:
commit a366053253
Author: Dan Winship <danw@gnome.org>
Date: Fri Aug 7 09:46:49 2015 -0400
glib: remove deprecated g_mem_is_system_malloc() check in gprintf.c
Having inconsistent OOM behaviour for the three impls is undesirable and
aborting on OOM is normal pratice for GLib APIs. Thus we must thus ensure
this happens in all impls of g_vasprintf.
Fixes#1622
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
These were introducing strict aliasing warnings. Remove them (in line
with other uses of `g_once_init_leave()`).
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Casting pointer types around is a bit fiddly when you also end up
dereferencing them. Take advantage of the fact that the
`__atomic_load()` and `__atomic_store()` built-ins are polymorphic, and
use `__typeof__()` to ensure that the atomic pointer macros use the
caller-provided types internally, and hence any type mismatches are
entirely the caller’s fault rather than ours.
This also means that the `__atomic_{load,store}()` built-ins have the
right alignment and width data from the caller-provided types, in case
that’s needed.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Various places that used atomic functions were using the wrong return
type. Fix that. This introduces no functional changes.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
If we're cross-compiling, the installed-tests are useful even if we
can't run them on the build machine: we can copy them to the host
machine (possibly via a distro package like Debian's libglib2.0-tests)
and run them there.
While I'm changing the build-tests condition anyway, deduplicate it.
Based on a patch by Helmut Grohne.
Bug-Debian: https://bugs.debian.org/941509
Signed-off-by: Simon McVittie <smcv@collabora.com>
`man dup2` says that on Linux, dup2() can return `EBUSY` if the
operation needs to be retried (in addition to returning `EINTR` in other
cases where it needs to be retried).
Signed-off-by: Philip Withnall <withnall@endlessm.com>
All uses of fdwalk in gspawn are between fork and exec, which means only
async-signal safe functions can be called if the parent process has
multiple threads. Since fdwalk is not a standard API, we should not
assume it is safe to use unless the manual of the system explicitly says
it is async-signal safe.
Fixes: #1638
See the mailing list thread <https://lists.fedoraproject.org/archives/list/
devel@lists.fedoraproject.org/thread/SZ676IHHSLOQD6UN2I5J5VKXJ5P5SOVO/>
"glib-2.0 G_CONST_RETURN causing GCC 'warning: const' on F31", where the GCC
diagnostic
> test.c:2:13: warning: const
> 2 | G_CONST_RETURN char * f();
> | ^~~~~~~
had confused me, and "Deprecated pre-processor symbol, repace with const" is
probably a better warning message than just "const".
(That recent GCC only prints "Deprecated pre-processor symbol, repace with "
appears to be a bug in GCC that GLIB_UNAVAILABLE_MACRO already suffers from,
too. Recent Clang correctly prints "Deprecated pre-processor symbol, repace
with const".)
Previously we used the old `__sync_fetch_*()` intrinsics for some of the
atomic operations, such as `g_atomic_int_compare_and_exchange()`. When
available, use the new `__atomic_*()` intrinsics for those instead.
As with the rest of our use of `__atomic_*()` intrinsics, we use the
`__ATOMIC_SEQ_CST` memory model. If people want to use a less
restrictive memory model to get better performance in certain
situations, they can use the compiler intrinsics directly themselves.
`g_atomic_*()` aim to be as fast as possible while remaining general
purpose.
Tested using:
```
meson test --repeat 1000 atomic atomic-test
```
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #1750
Android is emitting `-Wtautological-constant-out-of-range-compare`
warnings when compiling the validation functions for the enum types for
`GDate`. Fix that by comparing as integers.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
It was possible to pass in (for example) an invalid year to
g_date_time_new_week(), which would be passed on to g_date_time_new(),
which would (correctly) return `NULL` — but then
g_date_time_get_week_number() would try to dereference that.
Includes a test case.
oss-fuzz#17648
Signed-off-by: Philip Withnall <withnall@endlessm.com>
If the compiler doesn’t provide modern (C++11) atomic builtins (which is
now quite unlikely), we implement our own using the `__sync_synchronize()`
memory barrier. As Behdad and others have pointed out, though, the
implementation didn’t follow the same semantics as we use with the C++11
builtins — `__ATOMIC_SEQ_CST`.
Fix the use of memory barriers to provide `__ATOMIC_SEQ_CST` semantics.
In particular, this fixes the following common pattern:
```
GObject *obj = my_object_new ();
g_atomic_pointer_set (&shared_ptr, obj);
```
Previously this would have expanded to:
```
GObject *obj = my_object_new ();
*shared_ptr = obj;
__sync_synchronize ();
```
While the compiler would not have reordered the stores to `obj` and
`shared_ptr` within the code on one thread (due to the dependency
between them), the memory system might have made the write to
`shared_ptr` visible to other threads before the write to `obj` — if
they then dereferenced `shared_ptr` before seeing the write to `obj`,
that would be a bug.
Instead, the expansion is now:
```
GObject *obj = my_object_new ();
__sync_synchronize ();
*shared_ptr = obj;
```
This ensures that the write to `obj` is visible to all threads before
any write to `shared_ptr` is visible to any threads. For completeness,
`__sync_synchronize()` is augmented with a compiler barrier to ensure
that no loads/stores can be reordered locally before or after it.
Tested by disabling the C++11 atomic implementation and running:
```
meson test --repeat 1000 atomic atomic-test
```
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #1449
`g_assert_*()` provide more useful failure messages, and aren’t compiled
out when building with `G_DISABLE_ASSERT`, unlike `g_assert()`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
When compiling GLib with `-Wsign-conversion`, we get various warnings
about the atomic calls. A lot of these were fixed by
3ad375a629, but some remain. Fix them by
adding appropriate casts at the call sites.
Note that `g_atomic_int_{and,or,xor}()` actually all operate on `guint`s
rather than `gint`s (which is what the rest of the `g_atomic_int_*()`
functions operate on). I can’t find any written reasoning for this, but
assume that it’s because signedness is irrelevant when you’re using an
integer as a bit field. It’s unfortunate that they’re named a
`g_atomic_int_*()` rather than `g_atomic_uint_*()` functions.
Tested by compiling GLib as:
```
CFLAGS=-Wsign-conversion jhbuild make -ac |& grep atomic
```
I’m not going to add `-Wsign-conversion` to the set of default warnings
for building GLib, because it mostly produces false positives throughout
the rest of GLib.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #1565
Instead of calling close or fcntl on all possible file descriptors,
which is slow on systems with very high limit or even no limit on open
file descriptors, we can use closefrom or fcntl with F_CLOSEM to close
all unwanted file descriptors with a single system call.
This change only improves the performance when GSpawnChildSetupFunc is
NULL because there are applications known to abuse GSpawnChildSetupFunc
to unset FD_CLOEXEC on file descriptors. Since the change mentioned
above requires closing file descriptors directly, it cannot be used when
the caller may want to keep some of them open.
This patch was written by Sebastian Schwarz <seschwar@gmail.com> and
uploaded to https://gitlab.gnome.org/GNOME/glib/merge_requests/574.
It was later modified by Ting-Wei Lan <lantw@src.gnome.org> to address
code review issues.
Fixes: https://gitlab.gnome.org/GNOME/glib/issues/1638
While the introspection scanner can glean the transfer rule for the
return values by looking at their constness, adding an explicit
annotation has the advantage of gtk-doc writing out the transfer rule as
an additional bit of documentation, making the life of the documentation
reader easier.
While the XML specification doesn’t prescribe a limit, no reasonable bit
of XML is going to have more than 1000 attributes in a single XML
element.
Adding a limit reduces the changes of a runaway allocation loop caused
by dodgy input.
oss-fuzz#12960
Signed-off-by: Philip Withnall <withnall@endlessm.com>
It was possible to pass in (for example) an invalid hour to
g_date_time_new_ordinal(), which would be passed on to
g_date_time_new(), which would (correctly) return `NULL` — but then
g_date_time_new_ordinal() would try to dereference that.
Includes some test cases.
oss-fuzz#16103
oss-fuzz#17183
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Support for custom allocators was dropped in
commit 3be6ed60aa
Author: Alexander Larsson <alexl@redhat.com>
Date: Sat Jun 27 18:38:42 2015 +0200
Deprecate and drop support for memory vtables
The introductory doc text for the gmem APIs still warns against mixing
malloc/free with g_malloc/g_free. Clarify upfront in the docs that these
two sets of APIs are now guaranteed to use the same memory allocator &
can thus their usage can be freely mixed.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
g_utf8_get_char_validated() was not exactly matching its
documentation. The function was not checking if the sequence of
unicode characters was free of null bytes before performing a more
in-depth validation.
Fix issue #1052
The static analyser can’t yet work out how `g_autofree` works, so
disable those tests.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1767
The macros for the probes confuse the static analyser, and are often
called with arguments which the analyser things shouldn’t be used any
more (for example, the address of a block of memory which has just been
freed).
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1767
These squash various warnings from `scan-build`. None of them are
legitimate bugs, but some of them do improve code readability a bit.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1767
It’s confusing and often doesn’t help the user. Match the error code and
come up with a more UI-appropriate error message.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
If the user has `CAP_DAC_OVERRIDE` or similar (for example, if running
the tests as root), the `mkdir-with-parents-permission` test is skipped.
The check for `CAP_DAC_OVERRIDE` was by creating a subdirectory of the
test directory. That subdirectory, however, was never removed, which
caused a ‘directory not empty’ error when trying to delete the test
directory.
Fix that by correctly deleting the subdirectory if skipping the test.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
clang will warn with -Wunused-function if the g_autoptr macros are
ever used inside a .c file. This would break a lot of -Werror builds
with clang.
Mark such functions as explicitly unsed so clang will not warn about
them.
However, it's fine to call it when building for the debug target
(which uses the debug CRT and hence sets -D_DEBUG), so let's keep that
around.
The Windows App Certification Kit only runs on apps built in release
mode.
These macros wrap functions which were only introduced in certain
versions of GLib. The functions are correctly marked as introduced in
those versions, but the macros aren’t, which can result in not getting
appropriate deprecation warnings if you’re using those APIs when you
have said you’re targeting older GLib versions using
`GLIB_VERSION_MAX_ALLOWED`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Closes: #1860
We require a newer SDK version now, so this is not needed.
Specifically, we set _WIN32_WINNT to 0x0601, which sets our SDK
requirement to Windows 7+, and this code is only needed for MSVC 5.0,
which is ancient.
It’s confusing and often doesn’t help the user. Match the error code and
come up with a more UI-appropriate error message.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
If the dup(stderr) returns '-1' (an error occured), then the program
shouldn't call a 'close(stderr); dup(old_err);' after the exec() failed.
Fix issue #1880
A static analysis run noted that we weren't freeing the cmdline in the
error path here. We can just make this an assertion instead; I just
checked the kernel code, and it just usees a seq_printf() here which
will NUL terminate.
This fixes the following warning, by making the compiler checks for the
`pop` match those for the `push`:
```
[221/1124] Compiling C object 'glib/tests/d796b50@@mem-overflow@exe/mem-overflow.c.o'.
../glib/tests/mem-overflow.c:204:24: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
#pragma GCC diagnostic pop
```
Signed-off-by: Philip Withnall <withnall@endlessm.com>
When g_variant_get_child_value() is called for a child whose
serialisation is an empty byte string (which is possible), `bytes_data`
will be non-`NULL`, but `data` may be `NULL`. This results in a negative
offset being passed to `g_bytes_new_from_bytes()`, and a critical
warning.
So if `data` is `NULL`, set it to point to `bytes_data` so the offset is
calculated as zero. The actual value of the offset doesn’t matter, since
in this situation the size is always zero. An offset of zero is never
going to cause problems.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #1865
These are copies of the existing tests for `g_time_val_from_iso8601()`,
with the test strings which fail for `GDateTime` commented out. This is
OK, as it’s documented as only accepting a subset of ISO 8601 (and for
some of the test vectors, it’s debatable whether they’re actually valid
ISO 8601, depending on how you interpret the valid bounds of timezone
offsets — some interpretations of the available documentation would say
that timezone offsets should never be ≥24 hours or ≥60 minutes).
There is one test string which is not accepted by
`g_time_val_from_iso8601()` but which is accepted by `GDateTime`, as
`g_date_time_new_from_iso8601()` actually accepts RFC 3339, which is a
little more liberal than ISO 8601. Fun times. See
https://tools.ietf.org/html/rfc3339#section-5.6.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Some malformed ISO 8601 date/time strings were causing an assertion
failure when passed to `g_date_time_new_from_iso8601()`, due to a
mismatch between the bounds checking of timezone offsets in `GDateTime`
and `GTimeZone`. Fix that and add a unit test for it.
oss-fuzz#16101
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Since we transitioned from Bugzilla to GitLab, we have two forms of bug
references in the GLib source code: old (but still relevant) Bugzilla
links, and newer GitLab links. We can’t use a single base for the two,
so have to either build incorrect URIs, or provide the full URI in
g_test_bug().
It’s always seemed a bit of an over-optimisation to provide the bug base
separately from the bug ID, so relax the assertions and documentation
around g_test_bug_base() so that g_test_bug() can be used on its own.
The old usage patterns are still supported unchanged.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
On x86 (and apparently most other Linux architectures), the union
with the signal handler is the first member, but on MIPS Linux,
the first struct member is sa_flags (possibly done to be compatible
with IRIX). Zero out the struct and fill in the field we want by name.
Signed-off-by: Simon McVittie <smcv@collabora.com>
When using the mingw printf shims for C99 compat the msvc format specifiers don't work
and the build fails.
Ideally we would use glib functions which abstract this away, but in the error handler context
we shouldn't call back into glib. And for scanf we don't have a glib wrapper.
Instead call the "secure" versions provided by the win32 API (_snprintf_s/fprintf_s/sscanf_s)
which mingw doesn't replace.
We currently check in multiple places if vsnprintf/snprintf/printf are
good and if not use gnulib. This case was not checking for printf which
made the build fail with recent mingw-w64 where snprintf was improved to
pass all glib checks but printf still doesn't.
Commit 6f55306e04 unintendedly broke error handling for other
error conditions than ENOENT along the path, like EPERM. It wanted
to ignore ENOENT on all elements except the last in the path, but
in doing that it ignored any other error that might happen on the
last element.
https://gitlab.gnome.org/GNOME/glib/issues/1852
You may expect funny effects from passing invalid UTF-8, but not
that funny. The assert will probably be a better and more immediate
confirmation of an error than invalid writes under the address of the
string copy.
https://gitlab.gnome.org/GNOME/glib/issues/1863
This comment was correct until commit adf1f98f62, when the `GTimeVal`
which the result was put into (introducing the Y2038-unsafety) was
dropped.
The adjustment and scaling of the `FILETIME` should not make it
Y2038-unsafe: the maximum `FILETIME` is 2^64-1. Subtracting the epoch
adjustment and dividing by 10 gives the timestamp 1833029933770955161,
which is in June 58086408216 (at just after 3am UTC). I think that’s
enough time to be going on with.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1438
This requires some reworking of the internal g_date_time_new_from_unix()
function, since it previously operated in seconds, which wasn’t high
enough resolution — the g_get_current_time() code path used to operate
in microseconds.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1438
The former is now deprecated, so it makes sense to base its
implementation on the latter, rather than the other way around.
This introduces no functional changes.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1438
GTimeVal is subject to the year 2038 problem, since its `tv_sec` field
is a `glong`, which is 32 bits on 32-bit platforms.
Use `guint64` to represent microsecond-precision time since the Unix
epoch; or use `GDateTime` for full date/time representation.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #1438
It’s not Y2038-safe, as it’s 32-bit. While it was previously deprecated
in the documentation, now add the deprecation annotation for the
compiler.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1438
This is a simple wrapper around g_date_time_format_iso8601() which
always produces ISO 8601 dates, without people having to remember the
format string for them (and with the convenience of terminating UTC
dates with ‘Z’ rather than ‘+00’).
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1438
Only redefine g_message() and friends to use structured logging if the
compiling code is OK with depending on GLib functionality from ≥2.56.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #1847
Try to create the complete path right away and fall back
to creating all path elements one by one.
This also helps to avoid TOCTTOU problems and avoids walking
the path all the time, providing a nice performance gain, by
avoiding syscalls.
Ignore ENOENT errors up until the last element while trying to create each
of the path elements in case a restricted file-system is being used where
path elements can be hidden or non-accessible.
__atomic_load_8 and friends do not exist under clang. Use the generic
__atomic_load variant instead that are documented here:
https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html
These have the additional benefit that the exact size of gint (4 bytes)
or gpointer (4 or 8 bytes) no longer have to be checked.
I initially tried `__typeof__(*(atomic)) val;`, but that caused warnings
in Clang (-Wincompatible-pointer-types-discards-qualifiers) when
"atomic" points to a volatile variable. Aside from that, it is
apparently not supported everywhere, see the g_has_typeof macro.
Another reason not to use it are new warnings under Clang, including:
glib/deprecated/gthread-deprecated.c:683:11: warning: incompatible pointer types initializing 'typeof (*(&mutex->mutex.mutex))' (aka 'union _GMutex *') with an expression of type 'GRecMutex *' (aka 'struct _GRecMutex *') [-Wincompatible-pointer-types]
g_atomic_pointer_set (&mutex->mutex.mutex, result);
Hence, cast the atomic variable to gint/gpointer pointers, the size was
already statically asserted so the cast should be safe.
The macros use a (hopefully) rare "gaps_temp" name instead of something
like "val" to avoid an issue with GCC builds:
glib/tests/once.c:123:test_once4: assertion failed (val == "foo"): (NULL == "foo")
Closes#1843
We're using the `install` argument for configure_file() all over the
place.
The support for an `install` argument for configure_file() was added in
Meson 0.50, but we haven't bumped the minimum version of Meson we
require, yet; which means we're getting compatibility warnings when
using recent versions of Meson, and undefined behaviour when using older
versions.
The configure_file() object defaults to `install: false`, unless an
install directory is used. This means that all instances of an `install`
argument with an explicit `true` or `false` value can be removed,
whereas all instances of `install` with a value determined from a
configuration option must be turned into an explicit conditional.
If searching for an element which is smaller than every element in the
array (i.e. the element being searched for is not in the array), the
previous g_array_binary_search() implementation would underflow in the
calculation `right = middle - 1`, and end up trying to dereference an
element way off the right of the array.
Fix that by checking the additions/subtractions before doing them, and
bailing if the bounds are hit. We don’t need to check `middle <
G_MAXUINT`, as `middle` is bounded above by `right`, which is always `<=
_array->len - 1`, and `_array->len <= G_MAXUINT`.
Add some tests for that, and for not-present elements in the middle of
the array. Previously, the tests only checked for not-present elements
which were bigger than every element in the array.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
If `right` and `left` are both near `G_MAXUINT`, it’s possible for the
addition to overflow, even if the eventual result would fit in a
`guint`. Avoid that by operating on the difference instead.
The difference is guaranteed to be positive due to the prior `left <=
right` check.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
The allocation size was set correctly before, but not the array length,
so the copied array appeared to have zero elements.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
We need to enable building the dirent and gnulib sources for clang-cl,
as we are still using the Microsoft-style headers and lib's and CRT.
We need to also do this for the following, for similar reasoning:
-Symbol export (via __declspec(dllexport))
-Dependency discovery without pkg-config files
-long long and ssize_t detection
We do, however, enable the autoptr tests for clang-cl builds. Note that
at this point real MSVC builds are still better supported than clang-cl
builds, and it will likely remain so for at least the near future,
alhtough real MSVC builds of the GTK stack are consumable and are usable
by clang-cl.
Use the GCC-style definition for the deprecation warning macros so that
builds using those won't break due to how they are placed as clang-cl
seems to not support __declspec(deprecated) very well. Also make sure
that we do indeed support the temparary disabling of deprecation
warnings on clang-cl, as the MSVC ones don't really work on clang-cl.
clang-cl does support __attribute__((cleanup)), which is what is used
for the g_auto* macros, but neither it, nor clang.exe defines __GNUC__
when they are used in a MSVC cmd.exe environment. It does, however,
define __clang__.
So, check for the presence of the __clang__ macro to enable g_autoptr as
well, so that we can build things with MSVC builds that make use
of g_autoptr via pretending to be MSVC by using clang-cl.
Windows does not recognize the "America/Recife" as a valid timezone
identifier, so setting the TZ envvar to that will result in "UTC" to
be returned on Windows.
Instead, set TZ to be the Windows equivilant "SA Eastern Standard
Time", and see whether that is indeed our identifier when we create the
GTimeZone using that.
On Windows, we may be using the US DST boundaries by using the default
"Pacific Standard Time" for rules_from_windows_time_zone() in
rules_from_identifier(). This has the unfortunate side-effect of
hardcoding the out_identifier to "Pacific Standard Time", which is
likely not what we want.
Instead, upon retrieving the items successfully using
rules_from_windows_time_zone ("Pacific Standard Time", ...), we just
set the out_identifier to whatever identifier that was passed into
rules_from_identifier().
Update the gdatetime test program to make use of the updates that was
done in gtimezone.c in the previous commit, so that we don't have to
worry what language version of Windows the tests are being run in, but
instead be assured that we produce and check for the English-language
time zone name strings.
Also, instead of testing for "Pacific Standard Time" in
test_GDAteTime_printf(), use GetDynamicTimeZoneInformation() to get the
actual time zone string (where the system running the test program is)
we want to check for, because on Windows the actual result will be
dependent on which timezone the system running the test program is in.
https://bugzilla.gnome.org/show_bug.cgi?id=719344
The existing method of using RegQueryValueExW() to query the Std/Dlt
strings can only retrive the localized versions of those strings, so
that means they will vary by the language version of Windows. Instead,
use RegQueryValueExW() only as a fallback when RegLoadMUIStringW() fails,
as RegLoadMUIStringW() can query for the Std and Dlt strings in
whatever language we need by setting the locale stuff programatically on
the fly.
We are going to use RegLoadMUIStringW() in the next commit, since there
is no real RegLoadMUIStringA() function (it exists as a stub only).
This is done so that we are consistent along the way
Also fix rule_from_windows_time_zone_info() as we can't just do a strncpy()
of tzi->StandardName and tzi->DaylightName directly, as they are wchar_t/
gunichar2 strings, where we must convert to UTF-8 first.
https://bugzilla.gnome.org/show_bug.cgi?id=719344
We said the type must support being cleaned up and will be cleaned up in
an appropriate way, but in order to figure out how to do that, you had
to jump forward in the documentation to the other macros. Just say them.
This is a follow-up to a0c7f85437.
In addition LC_MESSAGES, we should remove LC_ALL from the environment
as well. Otherwise, LC_ALL overrides LC_MESSAGES, causing the test to
fail on FreeBSD when LC_ALL is set to a non-English locale.
This (dubious) testcase fails before the previous commit due to errno
being clobbered by the interrupted wait on the contended mutex. The
previous commit fixes that.
The testcase is dubious because, in theory (as per POSIX),
g_cond_wait_until() is permitted to return TRUE at any time for any
reason, due to so-called "spurious wakeups". Having a testcase that
asserts that the return value should be FALSE is therefore fundamentally
broken. We do it anyway, though.
We're only really trying to test a bug in our homemade Linux/futex
implementation here, and it takes a fair amount of effort to actually
convince the old code to fail (including some system stuff which
probably isn't available on Windows). There's also the spurious wakeup
situation mentioned above to worry about on other systems. For all of
those reasons, this test is only enabled on Linux.
The return value from `g_cond_wait_until()` is calculated, based on the
value of `errno` after reacquiring the mutex. This is a problem because
`errno` can be overwritten in the case the mutex is contended (in which
case the slow-path code will re-enter the kernel).
Perform the calculation before reacquiring the mutex.
See merge request GNOME/glib!958
Visual Studio 2010 and later support static_assert on both C and C++,
but we can only enable it for plain-C code on Visual Studio 2013 and
later, as apparently the static_assert macro implementation will carry
out something after defining certain variables.
For C++ code, we use static_assert on Visual Studio 2010 and later.
The return value for g_utf8_normalize may be null, but the return type
is not annotated as such. This is important for language bindings for
langs that are about nullability, such as Vala and Haskell.
g_test_trap_fork() is deprecated, but it’s still tested in the GLib
tests. If we don’t set the coredump rlimit to 0 to prevent coredumps,
the test for it creates a coredump (which is expected). On systems with
systemd-coredump, that causes systemd-coredump to spawn and process the
coredump for a while. That slows down the tests, and results in spurious
coredump lists in the system logs.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #436
The static assert message is much nicer to read, and is less likely to
be misinterpreted as a false positive.
glib is built with `-std=gnu89`, but this macro will be available to
projects that use glib with c11 or gnu11.
It is unclear that why the size of the buffer was chosen to be 100
because the commit introduced the code didn't mention the reason.
POSIX defines _POSIX_HOST_NAME_MAX to be 255 and provides a way to
determine the suitable value with sysconf, so we should use it instead
of hard-coding a small value.
The g_creat() call was done in *nix style in regards to the mode parameter,
which the Windows CRT isn't going to understand (nor like, on newer CRTs).
Update the program such that it will pass, by using Windows-style
permission mode parameters for g_creat(), on Windows.
On Windows, everything is assumed as UTF-8 in g_get_filename_charsets(),
so setting G_FILENAME_ENCODING envvar has no effect, so we can just skip
the test on g_get_filename_charsets() where we try to influence it by
setting the G_FILENAME_ENCODING envvar.
