Commit Graph

31038 Commits

Author SHA1 Message Date
Philip Withnall
4c3f06acbf Merge branch 'str-is-ascii-ifunc' into 'main'
gutf8: Add ifunc resolver for g_str_is_ascii() too

Closes #3511

See merge request GNOME/glib!4364
2024-10-22 17:37:32 +00:00
Michael Catanzaro
2c345bd8e2 Merge branch 'fuzz-fix' into 'main'
fuzzing: Fix buffer overread error in the fuzz test itself

See merge request GNOME/glib!4365
2024-10-22 16:00:36 +00:00
Michael Catanzaro
155d00a61e Merge branch '3470-unicode-16' into 'main'
Update to Unicode 16.0.0 and fix Unicode composition for its new codepoints

Closes #3470

See merge request GNOME/glib!4362
2024-10-22 15:29:42 +00:00
Philip Withnall
e795c715b5 Merge branch 'fix-namespace-UnixMountEntry' into 'main'
UnixMountEntry: Deprecate g_unix_mount_* API in favor of g_unix_mount_entry_* API for GUnixMountEntry methods

Closes #3492

See merge request GNOME/glib!4337
2024-10-22 14:48:12 +00:00
Jialu Zhou
aac56f1618 UnixMountEntry: Deprecate g_unix_mount_* API in favor of g_unix_mount_entry_*
This issue arises because the g_unix_mount_* naming convention does not match
the GUnixMountEntry instance type, confusing the introspection generator.

To resolve this, we are deprecating the g_unix_mount_* API functions that take
a GUnixMountEntry parameter and introducing equivalent g_unix_mount_entry_*
functions that correctly associate with the GUnixMountEntry instance. This change
ensures that introspection data correctly treats these as instance methods and
that documentation reflects proper ownership of returned data.

(Some minor tweaks by Philip Withnall.)

Fixes: #3492
2024-10-22 15:31:52 +01:00
Philip Withnall
0e0421cfe2
fuzzing: Fix buffer overread error in the fuzz test itself
valgrind spotted this one: a read off the end of the `stop_chars` buffer
when `stop_chars_len == -1`, due to the fuzzing test not sticking to the
requirement from `g_data_input_stream_read_upto()` that `stop_chars`
must be nul-terminated if `stop_chars_len < 0`.

This can happen when reading `fuzzing/README.md`, which is done as a
smoketest when the fuzzing tests are run without the fuzzer, as normal
unit tests. In this case, it made smoke.

Signed-off-by: Philip Withnall <pwithnall@gnome.org>
2024-10-22 13:18:59 +01:00
Philip Withnall
cf982177dd
gutf8: Add ifunc resolver for g_str_is_ascii() too
Just like how commit ad572e7780 added an
ifunc resolver for `g_utf8_validate()`, we also need to add one for
`g_str_is_ascii()`, as it also calls into the c-utf8 SIMD validation
code which causes false-positive buffer read overflow warnings from
valgrind and asan.

I thought about just adding the `strlen()` call into `g_str_is_ascii()`
unconditionally, as a simpler fix, but from a quick
codesearch.debian.net, it appears `g_str_is_ascii()` is used quite
widely, so this would have an unacceptable performance impact.

This should fix the valgrind failures on the `search-utils` test seen
here: https://gitlab.gnome.org/GNOME/glib/-/jobs/4423753.

Signed-off-by: Philip Withnall <pwithnall@gnome.org>
2024-10-22 12:46:16 +01:00
Philip Withnall
1755024caf
gutf8: Factor out complex type signature into a typedef
As suggested by Michael Catanzaro, this should make the return type of
the resolve function a bit easier for people to parse.

This introduces no functional changes.

Signed-off-by: Philip Withnall <pwithnall@gnome.org>
2024-10-22 12:45:20 +01:00
Philip Withnall
05fb05b49b
gutf8: Factor out ifunc attribute checks
It looks like these might get more complex in future, as compilers claim
to support the attribute (`__has_attribute(ifunc)` is true) but then
raise errors at compile time if the target architecture doesn’t support
ifuncs.

For example, see #3511.

This doesn’t fix #3511 (I don’t have time to test on musl right now), but
it should make it easier to update the platform preprocessor conditions
in future.

Signed-off-by: Philip Withnall <pwithnall@gnome.org>

Helps: #3511
2024-10-22 12:42:41 +01:00
Philip Withnall
f9f74efd76
tests: Improve Unicode composition code coverage
This adds various additional tests to cover branches of `gunidecomp.c`
which are not already covered, bringing our branch coverage of that file
up to 100% (if you ignore `g_utf8_normalize()`, which is tested by
`unicode-normalize.c` and I’m counting it separately).

Signed-off-by: Philip Withnall <pwithnall@gnome.org>

Helps: #3470
2024-10-21 19:32:50 +01:00
Philip Withnall
0125c58a05
tests: Test restricted result_len sizes for g_unichar_fully_decompose()
This pushes the code coverage of that function up to 100%.

And it found no bugs!

Signed-off-by: Philip Withnall <pwithnall@gnome.org>
2024-10-21 19:32:43 +01:00
Philip Withnall
84e3a9cde9
tests: Refactor g_unichar_fully_decompose() test to make it extensible
This introduces no functional changes, but allows the test to be easily
extended, in the following commit, to test restricted `result_len`
sizes.

Signed-off-by: Philip Withnall <pwithnall@gnome.org>
2024-10-21 19:32:37 +01:00
Philip Withnall
6eb648de1c
tests: Refactor g_unichar_compose() test to fix booleans
`g_assert_false (g_unichar_compose (…) && ch == 0)` will succeed if
`g_unichar_compose()` succeeds and returns a non-zero character (which
it will if it succeeds), so this isn’t really testing what we want it to
test. This regressed in commit ae4eea7a39.

Refactor out the repetitive calls to `g_unichar_compose()` and fix the
boolean checks.

Signed-off-by: Philip Withnall <pwithnall@gnome.org>
2024-10-21 19:32:30 +01:00
Philip Withnall
5ec1c2422c
gunidecomp: Add some explanatory comments to g_utf8_normalize()
Just to make it a bit easier for people to understand the logic in the
implementation in future, because it took me a while.

Signed-off-by: Philip Withnall <pwithnall@gnome.org>
2024-10-21 19:32:23 +01:00
Philip Withnall
32d3bddae4
gen-unicode-tables.pl: Add a new ‘either’ compose table
See the big comment in the code for details. Essentially, this adds a
new compose table specifically for the transitive closure of ‘either’
codepoints — codepoints which appear as the first codepoint in a
composition pair and as the second point in a composition pair
(potentially, but not necessarily, the same pair); or which appear in a
composition pair with an ‘either’ codepoint.

This new compose table has to be symmetrically indexed, as the
`COMPOSE_INDEX` macro doesn’t differentiate based on codepoint position
(first or second). It’s not possible to achieve that with the main
`compose_array` without making it absolutely huge (it’s currently about
150×40 in size and would have to become at least 150×150 in size). In
contrast, the new `compose_either_array` is currently 15×15.

Signed-off-by: Philip Withnall <pwithnall@gnome.org>

Helps: #3470
2024-10-21 19:32:16 +01:00
Philip Withnall
dc2491d224
gen-unicode-tables.pl: Add more error checking
We’re essentially trying to build a minimal perfect hash function, and
`vals` is the map which represents that function. If we redefine a
member of `vals`, the map is no longer a partial function — one input
value (a Unicode codepoint) has two output values (compose table
indices).

So it’s bad if a member of `vals` gets redefined, and we want to be
notified if that happens.

As it happens, some of the new codepoints in Unicode 16.0 cause these
checks to fail. For example, U+16121 Gurung Khema Vowel Sign U
decomposes to U+1611E U+1611E. This causes `vals{U+1611E}` to be defined
to an index from the `first` map, and then redefined to an index from
the `second` map.

The following few commits will fix this, but let’s get the checks in
first.

Signed-off-by: Philip Withnall <pwithnall@gnome.org>
2024-10-21 19:32:10 +01:00
Philip Withnall
ebd26727a8
gen-unicode-tables.pl: Add some internal documentation
Because how these big tables of numbers work is perhaps a bit hard to
figure out, and it would be useful to document the design decisions
involved in it.

Signed-off-by: Philip Withnall <pwithnall@gnome.org>
2024-10-21 19:32:03 +01:00
Philip Withnall
e2b96c8679
gunicode: Update to Unicode 16.0.0
All changes mechanically generated with:
```
./tools/update-unicode-data.sh ~/Downloads/UCD 16.0.0
```
using the data from https://www.unicode.org/Public/16.0.0/ucd/UCD.zip.

Signed-off-by: Philip Withnall <pwithnall@endlessos.org>

Fixes: #3470
2024-10-21 19:31:56 +01:00
Philip Withnall
07e191012d
gunicode: Add new scripts for Unicode 16.0
Manually added from the data in
https://www.unicode.org/Public/16.0.0/ucd/UCD.zip.

The following commit will mechanically update the Unicode tables to use
them.

Signed-off-by: Philip Withnall <pwithnall@gnome.org>

Helps: #3470
2024-10-21 19:31:49 +01:00
Philip Withnall
3065734dce
gunicode: Fix comment describing shortcode for Nag Mundari script
So it reflects the `sc` line in `PropertyValueAliases.txt` in the
Unicode database.

Signed-off-by: Philip Withnall <pwithnall@gnome.org>
2024-10-21 19:31:42 +01:00
Philip Withnall
230fdbb373
tools: Add missing license and SPDX header to update-unicode-data.sh
Only one other previous author, and my contribution just now is so
simple as to not be copyrightable.

Signed-off-by: Philip Withnall <pwithnall@gnome.org>

Helps: #1415
2024-10-21 19:31:35 +01:00
Philip Withnall
79e25aaa1b
tools: Update the normalisation test data when updating Unicode
Otherwise this easily gets forgotten.

Signed-off-by: Philip Withnall <pwithnall@gnome.org>

Helps: #3470
2024-10-21 19:31:26 +01:00
Philip Withnall
ad51ff8038
gunicode: Switch compose_array table from guint16 to gunichar
The time has finally come when Unicode has specified a codepoint above
U+FFFF which has a decomposition: U+16125 GURUNG KHEMA VOWEL SIGN AI, in
Unicode 16 which the following commits will add support for.

So far, we’ve managed to store the reverse-lookup from decomposed pairs
to their composed form using a 16-bit integer. Now we have to switch to
storing the composed form in a 32-bit `gunichar` as U+16125 won’t fit
otherwise.

This introduces no functional changes, but does double the in-memory
size of the `compose_array` table from 9176 bytes to 19932 bytes.

The code which uses this lookup table, in `gunidecomp.c`, was already
implicitly converting the loaded value to a `gunichar`, so needs no
changes.

When we update to Unicode 16, the new `NormalizationTest.txt` file
contains a test which will check that composed codepoints > U+FFFF work.
Specifically, U+11391 TULU-TIGALARI LETTER AU is tested.

Signed-off-by: Philip Withnall <pwithnall@gnome.org>

Helps: #3470
2024-10-21 19:30:38 +01:00
Philip Withnall
7ed7fe2867 Merge branch 'g_array_free-free_seg' into 'main'
free_seg -> free_segment for g_array_free() and g_ptr_array_free()

Closes #3508

See merge request GNOME/glib!4361
2024-10-21 11:49:10 +00:00
Philip Withnall
dec815df74 Merge branch '3477-32-bit-ci-docs' into 'main'
docs: Document 32-bit support policy

Closes #3477

See merge request GNOME/glib!4360
2024-10-21 11:46:14 +00:00
Marcel Telka
3a1eb4a0be free_seg -> free_segment for g_array_free() and g_ptr_array_free()
Signed-off-by: Marcel Telka <marcel@telka.sk>
2024-10-21 13:16:52 +02:00
Philip Withnall
e53659d546 Merge branch 'wip/smcv/i386-ci' into 'main'
Add a CI job for Debian stable i386 (32-bit)

See merge request GNOME/glib!4358
2024-10-19 19:11:49 +00:00
Simon McVittie
dbd7280c5e CI: Rebuild a slightly newer version of ninja for debian-stable-i386
The version of ninja-build in Debian 12 isn't built with large file
support, and therefore is not compatible with filesystems with large
inode numbers. Unfortunately, that includes the overlay mounts used by
Docker.

I've suggested a stable update for this as part of the next Debian 12
point release. Until/unless that happens, we can build our own.

Signed-off-by: Simon McVittie <smcv@debian.org>
2024-10-19 17:45:51 +01:00
Simon McVittie
20d54ba5f1 CI: Add a job that runs on 32-bit Debian stable
This is enough to detect most ILP32-specific issues. We previously
relied on 32-bit Windows to catch those, but the toolchains we're using
have increasingly minimal support for 32-bit Windows.

The combination of fedora-x86_64 and debian-stable-i386 between them
should cover nearly everything that debian-stable-x86_64 does, so demote
debian-stable-x86_64 to be run on a schedule or manually.

Helps: https://gitlab.gnome.org/GNOME/glib/-/issues/3477
Signed-off-by: Simon McVittie <smcv@debian.org>
2024-10-19 15:42:40 +01:00
Simon McVittie
630930a60b CI: Add a debian-stable-i386 CI image
This is identical to the debian-stable image, except that it uses
packages from the i386 dpkg architecture (i686-linux-gnu) instead of
amd64 (x86_64-linux-gnu). x86_64 Docker hosts with x86_64 kernels can
run i386 Docker images, so we can use our existing CI workers.

Instead of duplicating the content of the Dockerfile, add a layer of
architecture-switching so we can build essentially the same image
from a different base.

Signed-off-by: Simon McVittie <smcv@debian.org>
2024-10-19 15:42:40 +01:00
Simon McVittie
f8ecf25975 run-docker.sh: Document options
Signed-off-by: Simon McVittie <smcv@debian.org>
2024-10-19 15:42:40 +01:00
Philip Withnall
1334e585e7 Merge branch 'issue_3500' into 'main'
gmain: Include poll.h as early as possible, fixing build on AIX

Closes #3500

See merge request GNOME/glib!4355
2024-10-19 09:39:21 +00:00
Parth Patel
0924a2e881 gmain: Include poll.h as early as possible
On AIX, the system poll.h redefines the names of struct members,
for example `#define events reqevents`. This means that accesses
to GPollFD will fail to compile if poll.h was included after
glib/gpoll.h.

We can't simply add `#include <poll.h>` in glib/gpoll.h, because
that wouldn't work on platforms where poll.h doesn't exist, and
GLib supports some platforms in that category.

Resolves: https://gitlab.gnome.org/GNOME/glib/-/issues/3500
2024-10-19 01:52:32 -05:00
Philip Withnall
b465ca1e9e Merge branch 'gvarianttype-docs' into 'main'
gvarianttype: Add two missing (nullable) annotations and port docs to gi-docgen format

See merge request GNOME/glib!4326
2024-10-19 00:44:46 +00:00
Michael Catanzaro
c0d19d0bf7 Merge branch 'timezone-tests' into 'main'
tests: Use g_assert_*() rather than g_assert() in GDateTime tests

See merge request GNOME/glib!4359
2024-10-18 16:39:02 +00:00
Philip Withnall
3348e942b2
docs: Document 32-bit support policy
Signed-off-by: Philip Withnall <pwithnall@gnome.org>

Fixes: #3477
2024-10-18 13:59:23 +01:00
Philip Withnall
1ffa730035
tests: Use g_assert_*() rather than g_assert() in GDateTime tests
It won’t get compiled out with `G_DISABLE_ASSERT`.

Signed-off-by: Philip Withnall <pwithnall@gnome.org>
2024-10-18 13:24:13 +01:00
Philip Withnall
e9902a66a9 Merge branch 'bytes-docs' into 'main'
gbytes: Convert docs to gi-docgen linking syntax

See merge request GNOME/glib!4303
2024-10-18 12:04:12 +00:00
Philip Withnall
857b418b48 Merge branch 'wip/smcv/pst8pdt' into 'main'
gdatetime test: Fix regression with tzdata 2024b

Closes #3502

See merge request GNOME/glib!4356
2024-10-18 11:06:41 +00:00
Philip Withnall
e1278d62ad Merge branch 'msvc-ci-2019-arm64' into 'main'
CI: Add manual CI job for VS2019 ARM64 builds

See merge request GNOME/glib!4342
2024-10-18 10:45:49 +00:00
Simon McVittie
fe2699369f gdatetime test: Fall back if legacy System V PST8PDT is not available
On recent versions of Debian, PST8PDT is part of the tzdata-legacy
package, which is not always installed and might disappear in future.
Successfully tested with and without tzdata-legacy on Debian unstable.

Signed-off-by: Simon McVittie <smcv@debian.org>
2024-10-18 11:24:33 +01:00
Simon McVittie
30e9cfa573 gdatetime test: Try to make PST8PDT test more obviously correct
Instead of using timestamp 0 as a magic number (in this case interpreted
as 1970-01-01T00:00:00-08:00), calculate a timestamp from a recent
year/month/day in winter, in this case 2024-01-01T00:00:00-08:00.

Similarly, instead of using a timestamp 15 million seconds later
(1970-06-23T15:40:00-07:00), calculate a timestamp from a recent
year/month/day in summer, in this case 2024-07-01T00:00:00-07:00.

Signed-off-by: Simon McVittie <smcv@debian.org>
2024-10-18 11:22:49 +01:00
Rebecca N. Palmer
c0619f08e6 gdatetime test: Do not assume PST8PDT was always exactly -8/-7
In newer tzdata, it is an alias for America/Los_Angeles, which has a
slightly different meaning: DST did not exist there before 1883. As a
result, we can no longer hard-code the knowledge that interval 0 is
standard time and interval 1 is summer time, and instead we need to look
up the correct intervals from known timestamps.

Resolves: https://gitlab.gnome.org/GNOME/glib/-/issues/3502
Bug-Debian: https://bugs.debian.org/1084190
[smcv: expand commit message, fix whitespace]
Signed-off-by: Simon McVittie <smcv@debian.org>
2024-10-18 11:20:42 +01:00
Chun-wei Fan
b1f09a0b0d .gitignore: Add vs2019-arm64.txt
This file is generated during a CI run and is deleted once the CI run is
completed, so don't try to track that file.
2024-10-18 14:59:20 +08:00
Chun-wei Fan
85d8a618ad CI: Add CI job for VS2019 ARM64 builds
We need to write a Meson cross-compilation file on the fly here, hence the
additions in test-msvc.bat to set up the paths.

Like the 32-bit VS2019 CI job this is only run manually or weekly.
2024-10-18 14:58:53 +08:00
Philip Withnall
6cabc7bbf8 Merge branch 'wip/chergert/valgrind-utf8-check' into 'main'
glib/gutf8: use ifunc to check for valgrind

See merge request GNOME/glib!4344
2024-10-17 17:57:48 +00:00
Philip Withnall
ec7cf334db
gutf8: Add a comment explaining the ifunc and asan annotation
Why they’re necessary, why we _think_ the optimised implementation of
`g_utf8_validate()` is OK despite what valgrind and asan are telling us,
and how they work.

Signed-off-by: Philip Withnall <pwithnall@gnome.org>

Helps: #3493
2024-10-17 18:26:19 +01:00
Michael Catanzaro
85b53d6317 Merge branch 'data-input-stream-optimisation' into 'main'
gdatainputstream: Use memchr() for the multi-stop-char case too

See merge request GNOME/glib!4352
2024-10-17 15:42:03 +00:00
Philip Withnall
ca654b2a4e Merge branch 'codeowners-ci' into 'main'
docs: Add CI runner maintainers to CODEOWNERS

See merge request GNOME/glib!4353
2024-10-17 11:22:26 +00:00
Philip Withnall
2fd796ea94 Merge branch 'cm/broaden-suppression' into 'main'
glib.supp: Suppress more _g_io_module_get_default_type leaks

See merge request GNOME/glib!4354
2024-10-17 10:52:29 +00:00