It’s landed in kernel 5.9: http://lkml.iu.edu/hypermail/linux/kernel/2008.0/02649.html
Note, this is untested because I currently don’t have kernel 5.9. We can
fix anything up if it breaks once the new syscall is wrapped in glibc.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
This is very unlikely to happen, but add error handling to mirror the
other calls to `safe_open()`, and shut Coverity up.
Coverity CID: #1430611
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Use this to replace the much-hated `g_debug()` which told people that
`posix_spawn()` (the fast path) wasn’t being used for various reasons.
If people want to make their process spawning faster now, they’ll have
to use a profiling tool like sysprof to check their program’s
performance. Shocking.
I think I was wrong to put this `g_debug()` in there in the first place
— it hasn’t served its purpose of making people speed up their spawn
paths to use `posix_spawn()`, it’s only cluttered up logs and frustrated
people.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Allocate a working buffer before calling `fork()` to avoid calling
`malloc()` in the async-signal-safe context between `fork()` and
`exec()`, where it’s not safe to use.
In this case, the buffer is used to assemble a wrapper around `argv` so
it can be run under `/bin/sh`.
See `man 7 signal-safety`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #2140
Allocate a working buffer before calling `fork()` to avoid calling
`malloc()` in the async-signal-safe context between `fork()` and
`exec()`, where it’s not safe to use.
In this case, the buffer is used to assemble elements from `PATH` with
the binary from `argv[0]` to try executing them.
See `man 7 signal-safety`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #2140
Query the environment before calling `fork()` so that it doesn’t have to
be called in the async-signal-safe context between `fork()` and
`exec()`.
See `man 7 signal-safety`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #2140
They’re not safe to call in an async-signal-safe context on Linux.
`sysconf()` is safe to call on FreeBSD and OpenBSD (at least), so
continue doing that.
This will reduce performance in the (already low performance) fallback
case where `/proc` is inaccessible to a forked process on Linux, while
spawning a subprocess.
See `man 7 signal-safety`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #2140
Use the error handling infrastructure which already exists for other
failures in the async-signal-safe context.
`g_assert()` is unlikely to have caused problems in practice because it
is only async-signal-unsafe when the assertion condition fails.
See `man 7 signal-safety`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #2140
While `g_ascii_isdigit()` *is* currently async-signal-safe, it’s going
to be hard to remember to keep it that way if the implementation changes
in future.
It seems more robust to just reimplement it here, given that it’s not
much code.
See `man 7 signal-safety`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #2140
Use normal `close()` instead, which is guaranteed to be
async-signal-safe.
See `man 7 signal-safety`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #2140
Functions called between `fork()` and `exec()` have to be
async-signal-safe.
Add a comment to each function which is called in that context, and
`FIXME` comments to the non-async-signal-safe functions which end up
being called as leaves of the call graph.
The following commits will fix those `FIXME`s.
See `man 7 signal-safety`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #2140
This removes the limitation of select() that only FDs with values lower
than FD_SETSIZE can be used. Previously, if the out/err pipe FDs had
high values (which could happen if a large process, like Firefox, was
spawning subprocesses while having a lot of FDs open), GLib would abort
due to an assertion failure in libc.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #954
`man dup2` says that on Linux, dup2() can return `EBUSY` if the
operation needs to be retried (in addition to returning `EINTR` in other
cases where it needs to be retried).
Signed-off-by: Philip Withnall <withnall@endlessm.com>
All uses of fdwalk in gspawn are between fork and exec, which means only
async-signal safe functions can be called if the parent process has
multiple threads. Since fdwalk is not a standard API, we should not
assume it is safe to use unless the manual of the system explicitly says
it is async-signal safe.
Fixes: #1638
Instead of calling close or fcntl on all possible file descriptors,
which is slow on systems with very high limit or even no limit on open
file descriptors, we can use closefrom or fcntl with F_CLOSEM to close
all unwanted file descriptors with a single system call.
This change only improves the performance when GSpawnChildSetupFunc is
NULL because there are applications known to abuse GSpawnChildSetupFunc
to unset FD_CLOEXEC on file descriptors. Since the change mentioned
above requires closing file descriptors directly, it cannot be used when
the caller may want to keep some of them open.
This patch was written by Sebastian Schwarz <seschwar@gmail.com> and
uploaded to https://gitlab.gnome.org/GNOME/glib/merge_requests/574.
It was later modified by Ting-Wei Lan <lantw@src.gnome.org> to address
code review issues.
Fixes: https://gitlab.gnome.org/GNOME/glib/issues/1638
These squash various warnings from `scan-build`. None of them are
legitimate bugs, but some of them do improve code readability a bit.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1767
opendir and closedir are not async-signal-safe, these may call malloc
under the hood and cause a deadlock in a multi-threaded program.
This only affected Linux when /proc is mounted, other systems use a
slower path that iterates through all potential file descriptors.
Fixes a long-standing problem (since GLib 2.14.2).
Closes#945 and #1014
Philip Withnall suggests that glib should treat all negative
file descriptors as unset/invalid, rather than explicitly requiring
them to be -1.
This can simplify the logic for some users of this code.
https://gitlab.gnome.org/GNOME/gnome-shell/merge_requests/132
Function do_posix_spawn uses environ, but gspawn.c doesn't declare it.
Since there is no system header declaring this global variable, this
causes compilation error on FreeBSD.
Code added in this commit is copied from genviron.c.
G_SPAWN_LEAVE_DESCRIPTORS_OPEN must be set to enable the optimized
posix_spawn codepath, so this flag is likely to see more usage now.
Document that FD_CLOEXEC can be used to cause file descriptors to be
automatically closed while this flag is used.
When the amount of free memory on the system is somewhat low, gnome-shell
will sometimes fail to launch apps, reporting the error:
fork(): Cannot allocate memory
fork() is failing here because while cloning the process virtual address
space, Linux worries that the thread being forked may end up COWing the
entire address space of the parent process (gnome-shell, which is
memory-hungry), and there is not enough free memory to permit that to
happen.
In this case we are simply calling fork() in order to quickly call exec(),
which will throw away the entirity of the duplicated VM, so we should
look for ways to avoid the overcommit check.
The well known solution to this is to use clone(CLONE_VM) or vfork(), which
completely avoids creating a new memory address space for the child.
However, that comes with a bunch of caveats and complications:
https://gist.github.com/nicowilliams/a8a07b0fc75df05f684c23c18d7db234https://ewontfix.com/7/
In 2016, glibc's posix_spawn() was rewritten to use this approach
while also resolving the concerns.
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=9ff72da471a509a8c19791efe469f47fa6977410
I experimented with a similar approach in glib, but it was not practical
because glibc has several items of important internal knowledge (such as
knowing which signals should be given special treatment because they are
NPTL implementation details) that are not cleanly exposed elsewhere.
Instead, this patch adapts the gspawn code to use posix_spawn() where
possible, which will reap the benefits of that implementation.
The posix_spawn API is more limited than the gspawn API though,
partly due to natural limitations of using CLONE_VM, so the posix_spawn
path is added as a separate codepath which is only executed when the
conditions are right. Callers such as gnome-shell will have to be modified
to meet these conditions, such as not having a child_setup function.
In addition to allowing for the gnome-shell "Cannot allocate memory"
failure to be avoided, this should result in a general speedup in this
area, because fork()'s behaviour of cloning the entire VM space
has a cost which is now avoided. posix_spawn() has also recently
been optimized on OpenSolaris as the most performant way to spawn
a child process.
Add a new process spawning function variant which allows the caller
to pass specific file descriptors for stdin, stdout and stderr.
It is otherwise identical to g_spawn_async_with_pipes.
Allow the same fd to be passed in multiple parameters. To make this
workable, the child process logic that closes the fd after the first time
it has been dup2'ed needed tweaking; we now just set those fds to be
closed upon exec using the CLOEXEC flag. Add a test for this case.
This will be used by gnome-shell to avoid performing equivalent
dup2 actions in a child_setup function. Dropping use of child_setup will
enable use of an upcoming optimized process spawning codepath.
This will fix a few broken links in the documentation, and shut up a
load of gtk-doc warnings (but certainly not all of them).
Signed-off-by: Philip Withnall <withnall@endlessm.com>
https://bugzilla.gnome.org/show_bug.cgi?id=790015
The warnings issued when dealing with waitpid() raising ECHILD are
somewhat misleading: there are lots of reasons why waitpid() might
fail in this way, and we can't tell which one has happened.
In particular, passing a non-child or a non-pid, waiting for the same
pid elsewhere, or creating a duplicate watch for the same pid would
all fail in the same way.
Consolidate the restrictions into one place, and change all the other
places they were (or should have been!) mentioned to point to
that one place.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Reviewed-by: Philip Withnall <withnall@endlessm.com>
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=723743
All glib/*.{c,h} files have been processed, as well as gtester-report.
12 of those files are not licensed under LGPL:
gbsearcharray.h
gconstructor.h
glibintl.h
gmirroringtable.h
gscripttable.h
gtranslit-data.h
gunibreak.h
gunichartables.h
gunicomp.h
gunidecomp.h
valgrind.h
win_iconv.c
Some of them are generated files, some are licensed under a BSD-style
license and win_iconv.c is in the public domain.
Sub-directories inside glib/:
deprecated/: processed in a previous commit
glib-mirroring-tab/: already LGPLv2.1+
gnulib/: not modified, the code is copied from gnulib
libcharset/: a copy
pcre/: a copy
tests/: processed in a previous commit
https://bugzilla.gnome.org/show_bug.cgi?id=776504
And clarify that you must add a child watch or *not* use the
G_SPAWN_DO_NOT_REAP_CHILD flag, otherwise your child will become a
zombie on exit, and will not be reaped until the parent process exits.
Signed-off-by: Philip Withnall <philip@tecnocode.co.uk>
If we have an input parameter (or return value) we need to use (nullable).
However, if it is an (inout) or (out) parameter, (optional) is sufficient.
It looks like (nullable) could be used for everything according to the
Annotation documentation, but (optional) is more specific.
Adds the filename annotation for all file names
and things which can contain file names like
environment variables, argv-
On Unix they can contain anything while on Windows
they are always utf-8.
https://bugzilla.gnome.org/show_bug.cgi?id=767245
