As hidden file caches currently work, every look up on a directory caches
its .hidden file contents, and sets a 5s timeout to prune the directory
from the cache.
This creates a problem for usecases like Tracker Miners, which is in the
business of inspecting as many files as possible from as many directories
as possible in the shortest time possible. One timeout is created for each
directory, which possibly means gobbling thousands of entries in the hidden
file cache. This adds as many GSources to the glib worker thread, with the
involved CPU overhead in iterating those in its main context.
To fix this, use a unique timeout that will keep running until the cache
is empty. This will keep the overhead constant with many files/folders
being queried.
This incidentally also exercises the intended pattern for sending fds in
a D-Bus message: the fd list is meant to contain exactly those fds that
are referenced by a handle (type 'h') in the body of the message, with
numeric handle value n corresponding to g_unix_fd_list_peek_fds(...)[n].
Being able to send and receive file descriptors that are not referenced by
a handle (as in OpenFile here) is a quirk of the GDBus API, and while it's
entirely possible in the wire protocol, other D-Bus implementations like
libdbus and sd-bus typically don't provide APIs that make this possible.
Reproduces: https://gitlab.gnome.org/GNOME/glib/-/issues/2074
Signed-off-by: Simon McVittie <smcv@collabora.com>
Suppose we are sending a 5K message with fds (so data->blob points
to 5K of data, data->blob_size is 5K, and fd_list is non-null), but
the kernel is only accepting up to 4K with each sendmsg().
The first time we get into write_message_continue_writing(),
data->total_written will be 0. We will try to write the entire message,
plus the attached file descriptors; or if the stream doesn't support
fd-passing (not a socket), we need to fail with
"Tried sending a file descriptor on unsupported stream".
Because the kernel didn't accept the entire message, we come back in.
This time, we won't enter the Unix-specific block that involves sending
fds, because now data->total_written is 4K, and it would be wrong to try
to attach the same fds again. However, we also need to avoid failing
with "Tried sending a file descriptor on unsupported stream" in this
case. We just want to write out the data of the rest of the message,
starting from (blob + total_written) (in this exaple, the last 1K).
Resolves: https://gitlab.gnome.org/GNOME/glib/-/issues/2074
Signed-off-by: Simon McVittie <smcv@collabora.com>
This test ensures that g_socket_client_connect_to_host_async() fails if
it is cancelled, but it's not cancelled until after 1 millisecond. Our
CI testers are hitting that race window, and Milan is able to reproduce
the crash locally as well. Switching it from 1ms to 0ms is enough for
Milan to avoid the crash, but not enough for our CI, so let's move the
cancellation to a GSocketClientEvent callback where the timing is
completely deterministic.
Hopefully fixes#2221
`g_local_file_fstatat()` needs to fall back to returning an error if
`fstatat()` isn’t defined, which is the case on older versions of macOS
(as well as Windows, which was already handled). Callers shouldn’t call
`g_local_file_fstatat()` in these cases. (That’s already the case.)
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Fixes: #2203
The previous parsing code could read off the end of a URI if it had an
incorrect %-escaped character in.
Fix that, and more closely implement parsing for the syntax defined in
RFC 6874, which is the amendment to RFC 3986 which specifies zone ID
syntax.
This requires reworking some network-address tests, which were
previously treating zone IDs incorrectly.
oss-fuzz#23816
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Emptying trash over `gio trash` is a bit slow in comparison to plain
`rm -r`. On my system, it took about 3 min to empty the trash with a
folder containing 600 000 files, which is not ideal as `rm -r` call
took just a few seconds. I found that `g_file_delete` is implemented
differently for locations provided by the trash backend. The trash
backend prevents modifications of trashed content thus the delete
operation is allowed only for the top-level files and folders. So it
is not necessary to recursive delete all files as the permission
denied error is returned anyway. Let's call `g_file_delete` only for
top-level items, which reduces the time necessary for emptying trash
from minutes to seconds...
See: https://gitlab.gnome.org/GNOME/nautilus/-/issues/1589
Some filesystems don't have meaningful access times under at least some
circumstances (see #2189, #2205). In this situation the traditional stat()
and related kernel interfaces have to put something meaningless in the
st_atime field, and have no way to signal that it is meaningless.
However, statx() does have a way to signal that the atime is meaningless:
if the filesystem doesn't provide a useful access time, it will unset
the STATX_ATIME bit (as well as filling in the same meaningless value
for the stx_atime field that stat() would have used, for compatibility).
We don't actually *need* the atime, so never include it in the required
mask. This was already done for one code path in commit 6fc143bb
"gio: Allow no atime from statx" to fix#2189, but other callers were
left unchanged in that commit, and receive the same change here.
It is not actually guaranteed that *any* of the flags in the
returned stx_mask will be set (the only guarantee is that items in
STATX_BASIC_STATS have at least a harmless compatibility value, even if
their corresponding flag is cleared), so it might be better to follow
this up by removing the concept of the required mask entirely. However,
as of Linux 5.8 it looks as though STATX_ATIME is the only flag in
STATX_BASIC_STATS that might be cleared in practice, so this simpler
change fixes the immediate regression.
Resolves: https://gitlab.gnome.org/GNOME/glib/-/issues/2205
Signed-off-by: Simon McVittie <smcv@collabora.com>
It is not allowed to be `NULL` or unset if requested by the file
attribute matcher. Derive it from the basename. This doesn’t handle the
situation of a failed UTF-16 to UTF-8 conversion very well, but will at
least return something.
Note that the `g_filename_display_basename()` function can’t be used as
`GWinHttpFile` provides its URI in UTF-16 rather than in the file system
encoding.
This fixes a crash when using GIMP on Windows. Thanks to lillolollo for
in-depth debugging assistance.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #2194
For interoperability with libdbus, we want to use compatible timeouts.
In particular, this fixes a spurious failure of the `gdbus-server-auth`
test caused by libdbus and gdbus choosing to expire the key (cookie) at
different times, as diagnosed by Thiago Macieira. Previously, the libdbus
client would decline to use keys older than 7 minutes, but the GDBus
server would not generate a new key until the old key was 10 minutes old.
For completeness, also adjust the other arbitrary timeouts in the
DBUS_COOKIE_SHA1 mechanism to be the same as in libdbus. To make it
easier to align with libdbus, create internal macros with the same names
and values used in dbus-keyring.c.
* maximum time a key can be in the future due to clock skew between
systems sharing a home directory
- spec says "a reasonable time in the future"
- was 1 day
- now 5 minutes
- MAX_TIME_TRAVEL_SECONDS
* time to generate a new key if the newest is older
- spec says "If no recent keys remain, the server may generate a new
key", but that isn't practical, because in reality we need a grace
period during which an old key will not be used for new authentication
attempts but old authentication attempts can continue (in practice both
libdbus and GDBus implemented this logic)
- was 10 minutes
- now 5 minutes
- NEW_KEY_TIMEOUT_SECONDS
* time to discard old keys
- spec says "the timeout can be fairly short"
- was 15 minutes
- now 7 minutes
- EXPIRE_KEYS_TIMEOUT_SECONDS
* time allowed for a client using an old key to authenticate, before
that key gets deleted
- was at least 5 minutes
- now at least 2 minutes
- at least (EXPIRE_KEYS_TIMEOUT_SECONDS - NEW_KEY_TIMEOUT_SECONDS)
Based on a merge request by Philip Withnall.
Fixes: #2164
Thanks: Philip Withnall
Thanks: Thiago Macieira
Signed-off-by: Simon McVittie <smcv@collabora.com>
This doesn't trigger the cancellation assertion issue when run locally
(the task didn't return yet, so the error is simply overwritten), but
perhaps it ever does in CI. Anyhow, it's good to have a cancellation
test.
After a splice operation is finished, it attempts to 1) close input/output
streams, as per the given flags, and 2) return the operation result (maybe
an error, too).
However, if the operation gets cancelled early and the streams indirectly
closed, the splice operation will try to close both descriptors and return
on the task when both are already closed. The catch here is that getting the
streams closed under its feet is possible, so the completion callback would
find both streams closed after returning on the first close operation and
return the error, but then the second operation could be able to trigger
a second error which would be returned as well.
What happens here is up to further race conditions, if the task didn't
return yet, the returned error will be simply replaced (but the old one not
freed...), if it did already return, it'll result in:
GLib-GIO-FATAL-CRITICAL: g_task_return_error: assertion '!task->ever_returned' failed
Fix this by flagging the close_async() callbacks, and checking that both
close operations did return, instead of checking that both streams are
closed by who knows.
This error triggers a semi-frequent CI failure in tracker, see the summary at
https://gitlab.gnome.org/GNOME/tracker/-/issues/240
statx does not provide stx_atime when querying a file in a read-only
mounted file system. So call to statx should not expect it to be in
the mask. Otherwise we would fail with ERANGE for querying any file in
a read-only file system.
Fixes#2189.
The `make_pollfd()` call can’t fail because it only does so if
`cancellable == NULL`, and we’ve already checked that. Assert that’s the
case, to shut Coverity up and to catch behavioural changes in future.
Coverity CID: #1159433
Signed-off-by: Philip Withnall <withnall@endlessm.com>
`g_strsplit()` never returns `NULL`, although it can return an empty
strv (i.e. with its first element being `NULL`).
Drop a redundant `NULL` check.
Coverity CID: #1430976
Signed-off-by: Philip Withnall <withnall@endlessm.com>
If `statx()` is supported, query it for the file creation time and use
that if returned.
Incorporating some minor code rearrangement by Philip Withnall
<withnall@endlessm.com>.
Fixes: #1970
This currently just implements the same functionality as the existing
`stat()`/`fstat()`/`fstatat()`/`lstat()` calls, although where a reduced
field set is requested it may return faster.
Helps: #1970
It turns out that our async write operation implementation is broken
on non-O_NONBLOCK pipes, because the default async write
implementation calls write() after poll() said there were some
space. However, the semantics of pipes is that unless O_NONBLOCK is set
then the write *will* block if the passed in write count is larger than
the available space.
This caused a deadlock in https://gitlab.gnome.org/GNOME/glib/-/issues/2182
due to the loop-back of the app stdout to the parent, but even without such
a deadlock it is a problem that we may block the mainloop at all.
In the particular case of g_subprocess_communicate() we have full
control of the pipes after starting the app, so it is safe to enable
O_NONBLOCK (i.e. we can ensure all the code using the fd after this can handle
non-blocking mode).
This fixes https://gitlab.gnome.org/GNOME/glib/-/issues/2182
This has almost the same semantics as WSAECONNRESET and for all
practical purposes is handled the same. The main difference is about
*who* reset the connection: the peer or something in the network.
For UDP sockets this happens when receiving packets and previously sent
packets returned an ICMP "Time(-to-live) expired" message. This is
similar to WSAECONNRESET, which on UDP sockets happens when receiving
packets and previously sent packets returned an ICMP "Port Unreachable"
message.
This is a step towards supporting `statx()`, which allows the set of
fields it returns to be specified by the caller. Currently, the existing
`stat()` and `fstat()` calls continue to be made, and there are no
behavioural changes — but the new wrapper functions will be extended in
future.
Helps: #1970
Don’t call `g_file_query_exists()` followed by `g_file_delete()`. Just
call `g_file_delete()` and check the error.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Make `G_URI_FLAGS_PARSE_RELAXED` available instead, for the
implementations which need to handle user-provided or incorrect URIs.
The default should nudge people towards being compliant with RFC 3986.
This required also adding a new `G_URI_PARAMS_PARSE_RELAXED` flag, as
previously parsing param strings *always* used relaxed mode and there
was no way to control it. Now it defaults to using strict mode, and the
new flag allows for relaxed mode to be enabled if needed.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #2149
Add support for x-gvfs-notrash mount option, which allows to disable
trash functionality for certain mounts. This might be especially useful
e.g. to prevent trash folder creation on enterprise shares, which are
also accessed from Windows...
https://bugzilla.redhat.com/show_bug.cgi?id=1096200
There is already g_unix_mount_at function which allows to find certain
unix mount for given mount path. It would be useful to have similar
function for mount points, which will allow to replace custom codes in
gvfs. Let's add g_unix_mount_point_at.
