As hidden file caches currently work, every look up on a directory caches
its .hidden file contents, and sets a 5s timeout to prune the directory
from the cache.
This creates a problem for usecases like Tracker Miners, which is in the
business of inspecting as many files as possible from as many directories
as possible in the shortest time possible. One timeout is created for each
directory, which possibly means gobbling thousands of entries in the hidden
file cache. This adds as many GSources to the glib worker thread, with the
involved CPU overhead in iterating those in its main context.
To fix this, use a unique timeout that will keep running until the cache
is empty. This will keep the overhead constant with many files/folders
being queried.
Some filesystems don't have meaningful access times under at least some
circumstances (see #2189, #2205). In this situation the traditional stat()
and related kernel interfaces have to put something meaningless in the
st_atime field, and have no way to signal that it is meaningless.
However, statx() does have a way to signal that the atime is meaningless:
if the filesystem doesn't provide a useful access time, it will unset
the STATX_ATIME bit (as well as filling in the same meaningless value
for the stx_atime field that stat() would have used, for compatibility).
We don't actually *need* the atime, so never include it in the required
mask. This was already done for one code path in commit 6fc143bb
"gio: Allow no atime from statx" to fix#2189, but other callers were
left unchanged in that commit, and receive the same change here.
It is not actually guaranteed that *any* of the flags in the
returned stx_mask will be set (the only guarantee is that items in
STATX_BASIC_STATS have at least a harmless compatibility value, even if
their corresponding flag is cleared), so it might be better to follow
this up by removing the concept of the required mask entirely. However,
as of Linux 5.8 it looks as though STATX_ATIME is the only flag in
STATX_BASIC_STATS that might be cleared in practice, so this simpler
change fixes the immediate regression.
Resolves: https://gitlab.gnome.org/GNOME/glib/-/issues/2205
Signed-off-by: Simon McVittie <smcv@collabora.com>
statx does not provide stx_atime when querying a file in a read-only
mounted file system. So call to statx should not expect it to be in
the mask. Otherwise we would fail with ERANGE for querying any file in
a read-only file system.
Fixes#2189.
If `statx()` is supported, query it for the file creation time and use
that if returned.
Incorporating some minor code rearrangement by Philip Withnall
<withnall@endlessm.com>.
Fixes: #1970
This is a step towards supporting `statx()`, which allows the set of
fields it returns to be specified by the caller. Currently, the existing
`stat()` and `fstat()` calls continue to be made, and there are no
behavioural changes — but the new wrapper functions will be extended in
future.
Helps: #1970
Correct an off-by-one error in hex_unescape_string()'s computation of
the output string length.
(Turned into a git-format patch by Philip Withnall. Original patch
submitted on the Debian bug tracker, bug#962912.)
It's safe to assume an escaped string doesn't contain embedded null bytes,
but raw memory buffers (as returned by getxattr()) require more care.
If the length of the data to be escaped is known, use that knowledge instead
of invoking strlen().
(Turned into a git-format patch by Philip Withnall. One minor formatting
tweak. Original patch submitted on the Debian bug tracker, bug#962912.)
Fixes: #422
Since we (optionally) require nanosecond precision for this
(utimes() is used on *nix), use SetFileTime(), which nominally
has 100ns granularity (actual filesystem might be coarser), instead of
g_utime (), which only has 1-second granularity.
Expand our private statbuf structure with st_mtim, st_atim and st_ctim
fields, which are structs that contain tv_sec and tv_nsec fields,
representing a timestamp with 1-second precision (same value as st_mtime, st_atime
and st_ctime) and a fraction of a second (in nanoseconds) that adds nanosecond
precision to the timestamp.
Because FILEETIME only has 100ns precision, this won't be very precise,
but it's better than nothing.
The private _g_win32_filetime_to_unix_time() function is modified
to also return the nanoseconds-remainder along with the seconds timestamp.
The timestamp struct that we're using is named gtimespec to ensure that
it doesn't clash with any existing timespec structs (MinGW-w64 has one,
MSVC doesn't).
This reverts commit bfdc5fc4fc.
This changes the semantics of G_FILE_ATTRIBUTE_ID_UNIX_MODE, and we've
already found one user of the previous semantics (ostree).
Closes#1934
As with the previous commit, `st_mode` contains both the file type
(regular file, directory, symlink, special, etc.) and the file mode. For
`G_FILE_ATTRIBUTE_ID_UNIX_MODE`, we only want the file mode — so mask
`st_mode` with `~S_IFMT`.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
These were callers which explicitly specified the string length to
g_utf8_validate(), when it couldn’t be negative, and hence should be
able to unconditionally benefit from the increased string handling
length.
At least one call site would have previously silently changed behaviour
if called with strings longer than G_MAXSSIZE in length.
Another call site was passing strlen(string) to g_utf8_validate(), which
seems pointless: just pass -1 instead, and let g_utf8_validate()
calculate the string length. Its behaviour on embedded nul bytes
wouldn’t change, as strlen() stops at the first one.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Put the core readlink() code into a separate
_g_win32_readlink_handle_raw() function that takes a file handle,
can optionally ensure NUL-terminatedness of its output
(for cases where we need a NUL-terminator and do *not* need
to get the exact contents of the symlink as it is stored in FS)
and can either fill a caller-provided buffer *or* allocate
its own buffer, and can also read the reparse tag.
Put the rest of readlink() code into separate
functions that do UTF-16<->UTF-8, strip inconvenient prefix
and open/close the symlink file handle as needed.
Split _g_win32_stat_utf16_no_trailing_slashes() into
two functions - the one that takes a filename and the one
that takes a file descriptor. The part of these functions
that would have been duplicate is now split into the
_g_win32_fill_privatestat() funcion.
Add more comments explaining what each function does.
Only g_win32_readlink_utf8(), which is callable from outside
via private function interface, gets a real doc-comment,
the rest get normal, non-doc comments.
Change all callers to use the new version of the private
g_win32_readlink_utf8() function, which can now NUL-terminate
and allocate on demand - no need to call it in a loop.
Also, the new code should correctly get reparse tag when the
caller does fstat() on a symlink. Do note that this requires
the caller to get a FD for the symlink, not the target. Figuring
out how to do that is up to the caller.
Since symlink info (target path and reparse tag) are now always
read directly, via DeviceIoControl(), we don't need to use
FindFirstFileW() anymore.
On Windows NTFS symlinks are implemented as reparse points,
which are special kinds of files *or directories*. A directory
symlink should link to a directory. A file symlink should link
to a file. Mismatching (such as a file symlink pointing to a
directory) produces symlinks that simply do not function.
Therefore GFileType file vs directory vs symlink distinction is
too simplistic to correctly represent a NTFS filesystem object type.
Since we can't turn back time and choose a better way of representing
file types, make GFileType reflect the file vs directory type on
Windows, meaning that all FS objects are either files or
directories (or shortcuts, which are also files), but never symlinks.
A test for symlinkiness will have to be made via GFileInfo - it
tracks symlinkiness separately from file/directory/whatever.
The st_nlink field of the stat structure has meaning and should
be put into GFileInfo.
The st_mode field is far less meaningful, but could still be used
for some purposes, adjust the comment to clarify that.
G_FILE_ATTRIBUTE_DOS_IS_MOUNTPOINT allows mountpoints
(NTFS reparse points with IO_REPARSE_TAG_MOUNT_POINT tag) to
be told apart from symlinks (NTFS reparse points with
IO_REPARSE_TAG_SYMLINK tag), even though both are reported
by glib as "symlinks".
G_FILE_ATTRIBUTE_DOS_REPARSE_POINT_TAG allows the exact
reparse tag value to be obtained by the user. This way
even more exotic reparse points can be identified and
handled by the user (glib itself currently has no code
to work with any reparse points that are not symlinks
or mountpoints).
While mountpoints are *not* symlinks, strictly speaking,
they works in a similar enough way, so consider them to be
symlinks for the purpose of querying local file info.
On Windows st_ctime field is the file creation time.
POSIX mandates that field to be the file state change time.
Naturally, glib code interpreted st_ctime as POSIX suggested,
and the result was bad.
Fix this by introducing special W32-only logic for setting
attributes from st_ctime field.
Fixes issue #1452.
To be honest, I am not sure why, but in some special environments (e.g.
our CI integration) can happen, that file device number is different from
parent device number. Return "Unable to find or create trash directory for
%s" error from g_local_file_trash() in that case and also set
G_FILE_ATTRIBUTE_ACCESS_CAN_TRASH appropriately.
The previously implementation considered a file to be a mountpoint if
its parent is on a different device. / is its own parent, so by this
definition it is not a mountpoint.
But / is (generally) listed in fstab, and fstab(5) defines the
directories it contains to be mountpoints. This attribute should follow
that definition (and reasonable expectation): the root directory is a
mountpoint.
So, add a special-case for the case where the file's parent has the same
st_dev and st_ino as the file, which is true only at the root.
Test this attribute at / (only on POSIX), /proc (but only on Linux), and
at many files and directories created by the test suite (which cannot be
mountpoints).
Fix various warnings regarding unused variables, duplicated
branches etc by adjusting the ifdeffery and some missing casts.
gnulib triggers -Wduplicated-branches in one of the copied files,
disable as that just makes updating the code harder.
The warning indicating missing features are made none fatal through
pragmas. They still show but don't abort the build.
https://bugzilla.gnome.org/show_bug.cgi?id=793729
In MacOS the file selection dialog does not show folder icons.
With this fix the folder icons are shown. The bug is described
in:
https://bugzilla.gnome.org/show_bug.cgi?id=788936
This bug fix is only partial, because this fix is only the
last resort when no mime information is available.
The thumbnail attributes would previously only be set if thumbnail::path
was included in the query — so querying for just thumbnail::is-valid
would return no results.
This fixes the behaviour of
gio info -a thumbnail::is-valid ./some-file.png
vs
gio info -a thumbnail ./some-file.png
The first command would previously list nothing. The second would
previously list a thumbnail::path and thumbnail::is-valid.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
https://bugzilla.gnome.org/show_bug.cgi?id=791325
This commit adds new W32-only functions to gstdio.c,
and a new header file, gstdioprivate.h.
These functions are:
g_win32_stat_utf8()
g_win32_lstat_utf8()
g_win32_fstat()
and they fill a private structure, GWin32PrivateStat,
which has all the fields that normal stat has, as well as some
extras.
These functions are then used throughout glib and gio to get better
data about the system. Specifically:
* Full, 64-bit size, guaranteed (g_stat() is forced to use 32-bit st_size)
* Full, 64-bit file identifier (st_ino is 0 when normal stat() is used, and still is)
* W32 File attributes (which stat() doesn't report); in particular, this allows
symlinks to be correctly identified
* Full, 64-bit time, guaranteed (g_stat() uses 32-bit st_*time on 32-bit Windows)
* Allocated file size (as a W32 replacement for the missing st_blocks)
st_mode remains unchanged (thus, no S_ISLNK), so when these are given back to
glib users (via g_stat(), for example, which is now implemented by calling g_win32_stat_utf8),
this field does not contain anything unexpected.
g_lstat() now calls g_win32_lstat_utf8(), which works on symlinks the way it's supposed to.
Also adds the g_win32_readlink_utf8() function, which behaves like readlink()
(including its inability to return 0-terminated strings and inability to say how large
the output buffer should be; these limitations are purely for compatibility with
existing glib code).
Thus, symlink support should now be much better, although far from being complete.
A new W32-only test in gio/tests/file.c highlights the following features:
* allocated size
* 64-bit time
* unique file IDs
https://bugzilla.gnome.org/show_bug.cgi?id=788180
ssize_t is supported widely, but not universally, so use gssize instead.
Currently only one piece of code actually *needs* this change to be compilable
with MSVC, the rest are mostly in *nix parts of the code, but these are changed
too, for symmetry.
https://bugzilla.gnome.org/show_bug.cgi?id=788180
The return value from g_get_user_special_dir() might be NULL. Safest to
use g_strcmp0() uniformly everywhere.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
https://bugzilla.gnome.org/show_bug.cgi?id=661442
Prevent the situation where errno is set by function A, then function B
is called (which is typically _(), but could be anything else) and it
overwrites errno, then errno is checked by the caller.
errno is a horrific API, and we need to be careful to save its value as
soon as a function call (which might set it) returns. i.e. Follow the
pattern:
int errsv, ret;
ret = some_call_which_might_set_errno ();
errsv = errno;
if (ret < 0)
puts (strerror (errsv));
This patch implements that pattern throughout GLib. There might be a few
places in the test code which still use errno directly. They should be
ported as necessary. It doesn’t modify all the call sites like this:
if (some_call_which_might_set_errno () && errno == ESOMETHING)
since the refactoring involved is probably more harmful than beneficial
there. It does, however, refactor other call sites regardless of whether
they were originally buggy.
https://bugzilla.gnome.org/show_bug.cgi?id=785577
This is an implementation of most of GContentType using the OS X
UTType APIs.
Missing at this point is an implementation of
g_content_types_get_registered() and g_content_type_guess_for_tree().
https://bugzilla.gnome.org/show_bug.cgi?id=734946
Bug 13403 introduced support for the non-POSIX variants of these APIs
found on a system called "DG/UX". Meanwhile, the complicated checks
here are breaking cross-builds on systems that we actually care about.
Remove the complicated checks and replace them with AC_CHECK_FUNCS.
Remove the resulting dead code from a couple of .c files.
https://bugzilla.gnome.org/show_bug.cgi?id=756475
Previously, GLib returned text/plain for empty files.
This is important because people may want to open empty (eg:
just-created) text files with the text editor.
An unintended side-effect of b6fc1df022
caused GLib to start returning application/octet-stream instead of
text/plain for these files.
This commit is essentially a revert of that commit, with a different
solution: we move the special-case up a bit in the function and
hard-code it to text/plain.
This change does not exactly maintain the old behaviour: previously, a
"fast" lookup would have returned application/octet-stream on an empty
file and now it will return text/plain. I consider this to be an
improvement (since we're returning better data) and don't expect it to
cause problems.
https://bugzilla.gnome.org/show_bug.cgi?id=755795
This will prevent attempting to read from some files that appear normal but are
really device-like, such as those in /proc and /sys.
If we can't stat() the file then don't bother attempting to sniff, either.
https://bugzilla.gnome.org/show_bug.cgi?id=708525
FreeBSD and NetBSD have field st_birthtim and st_birthtime in struct stat,
respectively, which can be used to get file creation time on supported file
systems such as UFS2 and tmpfs.
https://bugzilla.gnome.org/show_bug.cgi?id=749492
'folder' is the name of the folder icon in the incon naming spec,
and the Adwaita icon theme doesn't include an inode-directory icon.
This fixes folders appearing as generic file in the file chooser.
https://bugzilla.gnome.org/show_bug.cgi?id=731996
As of e6af432, g_content_type_get_symbolic_icon() returns non-symbolic
fallbacks. Thus, we can't append another symbolic icon to the fallbacks.
The special case was a bit of a hack anyway. It was only applied to
themed icons and there was no generic fallback for mime types that are
not folders.
https://bugzilla.gnome.org/show_bug.cgi?id=726046
glocalfileinfo.c: In function '_g_local_file_info_get':
glocalfileinfo.c:1955:11: warning: passing argument 3 of
'get_thumbnail_attributes' from incompatible pointer type [enabled by
default]
get_thumbnail_attributes (path, info, &statbuf);
^
glocalfileinfo.c:1285:1: note: expected 'const struct GStatBuf *' but
argument is of type 'struct _stati64 *'
get_thumbnail_attributes (const char *path,
https://bugzilla.gnome.org/show_bug.cgi?id=711547
g_strconcat() allocates memory, it needs to be freed.
==10653== 1,400 bytes in 50 blocks are definitely lost in loss record
1,838 of 1,851
==10653== at 0x4A0645D: malloc (in
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==10653== by 0x54ACB22: g_malloc (gmem.c:102)
==10653== by 0x54ACE4D: g_malloc_n (gmem.c:343)
==10653== by 0x54C8463: g_strconcat (gstrfuncs.c:589)
==10653== by 0x4D6ED38: get_xattrs_from_fd (glocalfileinfo.c:660)
==10653== by 0x4D71622:
_g_local_file_info_get_from_fd (glocalfileinfo.c:2028)
==10653== by 0x4D731A0:
g_local_file_input_stream_query_info (glocalfileinputstream.c:356)
==10653== by 0x4C996D8:
g_file_input_stream_query_info (gfileinputstream.c:148)
==10653== by 0x4C863F6: file_copy_fallback (gfile.c:3120)
==10653== by 0x4C86DD2: g_file_copy (gfile.c:3398)
https://bugzilla.gnome.org/show_bug.cgi?id=722357
