In Windows development environments that have it, <unistd.h> is mostly
just a wrapper around several other native headers (in particular,
<io.h>, which contains read(), close(), etc, and <process.h>, which
contains getpid()). But given that some Windows dev environments don't
have <unistd.h>, everything that uses those functions on Windows
already needed to include the correct Windows header as well, and so
there is never any point to including <unistd.h> on Windows.
Also, remove some <unistd.h> includes (and a few others) that were
unnecessary even on unix.
https://bugzilla.gnome.org/show_bug.cgi?id=710519
There are two benefits to this:
1) We can centralize any operating system specific knowledge of
close-vs-EINTR handling. For example, while on Linux we should never
retry, if someone cared enough later about HP-UX, they could come by
and change this one spot.
2) For places that do care about the return value and want to provide
the caller with a GError, this function makes it convenient to do so.
Note that gspawn.c had an incorrect EINTR loop-retry around close().
https://bugzilla.gnome.org/show_bug.cgi?id=682819
$ sed -i s,determing,determining,g gio/gdrive.c
$ sed -i s,determing,determining,g gio/gdbusprivate.c
http://www.merriam-webster.com/dictionary/determining
For some reason according to `git log --follow` the whole file was created during some translation update.
commit c45b813504
Author: Timo Jyrinki <timo@debian.org>
Date: Mon Mar 12 11:02:04 2012 +0200
Finnish translation update from http://l10n.laxstrom.name/wiki/Gnome_3.4 translation sprint
Call g_simple_async_result_set_check_cancellable() after all
GSimpleAsyncResult creation in order to take advantage of the new
reliable cancellation feature.
The guarantee of reliable cancellation fixes a bug in dbusmenu (which
was already assuming that cancellation was reliable). See this bug:
https://bugs.launchpad.net/ubuntu/+source/libdbusmenu/+bug/953562https://bugzilla.gnome.org/show_bug.cgi?id=672013
Signed-off-by: David Zeuthen <davidz@redhat.com>
This was a regression in commit f41178c6c: flush_async_data wasn't
necessarily NULL in the "don't flush" case.
Also move initialization of these variables up so that it's
unconditional, since that's easier to verify than checking
that each branch gets it right.
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=664617
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
We didn't previously flush in a couple of cases where we should have
done:
* a write is running when flush is called: we should flush after it
finishes
* writes have been made since the last flush, but none are pending or
running right now: we should flush the underlying transport straight
away
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=662395
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Reviewed-by: Cosimo Alfarano <cosimo.alfarano@collabora.co.uk>
This makes it easier to schedule a flush, by putting it on the same code
path as writing and closing.
Also change message_written to expect the lock to be held, since all
that's left in that function either wants to hold the lock or doesn't
care, and it's silly to release the lock immediately before calling
message_written, which just takes it again.
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=662395
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Reviewed-by: Cosimo Alfarano <cosimo.alfarano@collabora.co.uk>
When we use this function to schedule a flush, it'll be called
with the lock held. Releasing and immediately re-taking the lock would
be pointless.
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=662395
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Reviewed-by: Cosimo Alfarano <cosimo.alfarano@collabora.co.uk>
maybe_write_next_message now also closes, and I'm about to make it
consider whether to flush as well, so its name is increasingly
inappropriate. Similarly, write_message_in_idle_cb is a wrapper around
it which could do any of those things.
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=662395
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Reviewed-by: Cosimo Alfarano <cosimo.alfarano@collabora.co.uk>
If the user calls flush_sync() with no messages in the queue, but an
async write call pending, then we ought to flush after that async write
returns (although we don't currently do that). If it was an async close
or flush that was pending, there's no need to flush (again) afterwards.
So, we need to distinguish.
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=662395
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Reviewed-by: Cosimo Alfarano <cosimo.alfarano@collabora.co.uk>
My previous fix for GNOME#662100 was incomplete: it seems that with some
timings, the stream can be closed with an async read in-flight. This
can make the read fail immediately with G_IO_ERROR_CLOSED instead of
becoming cancelled.
This happens reliably on an embedded device, and rarely on my laptop;
repeating the test 100 times in quick succession reliably reproduces
the bug on my laptop.
It seems as though what we really want is to ignore read errors, once
we've established that we want to close the connection anyway - this
means that after asking to close, you're immune to exit-on-close,
which seems like a good rule.
An additional subtlety is that continuing to read after we know we
want to close is still required, otherwise we'll never emit ::closed.
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=662100
Bug-NB: NB#287088
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Reviewed-by: Colin Walters <walters@verbum.org>
This was a regression caused by my previous work on GDBusWorker thread-safety
(Bug #651268). The symptom is that if you disconnect a GDBusConnection
locally, the default implementation of GDBusConnection::closed
terminates your process, even though it shouldn't do that for
locally-closed connections; this is because GDBusWorker didn't think a
cancelled read was a local close.
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=662100
Bug-NB: NB#287088
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Reviewed-by: David Zeuthen <davidz@redhat.com>
And remove the 'joinable' argument from g_thread_new() and
g_thread_new_full().
Change the wording in the docs. Clarify expectations for
(deprecated) g_thread_create().
Deprecate both g_thread_create functions and add
g_thread_new() and g_thread_new_full(). The new functions
expect a name for the thread.
Change GThreadPool, GMainContext and GDBus to create named threads.
https://bugzilla.gnome.org/show_bug.cgi?id=660635
Otherwise, we could use-after-free the GDBusWorker, if its last-unref
is immediately after _g_dbus_worker_new returns (before the worker thread
does its initial read).
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=651268
Bug-NB: NB#271520
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Signed-off-by: David Zeuthen <davidz@redhat.com>
This member is written in _g_dbus_worker_stop from arbitrary threads, and
read by the worker thread, so it should be accessed atomically.
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=651268
Bug-NB: NB#271520
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Signed-off-by: David Zeuthen <davidz@redhat.com>
We can't safely close the output part of the I/O stream until any
pending write or flush has been completed. In the worst case, this could
lead to an assertion failure in the worker (when the close wins the
race) or not closing the stream at all (when the write wins the race).
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=651268
Bug-NB: NB#271520
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Signed-off-by: David Zeuthen <davidz@redhat.com>
num_writes_pending was a counter, but it only took values 0 or 1, so make
it a boolean: it would never make sense to be trying to write out two
messages at the same time (they'd get interleaved).
Similarly, we can never be writing and flushing at the same time (that'd
mean we were flushing halfway through a message, which would be pointless)
so combine it with flush_pending too, calling the result output_pending.
Also assert that it takes the expected value whenever we change it,
and document the locking discipline used for it, including a subtle
case in write_message_in_idle_cb where it's not obvious at first glance
why we don't need the lock.
(Having the combined boolean at the top of the block of write-related
struct members improves struct packing on 64-bit platforms, by packing
read_num_ancillary_messages and output_pending into one word.)
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=651268
Bug-NB: NB#271520
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Signed-off-by: David Zeuthen <davidz@redhat.com>
... otherwise we might end up using the worker after it has been
freed. Reported by Dan Winship and Colin Walters.
This fix uncovered a bug in the /gdbus/nonce-tcp test case so "fix"
that as well to use a better way of having one thread wait for another
(using quotes for the word "fix" since it's pretty hackish to
busy-wait in one thread to wait for another).
Signed-off-by: David Zeuthen <davidz@redhat.com>
I can't see a reason to spin until the worker thread runs, so don't.
This avoids ugly sched_yield() calls that show up in strace and
annoy me; the code is cleaner now too.
We now grab the types needed for the WebKit workaround in the
thread creation area, but only release them when the thread itself
exits.
https://bugzilla.gnome.org/show_bug.cgi?id=651650
Turns out that GDBusWorker will issue callbacks (in its own thread)
even after g_dbus_worker_stop() has been called. This would rarely
happen (and unreffing a connection is even rarer) so only saw this bug
occasionally when running the gdbus-connection test case in a loop.
Fix up this issue by maintaining a set of GDBusConnection objects that
are currently "alive" and do nothing in the callbacks if the passed
user_data pointer is not in this set.
Also attempted to fix up a race condition with
_g_object_wait_for_single_ref_do() and its use of GObject toggle
references - for now, just resort to busy waiting, thereby
sidestepping the toggle reference mess altogether.
Signed-off-by: David Zeuthen <davidz@redhat.com>
The root problem is with GObject - for now, just work around it in
GDBus. Also include a test-case. See
https://bugzilla.gnome.org/show_bug.cgi?id=627724
for more information.
Signed-off-by: David Zeuthen <davidz@redhat.com>
Ryan pointed out that it's safe to do this because we have the
G_DBUS_SEND_MESSAGE_FLAGS_PRESERVE_SERIAL flag and that it simplifies
how filter functions work.
Signed-off-by: David Zeuthen <davidz@redhat.com>
Rework filter functions as per
https://bugzilla.gnome.org/show_bug.cgi?id=624546#c8
This commit breaks ABI. However, this ABI break affects only
applications using filter functions. The only known user of is dconf.
Signed-off-by: David Zeuthen <davidz@redhat.com>
... and use g_error_matches().
==29535== 1,360 (408 direct, 952 indirect) bytes in 17 blocks are definitely lost in loss record 1,252 of 1,264
==29535== at 0x4005BDC: malloc (vg_replace_malloc.c:195)
==29535== by 0x4057094: g_malloc (gmem.c:134)
==29535== by 0x406F2D6: g_slice_alloc (gslice.c:836)
==29535== by 0x406F31B: g_slice_alloc0 (gslice.c:848)
==29535== by 0x403A751: g_error_new_valist (gerror.c:54)
==29535== by 0x403AAD4: g_set_error (gerror.c:240)
==29535== by 0x41C06C8: g_socket_send_message (gsocket.c:2967)
==29535== by 0x421CB64: write_message_continue_writing (gdbusprivate.c:958)
==29535== by 0x421CE2A: write_message_async (gdbusprivate.c:1049)
==29535== by 0x421D4DD: maybe_write_next_message (gdbusprivate.c:1291)
==29535== by 0x421D26B: message_written (gdbusprivate.c:1187)
==29535== by 0x421D322: write_message_cb (gdbusprivate.c:1216)
Bug #628345.
==6793== 32 (24 direct, 8 indirect) bytes in 1 blocks are definitely lost in loss record 780 of 1,423
==6793== at 0x4005BDC: malloc (vg_replace_malloc.c:195)
==6793== by 0x4057094: g_malloc (gmem.c:134)
==6793== by 0x406F2D6: g_slice_alloc (gslice.c:836)
==6793== by 0x406F31B: g_slice_alloc0 (gslice.c:848)
==6793== by 0x413D5BB: g_type_create_instance (gtype.c:1867)
==6793== by 0x412372A: g_object_constructor (gobject.c:1482)
==6793== by 0x4122E1D: g_object_newv (gobject.c:1266)
==6793== by 0x4122B93: g_object_new (gobject.c:1178)
==6793== by 0x41DB582: g_unix_fd_list_new_from_array (gunixfdlist.c:191)
==6793== by 0x421BFD6: _g_dbus_worker_do_read_cb (gdbusprivate.c:590)
Bug #628329.
Allow modifying a GDBusMessage in a filter function and also add tests
for this. This breaks API but leaves ABI (almost) intact - at least
dconf's GSettings backend (the only big user I know of) will keep
working.
https://bugzilla.gnome.org/show_bug.cgi?id=624546
Signed-off-by: David Zeuthen <davidz@redhat.com>
Free the read buffer.
==26538== 4,096 bytes in 1 blocks are definitely lost in loss record 781 of 781
==26538== at 0x4005BDC: malloc (vg_replace_malloc.c:195)
==26538== by 0x4005C66: realloc (vg_replace_malloc.c:476)
==26538== by 0x405244D: g_realloc (gmem.c:181)
==26538== by 0x420E066: _g_dbus_worker_do_read_unlocked (gdbusprivate.c:780)
==26538== by 0x420E1D1: _g_dbus_worker_do_read (gdbusprivate.c:812)
==26538== by 0x420F14A: _g_dbus_worker_thread_begin_func (gdbusprivate.c:1318)
==26538== by 0x420D2ED: invoke_caller (gdbusprivate.c:266)
==26538== by 0x404DA7C: g_idle_dispatch (gmain.c:4224)
==26538== by 0x4049FCD: g_main_dispatch (gmain.c:2119)
==26538== by 0x404B2C1: g_main_context_dispatch (gmain.c:2672)
==26538== by 0x404B716: g_main_context_iterate (gmain.c:2750)
==26538== by 0x404BE7F: g_main_loop_run (gmain.c:2958)
==26538== by 0x420D2B5: shared_thread_func (gdbusprivate.c:248)
==26538== by 0x4077958: g_thread_create_proxy (gthread.c:1897)
==26538== by 0x57D918: start_thread (pthread_create.c:301)
==26538== by 0x4C6CBD: clone (clone.S:133)
Bug #627187.
If sending a lot of data and/or the other peer is not reading it, then
socket buffers can overflow. This is communicated from the kernel by
returning EAGAIN. In GIO, it is modelled by g_output_stream_write()
and g_socket_send_message() returning G_IO_ERROR_WOULD_BLOCK.
It is also problematic that that we're using synchronous IO in the
shared GDBus IO thread. It means that one GDBusConnection can lock up
others.
It turns out that by porting from g_output_stream_write() to
g_output_stream_write_async() we fix the EAGAIN issue. For GSocket, we
still need to handle things manually (by creating a GSource) as
g_socket_send_message() is used.
We check the new behavior in Michael's producer/consumer test case (at
/gdbus/overflow in gdbus-peer.c) added in the last commit.
Also add a test case that sends and receives a 20 MiB message.
Also add a new `transport' G_DBUS_DEBUG option so it is easy to
inspect partial writes:
$ G_DBUS_DEBUG=transport ./gdbus-connection -p /gdbus/connection/large_message
[...]
========================================================================
GDBus-debug:Transport:
>>>> WROTE 128000 bytes of message with serial 4 and
size 20971669 from offset 0 on a GSocketOutputStream
========================================================================
GDBus-debug:Transport:
>>>> WROTE 128000 bytes of message with serial 4 and
size 20971669 from offset 128000 on a GSocketOutputStream
========================================================================
GDBus-debug:Transport:
>>>> WROTE 128000 bytes of message with serial 4 and
size 20971669 from offset 256000 on a GSocketOutputStream
[...]
========================================================================
GDBus-debug:Transport:
>>>> WROTE 43669 bytes of message with serial 4 and
size 20971669 from offset 20928000 on a GSocketOutputStream
[...]
========================================================================
GDBus-debug:Transport:
<<<< READ 16 bytes of message with serial 3 and
size 20971620 to offset 0 from a GSocketInputStream
========================================================================
GDBus-debug:Transport:
<<<< READ 15984 bytes of message with serial 3 and
size 20971620 to offset 16 from a GSocketInputStream
========================================================================
GDBus-debug:Transport:
<<<< READ 16000 bytes of message with serial 3 and
size 20971620 to offset 16000 from a GSocketInputStream
[...]
========================================================================
GDBus-debug:Transport:
<<<< READ 144000 bytes of message with serial 3 and
size 20971620 to offset 20720000 from a GSocketInputStream
========================================================================
GDBus-debug:Transport:
<<<< READ 107620 bytes of message with serial 3 and
size 20971620 to offset 20864000 from a GSocketInputStream
OK
https://bugzilla.gnome.org/show_bug.cgi?id=626748
Signed-off-by: David Zeuthen <davidz@redhat.com>
