This was a regression in commit f41178c6c: flush_async_data wasn't
necessarily NULL in the "don't flush" case.
Also move initialization of these variables up so that it's
unconditional, since that's easier to verify than checking
that each branch gets it right.
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=664617
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
We didn't previously flush in a couple of cases where we should have
done:
* a write is running when flush is called: we should flush after it
finishes
* writes have been made since the last flush, but none are pending or
running right now: we should flush the underlying transport straight
away
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=662395
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Reviewed-by: Cosimo Alfarano <cosimo.alfarano@collabora.co.uk>
This makes it easier to schedule a flush, by putting it on the same code
path as writing and closing.
Also change message_written to expect the lock to be held, since all
that's left in that function either wants to hold the lock or doesn't
care, and it's silly to release the lock immediately before calling
message_written, which just takes it again.
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=662395
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Reviewed-by: Cosimo Alfarano <cosimo.alfarano@collabora.co.uk>
When we use this function to schedule a flush, it'll be called
with the lock held. Releasing and immediately re-taking the lock would
be pointless.
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=662395
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Reviewed-by: Cosimo Alfarano <cosimo.alfarano@collabora.co.uk>
maybe_write_next_message now also closes, and I'm about to make it
consider whether to flush as well, so its name is increasingly
inappropriate. Similarly, write_message_in_idle_cb is a wrapper around
it which could do any of those things.
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=662395
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Reviewed-by: Cosimo Alfarano <cosimo.alfarano@collabora.co.uk>
If the user calls flush_sync() with no messages in the queue, but an
async write call pending, then we ought to flush after that async write
returns (although we don't currently do that). If it was an async close
or flush that was pending, there's no need to flush (again) afterwards.
So, we need to distinguish.
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=662395
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Reviewed-by: Cosimo Alfarano <cosimo.alfarano@collabora.co.uk>
My previous fix for GNOME#662100 was incomplete: it seems that with some
timings, the stream can be closed with an async read in-flight. This
can make the read fail immediately with G_IO_ERROR_CLOSED instead of
becoming cancelled.
This happens reliably on an embedded device, and rarely on my laptop;
repeating the test 100 times in quick succession reliably reproduces
the bug on my laptop.
It seems as though what we really want is to ignore read errors, once
we've established that we want to close the connection anyway - this
means that after asking to close, you're immune to exit-on-close,
which seems like a good rule.
An additional subtlety is that continuing to read after we know we
want to close is still required, otherwise we'll never emit ::closed.
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=662100
Bug-NB: NB#287088
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Reviewed-by: Colin Walters <walters@verbum.org>
This was a regression caused by my previous work on GDBusWorker thread-safety
(Bug #651268). The symptom is that if you disconnect a GDBusConnection
locally, the default implementation of GDBusConnection::closed
terminates your process, even though it shouldn't do that for
locally-closed connections; this is because GDBusWorker didn't think a
cancelled read was a local close.
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=662100
Bug-NB: NB#287088
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Reviewed-by: David Zeuthen <davidz@redhat.com>
And remove the 'joinable' argument from g_thread_new() and
g_thread_new_full().
Change the wording in the docs. Clarify expectations for
(deprecated) g_thread_create().
Deprecate both g_thread_create functions and add
g_thread_new() and g_thread_new_full(). The new functions
expect a name for the thread.
Change GThreadPool, GMainContext and GDBus to create named threads.
https://bugzilla.gnome.org/show_bug.cgi?id=660635
Otherwise, we could use-after-free the GDBusWorker, if its last-unref
is immediately after _g_dbus_worker_new returns (before the worker thread
does its initial read).
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=651268
Bug-NB: NB#271520
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Signed-off-by: David Zeuthen <davidz@redhat.com>
This member is written in _g_dbus_worker_stop from arbitrary threads, and
read by the worker thread, so it should be accessed atomically.
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=651268
Bug-NB: NB#271520
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Signed-off-by: David Zeuthen <davidz@redhat.com>
We can't safely close the output part of the I/O stream until any
pending write or flush has been completed. In the worst case, this could
lead to an assertion failure in the worker (when the close wins the
race) or not closing the stream at all (when the write wins the race).
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=651268
Bug-NB: NB#271520
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Signed-off-by: David Zeuthen <davidz@redhat.com>
num_writes_pending was a counter, but it only took values 0 or 1, so make
it a boolean: it would never make sense to be trying to write out two
messages at the same time (they'd get interleaved).
Similarly, we can never be writing and flushing at the same time (that'd
mean we were flushing halfway through a message, which would be pointless)
so combine it with flush_pending too, calling the result output_pending.
Also assert that it takes the expected value whenever we change it,
and document the locking discipline used for it, including a subtle
case in write_message_in_idle_cb where it's not obvious at first glance
why we don't need the lock.
(Having the combined boolean at the top of the block of write-related
struct members improves struct packing on 64-bit platforms, by packing
read_num_ancillary_messages and output_pending into one word.)
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=651268
Bug-NB: NB#271520
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Signed-off-by: David Zeuthen <davidz@redhat.com>
... otherwise we might end up using the worker after it has been
freed. Reported by Dan Winship and Colin Walters.
This fix uncovered a bug in the /gdbus/nonce-tcp test case so "fix"
that as well to use a better way of having one thread wait for another
(using quotes for the word "fix" since it's pretty hackish to
busy-wait in one thread to wait for another).
Signed-off-by: David Zeuthen <davidz@redhat.com>
I can't see a reason to spin until the worker thread runs, so don't.
This avoids ugly sched_yield() calls that show up in strace and
annoy me; the code is cleaner now too.
We now grab the types needed for the WebKit workaround in the
thread creation area, but only release them when the thread itself
exits.
https://bugzilla.gnome.org/show_bug.cgi?id=651650
Turns out that GDBusWorker will issue callbacks (in its own thread)
even after g_dbus_worker_stop() has been called. This would rarely
happen (and unreffing a connection is even rarer) so only saw this bug
occasionally when running the gdbus-connection test case in a loop.
Fix up this issue by maintaining a set of GDBusConnection objects that
are currently "alive" and do nothing in the callbacks if the passed
user_data pointer is not in this set.
Also attempted to fix up a race condition with
_g_object_wait_for_single_ref_do() and its use of GObject toggle
references - for now, just resort to busy waiting, thereby
sidestepping the toggle reference mess altogether.
Signed-off-by: David Zeuthen <davidz@redhat.com>
The root problem is with GObject - for now, just work around it in
GDBus. Also include a test-case. See
https://bugzilla.gnome.org/show_bug.cgi?id=627724
for more information.
Signed-off-by: David Zeuthen <davidz@redhat.com>
Ryan pointed out that it's safe to do this because we have the
G_DBUS_SEND_MESSAGE_FLAGS_PRESERVE_SERIAL flag and that it simplifies
how filter functions work.
Signed-off-by: David Zeuthen <davidz@redhat.com>
Rework filter functions as per
https://bugzilla.gnome.org/show_bug.cgi?id=624546#c8
This commit breaks ABI. However, this ABI break affects only
applications using filter functions. The only known user of is dconf.
Signed-off-by: David Zeuthen <davidz@redhat.com>
... and use g_error_matches().
==29535== 1,360 (408 direct, 952 indirect) bytes in 17 blocks are definitely lost in loss record 1,252 of 1,264
==29535== at 0x4005BDC: malloc (vg_replace_malloc.c:195)
==29535== by 0x4057094: g_malloc (gmem.c:134)
==29535== by 0x406F2D6: g_slice_alloc (gslice.c:836)
==29535== by 0x406F31B: g_slice_alloc0 (gslice.c:848)
==29535== by 0x403A751: g_error_new_valist (gerror.c:54)
==29535== by 0x403AAD4: g_set_error (gerror.c:240)
==29535== by 0x41C06C8: g_socket_send_message (gsocket.c:2967)
==29535== by 0x421CB64: write_message_continue_writing (gdbusprivate.c:958)
==29535== by 0x421CE2A: write_message_async (gdbusprivate.c:1049)
==29535== by 0x421D4DD: maybe_write_next_message (gdbusprivate.c:1291)
==29535== by 0x421D26B: message_written (gdbusprivate.c:1187)
==29535== by 0x421D322: write_message_cb (gdbusprivate.c:1216)
Bug #628345.
==6793== 32 (24 direct, 8 indirect) bytes in 1 blocks are definitely lost in loss record 780 of 1,423
==6793== at 0x4005BDC: malloc (vg_replace_malloc.c:195)
==6793== by 0x4057094: g_malloc (gmem.c:134)
==6793== by 0x406F2D6: g_slice_alloc (gslice.c:836)
==6793== by 0x406F31B: g_slice_alloc0 (gslice.c:848)
==6793== by 0x413D5BB: g_type_create_instance (gtype.c:1867)
==6793== by 0x412372A: g_object_constructor (gobject.c:1482)
==6793== by 0x4122E1D: g_object_newv (gobject.c:1266)
==6793== by 0x4122B93: g_object_new (gobject.c:1178)
==6793== by 0x41DB582: g_unix_fd_list_new_from_array (gunixfdlist.c:191)
==6793== by 0x421BFD6: _g_dbus_worker_do_read_cb (gdbusprivate.c:590)
Bug #628329.
Allow modifying a GDBusMessage in a filter function and also add tests
for this. This breaks API but leaves ABI (almost) intact - at least
dconf's GSettings backend (the only big user I know of) will keep
working.
https://bugzilla.gnome.org/show_bug.cgi?id=624546
Signed-off-by: David Zeuthen <davidz@redhat.com>
Free the read buffer.
==26538== 4,096 bytes in 1 blocks are definitely lost in loss record 781 of 781
==26538== at 0x4005BDC: malloc (vg_replace_malloc.c:195)
==26538== by 0x4005C66: realloc (vg_replace_malloc.c:476)
==26538== by 0x405244D: g_realloc (gmem.c:181)
==26538== by 0x420E066: _g_dbus_worker_do_read_unlocked (gdbusprivate.c:780)
==26538== by 0x420E1D1: _g_dbus_worker_do_read (gdbusprivate.c:812)
==26538== by 0x420F14A: _g_dbus_worker_thread_begin_func (gdbusprivate.c:1318)
==26538== by 0x420D2ED: invoke_caller (gdbusprivate.c:266)
==26538== by 0x404DA7C: g_idle_dispatch (gmain.c:4224)
==26538== by 0x4049FCD: g_main_dispatch (gmain.c:2119)
==26538== by 0x404B2C1: g_main_context_dispatch (gmain.c:2672)
==26538== by 0x404B716: g_main_context_iterate (gmain.c:2750)
==26538== by 0x404BE7F: g_main_loop_run (gmain.c:2958)
==26538== by 0x420D2B5: shared_thread_func (gdbusprivate.c:248)
==26538== by 0x4077958: g_thread_create_proxy (gthread.c:1897)
==26538== by 0x57D918: start_thread (pthread_create.c:301)
==26538== by 0x4C6CBD: clone (clone.S:133)
Bug #627187.
If sending a lot of data and/or the other peer is not reading it, then
socket buffers can overflow. This is communicated from the kernel by
returning EAGAIN. In GIO, it is modelled by g_output_stream_write()
and g_socket_send_message() returning G_IO_ERROR_WOULD_BLOCK.
It is also problematic that that we're using synchronous IO in the
shared GDBus IO thread. It means that one GDBusConnection can lock up
others.
It turns out that by porting from g_output_stream_write() to
g_output_stream_write_async() we fix the EAGAIN issue. For GSocket, we
still need to handle things manually (by creating a GSource) as
g_socket_send_message() is used.
We check the new behavior in Michael's producer/consumer test case (at
/gdbus/overflow in gdbus-peer.c) added in the last commit.
Also add a test case that sends and receives a 20 MiB message.
Also add a new `transport' G_DBUS_DEBUG option so it is easy to
inspect partial writes:
$ G_DBUS_DEBUG=transport ./gdbus-connection -p /gdbus/connection/large_message
[...]
========================================================================
GDBus-debug:Transport:
>>>> WROTE 128000 bytes of message with serial 4 and
size 20971669 from offset 0 on a GSocketOutputStream
========================================================================
GDBus-debug:Transport:
>>>> WROTE 128000 bytes of message with serial 4 and
size 20971669 from offset 128000 on a GSocketOutputStream
========================================================================
GDBus-debug:Transport:
>>>> WROTE 128000 bytes of message with serial 4 and
size 20971669 from offset 256000 on a GSocketOutputStream
[...]
========================================================================
GDBus-debug:Transport:
>>>> WROTE 43669 bytes of message with serial 4 and
size 20971669 from offset 20928000 on a GSocketOutputStream
[...]
========================================================================
GDBus-debug:Transport:
<<<< READ 16 bytes of message with serial 3 and
size 20971620 to offset 0 from a GSocketInputStream
========================================================================
GDBus-debug:Transport:
<<<< READ 15984 bytes of message with serial 3 and
size 20971620 to offset 16 from a GSocketInputStream
========================================================================
GDBus-debug:Transport:
<<<< READ 16000 bytes of message with serial 3 and
size 20971620 to offset 16000 from a GSocketInputStream
[...]
========================================================================
GDBus-debug:Transport:
<<<< READ 144000 bytes of message with serial 3 and
size 20971620 to offset 20720000 from a GSocketInputStream
========================================================================
GDBus-debug:Transport:
<<<< READ 107620 bytes of message with serial 3 and
size 20971620 to offset 20864000 from a GSocketInputStream
OK
https://bugzilla.gnome.org/show_bug.cgi?id=626748
Signed-off-by: David Zeuthen <davidz@redhat.com>
This prints all GDBusMethodInvocation API usage and is normally used
with the `incoming' option. Example:
# G_DBUS_DEBUG=incoming,return ./polkitd --replace
Entering main event loop
Connected to the system bus
Registering null backend at priority -10
[...]
Acquired the name org.freedesktop.PolicyKit1
[...]
========================================================================
GDBus-debug:Incoming:
<<<< METHOD INVOCATION org.freedesktop.PolicyKit1.Authority.RegisterAuthenticationAgent()
on object /org/freedesktop/PolicyKit1/Authority
invoked by name :1.26
serial 299
========================================================================
GDBus-debug:Return:
>>>> METHOD ERROR org.freedesktop.PolicyKit1.Error.Failed
message `Cannot determine session the caller is in'
in response to org.freedesktop.PolicyKit1.Authority.RegisterAuthenticationAgent()
on object /org/freedesktop/PolicyKit1/Authority
to name :1.26
reply-serial 299
[...]
========================================================================
GDBus-debug:Incoming:
<<<< METHOD INVOCATION org.freedesktop.PolicyKit1.Authority.RegisterAuthenticationAgent()
on object /org/freedesktop/PolicyKit1/Authority
invoked by name :1.2402
serial 25
========================================================================
GDBus-debug:Return:
>>>> METHOD RETURN
in response to org.freedesktop.PolicyKit1.Authority.RegisterAuthenticationAgent()
on object /org/freedesktop/PolicyKit1/Authority
to name :1.2402
reply-serial 25
Signed-off-by: David Zeuthen <davidz@redhat.com>
Also add a 'address' G_DBUS_DEBUG option that will print out useful
debug information such as
GDBus-debug:Address: In g_dbus_address_get_for_bus_sync() for bus type `session'
GDBus-debug:Address: env var DBUS_SESSION_BUS_ADDRESS is not set
GDBus-debug:Address: env var DBUS_SYSTEM_BUS_ADDRESS is not set
GDBus-debug:Address: env var DBUS_STARTER_BUS_TYPE is not set
GDBus-debug:Address: Running `dbus-launch --autolaunch=05e508961149264c9b750a4c494aa6f7 --binary-syntax --close-stderr' to get bus address (possibly autolaunching)
GDBus-debug:Address: dbus-launch output:
0000: 75 6e 69 78 3a 61 62 73 74 72 61 63 74 3d 2f 74 unix:abstract=/t
0010: 6d 70 2f 64 62 75 73 2d 77 42 41 6f 4b 59 49 52 mp/dbus-wBAoKYIR
0020: 7a 75 2c 67 75 69 64 3d 30 34 30 64 31 33 66 33 zu,guid=040d13f3
0030: 30 61 30 62 35 32 63 32 30 66 36 32 63 34 31 63 0a0b52c20f62c41c
0040: 30 30 30 30 35 30 38 64 00 d2 38 00 00 01 00 40 0000508d..8....@
0050: 05 00 00 00 00 .....
GDBus-debug:Address: dbus-launch stderr output:
14542: Autolaunch enabled (using X11).
14542: --exit-with-session automatically enabled
14542: Connected to X11 display ':0.0'
14542: === Parent dbus-launch continues
14542: Waiting for babysitter's intermediate parent
14542: Reading address from bus
14542: Reading PID from daemon
14542: Saving x11 address
14542: Created window 88080385
14542: session file: /root/.dbus/session-bus/05e508961149264c9b750a4c494aa6f7-0
14542: dbus-launch exiting
GDBus-debug:Address: Returning address `unix:abstract=/tmp/dbus-wBAoKYIRzu,guid=040d13f30a0b52c20f62c41c0000508d' for bus type `session'
and
GDBus-debug:Address: In g_dbus_address_get_for_bus_sync() for bus type `session'
GDBus-debug:Address: env var DBUS_SESSION_BUS_ADDRESS is not set
GDBus-debug:Address: env var DBUS_SYSTEM_BUS_ADDRESS is not set
GDBus-debug:Address: env var DBUS_STARTER_BUS_TYPE is not set
GDBus-debug:Address: Running `dbus-launch --autolaunch=05e508961149264c9b750a4c494aa6f7 --binary-syntax --close-stderr' to get bus address (possibly autolaunching)
GDBus-debug:Address: dbus-launch output:
0000: 75 6e 69 78 3a 61 62 73 74 72 61 63 74 3d 2f 74 unix:abstract=/t
0010: 6d 70 2f 64 62 75 73 2d 77 42 41 6f 4b 59 49 52 mp/dbus-wBAoKYIR
0020: 7a 75 2c 67 75 69 64 3d 30 34 30 64 31 33 66 33 zu,guid=040d13f3
0030: 30 61 30 62 35 32 63 32 30 66 36 32 63 34 31 63 0a0b52c20f62c41c
0040: 30 30 30 30 35 30 38 64 00 d2 38 00 00 01 00 40 0000508d..8....@
0050: 05 00 00 00 00 .....
GDBus-debug:Address: dbus-launch stderr output:
14549: Autolaunch enabled (using X11).
14549: --exit-with-session automatically enabled
14549: Connected to X11 display ':0.0'
14549: dbus-daemon is already running. Returning existing parameters.
14549: dbus-launch exiting
GDBus-debug:Address: Returning address `unix:abstract=/tmp/dbus-wBAoKYIRzu,guid=040d13f30a0b52c20f62c41c0000508d' for bus type `session'
Note that things work exactly like libdbus, e.g. from the
dbus-launch(1) man page:
Whenever an autolaunch occurs, the application that had to start a
new bus will be in its own little world; it can effectively end up
starting a whole new session if it tries to use a lot of bus
services. This can be suboptimal or even totally broken, depending
on the app and what it tries to do.
[...]
You can always avoid autolaunch by manually setting
DBUS_SESSION_BUS_ADDRESS. Autolaunch happens because the default
address if none is set is "autolaunch:", so if any other address is
set there will be no autolaunch. You can however include autolaunch
in an explicit session bus address as a fallback, for example
DBUS_SESSION_BUS_ADDRESS="something:,autolaunch:" - in that case if
the first address doesn't work, processes will autolaunch. (The bus
address variable contains a comma-separated list of addresses to
try.)
Signed-off-by: David Zeuthen <davidz@redhat.com>
This commit adds the following G_DBUS_DEBUG flags
- emission
- incoming
- call
- signal
- payload
https://bugzilla.gnome.org/show_bug.cgi?id=620913
Signed-off-by: David Zeuthen <davidz@redhat.com>
These fixes makes udisks-daemon from udisks' gdbus-port branch, see
http://cgit.freedesktop.org/udisks/log/?h=gdbus-port
handle 200 add/remove uevents generated by e.g.
#!/bin/bash
DEV=mmcblk0p1
for n in `seq 200` ; do
udevadm trigger --sysname-match=$DEV --action=remove
udevadm trigger --sysname-match=$DEV --action=add
echo foo $n
done
without any substantial leaks.
Signed-off-by: David Zeuthen <davidz@redhat.com>
Without this guarantee, peer-to-peer connections are not very
useful. However, with this guarantee it's possible to export objects
in a handler for the GDBusServer::new-connection signal.
There are two caveats with this patch
- it won't work on message bus connections
- we don't queue up messages to be written
that can be addresses later if needed.
https://bugzilla.gnome.org/show_bug.cgi?id=623142
Signed-off-by: David Zeuthen <davidz@redhat.com>
This patch breaks some rarely-used public API (only known user is
dconf).
This patch is based on work from Peng Huang <shawn.p.huang@gmail.com>.
See https://bugzilla.gnome.org/show_bug.cgi?id=621945
Signed-off-by: David Zeuthen <davidz@redhat.com>
