We can't safely close the output part of the I/O stream until any
pending write or flush has been completed. In the worst case, this could
lead to an assertion failure in the worker (when the close wins the
race) or not closing the stream at all (when the write wins the race).
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=651268
Bug-NB: NB#271520
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Signed-off-by: David Zeuthen <davidz@redhat.com>
num_writes_pending was a counter, but it only took values 0 or 1, so make
it a boolean: it would never make sense to be trying to write out two
messages at the same time (they'd get interleaved).
Similarly, we can never be writing and flushing at the same time (that'd
mean we were flushing halfway through a message, which would be pointless)
so combine it with flush_pending too, calling the result output_pending.
Also assert that it takes the expected value whenever we change it,
and document the locking discipline used for it, including a subtle
case in write_message_in_idle_cb where it's not obvious at first glance
why we don't need the lock.
(Having the combined boolean at the top of the block of write-related
struct members improves struct packing on 64-bit platforms, by packing
read_num_ancillary_messages and output_pending into one word.)
Bug: https://bugzilla.gnome.org/show_bug.cgi?id=651268
Bug-NB: NB#271520
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Signed-off-by: David Zeuthen <davidz@redhat.com>
... otherwise we might end up using the worker after it has been
freed. Reported by Dan Winship and Colin Walters.
This fix uncovered a bug in the /gdbus/nonce-tcp test case so "fix"
that as well to use a better way of having one thread wait for another
(using quotes for the word "fix" since it's pretty hackish to
busy-wait in one thread to wait for another).
Signed-off-by: David Zeuthen <davidz@redhat.com>
I can't see a reason to spin until the worker thread runs, so don't.
This avoids ugly sched_yield() calls that show up in strace and
annoy me; the code is cleaner now too.
We now grab the types needed for the WebKit workaround in the
thread creation area, but only release them when the thread itself
exits.
https://bugzilla.gnome.org/show_bug.cgi?id=651650
Turns out that GDBusWorker will issue callbacks (in its own thread)
even after g_dbus_worker_stop() has been called. This would rarely
happen (and unreffing a connection is even rarer) so only saw this bug
occasionally when running the gdbus-connection test case in a loop.
Fix up this issue by maintaining a set of GDBusConnection objects that
are currently "alive" and do nothing in the callbacks if the passed
user_data pointer is not in this set.
Also attempted to fix up a race condition with
_g_object_wait_for_single_ref_do() and its use of GObject toggle
references - for now, just resort to busy waiting, thereby
sidestepping the toggle reference mess altogether.
Signed-off-by: David Zeuthen <davidz@redhat.com>
The root problem is with GObject - for now, just work around it in
GDBus. Also include a test-case. See
https://bugzilla.gnome.org/show_bug.cgi?id=627724
for more information.
Signed-off-by: David Zeuthen <davidz@redhat.com>
Ryan pointed out that it's safe to do this because we have the
G_DBUS_SEND_MESSAGE_FLAGS_PRESERVE_SERIAL flag and that it simplifies
how filter functions work.
Signed-off-by: David Zeuthen <davidz@redhat.com>
Rework filter functions as per
https://bugzilla.gnome.org/show_bug.cgi?id=624546#c8
This commit breaks ABI. However, this ABI break affects only
applications using filter functions. The only known user of is dconf.
Signed-off-by: David Zeuthen <davidz@redhat.com>
... and use g_error_matches().
==29535== 1,360 (408 direct, 952 indirect) bytes in 17 blocks are definitely lost in loss record 1,252 of 1,264
==29535== at 0x4005BDC: malloc (vg_replace_malloc.c:195)
==29535== by 0x4057094: g_malloc (gmem.c:134)
==29535== by 0x406F2D6: g_slice_alloc (gslice.c:836)
==29535== by 0x406F31B: g_slice_alloc0 (gslice.c:848)
==29535== by 0x403A751: g_error_new_valist (gerror.c:54)
==29535== by 0x403AAD4: g_set_error (gerror.c:240)
==29535== by 0x41C06C8: g_socket_send_message (gsocket.c:2967)
==29535== by 0x421CB64: write_message_continue_writing (gdbusprivate.c:958)
==29535== by 0x421CE2A: write_message_async (gdbusprivate.c:1049)
==29535== by 0x421D4DD: maybe_write_next_message (gdbusprivate.c:1291)
==29535== by 0x421D26B: message_written (gdbusprivate.c:1187)
==29535== by 0x421D322: write_message_cb (gdbusprivate.c:1216)
Bug #628345.
==6793== 32 (24 direct, 8 indirect) bytes in 1 blocks are definitely lost in loss record 780 of 1,423
==6793== at 0x4005BDC: malloc (vg_replace_malloc.c:195)
==6793== by 0x4057094: g_malloc (gmem.c:134)
==6793== by 0x406F2D6: g_slice_alloc (gslice.c:836)
==6793== by 0x406F31B: g_slice_alloc0 (gslice.c:848)
==6793== by 0x413D5BB: g_type_create_instance (gtype.c:1867)
==6793== by 0x412372A: g_object_constructor (gobject.c:1482)
==6793== by 0x4122E1D: g_object_newv (gobject.c:1266)
==6793== by 0x4122B93: g_object_new (gobject.c:1178)
==6793== by 0x41DB582: g_unix_fd_list_new_from_array (gunixfdlist.c:191)
==6793== by 0x421BFD6: _g_dbus_worker_do_read_cb (gdbusprivate.c:590)
Bug #628329.
Allow modifying a GDBusMessage in a filter function and also add tests
for this. This breaks API but leaves ABI (almost) intact - at least
dconf's GSettings backend (the only big user I know of) will keep
working.
https://bugzilla.gnome.org/show_bug.cgi?id=624546
Signed-off-by: David Zeuthen <davidz@redhat.com>
Free the read buffer.
==26538== 4,096 bytes in 1 blocks are definitely lost in loss record 781 of 781
==26538== at 0x4005BDC: malloc (vg_replace_malloc.c:195)
==26538== by 0x4005C66: realloc (vg_replace_malloc.c:476)
==26538== by 0x405244D: g_realloc (gmem.c:181)
==26538== by 0x420E066: _g_dbus_worker_do_read_unlocked (gdbusprivate.c:780)
==26538== by 0x420E1D1: _g_dbus_worker_do_read (gdbusprivate.c:812)
==26538== by 0x420F14A: _g_dbus_worker_thread_begin_func (gdbusprivate.c:1318)
==26538== by 0x420D2ED: invoke_caller (gdbusprivate.c:266)
==26538== by 0x404DA7C: g_idle_dispatch (gmain.c:4224)
==26538== by 0x4049FCD: g_main_dispatch (gmain.c:2119)
==26538== by 0x404B2C1: g_main_context_dispatch (gmain.c:2672)
==26538== by 0x404B716: g_main_context_iterate (gmain.c:2750)
==26538== by 0x404BE7F: g_main_loop_run (gmain.c:2958)
==26538== by 0x420D2B5: shared_thread_func (gdbusprivate.c:248)
==26538== by 0x4077958: g_thread_create_proxy (gthread.c:1897)
==26538== by 0x57D918: start_thread (pthread_create.c:301)
==26538== by 0x4C6CBD: clone (clone.S:133)
Bug #627187.
If sending a lot of data and/or the other peer is not reading it, then
socket buffers can overflow. This is communicated from the kernel by
returning EAGAIN. In GIO, it is modelled by g_output_stream_write()
and g_socket_send_message() returning G_IO_ERROR_WOULD_BLOCK.
It is also problematic that that we're using synchronous IO in the
shared GDBus IO thread. It means that one GDBusConnection can lock up
others.
It turns out that by porting from g_output_stream_write() to
g_output_stream_write_async() we fix the EAGAIN issue. For GSocket, we
still need to handle things manually (by creating a GSource) as
g_socket_send_message() is used.
We check the new behavior in Michael's producer/consumer test case (at
/gdbus/overflow in gdbus-peer.c) added in the last commit.
Also add a test case that sends and receives a 20 MiB message.
Also add a new `transport' G_DBUS_DEBUG option so it is easy to
inspect partial writes:
$ G_DBUS_DEBUG=transport ./gdbus-connection -p /gdbus/connection/large_message
[...]
========================================================================
GDBus-debug:Transport:
>>>> WROTE 128000 bytes of message with serial 4 and
size 20971669 from offset 0 on a GSocketOutputStream
========================================================================
GDBus-debug:Transport:
>>>> WROTE 128000 bytes of message with serial 4 and
size 20971669 from offset 128000 on a GSocketOutputStream
========================================================================
GDBus-debug:Transport:
>>>> WROTE 128000 bytes of message with serial 4 and
size 20971669 from offset 256000 on a GSocketOutputStream
[...]
========================================================================
GDBus-debug:Transport:
>>>> WROTE 43669 bytes of message with serial 4 and
size 20971669 from offset 20928000 on a GSocketOutputStream
[...]
========================================================================
GDBus-debug:Transport:
<<<< READ 16 bytes of message with serial 3 and
size 20971620 to offset 0 from a GSocketInputStream
========================================================================
GDBus-debug:Transport:
<<<< READ 15984 bytes of message with serial 3 and
size 20971620 to offset 16 from a GSocketInputStream
========================================================================
GDBus-debug:Transport:
<<<< READ 16000 bytes of message with serial 3 and
size 20971620 to offset 16000 from a GSocketInputStream
[...]
========================================================================
GDBus-debug:Transport:
<<<< READ 144000 bytes of message with serial 3 and
size 20971620 to offset 20720000 from a GSocketInputStream
========================================================================
GDBus-debug:Transport:
<<<< READ 107620 bytes of message with serial 3 and
size 20971620 to offset 20864000 from a GSocketInputStream
OK
https://bugzilla.gnome.org/show_bug.cgi?id=626748
Signed-off-by: David Zeuthen <davidz@redhat.com>
This prints all GDBusMethodInvocation API usage and is normally used
with the `incoming' option. Example:
# G_DBUS_DEBUG=incoming,return ./polkitd --replace
Entering main event loop
Connected to the system bus
Registering null backend at priority -10
[...]
Acquired the name org.freedesktop.PolicyKit1
[...]
========================================================================
GDBus-debug:Incoming:
<<<< METHOD INVOCATION org.freedesktop.PolicyKit1.Authority.RegisterAuthenticationAgent()
on object /org/freedesktop/PolicyKit1/Authority
invoked by name :1.26
serial 299
========================================================================
GDBus-debug:Return:
>>>> METHOD ERROR org.freedesktop.PolicyKit1.Error.Failed
message `Cannot determine session the caller is in'
in response to org.freedesktop.PolicyKit1.Authority.RegisterAuthenticationAgent()
on object /org/freedesktop/PolicyKit1/Authority
to name :1.26
reply-serial 299
[...]
========================================================================
GDBus-debug:Incoming:
<<<< METHOD INVOCATION org.freedesktop.PolicyKit1.Authority.RegisterAuthenticationAgent()
on object /org/freedesktop/PolicyKit1/Authority
invoked by name :1.2402
serial 25
========================================================================
GDBus-debug:Return:
>>>> METHOD RETURN
in response to org.freedesktop.PolicyKit1.Authority.RegisterAuthenticationAgent()
on object /org/freedesktop/PolicyKit1/Authority
to name :1.2402
reply-serial 25
Signed-off-by: David Zeuthen <davidz@redhat.com>
Also add a 'address' G_DBUS_DEBUG option that will print out useful
debug information such as
GDBus-debug:Address: In g_dbus_address_get_for_bus_sync() for bus type `session'
GDBus-debug:Address: env var DBUS_SESSION_BUS_ADDRESS is not set
GDBus-debug:Address: env var DBUS_SYSTEM_BUS_ADDRESS is not set
GDBus-debug:Address: env var DBUS_STARTER_BUS_TYPE is not set
GDBus-debug:Address: Running `dbus-launch --autolaunch=05e508961149264c9b750a4c494aa6f7 --binary-syntax --close-stderr' to get bus address (possibly autolaunching)
GDBus-debug:Address: dbus-launch output:
0000: 75 6e 69 78 3a 61 62 73 74 72 61 63 74 3d 2f 74 unix:abstract=/t
0010: 6d 70 2f 64 62 75 73 2d 77 42 41 6f 4b 59 49 52 mp/dbus-wBAoKYIR
0020: 7a 75 2c 67 75 69 64 3d 30 34 30 64 31 33 66 33 zu,guid=040d13f3
0030: 30 61 30 62 35 32 63 32 30 66 36 32 63 34 31 63 0a0b52c20f62c41c
0040: 30 30 30 30 35 30 38 64 00 d2 38 00 00 01 00 40 0000508d..8....@
0050: 05 00 00 00 00 .....
GDBus-debug:Address: dbus-launch stderr output:
14542: Autolaunch enabled (using X11).
14542: --exit-with-session automatically enabled
14542: Connected to X11 display ':0.0'
14542: === Parent dbus-launch continues
14542: Waiting for babysitter's intermediate parent
14542: Reading address from bus
14542: Reading PID from daemon
14542: Saving x11 address
14542: Created window 88080385
14542: session file: /root/.dbus/session-bus/05e508961149264c9b750a4c494aa6f7-0
14542: dbus-launch exiting
GDBus-debug:Address: Returning address `unix:abstract=/tmp/dbus-wBAoKYIRzu,guid=040d13f30a0b52c20f62c41c0000508d' for bus type `session'
and
GDBus-debug:Address: In g_dbus_address_get_for_bus_sync() for bus type `session'
GDBus-debug:Address: env var DBUS_SESSION_BUS_ADDRESS is not set
GDBus-debug:Address: env var DBUS_SYSTEM_BUS_ADDRESS is not set
GDBus-debug:Address: env var DBUS_STARTER_BUS_TYPE is not set
GDBus-debug:Address: Running `dbus-launch --autolaunch=05e508961149264c9b750a4c494aa6f7 --binary-syntax --close-stderr' to get bus address (possibly autolaunching)
GDBus-debug:Address: dbus-launch output:
0000: 75 6e 69 78 3a 61 62 73 74 72 61 63 74 3d 2f 74 unix:abstract=/t
0010: 6d 70 2f 64 62 75 73 2d 77 42 41 6f 4b 59 49 52 mp/dbus-wBAoKYIR
0020: 7a 75 2c 67 75 69 64 3d 30 34 30 64 31 33 66 33 zu,guid=040d13f3
0030: 30 61 30 62 35 32 63 32 30 66 36 32 63 34 31 63 0a0b52c20f62c41c
0040: 30 30 30 30 35 30 38 64 00 d2 38 00 00 01 00 40 0000508d..8....@
0050: 05 00 00 00 00 .....
GDBus-debug:Address: dbus-launch stderr output:
14549: Autolaunch enabled (using X11).
14549: --exit-with-session automatically enabled
14549: Connected to X11 display ':0.0'
14549: dbus-daemon is already running. Returning existing parameters.
14549: dbus-launch exiting
GDBus-debug:Address: Returning address `unix:abstract=/tmp/dbus-wBAoKYIRzu,guid=040d13f30a0b52c20f62c41c0000508d' for bus type `session'
Note that things work exactly like libdbus, e.g. from the
dbus-launch(1) man page:
Whenever an autolaunch occurs, the application that had to start a
new bus will be in its own little world; it can effectively end up
starting a whole new session if it tries to use a lot of bus
services. This can be suboptimal or even totally broken, depending
on the app and what it tries to do.
[...]
You can always avoid autolaunch by manually setting
DBUS_SESSION_BUS_ADDRESS. Autolaunch happens because the default
address if none is set is "autolaunch:", so if any other address is
set there will be no autolaunch. You can however include autolaunch
in an explicit session bus address as a fallback, for example
DBUS_SESSION_BUS_ADDRESS="something:,autolaunch:" - in that case if
the first address doesn't work, processes will autolaunch. (The bus
address variable contains a comma-separated list of addresses to
try.)
Signed-off-by: David Zeuthen <davidz@redhat.com>
This commit adds the following G_DBUS_DEBUG flags
- emission
- incoming
- call
- signal
- payload
https://bugzilla.gnome.org/show_bug.cgi?id=620913
Signed-off-by: David Zeuthen <davidz@redhat.com>
These fixes makes udisks-daemon from udisks' gdbus-port branch, see
http://cgit.freedesktop.org/udisks/log/?h=gdbus-port
handle 200 add/remove uevents generated by e.g.
#!/bin/bash
DEV=mmcblk0p1
for n in `seq 200` ; do
udevadm trigger --sysname-match=$DEV --action=remove
udevadm trigger --sysname-match=$DEV --action=add
echo foo $n
done
without any substantial leaks.
Signed-off-by: David Zeuthen <davidz@redhat.com>
Without this guarantee, peer-to-peer connections are not very
useful. However, with this guarantee it's possible to export objects
in a handler for the GDBusServer::new-connection signal.
There are two caveats with this patch
- it won't work on message bus connections
- we don't queue up messages to be written
that can be addresses later if needed.
https://bugzilla.gnome.org/show_bug.cgi?id=623142
Signed-off-by: David Zeuthen <davidz@redhat.com>
This patch breaks some rarely-used public API (only known user is
dconf).
This patch is based on work from Peng Huang <shawn.p.huang@gmail.com>.
See https://bugzilla.gnome.org/show_bug.cgi?id=621945
Signed-off-by: David Zeuthen <davidz@redhat.com>
See https://bugzilla.gnome.org/show_bug.cgi?id=621838 for the whole
story. The problem was that we ended up reading data from arrays of
arrays when we were just supposed to be aligning the buffers.
Also add a host of debug infrastructure that was needed to find the
root cause. For now it can be turned on only via defining
DEBUG_SERIALIZER. In the future we might want to make it work via
G_DBUS_DEBUG. In a nutshell, the added debug info looks like this
Parsing blob (blob_len = 0x0084 bytes)
0000: 6c 01 00 01 3c 00 00 00 41 00 00 00 37 00 00 00 l...<...A...7...
0010: 08 01 67 00 08 61 61 79 61 7b 73 76 7d 00 00 00 ..g..aaya{sv}...
0020: 01 01 6f 00 08 00 00 00 2f 66 6f 6f 2f 62 61 72 ..o...../foo/bar
0030: 00 00 00 00 00 00 00 00 03 01 73 00 06 00 00 00 ..........s.....
0040: 4d 65 6d 62 65 72 00 00 00 00 00 00 34 00 00 00 Member......4...
0050: 03 00 00 00 63 77 64 00 01 73 00 00 23 00 00 00 ....cwd..s..#...
0060: 2f 68 6f 6d 65 2f 64 61 76 69 64 7a 2f 48 61 63 /home/davidz/Hac
0070: 6b 69 6e 67 2f 67 6c 69 62 2f 67 69 6f 2f 74 65 king/glib/gio/te
0080: 73 74 73 00 sts.
Parsing headers (blob_len = 0x0084 bytes)
Reading type a{yv} from offset 0x000c: array spans 0x0037 bytes
Reading type {yv} from offset 0x0010
Reading type y from offset 0x0010: 0x08 '
Reading type v from offset 0x0011
Reading type g from offset 0x0014: 'aaya{sv}'
Reading type {yv} from offset 0x001e
Reading type y from offset 0x0020: 0x01 ''
Reading type v from offset 0x0021
Reading type o from offset 0x0024: '/foo/bar'
Reading type {yv} from offset 0x0031
Reading type y from offset 0x0038: 0x03 ''
Reading type v from offset 0x0039
Reading type s from offset 0x003c: 'Member'
Parsing body (blob_len = 0x0084 bytes)
Reading type (aaya{sv}) from offset 0x0047
Reading type aay from offset 0x0048: array spans 0x0000 bytes
Reading type a{sv} from offset 0x004c: array spans 0x0034 bytes
Reading type {sv} from offset 0x0050
Reading type s from offset 0x0050: 'cwd'
Reading type v from offset 0x0058
Reading type s from offset 0x005b: '/home/davidz/Hacking/glib/gio/tests'
OK
Signed-off-by: David Zeuthen <davidz@redhat.com>
While we are already propagating the error to the user via the
GDBusConnection::disconnected signal (because the only safe thing is
to disconnect the other peer), changes are the user is simply not
listening to this signal.
This should never ever happen unless there's a bug in the
serializaer/deserializer so it's fine to complain via g_warning()
here.
Bug 621838, see
https://bugzilla.gnome.org/show_bug.cgi?id=621838
is related to this.
Signed-off-by: David Zeuthen <davidz@redhat.com>
This allows the caller to specify the reply type that they are expecting
for this call. If the reply comes back with the wrong type, GDBus will
generate an appropriate error internally.
- add a GVariantType * argument to g_dbus_connection_call() and
_call_sync().
- move the internal API for computing message types from introspection
data to be based on GVariantType instead of strings. Update users
of this code.
- have GDBusProxy pass this calculated GVariantType into
g_dbus_connection_call(). Remove the checks done in GDBusProxy.
- Update other users of the code (test cases, gdbus-tool, GSettings
tool, etc). In some cases, remove redundant checks; in some other
cases, we are fixing bugs because no checking was done where it
should have been.
Closes bug #619391.
- Fix various #include issues
- Change #error to #warning for the EXTERNAL authentication mechanism.
It is not clear if this should work on Win32 at all.
- Call close() before unlink() for the SHA1 keyring
- Change #error to #warning so we don't forget to do
permission checking of the .dbus-keyrings directory
- Use Win32 SID for the SHA1 auth mech
- Apparently we can't use word 'interface' as an identifier
- Implement a _g_dbus_win32_get_user_sid() function. For now it's
private. Don't know if it should be public somewhere. Maybe in
a future GCredentials support for Win32? I don't know.
- GFileDescriptorBased is not available on Win32. So avoid using
it in GLocalFile stuff. Now, Win32 still uses GLocalFile + friends
(which works with file descriptors) so expose a private function
to get the fd for an OutputStream so things still work.
- Fixup gio.symbols
- Fixup tests/gdbus-peer.c so it builds
With this, at least things compile and the gdbus-peer.exe test case
passes. Which is a great start. I've tested this by cross-compiling on
a x86_64 Fedora 13 host using mingw32 and running the code on a 32-bit
Windows 7 box.
https://bugzilla.gnome.org/show_bug.cgi?id=619142
Signed-off-by: David Zeuthen <davidz@redhat.com>
This is needed to e.g. allow encoding maybe types (once we add
G_DBUS_CAPABILITY_FLAGS_MAYBE_TYPES) if, and only if, that capability
has been negotiated with the peer (via authentication).
Things compile and the test-suite passes. Still need to hook up
gio.symbols and docs. There are still a bunch of TODOs left in the
sources that needs to be addressed.
Signed-off-by: David Zeuthen <davidz@redhat.com>
