My application (OSTree) has suffered really bad performance
degredation recently, and doing some analysis of the problem, I think
a lot stems from the recent order of magnitude increase in the default
GLib worker threads.
OSTree does a *lot* of async I/O (directory enumeration, calling
link(), reading/writing files) etc. It very quickly reaches 100
threads, but there is up/down thread churn as threads complete, but
process gets I/O bound and thus the threads are starved of work, but
then the main thread manages to fill the queue more, suddenly spinning
up lots of worker threads again.
Now, as this patch forces users to specify, there are several distinct
classes of tasks. It basically never makes sense to run more
CPU-bound threads than there are CPUs. Spawning 90+ threads to do
SHA256 calculation for example is just creating pointless extra
contention on a dual-processor system.
Therefore, the limit of the thread pool for G_TASK_THREAD_KIND_CPU is
the number of processors.
Similarly, there's a real limit to how much I/O traffic it makes sense
to schedule simultaneously. I need to do more research here - what
makes sense for my laptop with 1 magnetic hard drive is likely
different from a fast server with RAID on top of SSDs. For the
moment, I've chosen to limit I/O bound threads to 4.
The limit for _DEFAULT remains 100 - but we can solve this better if
we have e.g. a way for tasks to express dependencies, among other things.
But for now, with this patch, and OSTree modified to use
G_TASK_THREAD_KIND_IO for its rename()/link() threads, I am seeing
better performance.
https://bugzilla.gnome.org/show_bug.cgi?id=687223
Very many testcases, some GLib tools (resource compiler, etc) and
GApplication were calling g_type_init().
Remove those uses, as they are no longer required.
https://bugzilla.gnome.org/show_bug.cgi?id=686161
GTask is a replacement for GSimpleAsyncResult and GIOScheduler, that
also allows for making cancellable wrappers around non-cancellable
functions (as in GThreadedResolver).
https://bugzilla.gnome.org/show_bug.cgi?id=661767
