Document g_test_run() order better, and how it changed

But also note that the ordering is strictly an aesthetic/convenience
thing, and that tests should not be written to depend on it.

https://bugzilla.gnome.org/show_bug.cgi?id=721624
This commit is contained in:
Dan Winship 2014-01-07 11:18:54 -05:00
parent 0e109fceab
commit 86497649bb
2 changed files with 43 additions and 1 deletions

View File

@ -67,6 +67,24 @@ and attach the patch to that bug report.
Patches should be in unified diff form. (The -up option to GNU diff.)
Notes about GLib 2.40
=====================
* g_test_run() no longer runs tests in exactly the order they are
registered; instead, it groups them according to test suites (ie,
path components) like the documentation always claimed it did. In
some cases, this can result in a sub-optimal ordering of tests,
relative to the old behavior. The fix is to change the test paths to
properly group together the tests that should run together. (eg, if
you want to run test_foo_simple(), test_bar_simple(), and
test_foo_using_bar() in that order, they should have test paths like
"/simple/foo", "/simple/bar", "/complex/foo-using-bar", not
"/foo/simple", "/bar/simple", "/foo/using-bar" (which would result
in test_foo_using_bar() running before test_bar_simple()).
(The behavior actually changed in GLib 2.36, but it was not
documented at the time, since we didn't realize it mattered.)
Notes about GLib 2.36
=====================

View File

@ -1519,6 +1519,28 @@ g_test_get_root (void)
* g_test_run_suite() or g_test_run() may only be called once
* in a program.
*
* In general, the tests and sub-suites within each suite are run in
* the order in which they are defined. However, note that prior to
* GLib 2.36, there was a bug in the <literal>g_test_add_*</literal>
* functions which caused them to create multiple suites with the same
* name, meaning that if you created tests "/foo/simple",
* "/bar/simple", and "/foo/using-bar" in that order, they would get
* run in that order (since g_test_run() would run the first "/foo"
* suite, then the "/bar" suite, then the second "/foo" suite). As of
* 2.36, this bug is fixed, and adding the tests in that order would
* result in a running order of "/foo/simple", "/foo/using-bar",
* "/bar/simple". If this new ordering is sub-optimal (because it puts
* more-complicated tests before simpler ones, making it harder to
* figure out exactly what has failed), you can fix it by changing the
* test paths to group tests by suite in a way that will result in the
* desired running order. Eg, "/simple/foo", "/simple/bar",
* "/complex/foo-using-bar".
*
* However, you should never make the actual result of a test depend
* on the order that tests are run in. If you need to ensure that some
* particular code runs before or after a given test case, use
* g_test_add(), which lets you specify setup and teardown functions.
*
* Returns: 0 on success, 1 on failure (assuming it returns at all),
* 77 if all tests were skipped with g_test_skip().
*
@ -2185,7 +2207,9 @@ g_test_run_suite_internal (GTestSuite *suite,
* Execute the tests within @suite and all nested #GTestSuites.
* The test suites to be executed are filtered according to
* test path arguments (-p <replaceable>testpath</replaceable>)
* as parsed by g_test_init().
* as parsed by g_test_init(). See the g_test_run() documentation
* for more information on the order that tests are run in.
*
* g_test_run_suite() or g_test_run() may only be called once
* in a program.
*