------------------------------------------------------------------- Tue Sep 17 11:09:08 UTC 2019 - coolo@suse.com - Update to version 4.6.1568387059.db88ff48d: * Add Storable dependency * Remove debugging * Create a deepcopy of the settings * Always sort keys to make behaviour reproducible * Demonstrate what's going wrong on staging yaml * Fix duplicate path name * Update YAML guide and schema to explain custom job template names * Publish API schema(s) along with other docs * Fix some issues in the doc markup reported by asciidoc * Always build docs and allow local generation * Fix warning caused by filename getting removed from reused form hashref in Mojo::UserAgent * Cleanup disabled code about 'Sereal' * Non-experimental YAML routes with explicit schema * Revert back to duplicating dependencies in dockerfile (#2315) * Extend the requires of openQA-devel and make them easier to review (#2313) * Implement optional defaults.settings key in YAML * Fix container image to use repository paths (#2310) * Improve test coverage for more background jobs * Increase coverage by allowing Devel::Cover to collect stats for background jobs * Fix printing usage when invoking openqa-clone-job --help * Remove codecov hack that was only added to increase coverage percentage, but does not actually test anything useful * Scenarios can have different settings * docker: Use devel package instead of spelling out duplicate dependencies (#2308) * Cut the test result directory shorter when its length is more than 255 * Support branch url on openqa-clone-custom-git-refspec * docs: Hint to 'openQA-devel' for pulling in dependencies * Delete unused screenshots in separate loop * docs: Render code correctly (#2306) * Allow CodeMirror to render off screen * Show context for REST-API errors to make debugging a little bit easier * Fix artefact upload errors to show the message instead of the hash address * Reduce coverage requirement a bit until we can make Devel::Cover include subprocesses in stats * Don't use 3rd set of names for passing dependencies to JS * Use capitalized names for module-scoped variables * Consider directly chained deps in several places * Support _START_DIRECTLY_AFTER_JOBS * Add dependency type for START_DIRECTLY_AFTER_TEST * Failed isotovideo status checks are fairly harmless, but a log message can be useful for debugging * Since we now expect there to be more subprocesses, there is no reason to log them as unexpected anymore * Do not try to reconnect the WebSocket connection in subprocesses * Make annoying log message debug level instead of info for now * Test API and upload errors * Test asset uploads too * Move artefact upload into a webui connection method so it can be easily mocked for testing * Add more worker upload tests * Fix fake WebSocket transaction to correctly defer callback execution * Improve test coverage of the worker job life cycle * Move all direct uses of Mojo::UserAgent from OpenQA::Worker::Job to client modules * Add new worker tests and an isotovideo client module that can be mocked easily * Perform isotovideo status checks non-blocking too * Try to defer all callbacks correctly to the next reactor tick * Handle exceptions in uploads * Make image uploads non-blocking too * Callbacks cannot be optional in an async application * Update mock worker tests to be able to deal with multiple processes handling uploads * Handle asset and logfile uploads in a subprocess * Split up upload handling into more manageable parts * Reuse t/test_postgresql in container test as well * Harmonize dependencies in cpanfile+openQA.spec * spec: Specify more dependencies needed for local development * spec: Fix test dependencies after os-autoinst removed dependencies * Move codemirror javascript file to javascript label (#2297) * spec: Fix 'devel' package (for real) with proper macro definition * spec: Fix error about python shebang * Fix 'devel' package * docs: Update openSUSE repos using $releasever, priorities, better names * spec: Add 'devel' package * worker job: Add log of second virtio console to upload list * Fix accidentally removed dependency * Add missing apparmor permission (#2285) * Remove shellcheck disable * Modernize bash tests * Fix test that did nothing * Fix argument handling of a function and turn background job into a real plugin * Remove print from test * Avoid using HAVING on ScreenshotLinks * No longer delete job modules one by one * Remove support for old migration * Use IPC::Run instead of IPC::Run::Simple * More code style fixes * Tidy developer mode test * Test assert_screen args with array of needle tags * Code style changes and unit test for current_api_function handling * Increase lock timeouts a bit because some jobs might just need a little more time and the risk or locks not getting cleared at the end is very low (#2280) * Show needle tags for assert_screen/check_screen * Validate tag input field in needleeditor * Render a diff of the job group YAML in preview/ save * Use purple for blocked jobs * Distinguish scheduled and blocked jobs on test result overview * Delete vim footer in files that still have it * Makefile: Allow again to select single test cases with PROVE_ARGS * Preserve QEMU PID file on pool dir cleanup if QEMU still running * Improve tests for triggering pool directory cleanup * Test support for non-ASCII chars in needle JSON only in description * Tweaks according to code review * Handle newline characters in form parameters correctly * Use single gru job * Always include the group property editor * Do not delete assets based on empty size * Remove useless quoting * Check finished job cout in async test * Increase timeout in async test * Remove unconditional sleeps from async test * Add Gru Test with Mock Api Server * Split Folders and Gru controllers * Add Rsync Queue UI * Use Gru for rsync * ObsRsync handle requests asynchronously * job cancel: Explicitly return a number to avoid warning * Fix warning about undefined $testresdir * Fix flaky test t/api/11-bugs.t * scheduler: Fix warnings about undefined variables * Fix rendering of the openqa.cmd * openqa-bootstrap: Fix logic of git clone * Add machine filter in 'Test result overview' * docs: Mention most simple installation instructions and "clone-job" first * docs: Add documentation for immediate bootstrap+clone * docs: Fix internal link * branding: Prevent undefined flavor in external_reporting.html.ep * apparmor: Allow virtio_console based on named PIPE * spec: Split out all dependencies properly per sub-package * apparmor: Allow loading of client.conf in usual home of geekotest * apparmor: Allow loading override templates in /etc/openqa/templates/ * branding: Prevent warnings about undefined $distri in external_reporting.html.ep * Test worker behavior if _upload_log_file fails * Improve needle editor test * Change iso scheduling behavior to not obsolete older jobs anymore * Improve calculating preferred machine * Fix error when saving needle JSON with non-ASCII characters * Modify workaround property to support adding reason * Fix 30-test_parser.t warnings * Remove exported functions from OpenQA::File and stop using private methods as part of the public API * docs: Fix explanation for masquerading which is needed on all involved zones * docs: Simplify firewalld commands by using the runtime variants * docs: Make the current "firewalld" approach more prominent than old SuSEfirewall2 * Fix PROVE_ARGS in spec file * docs: Remove obsolete section about "wicked < 0.6.23" * load_templates: fix --clean and reading perl templates * Do not handle arguments in reverse and move OpenQA::Files class into a proper package * Make worker retry-delay configurable * Test retry behavior of worker * Extend retry delay of worker to 60 seconds if web UI busy * Adapt codecov target to current coverage of 85% * Allow * placeholders in product names * docs: Fix marking for 2nd level list about "wicked pros" * Prevent over-escaping of bug titles in test result overview * Adjust web socket server test * Avoid 'return;' in web socket code * Fix sending web socket message when job not exists * Improve transaction handling in web socket server * AMQP: allow no message prefix * Fix '--no-cleanup' on worker start regression * Remove OpenQA::Schema::Result::Jobs::set_scheduling_worker * Replace Text::Markdown and HTML::Restrict with CommonMark for much better comment rendering speed and security * Adding a new parameter to support not schedule parent test suite * Timeout for wait_for_ajax * Let prove handle @INC for us * Expose saving and hiding of the legacy editor in the UI * Scrub dangerous HTML from comments * Abstract out markdown handling into a new class * worker: Consider ws disconnects as uncritical during job execution * Fix worker crash when sending job ID * YAML: don't merge scenarios with multiple machines with defaults * Add dependency on CommonMark, which will soon replace Text::Markdown for a 1000% performance increase (#2228) * Ignore cpu_modelname in worker test * Let worker pass current job ID on re-registration * Allow worker to pass its current job on registration * Improve test for checking job status and output * Improve error message when rejecting status update * Add YAML support to {dump,load}_templates * Add systemd timer to trigger daily audit event cleanup * Delete old audit events according to configured storage duration * AMQP: don't encode message body to JSON before publishing * Handle reusing existing connection in devel fullstack test * Improve documentation about running test of openQA itself * Timeout for wait_for_developer_console_contains_log_message * Override default exception handler in extra daemons * Fix default route of live handler to prevent template errors * Docs for the job group editor and YAML REST * AMQP: allow passing headers to publish_amqp * Add note about openQA's CLI interface * openqa-bootstrap: Fix lookup of configure-web-proxy * Improve rendering summary in test result overview * t/api/03-auth.t: don't actually wipe asset files from disk * Mojo::Pg 4.14 requires the "delete $dbh->{private_...}" feature from DBI 1.632 * Log errors when reading worker config * Add type/additionalProperties to products spec * spec: Fix SLE12 build with patched out python subpackage * Start a documentation section on how to build openQA plugins * templates: Update references to documentation in help texts * docs: Update references to target versions in our issue tracker * Document adding new dependencies * Improve doc section about starting webserver from Git * Move testing with production data to database section * Improve documentation about OPENQA_BASEDIR * Replay cmds in devel mode console only if connected to os-autoinst * Make sure all four levels of API access have named routes for plugins and tests * Add undef to return * Prevent cleaning pool directory in worker availability check * Add helper script 'openqa-label-all' * Set log for failing to stop cmd srv to info level * Do not fail fullstack test when connection closed * Split api request to own route * Extend unit tests for OpenQA::Worker * Do not stop current job twice when stopping during setup * Remove unused function from worker * Remove check for cpu_opmode from worker test * Stop job directly with reason 'api-failure' on API failures * Give the API authentication routes plugins will use names and test them * Log sent commands in the developer console as well * Replay commands send though developer console while offline * Add put method with disabled test * Fix test doing an equality check on an exception instead of a regular expression comparison * Fix html * Adapt codecov target to current coverage * Small tweaks * Default to HTTPS instead of HTTP with openqa-client * Add Operator menu item * Remove duplicate use * Small CR adjustment * Make routes restricted * Add ObsRsync plugin * Expand Machine and Medium Type settings when creating job with 'jobs post' When users created jobs using 'client jobs post', the jobs settings did not expand Machine and Medium Type settings. This PR is used to support to expand settings from Machine and Medium Type when creating jobs with 'jobs post'. * .travis.yml: Save travis CI some testing time * Add Test::More based error handler to SeleniumTest ------------------------------------------------------------------- Mon Jul 15 16:24:19 UTC 2019 - coolo@suse.com - Update to version 4.6.1563206570.e00d3964: * Make the websocket server tests less noisy * docker: Add back 'Test::Compile' as needed for os-autoinst * spec: Fix invalid '%perl' introduced with 3aadc34c * Test handling worker job status changes * Fix passing --no-cleanup worker CLI option * fetchneedles: Use subshells efficiently for dir changes * fetchneedles: Fix indention * fetchneedles: Prevent noisy output about auto-packing git repos * Ensure query in jobs API does not become too big * Remove group name from job templates example * Adapt order of job template example * Remove dead auth tests from a DBus refactoring 4 years ago * Give the authentication routes plugins will use names and test them * Improve and simplify compile check * Add Test::Strict to Docker container * Allow plugins to add links to the menu * Allow any suffix for TAP source files * .travis.yml: Make documentation generation an explicit job, only on master * .travis.yml: Give readable names to test jobs * .travis.yml: Put handling of cache dir together * Extend filter for new/updated admin table rows * Make log messages of worker tests less verbose * generate-documentation: Fix wrong script path introduced by 81df2fd * docs: Fix quotes and apostrophes for asciidoctor/github style * .travis.yml: Workaround flaky tests with 'travis_test' * Test kvm module, only if it is not built-in * Make containers non-x86 archs friendly and update to leap 15.1 * Remove another useless prototype and an unused function from OpenQA::Utils * Remove obsolete prototypes * Polish worker code * travis_test: Fix typo in 'python3-future' * Explicitly specify perl module 'Module::Pluggable' to fix warning * openqa-clone-job: Improve handling --from parameter * README: Clarify how releases are made * Adjust testrules.yml to changed worker tests * Post upload progress after ignoring known images * Remove dead code for handling known images * Improve formatting of comments for WebUIConnection attributes * Remove leftovers from interactive mode * Fix reporting number of images to send to liveview handler * Allow stopping worker during job setup * Rename OpenQA::Worker::Client to OpenQA::Worker::WebUIConnection * Improve use of attributes in worker code * Log specific message if ignoring job while already busy * Enable verbose option in compile check * Restructure worker * travis_test: Add dependencies for 'python-scripts' * Add YAML diff to jobtemplate_create events ------------------------------------------------------------------- Sat Jul 06 18:34:37 UTC 2019 - coolo@suse.com - Update to version 4.6.1562333920.03805636: * openqa-bootstrap-container: Fix package list passed as single string * Add Text::Diff dependency to spec and Dockerfile * Fix handling of unknown workers in the websocket server * Activate strictures in tests as soon as possible * openqa-clone-job: Add support for full test URLs * tidy: Apply to all regular perl modules and scripts * worker: Improve log messages, e.g. avoid contractions * worker: Do not use bare returns * docs: Point to wrapper client-script for shorter path * docs: Replace the confusing table of triplets with a simple scenario list * docs: Revamp the "job templates" section * docs: Combine "asset cleanup" with "asset handling" * docs: Stress older versions of Leap less * docs: Better classify the different installation methods * docs: Clarify job groups display on index page * docs: Ensure "Installation & Users guide" are prominently featured in "Getting Started" * docs: Include all feature descriptions from wiki * docs: Also explain "medium types" directly in the glossary * Delete unused helper "limit_previous_link" * t: Fix warning in full-stack.t * t: Output isotovideo log files in case of incomplete jobs within the full stack test * t: Delete superfluous setting delete * docs: Add hint how to get coverage for individual tests * Do not trigger test cleanup code from forked processes * Always use spaces in CodeMirror editor * fetchneedles: Go back to old, quiet mode * t: Fix warning in full-stack.t * t: Delete superfluous setting delete * docs: Add hint how to get coverage for individual tests * Move 'create_worker' method to single place where used * client: Prevent warnings when receiving no HTML status code at all * Add package self-test for worker as well * Add make target "test-with-database" for easier test database setup * t: Fix typo in testrules.yml * Makefile: Use explicit target name for non-travis specifics * Lock test suites in YAML-based groups for manual updates and deletion ------------------------------------------------------------------- Wed Jul 03 14:34:58 UTC 2019 - coolo@suse.com - Update to version 4.6.1562157181.8465317e: * Fix tests that were assuming Test::Mojo methods had special return values * script: Fix shellcheck issues with old shellcheck version * Fix shellcheck warnings on docker/openqa/entrypoint.sh * travis_test: Use consistent spaces instead of tabs * Make all shell scripts pass default level (style) of shellcheck * Make all shell scripts pass 'warning' level of shellcheck * fetchneedles: Use consistent spaces style * Makefile: Add test for shell scripts using "shellcheck" * openqa-clone-custom-git-refspec: Include further suggestions by shellcheck * openqa-clone-custom-git-refspec: Fix use of "-n" on GITHUB_TOKEN * Trim whitespace characters from both ends of string * Add script for simple setup of a single instance * Revert to usual wording "distri" in YAML schedules ------------------------------------------------------------------- Mon Jul 01 06:01:32 UTC 2019 - coolo@suse.com - Update to version 4.6.1561959871.3b6d67be: * Fix deletion of scripts during install * travis_test: Ensure shellcheck is installed as part of dependencies (#2136) * Strip unnecessary extension from scripts (#2138) * Integrate clone-job into bootstrap ------------------------------------------------------------------- Tue Jun 25 07:23:01 UTC 2019 - coolo@suse.com - Update to version 4.6.1561118372.b9b694d3: * Move websocket server helpers to the model * Prevent blank line in worker caching output * Fix missing dependency in package buildtime tests as well * Rename architectures to scenarios * Defer loading the schema in OpenQA::Schema::Profiler, so the worker does not trigger a migration * Fix missing runtime dependency on "DBIx::Class::DeploymentHandler" * apparmor: Allow unzip-plain to fix worker startup ------------------------------------------------------------------- Wed Jun 19 07:54:35 UTC 2019 - coolo@suse.com - Update to version 4.6.1560873517.08f60a7b: * Calculate elapsed time more correctly * Add some documentation for OpenQA::Schema::Profiler * Refactor DBIx::Class profiler to use current openQA best practices * Remove dead code and move rng functions still in use to OpenQA::Utils * docs: Fix shell code box rendering with line break * Support storing YAML job templates in the database * docs: Improve language of Networking document * docs: Consistently use QEMU, the offical way of writing ------------------------------------------------------------------- Sun Jun 16 23:54:22 UTC 2019 - coolo@suse.com - Update to version 4.6.1560549641.c09044ae: * docs: No need to start openqa-gru explicitly * Add support for OPENQA_BASE_PORT environment variable ------------------------------------------------------------------- Fri Jun 14 15:54:02 UTC 2019 - coolo@suse.com - Update to version 4.6.1560497112.f26a0400: * Don't take the full cluster as possibly blocked (#2109) * Make localhost authentication more testable * Treat IPv4 and IPv6 the same for local host authentication * Blocked by calculation: Fix cyclic behaviour (#2104) * Fix travis cache permissions for docker user * Fix a problem where an Mojo::RabbitMQ::Client::Publisher instance goes out of scope too early (#2107) ------------------------------------------------------------------- Tue Jun 11 18:22:27 UTC 2019 - coolo@suse.com - Update to version 4.6.1560026218.807c3c09: * Update MM setup documentation * Test the scheduler API * Cache: Fix missing download progress output and warning * Provide the script `openqa-clone-job` already in source * Remove dead code from OpenQA.pm * Remove imports that are no longer needed * Remove all remaining traces of D-Bus * Share authentication controller between websocket server and scheduler * Remove old script that had been broken for quite some time (and was therefore probably not used anymore) * Turn scheduler core into a model class * Replace D-Bus with HTTP * Remove imports that are no longer needed * Add a scheduler client * Update architecture diagram ------------------------------------------------------------------- Wed Jun 05 14:36:54 UTC 2019 - coolo@suse.com - Update to version 4.6.1559744346.36d65edf: * docs: Add note how to fix broken CSS * Remove tidy from spec file * Use the latest version of perltidy * Improve tests for admin tables * spec: Fix Perl::Tidy mismatch in build tests * Follow updated admin table rows * Clear admin table filter before adding new row * apparmor: Fix worker startup on openSUSE Leap 15.1 * Don't log jobtemplate_create event if in preview mode * docs: Harmonize use of no colon in titles ------------------------------------------------------------------- Fri May 31 18:17:47 UTC 2019 - coolo@suse.com - Update to version 4.6.1559047081.8050f3f0: * Fix filtering settings in admin tables * Document latest/scope{relevant,current}/limit in jobs ------------------------------------------------------------------- Wed May 22 14:34:05 UTC 2019 - coolo@suse.com - Update to version 4.6.1558532409.ab91f319: * Install qemu-uefi-aarch64 on aarch64 in bootstrap * Use less noisy perl style for $comment->label * Prevent accessing comment label without check for undef * Ensure that module names are escaped * List remaining failed modules on mouseover * Allow listing bugs by the time of reporting * Require YAML::XS >= 0.67 ------------------------------------------------------------------- Mon May 20 07:41:15 UTC 2019 - coolo@suse.com - Update to version 4.6.1558084113.18c6f052: * Revert "Improve code for settings expansion" * Update documentation about job scheduling * Add job template settings when scheduling new jobs * Show job template parameter via regular REST API * Allow YAML import/export of job template settings * Support scaling MAX_JOB_TIME * Add job template settings table * Add test for saving needles via Git * Validate properties of OpenQA::Git before using them * Fix comparision of cluster ID for dependency graph (#2068) * Set user correctly in OpenQA::Git instantiation ------------------------------------------------------------------- Mon May 13 12:07:50 UTC 2019 - coolo@suse.com - Update to version 4.6.1557739992.f9fbeb06: * Avoid leaking GITHUB_TOKEN * Skip UEFI_PFLASH_VARS asset when UEFI is disabled * Add an own result jobs which exceeded the timeout * Split off git support into its own class * Adapt 13-admin.t to the new sorting-enabled tables * Fix remaining jshint warnings in admintable.js * Adjust sorting of admin tables so empty strings come last * Improve styling of help icon beside search box * Improve regex-enabled search of admin tables * Tweak sorting and filtering in admin data tables * Remove admin table code for column type settings * Make admin tables use DataTables correctly * Improve code for settings expansion * Move job settings expansion to its own module * Use CodeMirror with YAML editor * Inherit specified settings when create job with 'Jobs post' * Show an error message in the editor if the group changed * Clean up 08-jobtemplates.t * Do not add new testsuites when importing job templates * Add YAML-based template editor UI * Add non-trivial test suites to exercise YAML validation * Import of job template specification from YAML * Support click_point in needle editor controller code * Fix issues reported by jshint in shapes.js * Test adding/removing a click point * Allow only one click point per needle * Allow adding click coordinates in the needle editor * Improve JSON parse errors in openqa-clone-custom-git-refspec * Retry after rsync exits with status 24 * Add non-fatal messages during scheduling to results * Allow authenticated calls to the github api * Remove deprecated qemu-kvm and use qemu-arm, qemu-ppc and qemu-x86 instead * apparmor: rk permission on hdd contents ------------------------------------------------------------------- Wed Apr 17 14:34:07 UTC 2019 - coolo@suse.com - Update to version 4.6.1555478309.f6dfdd9b: * Clone only the latest revision of the SLE needles (#2048) * Revert "openQA.spec: multiline macros for services and dependencies" (#2051) * Update architecture diagram in accordance with PR 2043 * Fix the remaining issues of deleting workers * Get rid of a few warnings that make test results on Travis hard to read * Be a little more strict about WebSocket server response codes * There should be no need for AssetPack in the websocket server * Trust connections from localhost to the websocket server to avoid complex config file requirements * The websocket server is no longer a dbus service * Clean up tests and remove a few connection warnings * The websocket server is no longer a dbus service * Move ws_send from dbus to HTTP * Move ws_send_job from dbus to HTTP * Remove ws_send_all dbus function, which was actually unused * Remove first dbus function from the websocket server * Add an HTTP API to the websocket server * Don't return status 402 on worker deletion failure * Fix indentation in admin_worker.js * Make OpenQA::Events::emit_event a non-static member function * Update AppArmor profile for usrMerge (boo#1132350) * Add cacheservice as part of openqa-worker.target * openQA.spec: multiline macros for services and dependencies ------------------------------------------------------------------- Sun Apr 14 17:57:10 UTC 2019 - coolo@suse.com - Update to version 4.6.1554809752.3a77164a: * Fix mock session logins to use only public Mojolicious APIs and create users with a proper resultset * Prevent multiple limit_* tasks from running in parallel * Allow deleting offline workers via API and web UI * Don't duplicate settings in iso_create/cancel events * Adapt product log test * Add test for re-scheduling product * Migrate 'iso_create' events to scheduled products * Show scheduled products from new table in UI * Test async flag for iso post route * Move constants for job dependencies to their own file * Add API route to query scheduled product * Add async flag to ISO post route and table to keep track * Cleanup scheduling functions ------------------------------------------------------------------- Sun Mar 31 17:49:35 UTC 2019 - coolo@suse.com - Update to version 4.6.1553599469.57d6b3ca: * Properly escape bug titles * YAML output for job templates and schema validation (#1999) * Make the websocket server testable as a normal Mojolicious application * Add a model class to store worker state * Move more functions into helpers * Move dbus functions into helpers * Add controllers to the websocket server application * Install test distri deps on bootstrap * Turn websocket server into a real Mojolicious app * Use Mojo::Promise in enqueue_and_keep_track * docker: Add IO::Scalar dependency * doc: Reflect renaming virtio_screen.pm to serial_screen.pm * worker: Upload serial_terminal.txt for svirt backend * Unify schema access in OpenQA::WebAPI and fix the issue where the db helper was called on the application * Use result_p to track Minion job instead of timers * Improve logging non-scalar error info of GRU/Minion tasks * Extract common code for saving and deleting needles * Fix test for dependency graph * Don't let needle-related tasks fail if the cause is uncritical * Make needle deletion a Minion job * Setting to not cancel parallel parents with still-pending children * Add hints to fix dependencies when using devel ------------------------------------------------------------------- Fri Mar 15 05:22:32 UTC 2019 - coolo@suse.com - Update to version 4.6.1552567428.80082fe5: * Test create and delete events for job templates * Unify schema access with a singleton method * Use route placeholder types whenever possible for cleaner routes * Add link to relevant documentation in the job group settings * Cache result of OpenQA::..::JobGroup::load_properties * Allow disabling bug carry over on job group level * Remove dead validation code and replace it with route placeholder types * Pass affected_rows correctly in openqa_jobtemplate_create ------------------------------------------------------------------- Tue Mar 12 21:22:25 UTC 2019 - coolo@suse.com - Update to version 4.6.1552406543.b3e49dcb: * Remove check for $OpenQA::Utils::app * Move get_bug from Result::Bugs to ResultSet::Bugs * Add bug to db already on comment create or update * Add support for links to jira tickets * Rename tmp_schema to search_path_for_tests, which is a little more descriptive * Allow monitoring of Gru/Minion with a new public /admin/influxdb/minion API endpoint * Log job group events via audit log * Make openqa-webui.service depend on openqa-gru.service ------------------------------------------------------------------- Sun Mar 10 13:22:04 UTC 2019 - coolo@suse.com - Update to version 4.6.1552078716.6b292c9a: * Test deleting job from worker's job history * Move another class into its own .pm file and do not use private functions/methods across modules/classes anymore * Remove all uses of the enterprise operator and use a more consistent coding style in the affected classes * Add loading indication when saving needle ------------------------------------------------------------------- Fri Mar 08 05:22:22 UTC 2019 - coolo@suse.com - Update to version 4.6.1551963862.0a6b30be: * Prevent deletion of worker when deleting its current job * Prevent broken rendering of unknown result types * Add unxz/xz to apparmor profile to fix poo#48743 * Don't hardcode remote and branch names for saving needles ------------------------------------------------------------------- Tue Mar 05 21:22:30 UTC 2019 - coolo@suse.com - Update to version 4.6.1551797932.a5aadf24: * Remove safe_call from OpenQA::Utils since it is no longer needed * Cover more Gru special cases with tests * Remove resource allocator from architecture diagram * Mention the possible race condition in a comment * Bypass Gru special cases for non-Gru tasks * Use a completely new task for testing so we can replicate more edge cases in the future * Be a little more defensive with removing Gru tasks * Allow pure Minion jobs (without a Gru id) to bypass special case handling * Do not delete Gru tasks if they are not finished in Minion yet ------------------------------------------------------------------- Sat Mar 02 17:37:11 UTC 2019 - coolo@suse.com - Update to version 4.6.1551454083.e27d66d8: * Extend apparmor profile for all Git commands we use * Correct limit_assets task arguments * Allow test vars for openqa-clone-custom-git-refs * Fix updating needle Git repository * Fix uniform box widths of dependency graph in current Firefox * Move JSON::Validator and YAML::XS to t_requires * Drop the `cpanfile` changes * Add YAML::XS and JSON::Validator dependencies * worker: Skip announcing stop if isotovideo already exited * worker: Fix "final status update" * worker: Refactor _stop_job_kill_and_upload * worker: Remove extra handling for dead children * worker: Make API calls when stopping job non-critical * worker: Treat upload failures as API failures * Raise an exception if deploying the SQLite database fails and handle an existing database more gracefully * Do not reset the job queue every time the cache service is restarted * Test cache service migrations * Use Mojo::SQLite migrations to handle schema upgrades in the cache service gracefully * Remove unused imports and IPC objects * Remove the resource allocator * Move barrier handling out of the resource allocator * Move mutex handling out of the resource allocator * Remove first dbus method and use OpenQA::Schema directly * Declare the DBIx::Class dependency properly * Move OpenQA::Resource::Jobs handling from the resource allocator to the webapi ------------------------------------------------------------------- Tue Feb 26 21:22:42 UTC 2019 - coolo@suse.com - Update to version 4.6.1551101052.7f891940: * Use well-defined worker config for fullstack test(s) * Adapt tests to default WORKER_CLASS * Assign default WORKER_CLASS when posting job * Assign default WORKER_CLASS when scheduling ISO ------------------------------------------------------------------- Sat Feb 23 17:39:53 UTC 2019 - coolo@suse.com - Update to version 4.6.1550673294.3fb7464e: * worker: Log error returned by JSON API * Render error in all cases in create_artefact ------------------------------------------------------------------- Tue Feb 19 14:11:16 UTC 2019 - coolo@suse.com - Update to version 4.6.1550584043.5098c32d: * Log isotovideo startup more detailed ------------------------------------------------------------------- Tue Feb 12 21:22:24 UTC 2019 - coolo@suse.com - Update to version 4.6.1549966436.c3938033: * Revert support for ASSET_\d+_URL * openqa-clone-custom-git-refspec: make use of GROUP var * Improve error handling when changing priority ------------------------------------------------------------------- Sun Feb 10 13:22:09 UTC 2019 - coolo@suse.com - Update to version 4.6.1549799415.315b994a: * Fix modules filter wrong results while using comma * Show bug reporting links also on text results * Prevent download tasks for the same asset running in parallel * Improve Minion error handling when saving needle * Allow to run Minion jobs in UI tests * Migrate intermediate database changes * Don't use might_have relationship for job scenario * Show externally skipped module count in resultstats * Use same style for needle editor warnings as for regular flash messages * Save needles via Minion * Format Git errors more nicely when committing needle * Rebase needle directory before commiting changes * Use commit_git_return_error in Step.pm to propagate error * Skip result skipped on calculate overall result ------------------------------------------------------------------- Tue Feb 05 15:29:17 UTC 2019 - coolo@suse.com - Update to version 4.6.1549379724.2b90641c: * Split configured hosts by whitespace * stopping job: Proceed on errors during announcement * Let devel mode fullstack test also pass on state cancelled * stopping job: Change order of POST requests * Prevent error about cmd srv connection when test ends * Make code for engine startup more clear * Add architecture diagram ------------------------------------------------------------------- Sun Feb 03 07:29:14 UTC 2019 - coolo@suse.com - Update to version 4.6.1549026855.08485c7c: * Remove SQL files for 2017 * Update serial terminal function (#1982) * Remove image migration jobs that were only used for very old migrations * Lift the parallel job limit for the gru worker and add more fine grained limits to individual tasks ------------------------------------------------------------------- Thu Jan 31 12:36:58 UTC 2019 - coolo@suse.com - Update to version 4.6.1548936964.47070d70: * spec: Fix missing mention of 'openqa-clone-custom-git-refspec' in script/ * Add test check for scenario within error reporting * docs: Add documentation for triggering jobs based on any git refspec * Add script 'openqa-clone-custom-git-refspec' * Consider parent group comments for important builds * Cache result of function call * Add scenario description to infopanel of test * Fix typo in reference to DeveloperSessions * Fix another regression from c9735699 when no sync is needed * apparmor: Fix access to git templates * Switch to four spaces as requested, modify error ignoring * Fix flaky test regarding needle update * Improve error handling in worker cache /status route * openqa-bootstrap-container: do more error checking, fix a hang ------------------------------------------------------------------- Sat Jan 26 02:08:43 UTC 2019 - coolo@suse.com - Update to version 4.6.1548420629.e8e80a4b: * Fix regression of test caching failing introduced by c9735699 * worker: Fix two perl warnings ------------------------------------------------------------------- Wed Jan 23 17:22:28 UTC 2019 - coolo@suse.com - Update to version 4.6.1548078204.c9f60161: * Use nicer syntax for default vars * Add (carried over) bugref to job.done AMQP event ------------------------------------------------------------------- Fri Jan 18 17:28:35 UTC 2019 - coolo@suse.com - Update to version 4.6.1547830304.62f9aaaa: * Allow overwriting container bootstrap repo * Remove Failed modules filter from UI * Add UI tests for filter by module * Add search by module in all jobs of job group * Prevent github to think we have perl6 code * Better check for existing DBs in bootstrap * bootstrap: make errors fatal and deal with re-runs * apparmor: Further fixes for custom git clones * Fix external reporting if scenario description not present * Improve code for scenario description and add test * Add test suite description for bug reporting * apparmor: Fix custom git repo clone "env" denied * clone_job: Add convenience option '--within-instance' * Also set assigned jobs to parallel_failed (#1951) * Fix a few issues reported by jshint * Prevent the live log from becoming too long * Handle undefined `$text` in `find_bugref` and `find_bugrefs` * Wrap text in live log so long lines are readable * Allow custom CASEDIR/PRODUCTDIR/ASSETDIR ------------------------------------------------------------------- Thu Jan 10 10:08:18 UTC 2019 - coolo@suse.com - Update to version 4.6.1547114886.81dabdea: * 21-needles.t: Use `discard_changes` instead of re-calling `find` * Add bootstrap hint for Leap 15.0 * Fix "Can't locate object method new via package JSON" introduced by 95cb6601 * Check for correct perltidy version on execution * Fix if condition and leap repo prios for bootstrap * Improve a few details in worker code * worker: Improve handling API failures * Handle unexpected errors on status update more nicely * worker: Fix handling errors caused by connection issues with web UI * Show currently executed function in developer mode * Allow to pause on the next command ------------------------------------------------------------------- Sat Dec 22 13:41:18 UTC 2018 - coolo@suse.com - Update to version 4.6.1545406149.53968c1e: * Activate strict and warnings for all tests and do not use indirect object syntax * Add tool to bootstrap openQA with only one command * Fix comments and reference the usage messages of the minion job/worker commands in usage messages * Test setting last needle update * Add examples for run and list commands * Add Minion tests and make the run command a subclass of the worker command * Use better variable names in t/21-needles.t * Prevent wrong 'New needle ...' notifications in needle editor * Rename Docker-related targets and scripts * Improve documentation about running tests without Docker * Remove obsolete docker/testing * Improve documentation regarding Docker * Use --oneshot to make it more obvious what is happening * Improve job restart * Allow submitting margin and match level via return key * Hide tooltips when hiding developer mode elements * Remove 'Cancel job' button from developer mode * Reuse many more parts of Minion for Gru * Add a new OpenQA::WebAPI::Command namespace so there is no need for hidden classes in plugins ------------------------------------------------------------------- Thu Dec 20 05:41:42 UTC 2018 - coolo@suse.com - Update to version 4.6.1545230885.56b79eeb: * Use OPENQA_TEST_IPC mock in 37-limit_assets.t (#1935) * Reformat with new perltidy version * Adjust comment for pauseOnScreenMismatch * Allow test in devel mode fullstack test to pass * Remove 'Error from livehandler daemon' prefix * devel mode: Test version check * devel mode: Check whether os-autoinst is compatible * devel mode: Allow to pause on check_screen timeout * Improve and fix asset caching doc * Fix typos * Makefile: Prevent docker "build" and "run" to run in parallel * Add basic documentation for OpenQA::Events ------------------------------------------------------------------- Mon Dec 17 13:44:07 UTC 2018 - coolo@suse.com - Update to version 4.6.1545054233.21210ce3: * Use a fixed version of perltidy * Prevent 'Summary of HASH(...)' if multiple distris/versions specified * .travis.yml: Delete not necessary stage overrides * docs: Make commands to call more prominent with backticks * Makefile: Delete unused target docker-test-travis * Do not set the MOJO_LOG_LEVEL environment variable by default for more granular logging * Reformat everything for the latest version of perltidy * Support filtering by test in group overview again * worker: Validate job sent by web UI "job_grab" * Use localized signal handling instead of global sigtrap for modules * Set transaction isolation level for asset cleanup accordingly * asset cleanup: Check whether $res_max_job is set before comparing * Fix copyright year * Add license to new module * Move memory limit handling into a plugin * Move all global events to OpenQA::Events * Add OpenQA::Events module to replace random events being added to Mojo classes * Improve documentation for schedule control * Extend and improve asset documentation * Don't treat UEFI_PFLASH_VARS as asset if path is absolute ------------------------------------------------------------------- Thu Dec 13 05:40:52 UTC 2018 - coolo@suse.com - Update to version 4.6.1544609268.19a4cf68: * Fix test for popovers in 15-admin-workers.t for real * Improve "Refusing 'grab_job'" debug message * Test worker broken state * Add 'broken' state to the worker * Prevent worker to accept jobs if caching not available * Update specfile line for restarting apparmor ------------------------------------------------------------------- Tue Dec 11 07:06:36 UTC 2018 - coolo@suse.com - Update to version 4.6.1544510066.913d8f26: * Update license of test spec-file to current SPDX format (#1915) * Use the boring standard return value for all Perl modules * Merge hashes properly * Test handling of assets with invalid name * Prevent registering empty assets * Skip cleanup of assets with empty name ------------------------------------------------------------------- Fri Dec 07 09:40:53 UTC 2018 - coolo@suse.com - Update to version 4.6.1544166689.47296c23: * Disable INSTALL_FROM_CPAN by default (#1863) * Don't leave a limit_assets task hanging after 14-grutasks.t (#1882) * Do not require perlcritic on sle-12 (#1910) * When detecting asset keys, don't use the _URL form for ASSET (#1907) * Add Perl::Critic::Freenode to the test container (#1906) * Fix devel fullstack mode test * Make devel fullstack test more verbose * openQA.spec: Add missing runtime requirement for client (#1905) * doc: Mention select_serial_terminal() in serial terminal doc * Test rendering external table * Don't show categories in table for external results * Add filter for external results table * Use circle to display results in external table * Add separate table for external results from parser * Move logic for test detail steps from view into controller * Fedmsg: use Python 3 logger command * Extend documentation on the developer mode panel * Add WORKER_HOSTNAME to workers.ini template * Document re-needling workflow ------------------------------------------------------------------- Mon Dec 03 10:10:38 UTC 2018 - coolo@suse.com - Update to version 4.6.1543830229.359b5dd9: * Add Perl::Critic::Freenode to openQA.spec * Use Perl::Critic::Freenode to determine good coding style * Make all modules pass Perl::Critic::Freenode with severity level 4 (stern) * Activate strict and warnings consistently for all modules * Fix network dependency for cache service * Remove DEBUG and LOGDIR environment variables and change cache services to use the test mode * Test skipping timeout in developer fullstack test * devel fullstack test: Don't fail if renamed needles already there * devel mode: Allow to skip assert/check_screen timeout * Remove srand workaround that does nothing anymore since Mojolicious 5.76 (2015) ------------------------------------------------------------------- Wed Nov 28 11:53:44 UTC 2018 - coolo@suse.com - Update to version 4.6.1543400978.50e006bc: * Fix worker-cacheservice failing on startup due to no network * docs: Extend development tricks with dynamic test override * Fix typo in installation documentation * Replace Mojolicious::Commands with Mojo::Server::Daemon in test modules, make sure all test modules use strict, and fix a few issues caused by not using strict before * Restart openqa-worker-cacheservice on failure * Allow caching of externally downloaded assets ------------------------------------------------------------------- Mon Nov 26 21:40:33 UTC 2018 - coolo@suse.com - Update to version 4.6.1543249675.02bb8eff: * Exit worker with non-zero return code in error cases * Do not import from_json, there is already an inherited method with the same name * Add comments to point at Mojo::JSON being used to set up booleans * Debug 25-cache-service.t * Make perltidy happy * Convert a few cases back to Cpanel::JSON::XS that require special configurations (mostly for tests, but there might be underlying protocol requirements too) * Document steps to debug developer mode setup * Convert most uses of Cpanel::JSON::XS to Mojo::JSON, which will take care of using an optimized encoder and avoiding boolean warnings * Use fixed header for test result overview table * Check for JavaScript errors while waiting for result panel * Prevent selenium test to accidentally click on the VNC notice * Move note about VNC into developer panel * Update existing unique flash messages * Expand devel controls by default if paused and owning the session (#1880) * Add rsync BuildRequires and Recommends to RPM spec (#1881) * Add a new is_admin_js helper to generate JavaScript booleans for use in templates * Add a test for browsing the admin pages in readonly mode * Use OPENQA_TEST_IPC in developer mode unit test (#1868) * Add test to check link for entire build * Add link to test results for multiple job grops * Call _reschedule after schedule concluded * Remove unused variable $force in _reschedule() * Fix comment in scheduler code * Fix various uses of the is_admin helper in templates to generate JavaScript that resulted in empty lists, because the helper returns a Perl boolean value instead of a JavaScript boolean * Revert "Fix deprecation warnings regarding Mojo::Transaction::error" * Fix line break in failed module box * Fix formatting * Fix deprecation warnings regarding Mojo::Transaction::error * Combine multiple distri/version/flavor/build filter with logical or * Fix filter for architecture * Upload video timestamp file when it exists (#1873) * Fix formatting * Use Mojo::File::slurp in check_tabmode * Improve error handling of Step controller * Handle `app` and `tx` attributes of Controller being weakened * Do not check group name by dragging group * Minor fixes for WritingTests doc (#1865) * Add 2 new dependencies to the travis container: (#1864) * Update all uses of Mojo::Transaction::success to their Mojolicious 8.06 equivalent * Fix a few new uses of tempdir * Fix various incorrect uses of Mojo::File::tempdir * docs: Extend definition of *softfailed* to "known issues" (#1837) * Refuse updating job group name with empty or blank (#1844) * Apparmor: Add /dev/hugepages (#1857) * Fix broken in section of inter machine dependencies (#1859) * Wrap up and polish * Do not use wantarray in GRU's enqueue() * Update POD OpenQA::Worker::Cache::Client's SYNOPSIS * Avoid to race to get available minion workers * Drop PID from log * Move Minion workers under a single service * Cleanup * Fix typo * Adapt gru tests * Generate same lock for asset/host in the request object * Fix rsync output in minion task * Refactor and update Documentation * Use the cache service to perform rsync tests caching * Create Minion task for rsync execution * Use the host folder to cache tests/needles * Do not check if child is running if it's not there * Reduce the lock expiration time * Trim spaces from left and right in assets name * Hold the db handle during transactions * Add test for Minion task registration and execution * specfile: DBD-SQLite needs to be > 1.51, required by Minion::Backend::SQLite * Add tests and cover also subprocess execution * Add function to generate session token * Read and set limit from config file, defaults to 50GB * Makefile: install Worker Cache Service systemd unit * Add POD for Cache Service and extend docs/Installing.asciidoc * Add service file and adapt openQA.spec * Avoid useless quotes on hash key * Debug - adapt compile-check-all * Address to PR comments - also add test for checking output in cache-service.t * Drop unused function * Write cache output into os-autoinst log * Make possible to retrieve cache output * Add debug messages while enqueuing download * Asset path needs to be a string for vars * Make dequeue and status post * Adapt fullstack to cache path change * Extract domain parsing to _base_url and use it also in the client to resolve asset path * Adapt tests * Get the cache path right * Incomplete job if cache service is down * tests: Wait also for worker minion in fullstack * Use defaults in isotovideo engine * Update defaults and remove protoyped function from Cache::Client * tests: Add test for available_workers() * Add available_workers() to Cache::Client * Fullstack: move sqlite cache check up * Cache: make retry() return always the response object * Fullstack: Wait cache to be ready * Adapt tests and utils * Drop unneeded default * Use OpenQA::Worker::Client inside the cache task * Start to adapt fullstack test * Adapt to PBP * Add dep on Minion::Backend::SQLite * Move common testing functions to Test::Utils as they will be reused also in fullstack test * Add Mojo::SQLite dependency * Use the Cache service within the worker * Add accessors to retrieve asset path and check if exists * Remove unneded log line and cleanup check_limits() * Finishing porting the Cache.pm code to Mojo::SQLite * Rework cleanup strategy, avoid endless loops - update etag fails now * Use Minion Admin plugin with the cache service * Drop downloading, Cache.pm now records just etag and size * Use Devel::Cover in cache-service tests to enhance coverage * Adapt cache tests - still fails on size calculation * Move Cache.pm code to Mojo::SQLite * Add dequeue() and tests for it * Fix and test enqueued tasks * Do not use list_jobs from Minion to gather statuses, causes races * Add default usage tests and use minion lock to detect if it's in use * Fix test copyright header * Add standard client usage * Add Cache service client * Do not lock anymore on assets * Add test for cache service * Add script to start cache service * Create separate class for Minion Server and worker * Do not log an error when trying to unlink an nonexisting file (#1854) * Prevent filter on test overview to reveal old jobs * Allow to create dependency for jobs on different machines * Test that filtering overview does not reveal old jobs * Add dependency for test in os-autoinst (#1851) * Handle skipped test result in IPA parser * Show bug title on mouseover * Fix job templates JavaScript * Fix typo in networking documentation * Regenerate docs * devel mode: Suppress connection errors if no module running * Make 13-admin.t more verbose in case of failure * Use $LEAP_VERSION in documentation consistently * Use 'not yet' instead of 'never started' on /admin/workers/X (#1841) * Extend unit tests for developer mode UI * Don't replace preview container when updating results * Load new test details when paused in the middle of module * Add START_AFTER_TEST/PARALLEL_WITH to dep graph tooltip (#1840) * Allow to scroll up and down using arrow keys on test details (#1842) * Fix testhelper wait_for_session_info * Lazy-initialize devel mode when switching to live view tab * Make 'can't remove ...' warnings debug messages (#1836) * Create new db schema version * Add tests for test module flags * Add special icon for the always rollback flag * Add always_rollback flag description * Add regex filter for admin tables (#1831) * Simplify Schema::Result::Jobs::running_modinfo * Test progress bars on /tests * Fix progress bars on /tests * Fix prio field in /tests and add a test for it (#1834) * Hide cloned jobs from job dependency graph (#1833) * Add route to poll stats of running and scheduled jobs (#1829) * Allow softfail and softfailed as step result * Prevent rendering undef/refs as resborder * Move errors regarding developer mode into the devel panel (#1830) * Test error handling of status/image/progress upload * Recompute asset status after cleanup for /admin/assets * Fix misleading error message on non-UEFI jobs (#1825) * Save asset status cache under /var/lib/openqa/webui/cache * Don't serve assets table while cleanup is ongoing * Update apparmor profile for current Tumbleweed for x86 workers (#1827) * worker: Stop job if considered dead by the web UI (2) * worker: Stop job if considered dead by the web UI (#1824) * Prevent multiple limit_assets tasks to run in parallel * Spawn limit_assets task in a consistent way * Polish a few details on the /admin/assets table * Allow to trigger asset cleanup task on /admin/assets * Prevent warning when rendering assigned jobs on 'All tests' page (#1823) * Fix issues noted by jshint * Make displaying assets by group faster * Set asset table width to 100% * Show last update of assets table * Improve speed of /admin/assets * Fix dependencies of openQA-client package (#1819) * Use less-ugly gravatar icons already used in OBS and progress (#1822) * Fix padding of tabs in testresults * Polish code for dependency graph * Clean CSS * Ensure filter icon and last tab button don't overlap * Add help text to dependency graph * Rename part_of_cluster to has_dependencies * Remove old dependency table * Test dependency graph * Improve appearance of dependency graph * Compute cluster for dependency graph * Fix clipping and alignment of dependency graph * Make dependency graph a bit more fancy * Display dependency graph using dagre-d3 * Handle logging errors (#1815) * Use italic font to display default prio (#1818) * Fix openQA-client package for openSUSE (#1816) * Stop worker when an unhandled exception occurs (#1809) * Limit new needles sent to os-autoinst on resume to 100 * Use Mojo::File in t/ui/07-file.t * Add 'perl(File::Touch)' to Dockerfile * Test needle download * Allow command server to download new needles on resume ------------------------------------------------------------------- Wed Oct 03 06:37:58 UTC 2018 - coolo@suse.com - Update to version 4.6.1538413557.f9e2a131: * Adapt tests to logging behavior of Mojolicious 8 (#1814) * Use transaction when computing the asset status * Make only cleanup fail on inconsistent asset status * Print max job when detecting ongoing scheduling during cleanup * Fix removal of scheduling assets * Document asset cleanup ------------------------------------------------------------------- Sat Sep 29 18:38:25 UTC 2018 - coolo@suse.com - Update to version 4.6.1538057874.2d8d0618: * Update AppArmor profile for Leap 15.0 (#1810) * Allow overwriting job priority when posting isos (#1807) * Dump the JSON templates as ascii * Fix jobs fixtures * Increase debugging information in asset removal * Round the remaining days to make tests less blinky * Fix wrongly assigned states in existing data * Prevent setting the state to SKIPPED * Test whether t_finished is assigned when setting a final job state ------------------------------------------------------------------- Wed Sep 26 06:38:31 UTC 2018 - coolo@suse.com - Update to version 4.6.1537939546.7aa7ed58: * all tests page: Fix parameter passing for timestamp rendering (#1806) * Make asset cleanup more verbose (#1803) * Fix 'last_match' param of AJAX route for needle table (#1802) * Use AJAX for all tables on /tests * Don't pre-compute IDs for finished jobs when loading HTML * Optimize all tests page * doc: remove mention of openQA-devel package (#1800) * Make sure multimachine jobs get a worker eventually * Some code restructing in the scheduler function * Delete empty settings in clone job * Display progress bar when downloading asset * Cache infos about scheduled jobs * Limit logs and results for groupless jobs (#1798) * Release job networks more often (#1795) * Fix removal of untracked assets (#1796) * Enable the scheduler on updates (and the webui is enabled) (#1785) * docs: Write Leap installation repo in more timeless way (#1791) * Run inserting screenshots in a transaction (#1792) * Update AppArmor profile to allow worker to use cgroups * Set specific slice for worker service * Better detect of systemd slice * Inform worker about devel session to reduce posting status info (#1789) * Correct typo (#1790) * Require RabbitMQ::Client 0.2 * Make AMQP sending less fragile * Make web socket and livehandler daemons listen on IPv4 and IPv6 (#1787) * Prevent worker to endlessly retry non-critical API calls (#1788) * Fix typo in t/api/02-iso.t * Make asset download feature usable by both "jobs post" and "isos post" * docs: Cover different asset types and asset options ------------------------------------------------------------------- Thu Sep 13 15:13:01 UTC 2018 - coolo@suse.com - Update to version 4.6.1536756702.f75303c1: * Improve error log of AMQP plugin (#1784) * Treat failure when posting upload progress as non-critical * Allow regular HTTP and WS being passed though reverse proxy * Fix random failures in developer mode unit tests * Add unit test for posting progress upload * Add HMAC time stamp also on livehandler daemon * Store upload progress in database * Prevent use of uninitialized value when posting upload progress * Check for upload progress in developer fullstack test * Use API authentification for post upload progress route * Test handling of upload progress in UI * Show needle editor button not unless upload complete * Pass upload progress from worker via liveviewhandler to browser * Open needle editor in developer fullstack test * Fix one more corner case for blocked_by (#1781) * Simplify reschedule code * Make the overview page responsive again (#1780) * Sort scheduled jobs via database (#1779) * Fedmsg plugin: emit 'standardized' fedmsg (#1758) * Fix blocked jobs not appearing in scheduled table * Increase max execution time per job in fullstack test * fullstack.t: Distinguish between not found and wrong permissions * Add more checks in 15-comments.t to debug random failure * OpenQA::Parser::Format::JUnit include error and skipped details (#1778) * Update serial failures detection documentation * Prevent asset-releated tests being disturbed by unrelated files * Test computing asset status * Optimize querying asset status for admin table * Allow rendering asset table as JSON ------------------------------------------------------------------- Sun Sep 09 11:03:30 UTC 2018 - coolo@suse.com - Update to version 4.6.1536432963.ac9f540e: * Fix the vlan assignment and release * Remove debug that prints to STDERR * Clean up 05-scheduler-dependencies * Take the full cluster into account, not just the first level * Avoid undefined warnings from rescheduling * Do not reschedule cluster jobs multiple times * Do not require websocket in scheduler (#1777) * Extend the test coverage for blocked_by * Fixed blocked_by calculation not to create loops * Schedule the jobs manually in fullstack test * git clone only the HEAD of os-autoinst * Do only copy back the cache on travis * Make 2 debug messages in WS server info * Wait for the scheduler to reshedule job 3 * Die early in full-stack test * Don't output 'Scheduling new jobs' if there are 0 free workers * Add some means to debug full-stack test * Only talk about cpanm if it's actually true * Run chromium with --no-sandbox not to require admin caps * Revert PR 1747 * Allow to enable monitoring using mojo-status (#1763) * Catch misconfigured test suites creating cycles (#1775) * Make tidy fast by only looking at changed files (#1773) * Ignore file docker.env created by make target docker.env * Update docker-related documentation * fullstack/Docker test: Skip os-autoinst build only conditionally * fullstck test: Improve ignoring JS errors * fullstack test: Prevent auto page reload when waiting for result panel (#1770) * Improve help popover for job group/template prio * Prevent Firefox to misalign arrows of prio input * Disable job template prio editor for non-admins * Test using default prio from job group when scheduling ISO * Add tests for setting prio via job templates API * Improve error msg and response for update of job template API * Allow job template to inherit prio from group * Allow amending job template priority * Use default prio when adding new job template to existing medium * Avoid using Mojo::ByteStream directly when rendering JSON * Adapt t/16-utils-runcmd.t for e2771ee * Fix sorting worker jobs by finished * Make 33-developer-mode.t more stable * Prevent use of uninitialized $current_running in Worker/Jobs.pm * Test opening needle editor for running tests * Improve 404 error pages of needle editor for running jobs * Test pausing on assert screen timeout * Consider only new console messages in fullstack test * Adjust upload_status for pausing on assert_screen timeout * Fix typo occured -> occurred * Test UI changes for pausing on assert_screen timeout * devel mode: Hide executed modules when test status updated * devel mode: Show reason for pause * needle editor: Show 'back to live view' if in devel mode * devel mode: Allow to open needle editor when paused * devel mode: Add UI elements for pausing at timeout * Use git -C for commits * Don't check for .git directory when making git commits ------------------------------------------------------------------- Tue Aug 21 18:55:46 UTC 2018 - coolo@suse.com - Update to version 4.6.1534873283.be929573: * Remove entrypoint from specfile sources * Makefile: Build env file for docker with needed environmental variables * Fix Mojolicious version requirement * Emit parent group ID in AMQP comment event * Use subtests in AMQP test * Makefile: Avoid also to parse env with spaces to travis * tests: Fix builds on master * Adapt fullstack tests to changes in os-autoinst * Make javascript_console method less verbose when there are no errors * test: disable broken test * Do not call unblock introduced in 184d0c5d5 * Revert "Calculate blocked by jobs on creation in job resultset" * Do not display blocked count * Do not calculate blocked jobs * Drop unused import * Revert "Merge pull request #1729 from mudler/blocked_by" * Revert "Fix usage of $blocked in test results template (#1740)" * Revert "Merge pull request #1741 from coolo/fix_blocked_by" * Makefile: cleanup * tests: INSTALL_FROM_CPAN by default * docker: Pass environment to docker * docker: try to build our image with package upgrades and custom entrypoint while running tests * docker: Upgrade container packages before running tests * Fix suggested command for adding devel repo ------------------------------------------------------------------- Fri Aug 10 15:29:12 UTC 2018 - coolo@suse.com - Update to version 4.6.1533914931.14b73490: * Use the transaction to check if it's empty (and avoid warnings) * Upload: Keep the transaction around * Calculate blocked_by status of restarted jobs * Remove systemd dependencies on scheduler * Fix usage of $blocked in test results template (#1740) * Make limit for keeping untracked assets configurable * Test inheritance and overriding of job/parent group properties * Read default job/parent group properties for cleanup from settings * Update openQA documentation regarding serial failures detection * Make UEFI_PFLASH_VARS non critical on local workers * Print autoinst-log.txt after Travis testrun * Fix full-stack.t * Adapt test of group overview JSON route for parent comments * Test tagging on parent group level * Test commenting on parent group level * Allow pinned comments on parent group overview * Support commenting on parent-group level * Test overrides for job settings passed to clone job script * Reduce indentation in clone_job_apply_settings * Require at least IO Socket SSL 2.009 * Pass worker setting WORKER_HOSTNAME to web UI * Add tests for comments carryover if module contains bugref in details * developer mode: Don't show same error twice * Unify disable_bootstrap_animations and disable_bootstrap_fade_animation * Carry comment over if test suite failed with same serial failure * developer mode: Show flash messages on errors * Test processing state changes from os-autoinst * Consider fullpath as a valid asset * Minor refactor for clone_job.pl and add unit test * Add global settings support for clone job * Treat UEFI_PFLASH_VARS as non critical asset * Change striped name from IPA results * Remove debug leftover * Highlight blocked jobs with orange colour * Do not hide scheduled jobs if blocked * Calculate blocked by jobs on creation in job resultset * Release vlan networks only if other jobs in the same cluster aren't running anymore * Disable worker shuffling in 05-scheduler-* tests * Add scheduler function to disable worker shuffling during computation * Adapt job_grab tests * Adapt scheduler fullstack test to latest changes * Follow the rename of get_websocket_api_version * blocked_by calculation: Don't block on self * Fix timer reset on quit * Calculate blocked_by state later in the process * Reset the scheduler tick if not summoned * Simply scheduler logic - but a lot * Fix blocked_by for chained parallel jobs * Disable pp in IPC - a little too noisy on allocating * Filtering for scheduler * Prepare a hash of jobs * Select all scheduled jobs and filter them for free workers * Remove BUSY_BACKOFF and CONGESTION_CONTROL * Remove CAPTURE_LOOP_AVOIDANCE option * Remove OPENQA_SCHEDULER_FIND_JOB_ATTEMPTS option * Add UEFI_PFLASH_VARS to detect_asset_keys * Mark all restarted (cluster) jobs in UI * Add IPA format to OpenQA Parser * Improve logging in LiveViewHandler * Assign os-autoinst tx not before actually connected * Ignore 'Data frame received after close' for now * Extend unit tests for live view handler * Refactor LiveViewHandler * devel mode: Prevent endless, instant reconnects on error * Mention how to actually enable openqa-ssl.conf under openSUSE * Use wss protocol for websocket connection when page loaded via https * Fix javascript errors on running.js (#1719) * Update README.md ------------------------------------------------------------------- Tue Jul 10 13:39:38 UTC 2018 - coolo@suse.com - Update to version 4.6.1531161673.f7fc21dd: * Remove docker-tests during install * Don't display blocked jobs as scheduled * Make registering assets transactional (#1711) * docs: Add note on sending new lines to a serial terminal (#1714) * clone_job: Make _GROUP and _GROUP_ID override each other (#1716) * Add blocked_by to jobs - calculated on job changes * WIP: Move commands to run docker tests to separate script * Make parser test less verbose * Make error messages in setup_share_dir more specific * Allow to use a custom os-autoinst checkout * Remove print() leftover from debugging * ws proxy: Add tx after last client-specific error handled * Don't fail full-stack.t when already assigned * Prevent use of global variables in LiveViewHandler.pm * Make developer mode accessible to operators (#1707) * needle editor: Assign new background image instantly * developer mode: Show only modules which are still ahead * Adjust overflow behavior of dashboard progress bars * Use _GROUP_ID in clone_job.pl * Refactor test code for jobs post * Test specifying group by ID on jobs post * live view: Connect to ws server when testStatus.running is set * Fix bugs in form controls for devel mode * Show developer mode form only when connected via web sockets * Fix help popover in developer panel * devel mode: Handle case when module list still unknown * Never assume it is the own devel session when not logged in * Don't update developer mode controls until confirmed * Rephrase to avoid 'developer session' * Make full-stack test more stable (#1701) * Replace 'opened by' with 'owned by' * Prevent resetting the module to pause at on page load * Show link to ws console only in development mode * Remove useless debug printing in live view handler * Don't use Test::MockModule::redefine() * Improve error handling of livehandler daemon * Test UI controls also in developer fullstack test * Cancel job when quitting the developer session * Don't quit developer session when disconnecting from os-autoinst * Add tests for developer UI controls * Fix importing determine_web_ui_web_socket_url * Start developer session via submit button * Set status code when finishing ws connections * Avoid using Mojo::IOLoop->one_tick() in developer unit tests * Adapt to removal of waiting state * Fix style issues mentioned by @foursixnine in review * Extend unit tests for status-only route * Fix broken HTML on test details page * Check that VNC instructions are shown when test is paused * Document variables for developer mode * Hide entire developer panel when test not running * Adapt existing developer test to UI changes * Update module select for devel mode like modules under details * Handle reconnects in the client * Notify clients about removed sessions as well * Keep track of status-only and devel ws tx separately * Show VNC instruction only for running jobs * Pick the right route (either developer session or status only) * Add UI controls for developer session * Move methods to build proxy url to OpenQA::Utils * Remove deprecated ui test * Allow developer-instructions to be modified by status updates * Add status route to websocket proxy * Fix typos * Connect via livehandler proxy to isotovideo to get updates * Update service file for livehandler service * Remove unnecesary options for livehandler service * Update instructions * Expose VNC port to developer * Add UI controls for developer session * Fix formatting of Contributing.asciidoc * Document running UI tests with Docker in non-headless mode ------------------------------------------------------------------- Tue Jun 26 12:54:52 UTC 2018 - coolo@suse.com - Update to version 4.6.1530017682.0f384112: * Allow 'assigned' state in fullstack.t (#1699) * Move constants from Schema::Result::Jobs to a separate package (#1696) * Remove leftovers of interacitve mode (#1697) * Adapt fullstack test for the logging changes * Adapt tests for the display of pid in the log in debug mode * Print pids of processes in logs if log level is debug * Avoid db locks on asset_lookup * Treat the case when we could not update asset in critical section * More protection for critical section in the cache * Add other cache tests and more refactoring * Try to bring coverage back * Add locking test, and remove useless query in fullstack.t * Reduce the critical section * Avoid Perl 5.18 complain * No need to import getcwd * Do not run the cache test in parallel * Reduce parallel processes for concurrency test to avoid travis to crash * Wrap also toggle_asset_lock in lock_section * Flip download = 0 from the same function used to toggle on/off * Make the cache test more stable - add also option for stress test * Add concurrent test for cache * No need to init explictly anymore * No need to export get_asset anymore * Adapt cache and cache test to more OOP style * Use transactions and wrap rollbacks in eval calls for further debug. Also enable exclusive transaction in toggle_asset_lock * Fix wrapping of parser results (#1695) ------------------------------------------------------------------- Tue Jun 19 18:37:47 UTC 2018 - coolo@suse.com - Update to version 4.6.1529433451.a70c6e82: * Remove running_or_waiting method * Remove waiting state that was leftover from 5e65496f * Ensure proxy is connected to os-autoinst in devel test * Allow status as a ws_proxy command * Make waiting for paused more robust against disconnects * Remove unused parameter in wait_for_developer_console_contains_log_message * Let the web socket console automatically reconnect * Ensure ws connection won't timeout in fullstack tests * Improve check for developer console in fullstack test * Remove link to developer console * Use docker image from OBS (#1685) * Let tmpfiles.d create the mount point "share" (#1689) * Add basic barrier documentation * Improve documentation about mutexes * Remove $isodir from Utils (only used in tests) * Remove /tests/X/iso route * Remove hdddir and otherdir from Utils * Fix waiting for development session * Don't fake D-Bus when running the devel fullstack test * Propagate os-autoinst URL from worker to web UI * Fix CSS for developer console * Fix path for liveviewhandler in Apache config * Adapt execution of developer test to latest changes * Fix perl critic error * Open the 2nd window/tab in devel mode test correctly * Handle errors happening before dev session correctly * Add unit test for developer mode * Remove unneccassary D-bus env for livehandler * Refactor live view handler to use regular methods * developer console: Handle case when port is NaN * Refactor full-stack.t and 33-developer_mode.t * Add test for developer mode to Travis test matrix * Add test for developer mode * Workaround broken syntax highlighting of full-stack.t in Kate * Allow to start Selenium driver directly via start_driver * Improve passing parameter to OpenQA::Test::Database::create() * Fix ID of developer session info * Increase inactivity timeout for openqa-livehandler * ws client: Broadcast messages to all JavaScript client * ws proxy: Try/catch only JSON decoding * Allow login as a 2nd administrator via fake auth * Make logout accessible via regular GET * Install service file for openqa-livehandler * Use transaction to delete development session * Move ws connection for developer session to its own service * Keep track of developer session, limit to one session per job * Validate commands before passing to os-autoinst * Make the websocket proxy route basically work * Allow to use websocket console for proxy route as well * Add basic structure for developer mode web socket route * Make fullstack test more verbose * Prevent warning in fullstack test * Test pausing and resuming jobs * Add web socket console for isotovideo commands * Display incomplete jobs in next&previous tab (#1682) * Update documentation with NAT configuration using firewalld (#1683) * Categorize UEFI PFLASH drives as HDD assets (#1684) * Try caching * Easier way to pass exit status from docker-tests * Change message level when cgroups cannot be created (#1681) * Fix typo in the travis.yml and fix the grep regular expression for the status checking (#1679) * Adjust popover margin on audit log (#1678) * Change from su to sudo in the docker image * Docker shell scripts cleanup * Allow where parameter for mutex unlocking (#1677) * Documentation for running tests in docker * Enable codecoverage * Add support for os-autoinst * Add DBUS support. * Force user permissions 1000 to be compatible with opensuse * Moved the DB from the host to the container * First release of the dockerfile to create container to run unit and integration tests * Remove job_set_running - as jobs are no longer waiting * Remove job_set_waiting - jobs aren't WAITING anymore * Remove current interactive mode ------------------------------------------------------------------- Sat Jun 16 05:41:15 UTC 2018 - coolo@suse.com - initial package