ALP-pool/salt
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
main
salt/early-feature-support-config.patch

3785 lines
114 KiB
Diff
Raw Permalink Blame History

From 7b47e6f19b38d773a6ec744209753f3d29b094ea Mon Sep 17 00:00:00 2001
From: Alexander Graul <agraul@suse.com>
Date: Tue, 18 Jan 2022 16:40:45 +0100
Subject: [PATCH] early feature: support-config
Add support script function
Add salt-support starter
Initial support wrapper
Add data collector skeleton
Add default scenario of the support configuration
Add main flow for the collector.
Move support library to its own package
Add default support collection scenario
Add logging
Handle CLI error.
Update format of the default support scenario
Default archive name
Finalise local data collection
Write archive from memory objects.
Add colored console outputter for salt-support.
Use colored outputter
Add message output class
Remove try/except capture from the scripts and move to the runner directly
Implement output highlighter methods for CLI output
Move scenarios to profiles
Get return section from the output. Tolerate raw data.
Implement internal data collector
Add network stack examination to the default profile
Add an internal filetree function
Add a method to discard current session
Add a method to link a static file to the resulting archive
Implement internal function caller
Add internal functions
Add default root for the one-file support data
Set output device
Separate dynamic data and static files on the fs
Update color theme
Add ident to the error message
Report rejected files with the ident
Reuse system error exceptions and reduce stat on the file check
Use socket name of the host machine
Add options for profile and archive settings
Use archive name from options.
Get profile by config/options
Cleanup broken archive on crash/exception
Use profile from the options/configuration
Add more colored messages :-)
Initial implementation of get static profiles
Update docstring
Move PostgreSQL profile to its own
Handle profile listing, do not yield sys.exit on specific module
Add network profile
Add Salt's profile
Uncomment package profile
Allow several profiles to be specified
Remove comments, add parameter to get more profiles
Implement existing configuration finder
Add options to handle unit configurations
Pre-parse options prior run() to choose proper configuration target
Handle arg parse generic errors, unit mis-choose
Let cleanup be aware of pre-config state
Fix imports
Handle exit codes properly
Allow to overwrite existing archive
Use py2/3 exceptions equally
Include exit exception on debugging
Render profiles as Jinja2, add basic recursive caller to the template of the profile
Add "users" profile
Implement basic caller for the profile template
Add table output renderer
Fix typo
Remove table outputter
Allow default outputters and specify outputters inside the profile
Remove group.getent from the loop per each user
Add table outputter to network profile
Add text outputter to hostname/fqdn data
Remove network part from the default profile. Add text/table outputters.
Fix Py3 compat
Collect status (initial)
Avoid irrelevant to profile files
Add job profiles
Add profile template trace
Add inspection through the runners
Allow parameters in callers and runners
Handle non-dict iterables
Highlight template content in the trace log
Add return extractor from the local call returns
Move local runner to its own namespace
Lintfix: PEP8
Remove duplicate code
Fix caller return
Add description tag to the scenario
Add generic colored message
Add wrapping function. NOTE: it should be refactored with the other similar functions
Print description while processing the scenario
Turn off default profile and print help instead
Move command-line check before collector
Do not verify archive if help needs to be printed
Add console output unit test for indent output
Fix docstring
Rename test class
Refactor test to add setup/teardown
Add unit test to verify indent
Use direct constants instead of encoded strings
Add unit test for color indent rotation check
Add a test case for Collector class
Add unit test for closing the archive
Add unit test for add/write sections on the collector object
Add test for linking an external file
Cleanup tests on tear-down method
Add call count check
Add unit test for support collection section discard
Add unittest for SaltSupport's function config preparation
Fix docstring
Add unit test for local caller
Add unit test for local runner
Add unit test for internal function call
Add unit test for getting an action description from the action meta
Add unit test for internal function call
Add unit test for return extration
Add unit test for determine action type from the action meta
Add unit test for cleanup routine
Fix typo of method name
Add unit test for check existing archive
Add test suite for profile testing
Add unit test for default profile is YAML-parseable
Add unit test for user template profile rendering
Update unit test for all non-template profiles parse check
Add function to render a Jinja2 template by name
Use template rendering function
Add unit test on jobs-trace template for runner
Move function above the tests
Add current logfile, if defined in configuration
Bugfix: ignore logfile, if path was not found or not defined or is None
Lintfix: iteration over .keys()
Remove template "salt" from non-template checks
Lintfix: use salt.utils.files.fopen for resource leak prevention
Lintfix: PEP8 E302: expected 2 blank lines, found 0
Lintfix: use salt.utils.files.fopen instead of open
Lintfix: PEP8 E303: too many blank lines (3)
Lintfix: Uses of an external blacklisted import 'six': Please use 'import salt.ext.six as six'
Lintfix: use salt.utils.files.fopen instead of open
Fix unit tests
Fix six import
Mute pylint: file handler explicitly needed
Lintfix: explicitly close filehandle
Lintfix: mute fopen warning
Remove development stub. Ughh...
Removed blacklist of pkg_resources
Make profiles a package.
Add UTF-8 encoding
Add a docstring
Support-config non-root permission issues fixes (U#50095)
Do not crash if there is no configuration available at all
Handle CLI and log errors
Catch overwriting exiting archive error by other users
Suppress excessive tracebacks on error log level
Add multi-file support and globbing to the filetree (U#50018)
Add more possible logs
Support multiple files grabbing
Collect system logs and boot logs
Support globbing in filetree
Add supportconfig module for remote calls and SaltSSH
Add log collector for remote purposes
Implement default archive name
Fix imports
Implement runner function
Remove targets data collector function as it is now called by a module instead
Add external method decorator marker
Add utility class for detecting exportable methods
Mark run method as an external function
Implement function setter
Fix imports
Setup config from __opts__
Use utility class
Remove utils class
Allow specify profile from the API parameter directly
Rename module by virtual name
Bypass parent subclass
Implement profiles listing (local only for now)
Specify profile from the state/call
Set default or personalised archive name
Add archives lister
Add personalised name element to the archive name
Use proper args/kwargs to the exported function
Add archives deletion function
Change log level when debugging rendered profiles
Add ability to directly pass profile source when taking local data
Add pillar profile support
Remove extra-line
Fix header
Change output format for deleting archives
Refactor logger output format
Add time/milliseconds to each log notification
Fix imports
Switch output destination by context
Add last archive function
Lintfix
Return consistent type
Change output format for deleted archives report
Implement report archive syncing to the reporting node
Send multiple files at once via rsync, instead of send one after another
Add sync stats formatter
Change signature: cleanup -> move. Update docstring.
Flush empty data from the output format
Report archfiles activity
Refactor imports
Do not remove retcode if it is EX_OK
Do not raise rsync error for undefined archives.
Update header
Add salt-support state module
Move all functions into a callable class object
Support __call__ function in state and command modules as default entrance that does not need to be specified in SLS state syntax
Access from the outside only allowed class methods
Pre-create destination of the archive, preventing single archive copied as a group name
Handle functions exceptions
Add unit test scaffold
Add LogCollector UT for testing regular message
Add LogCollector UT for testing INFO message
Add LogCollector UT for testing WARNING message
Replace hardcoded variables with defined constants
Add LogCollector UT for testing ERROR message
Test title attribute in msg method of LogCollector
Add UT for LogCollector on highlighter method
Add UT for LogCollector on put method
Fix docstrings
Add UT for archive name generator
Add UT for custom archive name
Fix docstring for the UT
Add UT for checking profiles list format
Add Unit Test for existing archives listing
Add UT for the last archive function
Create instance of the support class
Add UT for successfully deleting all archives
Add UT for deleting archives with failures
Add UI for formatting sync stats and order preservation
Add UT for testing sync failure when no archives has been specified
Add UT for last picked archive has not found
Add UT for last specified archive was not found
Bugfix: do not create an array with None element in it
Fix UT for found bugfix
Add UT for syncing no archives failure
Add UT for sync function
Add UT for run support function
Fix docstring for function "run"
lintfix: use 'salt.support.mock' and 'patch()'
Rewrite subdirectory creation and do not rely on Python3-only code
Lintfix: remove unused imports
Lintfix: regexp strings
Break-down oneliner if/else clause
Use ordered dictionary to preserve order of the state.
This has transparent effect to the current process: OrderedDict is the
same as just Python dict, except it is preserving order of the state
chunks.
Refactor state processing class.
Add __call__ function to process single-id syntax
Add backward-compatibility with default SLS syntax (id-per-call)
Lintfix: E1120 no value in argument 'name' for class constructor
Remove unused import
Check last function by full name
---
doc/ref/modules/all/index.rst | 1 +
doc/ref/states/all/index.rst | 1 +
salt/cli/support/__init__.py | 76 +++
salt/cli/support/collector.py | 563 ++++++++++++++++++++++
salt/cli/support/console.py | 184 +++++++
salt/cli/support/intfunc.py | 51 ++
salt/cli/support/localrunner.py | 33 ++
salt/cli/support/profiles/__init__.py | 4 +
salt/cli/support/profiles/default.yml | 78 +++
salt/cli/support/profiles/jobs-active.yml | 3 +
salt/cli/support/profiles/jobs-last.yml | 3 +
salt/cli/support/profiles/jobs-trace.yml | 7 +
salt/cli/support/profiles/network.yml | 27 ++
salt/cli/support/profiles/postgres.yml | 11 +
salt/cli/support/profiles/salt.yml | 9 +
salt/cli/support/profiles/users.yml | 22 +
salt/loader/lazy.py | 6 +-
salt/modules/saltsupport.py | 405 ++++++++++++++++
salt/scripts.py | 15 +
salt/state.py | 38 +-
salt/states/saltsupport.py | 225 +++++++++
salt/utils/args.py | 3 +-
salt/utils/decorators/__init__.py | 24 +
salt/utils/parsers.py | 114 +++++
scripts/salt-support | 11 +
setup.py | 2 +
tests/pytests/unit/cli/test_support.py | 553 +++++++++++++++++++++
tests/unit/modules/test_saltsupport.py | 496 +++++++++++++++++++
28 files changed, 2958 insertions(+), 7 deletions(-)
create mode 100644 salt/cli/support/__init__.py
create mode 100644 salt/cli/support/collector.py
create mode 100644 salt/cli/support/console.py
create mode 100644 salt/cli/support/intfunc.py
create mode 100644 salt/cli/support/localrunner.py
create mode 100644 salt/cli/support/profiles/__init__.py
create mode 100644 salt/cli/support/profiles/default.yml
create mode 100644 salt/cli/support/profiles/jobs-active.yml
create mode 100644 salt/cli/support/profiles/jobs-last.yml
create mode 100644 salt/cli/support/profiles/jobs-trace.yml
create mode 100644 salt/cli/support/profiles/network.yml
create mode 100644 salt/cli/support/profiles/postgres.yml
create mode 100644 salt/cli/support/profiles/salt.yml
create mode 100644 salt/cli/support/profiles/users.yml
create mode 100644 salt/modules/saltsupport.py
create mode 100644 salt/states/saltsupport.py
create mode 100755 scripts/salt-support
create mode 100644 tests/pytests/unit/cli/test_support.py
create mode 100644 tests/unit/modules/test_saltsupport.py
diff --git a/doc/ref/modules/all/index.rst b/doc/ref/modules/all/index.rst
index cbd8b0cdc5..abd40e0bc7 100644
--- a/doc/ref/modules/all/index.rst
+++ b/doc/ref/modules/all/index.rst
@@ -416,6 +416,7 @@ execution modules
salt_version
saltcheck
saltcloudmod
+ saltsupport
saltutil
schedule
scp_mod
diff --git a/doc/ref/states/all/index.rst b/doc/ref/states/all/index.rst
index 13ff645b59..7a062c227b 100644
--- a/doc/ref/states/all/index.rst
+++ b/doc/ref/states/all/index.rst
@@ -283,6 +283,7 @@ state modules
rvm
salt_proxy
saltmod
+ saltsupport
saltutil
schedule
selinux
diff --git a/salt/cli/support/__init__.py b/salt/cli/support/__init__.py
new file mode 100644
index 0000000000..59c2609e07
--- /dev/null
+++ b/salt/cli/support/__init__.py
@@ -0,0 +1,76 @@
+"""
+Get default scenario of the support.
+"""
+import logging
+import os
+
+import jinja2
+import salt.exceptions
+import yaml
+
+log = logging.getLogger(__name__)
+
+
+def _render_profile(path, caller, runner):
+ """
+ Render profile as Jinja2.
+ :param path:
+ :return:
+ """
+ env = jinja2.Environment(
+ loader=jinja2.FileSystemLoader(os.path.dirname(path)), trim_blocks=False
+ )
+ return (
+ env.get_template(os.path.basename(path))
+ .render(salt=caller, runners=runner)
+ .strip()
+ )
+
+
+def get_profile(profile, caller, runner):
+ """
+ Get profile.
+
+ :param profile:
+ :return:
+ """
+ profiles = profile.split(",")
+ data = {}
+ for profile in profiles:
+ if os.path.basename(profile) == profile:
+ profile = profile.split(".")[0] # Trim extension if someone added it
+ profile_path = os.path.join(
+ os.path.dirname(__file__), "profiles", profile + ".yml"
+ )
+ else:
+ profile_path = profile
+ if os.path.exists(profile_path):
+ try:
+ rendered_template = _render_profile(profile_path, caller, runner)
+ log.debug("\n{d}\n{t}\n{d}\n".format(d="-" * 80, t=rendered_template))
+ data.update(yaml.load(rendered_template))
+ except Exception as ex:
+ log.debug(ex, exc_info=True)
+ raise salt.exceptions.SaltException(
+ "Rendering profile failed: {}".format(ex)
+ )
+ else:
+ raise salt.exceptions.SaltException(
+ 'Profile "{}" is not found.'.format(profile)
+ )
+
+ return data
+
+
+def get_profiles(config):
+ """
+ Get available profiles.
+
+ :return:
+ """
+ profiles = []
+ for profile_name in os.listdir(os.path.join(os.path.dirname(__file__), "profiles")):
+ if profile_name.endswith(".yml"):
+ profiles.append(profile_name.split(".")[0])
+
+ return sorted(profiles)
diff --git a/salt/cli/support/collector.py b/salt/cli/support/collector.py
new file mode 100644
index 0000000000..1879cc5220
--- /dev/null
+++ b/salt/cli/support/collector.py
@@ -0,0 +1,563 @@
+import builtins as exceptions
+import copy
+import json
+import logging
+import os
+import sys
+import tarfile
+import time
+from io import BytesIO
+from io import IOBase as file
+
+import salt.cli.caller
+import salt.cli.support
+import salt.cli.support.console
+import salt.cli.support.intfunc
+import salt.cli.support.localrunner
+import salt.defaults.exitcodes
+import salt.exceptions
+import salt.ext.six as six
+import salt.output.table_out
+import salt.runner
+import salt.utils.files
+import salt.utils.parsers
+import salt.utils.platform
+import salt.utils.process
+import salt.utils.stringutils
+import salt.utils.verify
+import yaml
+
+salt.output.table_out.__opts__ = {}
+log = logging.getLogger(__name__)
+
+
+class SupportDataCollector:
+ """
+ Data collector. It behaves just like another outputter,
+ except it grabs the data to the archive files.
+ """
+
+ def __init__(self, name, output):
+ """
+ constructor of the data collector
+ :param name:
+ :param path:
+ :param format:
+ """
+ self.archive_path = name
+ self.__default_outputter = output
+ self.__format = format
+ self.__arch = None
+ self.__current_section = None
+ self.__current_section_name = None
+ self.__default_root = time.strftime("%Y.%m.%d-%H.%M.%S-snapshot")
+ self.out = salt.cli.support.console.MessagesOutput()
+
+ def open(self):
+ """
+ Opens archive.
+ :return:
+ """
+ if self.__arch is not None:
+ raise salt.exceptions.SaltException("Archive already opened.")
+ self.__arch = tarfile.TarFile.bz2open(self.archive_path, "w")
+
+ def close(self):
+ """
+ Closes the archive.
+ :return:
+ """
+ if self.__arch is None:
+ raise salt.exceptions.SaltException("Archive already closed")
+ self._flush_content()
+ self.__arch.close()
+ self.__arch = None
+
+ def _flush_content(self):
+ """
+ Flush content to the archive
+ :return:
+ """
+ if self.__current_section is not None:
+ buff = BytesIO()
+ buff._dirty = False
+ for action_return in self.__current_section:
+ for title, ret_data in action_return.items():
+ if isinstance(ret_data, file):
+ self.out.put(ret_data.name, indent=4)
+ self.__arch.add(ret_data.name, arcname=ret_data.name)
+ else:
+ buff.write(salt.utils.stringutils.to_bytes(title + "\n"))
+ buff.write(
+ salt.utils.stringutils.to_bytes(("-" * len(title)) + "\n\n")
+ )
+ buff.write(salt.utils.stringutils.to_bytes(ret_data))
+ buff.write(salt.utils.stringutils.to_bytes("\n\n\n"))
+ buff._dirty = True
+ if buff._dirty:
+ buff.seek(0)
+ tar_info = tarfile.TarInfo(
+ name="{}/{}".format(
+ self.__default_root, self.__current_section_name
+ )
+ )
+ if not hasattr(buff, "getbuffer"): # Py2's BytesIO is older
+ buff.getbuffer = buff.getvalue
+ tar_info.size = len(buff.getbuffer())
+ self.__arch.addfile(tarinfo=tar_info, fileobj=buff)
+
+ def add(self, name):
+ """
+ Start a new section.
+ :param name:
+ :return:
+ """
+ if self.__current_section:
+ self._flush_content()
+ self.discard_current(name)
+
+ def discard_current(self, name=None):
+ """
+ Discard current section
+ :return:
+ """
+ self.__current_section = []
+ self.__current_section_name = name
+
+ def _printout(self, data, output):
+ """
+ Use salt outputter to printout content.
+
+ :return:
+ """
+ opts = {"extension_modules": "", "color": False}
+ try:
+ printout = salt.output.get_printout(output, opts)(data)
+ if printout is not None:
+ return printout.rstrip()
+ except (KeyError, AttributeError, TypeError) as err:
+ log.debug(err, exc_info=True)
+ try:
+ printout = salt.output.get_printout("nested", opts)(data)
+ if printout is not None:
+ return printout.rstrip()
+ except (KeyError, AttributeError, TypeError) as err:
+ log.debug(err, exc_info=True)
+ printout = salt.output.get_printout("raw", opts)(data)
+ if printout is not None:
+ return printout.rstrip()
+
+ return salt.output.try_printout(data, output, opts)
+
+ def write(self, title, data, output=None):
+ """
+ Add a data to the current opened section.
+ :return:
+ """
+ if not isinstance(data, (dict, list, tuple)):
+ data = {"raw-content": str(data)}
+ output = output or self.__default_outputter
+
+ if output != "null":
+ try:
+ if isinstance(data, dict) and "return" in data:
+ data = data["return"]
+ content = self._printout(data, output)
+ except Exception: # Fall-back to just raw YAML
+ content = None
+ else:
+ content = None
+
+ if content is None:
+ data = json.loads(json.dumps(data))
+ if isinstance(data, dict) and data.get("return"):
+ data = data.get("return")
+ content = yaml.safe_dump(data, default_flow_style=False, indent=4)
+
+ self.__current_section.append({title: content})
+
+ def link(self, title, path):
+ """
+ Add a static file on the file system.
+
+ :param title:
+ :param path:
+ :return:
+ """
+ # The filehandler needs to be explicitly passed here, so PyLint needs to accept that.
+ # pylint: disable=W8470
+ if not isinstance(path, file):
+ path = salt.utils.files.fopen(path)
+ self.__current_section.append({title: path})
+ # pylint: enable=W8470
+
+
+class SaltSupport(salt.utils.parsers.SaltSupportOptionParser):
+ """
+ Class to run Salt Support subsystem.
+ """
+
+ RUNNER_TYPE = "run"
+ CALL_TYPE = "call"
+
+ def _setup_fun_config(self, fun_conf):
+ """
+ Setup function configuration.
+
+ :param conf:
+ :return:
+ """
+ conf = copy.deepcopy(self.config)
+ conf["file_client"] = "local"
+ conf["fun"] = ""
+ conf["arg"] = []
+ conf["kwarg"] = {}
+ conf["cache_jobs"] = False
+ conf["print_metadata"] = False
+ conf.update(fun_conf)
+ conf["fun"] = conf["fun"].split(":")[-1] # Discard typing prefix
+
+ return conf
+
+ def _get_runner(self, conf):
+ """
+ Get & setup runner.
+
+ :param conf:
+ :return:
+ """
+ conf = self._setup_fun_config(copy.deepcopy(conf))
+ if not getattr(self, "_runner", None):
+ self._runner = salt.cli.support.localrunner.LocalRunner(conf)
+ else:
+ self._runner.opts = conf
+ return self._runner
+
+ def _get_caller(self, conf):
+ """
+ Get & setup caller from the factory.
+
+ :param conf:
+ :return:
+ """
+ conf = self._setup_fun_config(copy.deepcopy(conf))
+ if not getattr(self, "_caller", None):
+ self._caller = salt.cli.caller.Caller.factory(conf)
+ else:
+ self._caller.opts = conf
+ return self._caller
+
+ def _local_call(self, call_conf):
+ """
+ Execute local call
+ """
+ try:
+ ret = self._get_caller(call_conf).call()
+ except SystemExit:
+ ret = "Data is not available at this moment"
+ self.out.error(ret)
+ except Exception as ex:
+ ret = "Unhandled exception occurred: {}".format(ex)
+ log.debug(ex, exc_info=True)
+ self.out.error(ret)
+
+ return ret
+
+ def _local_run(self, run_conf):
+ """
+ Execute local runner
+
+ :param run_conf:
+ :return:
+ """
+ try:
+ ret = self._get_runner(run_conf).run()
+ except SystemExit:
+ ret = "Runner is not available at this moment"
+ self.out.error(ret)
+ except Exception as ex:
+ ret = "Unhandled exception occurred: {}".format(ex)
+ log.debug(ex, exc_info=True)
+
+ return ret
+
+ def _internal_function_call(self, call_conf):
+ """
+ Call internal function.
+
+ :param call_conf:
+ :return:
+ """
+
+ def stub(*args, **kwargs):
+ message = "Function {} is not available".format(call_conf["fun"])
+ self.out.error(message)
+ log.debug(
+ 'Attempt to run "{fun}" with {arg} arguments and {kwargs} parameters.'.format(
+ **call_conf
+ )
+ )
+ return message
+
+ return getattr(salt.cli.support.intfunc, call_conf["fun"], stub)(
+ self.collector, *call_conf["arg"], **call_conf["kwargs"]
+ )
+
+ def _get_action(self, action_meta):
+ """
+ Parse action and turn into a calling point.
+ :param action_meta:
+ :return:
+ """
+ conf = {
+ "fun": list(action_meta.keys())[0],
+ "arg": [],
+ "kwargs": {},
+ }
+ if not len(conf["fun"].split(".")) - 1:
+ conf["salt.int.intfunc"] = True
+
+ action_meta = action_meta[conf["fun"]]
+ info = action_meta.get("info", "Action for {}".format(conf["fun"]))
+ for arg in action_meta.get("args") or []:
+ if not isinstance(arg, dict):
+ conf["arg"].append(arg)
+ else:
+ conf["kwargs"].update(arg)
+
+ return info, action_meta.get("output"), conf
+
+ def collect_internal_data(self):
+ """
+ Dumps current running pillars, configuration etc.
+ :return:
+ """
+ section = "configuration"
+ self.out.put(section)
+ self.collector.add(section)
+ self.out.put("Saving config", indent=2)
+ self.collector.write("General Configuration", self.config)
+ self.out.put("Saving pillars", indent=2)
+ self.collector.write(
+ "Active Pillars", self._local_call({"fun": "pillar.items"})
+ )
+
+ section = "highstate"
+ self.out.put(section)
+ self.collector.add(section)
+ self.out.put("Saving highstate", indent=2)
+ self.collector.write(
+ "Rendered highstate", self._local_call({"fun": "state.show_highstate"})
+ )
+
+ def _extract_return(self, data):
+ """
+ Extracts return data from the results.
+
+ :param data:
+ :return:
+ """
+ if isinstance(data, dict):
+ data = data.get("return", data)
+
+ return data
+
+ def collect_local_data(self, profile=None, profile_source=None):
+ """
+ Collects master system data.
+ :return:
+ """
+
+ def call(func, *args, **kwargs):
+ """
+ Call wrapper for templates
+ :param func:
+ :return:
+ """
+ return self._extract_return(
+ self._local_call({"fun": func, "arg": args, "kwarg": kwargs})
+ )
+
+ def run(func, *args, **kwargs):
+ """
+ Runner wrapper for templates
+ :param func:
+ :return:
+ """
+ return self._extract_return(
+ self._local_run({"fun": func, "arg": args, "kwarg": kwargs})
+ )
+
+ scenario = profile_source or salt.cli.support.get_profile(
+ profile or self.config["support_profile"], call, run
+ )
+ for category_name in scenario:
+ self.out.put(category_name)
+ self.collector.add(category_name)
+ for action in scenario[category_name]:
+ if not action:
+ continue
+ action_name = next(iter(action))
+ if not isinstance(action[action_name], str):
+ info, output, conf = self._get_action(action)
+ action_type = self._get_action_type(
+ action
+ ) # run:<something> for runners
+ if action_type == self.RUNNER_TYPE:
+ self.out.put("Running {}".format(info.lower()), indent=2)
+ self.collector.write(info, self._local_run(conf), output=output)
+ elif action_type == self.CALL_TYPE:
+ if not conf.get("salt.int.intfunc"):
+ self.out.put("Collecting {}".format(info.lower()), indent=2)
+ self.collector.write(
+ info, self._local_call(conf), output=output
+ )
+ else:
+ self.collector.discard_current()
+ self._internal_function_call(conf)
+ else:
+ self.out.error(
+ 'Unknown action type "{}" for action: {}'.format(
+ action_type, action
+ )
+ )
+ else:
+ # TODO: This needs to be moved then to the utils.
+ # But the code is not yet there (other PRs)
+ self.out.msg(
+ "\n".join(salt.cli.support.console.wrap(action[action_name])),
+ ident=2,
+ )
+
+ def _get_action_type(self, action):
+ """
+ Get action type.
+ :param action:
+ :return:
+ """
+ action_name = next(iter(action or {"": None}))
+ if ":" not in action_name:
+ action_name = "{}:{}".format(self.CALL_TYPE, action_name)
+
+ return action_name.split(":")[0] or None
+
+ def _cleanup(self):
+ """
+ Cleanup if crash/exception
+ :return:
+ """
+ if (
+ hasattr(self, "config")
+ and self.config.get("support_archive")
+ and os.path.exists(self.config["support_archive"])
+ ):
+ self.out.warning("Terminated earlier, cleaning up")
+ try:
+ os.unlink(self.config["support_archive"])
+ except Exception as err:
+ log.debug(err)
+ self.out.error("{} while cleaning up.".format(err))
+
+ def _check_existing_archive(self):
+ """
+ Check if archive exists or not. If exists and --force was not specified,
+ bail out. Otherwise remove it and move on.
+
+ :return:
+ """
+ if os.path.exists(self.config["support_archive"]):
+ if self.config["support_archive_force_overwrite"]:
+ self.out.warning(
+ "Overwriting existing archive: {}".format(
+ self.config["support_archive"]
+ )
+ )
+ try:
+ os.unlink(self.config["support_archive"])
+ except Exception as err:
+ log.debug(err)
+ self.out.error(
+ "{} while trying to overwrite existing archive.".format(err)
+ )
+ ret = True
+ else:
+ self.out.warning(
+ "File {} already exists.".format(self.config["support_archive"])
+ )
+ ret = False
+ else:
+ ret = True
+
+ return ret
+
+ def run(self):
+ exit_code = salt.defaults.exitcodes.EX_OK
+ self.out = salt.cli.support.console.MessagesOutput()
+ try:
+ self.parse_args()
+ except (Exception, SystemExit) as ex:
+ if not isinstance(ex, exceptions.SystemExit):
+ exit_code = salt.defaults.exitcodes.EX_GENERIC
+ self.out.error(ex)
+ elif isinstance(ex, exceptions.SystemExit):
+ exit_code = ex.code
+ else:
+ exit_code = salt.defaults.exitcodes.EX_GENERIC
+ self.out.error(ex)
+ else:
+ if self.config["log_level"] not in ("quiet",):
+ self.setup_logfile_logger()
+ salt.utils.verify.verify_log(self.config)
+ salt.cli.support.log = log # Pass update logger so trace is available
+
+ if self.config["support_profile_list"]:
+ self.out.put("List of available profiles:")
+ for idx, profile in enumerate(
+ salt.cli.support.get_profiles(self.config)
+ ):
+ msg_template = " {}. ".format(idx + 1) + "{}"
+ self.out.highlight(msg_template, profile)
+ exit_code = salt.defaults.exitcodes.EX_OK
+ elif self.config["support_show_units"]:
+ self.out.put("List of available units:")
+ for idx, unit in enumerate(self.find_existing_configs(None)):
+ msg_template = " {}. ".format(idx + 1) + "{}"
+ self.out.highlight(msg_template, unit)
+ exit_code = salt.defaults.exitcodes.EX_OK
+ else:
+ if not self.config["support_profile"]:
+ self.print_help()
+ raise SystemExit()
+
+ if self._check_existing_archive():
+ try:
+ self.collector = SupportDataCollector(
+ self.config["support_archive"],
+ output=self.config["support_output_format"],
+ )
+ except Exception as ex:
+ self.out.error(ex)
+ exit_code = salt.defaults.exitcodes.EX_GENERIC
+ log.debug(ex, exc_info=True)
+ else:
+ try:
+ self.collector.open()
+ self.collect_local_data()
+ self.collect_internal_data()
+ self.collector.close()
+
+ archive_path = self.collector.archive_path
+ self.out.highlight(
+ '\nSupport data has been written to "{}" file.\n',
+ archive_path,
+ _main="YELLOW",
+ )
+ except Exception as ex:
+ self.out.error(ex)
+ log.debug(ex, exc_info=True)
+ exit_code = salt.defaults.exitcodes.EX_SOFTWARE
+
+ if exit_code:
+ self._cleanup()
+
+ sys.exit(exit_code)
diff --git a/salt/cli/support/console.py b/salt/cli/support/console.py
new file mode 100644
index 0000000000..266b645479
--- /dev/null
+++ b/salt/cli/support/console.py
@@ -0,0 +1,184 @@
+"""
+Collection of tools to report messages to console.
+
+NOTE: This is subject to incorporate other formatting bits
+ from all around everywhere and then to be moved to utils.
+"""
+
+
+import os
+import sys
+import textwrap
+
+import salt.utils.color
+
+
+class IndentOutput:
+ """
+ Paint different indends in different output.
+ """
+
+ def __init__(self, conf=None, device=sys.stdout):
+ if conf is None:
+ conf = {0: "CYAN", 2: "GREEN", 4: "LIGHT_BLUE", 6: "BLUE"}
+ self._colors_conf = conf
+ self._device = device
+ self._colors = salt.utils.color.get_colors()
+ self._default_color = "GREEN"
+ self._default_hl_color = "LIGHT_GREEN"
+
+ def put(self, message, indent=0):
+ """
+ Print message with an indent.
+
+ :param message:
+ :param indent:
+ :return:
+ """
+ color = self._colors_conf.get(
+ indent + indent % 2, self._colors_conf.get(0, self._default_color)
+ )
+
+ for chunk in [" " * indent, self._colors[color], message, self._colors["ENDC"]]:
+ self._device.write(str(chunk))
+ self._device.write(os.linesep)
+ self._device.flush()
+
+
+class MessagesOutput(IndentOutput):
+ """
+ Messages output to the CLI.
+ """
+
+ def msg(self, message, title=None, title_color=None, color="BLUE", ident=0):
+ """
+ Hint message.
+
+ :param message:
+ :param title:
+ :param title_color:
+ :param color:
+ :param ident:
+ :return:
+ """
+ if title and not title_color:
+ title_color = color
+ if title_color and not title:
+ title_color = None
+
+ self.__colored_output(title, message, title_color, color, ident=ident)
+
+ def info(self, message, ident=0):
+ """
+ Write an info message to the CLI.
+
+ :param message:
+ :param ident:
+ :return:
+ """
+ self.__colored_output("Info", message, "GREEN", "LIGHT_GREEN", ident=ident)
+
+ def warning(self, message, ident=0):
+ """
+ Write a warning message to the CLI.
+
+ :param message:
+ :param ident:
+ :return:
+ """
+ self.__colored_output("Warning", message, "YELLOW", "LIGHT_YELLOW", ident=ident)
+
+ def error(self, message, ident=0):
+ """
+ Write an error message to the CLI.
+
+ :param message:
+ :param ident
+ :return:
+ """
+ self.__colored_output("Error", message, "RED", "LIGHT_RED", ident=ident)
+
+ def __colored_output(self, title, message, title_color, message_color, ident=0):
+ if title and not title.endswith(":"):
+ _linesep = title.endswith(os.linesep)
+ title = "{}:{}".format(title.strip(), _linesep and os.linesep or " ")
+
+ for chunk in [
+ title_color and self._colors[title_color] or None,
+ " " * ident,
+ title,
+ self._colors[message_color],
+ message,
+ self._colors["ENDC"],
+ ]:
+ if chunk:
+ self._device.write(str(chunk))
+ self._device.write(os.linesep)
+ self._device.flush()
+
+ def highlight(self, message, *values, **colors):
+ """
+ Highlighter works the way that message parameter is a template,
+ the "values" is a list of arguments going one after another as values there.
+ And so the "colors" should designate either highlight color or alternate for each.
+
+ Example:
+
+ highlight('Hello {}, there! It is {}.', 'user', 'daytime', _main='GREEN', _highlight='RED')
+ highlight('Hello {}, there! It is {}.', 'user', 'daytime', _main='GREEN', _highlight='RED', 'daytime'='YELLOW')
+
+ First example will highlight all the values in the template with the red color.
+ Second example will highlight the second value with the yellow color.
+
+ Usage:
+
+ colors:
+ _main: Sets the main color (or default is used)
+ _highlight: Sets the alternative color for everything
+ 'any phrase' that is the same in the "values" can override color.
+
+ :param message:
+ :param formatted:
+ :param colors:
+ :return:
+ """
+
+ m_color = colors.get("_main", self._default_color)
+ h_color = colors.get("_highlight", self._default_hl_color)
+
+ _values = []
+ for value in values:
+ _values.append(
+ "{p}{c}{r}".format(
+ p=self._colors[colors.get(value, h_color)],
+ c=value,
+ r=self._colors[m_color],
+ )
+ )
+ self._device.write(
+ "{s}{m}{e}".format(
+ s=self._colors[m_color],
+ m=message.format(*_values),
+ e=self._colors["ENDC"],
+ )
+ )
+ self._device.write(os.linesep)
+ self._device.flush()
+
+
+def wrap(txt, width=80, ident=0):
+ """
+ Wrap text to the required dimensions and clean it up, prepare for display.
+
+ :param txt:
+ :param width:
+ :return:
+ """
+ ident = " " * ident
+ txt = (txt or "").replace(os.linesep, " ").strip()
+
+ wrapper = textwrap.TextWrapper()
+ wrapper.fix_sentence_endings = False
+ wrapper.initial_indent = wrapper.subsequent_indent = ident
+
+ return wrapper.wrap(txt)
diff --git a/salt/cli/support/intfunc.py b/salt/cli/support/intfunc.py
new file mode 100644
index 0000000000..a9f76a6003
--- /dev/null
+++ b/salt/cli/support/intfunc.py
@@ -0,0 +1,51 @@
+"""
+Internal functions.
+"""
+# Maybe this needs to be a modules in a future?
+
+import glob
+import os
+
+import salt.utils.files
+from salt.cli.support.console import MessagesOutput
+
+out = MessagesOutput()
+
+
+def filetree(collector, *paths):
+ """
+ Add all files in the tree. If the "path" is a file,
+ only that file will be added.
+
+ :param path: File or directory
+ :return:
+ """
+ _paths = []
+ # Unglob
+ for path in paths:
+ _paths += glob.glob(path)
+ for path in set(_paths):
+ if not path:
+ out.error("Path not defined", ident=2)
+ elif not os.path.exists(path):
+ out.warning("Path {} does not exists".format(path))
+ else:
+ # The filehandler needs to be explicitly passed here, so PyLint needs to accept that.
+ # pylint: disable=W8470
+ if os.path.isfile(path):
+ filename = os.path.basename(path)
+ try:
+ file_ref = salt.utils.files.fopen(path) # pylint: disable=W
+ out.put("Add {}".format(filename), indent=2)
+ collector.add(filename)
+ collector.link(title=path, path=file_ref)
+ except Exception as err:
+ out.error(err, ident=4)
+ # pylint: enable=W8470
+ else:
+ try:
+ for fname in os.listdir(path):
+ fname = os.path.join(path, fname)
+ filetree(collector, [fname])
+ except Exception as err:
+ out.error(err, ident=4)
diff --git a/salt/cli/support/localrunner.py b/salt/cli/support/localrunner.py
new file mode 100644
index 0000000000..ad10eda0b0
--- /dev/null
+++ b/salt/cli/support/localrunner.py
@@ -0,0 +1,33 @@
+"""
+Local Runner
+"""
+
+import logging
+
+import salt.runner
+import salt.utils.platform
+import salt.utils.process
+
+log = logging.getLogger(__name__)
+
+
+class LocalRunner(salt.runner.Runner):
+ """
+ Runner class that changes its default behaviour.
+ """
+
+ def _proc_function(self, fun, low, user, tag, jid, daemonize=True):
+ """
+ Same as original _proc_function in AsyncClientMixin,
+ except it calls "low" without firing a print event.
+ """
+ if daemonize and not salt.utils.platform.is_windows():
+ salt.log.setup.shutdown_multiprocessing_logging()
+ salt.utils.process.daemonize()
+ salt.log.setup.setup_multiprocessing_logging()
+
+ low["__jid__"] = jid
+ low["__user__"] = user
+ low["__tag__"] = tag
+
+ return self.low(fun, low, print_event=False, full_return=False)
diff --git a/salt/cli/support/profiles/__init__.py b/salt/cli/support/profiles/__init__.py
new file mode 100644
index 0000000000..b86aef30b8
--- /dev/null
+++ b/salt/cli/support/profiles/__init__.py
@@ -0,0 +1,4 @@
+# coding=utf-8
+'''
+Profiles for salt-support.
+'''
diff --git a/salt/cli/support/profiles/default.yml b/salt/cli/support/profiles/default.yml
new file mode 100644
index 0000000000..3defb5eef3
--- /dev/null
+++ b/salt/cli/support/profiles/default.yml
@@ -0,0 +1,78 @@
+sysinfo:
+ - description: |
+ Get the Salt grains of the current system.
+ - grains.items:
+ info: System grains
+
+packages:
+ - description: |
+ Fetch list of all the installed packages.
+ - pkg.list_pkgs:
+ info: Installed packages
+
+repositories:
+ - pkg.list_repos:
+ info: Available repositories
+
+upgrades:
+ - pkg.list_upgrades:
+ info: Possible upgrades
+
+## TODO: Some data here belongs elsewhere and also is duplicated
+status:
+ - status.version:
+ info: Status version
+ - status.cpuinfo:
+ info: CPU information
+ - status.cpustats:
+ info: CPU stats
+ - status.diskstats:
+ info: Disk stats
+ - status.loadavg:
+ info: Average load of the current system
+ - status.uptime:
+ info: Uptime of the machine
+ - status.meminfo:
+ info: Information about memory
+ - status.vmstats:
+ info: Virtual memory stats
+ - status.netdev:
+ info: Network device stats
+ - status.nproc:
+ info: Number of processing units available on this system
+ - status.procs:
+ info: Process data
+
+general-health:
+ - ps.boot_time:
+ info: System Boot Time
+ - ps.swap_memory:
+ info: Swap Memory
+ output: txt
+ - ps.cpu_times:
+ info: CPU times
+ - ps.disk_io_counters:
+ info: Disk IO counters
+ - ps.disk_partition_usage:
+ info: Disk partition usage
+ output: table
+ - ps.disk_partitions:
+ info: Disk partitions
+ output: table
+ - ps.top:
+ info: Top CPU consuming processes
+
+boot_log:
+ - filetree:
+ info: Collect boot logs
+ args:
+ - /var/log/boot.*
+
+system.log:
+ # This works on any file system object.
+ - filetree:
+ info: Add system log
+ args:
+ - /var/log/syslog
+ - /var/log/messages
+
diff --git a/salt/cli/support/profiles/jobs-active.yml b/salt/cli/support/profiles/jobs-active.yml
new file mode 100644
index 0000000000..508c54ece7
--- /dev/null
+++ b/salt/cli/support/profiles/jobs-active.yml
@@ -0,0 +1,3 @@
+jobs-active:
+ - run:jobs.active:
+ info: List of all actively running jobs
diff --git a/salt/cli/support/profiles/jobs-last.yml b/salt/cli/support/profiles/jobs-last.yml
new file mode 100644
index 0000000000..e3b719f552
--- /dev/null
+++ b/salt/cli/support/profiles/jobs-last.yml
@@ -0,0 +1,3 @@
+jobs-last:
+ - run:jobs.last_run:
+ info: List all detectable jobs and associated functions
diff --git a/salt/cli/support/profiles/jobs-trace.yml b/salt/cli/support/profiles/jobs-trace.yml
new file mode 100644
index 0000000000..00b28e0502
--- /dev/null
+++ b/salt/cli/support/profiles/jobs-trace.yml
@@ -0,0 +1,7 @@
+jobs-details:
+ {% for job in runners('jobs.list_jobs') %}
+ - run:jobs.list_job:
+ info: Details on JID {{job}}
+ args:
+ - {{job}}
+ {% endfor %}
diff --git a/salt/cli/support/profiles/network.yml b/salt/cli/support/profiles/network.yml
new file mode 100644
index 0000000000..268f02e61f
--- /dev/null
+++ b/salt/cli/support/profiles/network.yml
@@ -0,0 +1,27 @@
+network:
+ - network.get_hostname:
+ info: Hostname
+ output: txt
+ - network.get_fqdn:
+ info: FQDN
+ output: txt
+ - network.default_route:
+ info: Default route
+ output: table
+ - network.interfaces:
+ info: All the available interfaces
+ output: table
+ - network.subnets:
+ info: List of IPv4 subnets
+ - network.subnets6:
+ info: List of IPv6 subnets
+ - network.routes:
+ info: Network configured routes from routing tables
+ output: table
+ - network.netstat:
+ info: Information on open ports and states
+ output: table
+ - network.active_tcp:
+ info: All running TCP connections
+ - network.arp:
+ info: ARP table
diff --git a/salt/cli/support/profiles/postgres.yml b/salt/cli/support/profiles/postgres.yml
new file mode 100644
index 0000000000..2238752c7a
--- /dev/null
+++ b/salt/cli/support/profiles/postgres.yml
@@ -0,0 +1,11 @@
+system.log:
+ - filetree:
+ info: Add system log
+ args:
+ - /var/log/syslog
+
+etc/postgres:
+ - filetree:
+ info: Pick entire /etc/postgresql
+ args:
+ - /etc/postgresql
diff --git a/salt/cli/support/profiles/salt.yml b/salt/cli/support/profiles/salt.yml
new file mode 100644
index 0000000000..4b18d98870
--- /dev/null
+++ b/salt/cli/support/profiles/salt.yml
@@ -0,0 +1,9 @@
+sysinfo:
+ - grains.items:
+ info: System grains
+
+logfile:
+ - filetree:
+ info: Add current logfile
+ args:
+ - {{salt('config.get', 'log_file')}}
diff --git a/salt/cli/support/profiles/users.yml b/salt/cli/support/profiles/users.yml
new file mode 100644
index 0000000000..391acdb606
--- /dev/null
+++ b/salt/cli/support/profiles/users.yml
@@ -0,0 +1,22 @@
+all-users:
+ {%for uname in salt('user.list_users') %}
+ - user.info:
+ info: Information about "{{uname}}"
+ args:
+ - {{uname}}
+ - user.list_groups:
+ info: List groups for user "{{uname}}"
+ args:
+ - {{uname}}
+ - shadow.info:
+ info: Shadow information about user "{{uname}}"
+ args:
+ - {{uname}}
+ - cron.raw_cron:
+ info: Cron for user "{{uname}}"
+ args:
+ - {{uname}}
+ {%endfor%}
+ - group.getent:
+ info: List of all available groups
+ output: table
diff --git a/salt/loader/lazy.py b/salt/loader/lazy.py
index d319fe54b4..5de995d446 100644
--- a/salt/loader/lazy.py
+++ b/salt/loader/lazy.py
@@ -972,8 +972,10 @@ class LazyLoader(salt.utils.lazy.LazyDict):
mod_names = [module_name] + list(virtual_aliases)
for attr in funcs_to_load:
- if attr.startswith("_"):
- # private functions are skipped
+ if attr.startswith("_") and attr != "__call__":
+ # private functions are skipped,
+ # except __call__ which is default entrance
+ # for multi-function batch-like state syntax
continue
func = getattr(mod, attr)
if not inspect.isfunction(func) and not isinstance(func, functools.partial):
diff --git a/salt/modules/saltsupport.py b/salt/modules/saltsupport.py
new file mode 100644
index 0000000000..e800e3bf1f
--- /dev/null
+++ b/salt/modules/saltsupport.py
@@ -0,0 +1,405 @@
+#
+# Author: Bo Maryniuk <bo@suse.de>
+#
+# Copyright 2018 SUSE LLC
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+:codeauthor: :email:`Bo Maryniuk <bo@suse.de>`
+
+Module to run salt-support within Salt.
+"""
+# pylint: disable=W0231,W0221
+
+
+import datetime
+import logging
+import os
+import re
+import sys
+import tempfile
+import time
+
+import salt.cli.support
+import salt.cli.support.intfunc
+import salt.defaults.exitcodes
+import salt.exceptions
+import salt.utils.decorators
+import salt.utils.dictupdate
+import salt.utils.odict
+import salt.utils.path
+import salt.utils.stringutils
+from salt.cli.support.collector import SaltSupport, SupportDataCollector
+
+__virtualname__ = "support"
+log = logging.getLogger(__name__)
+
+
+class LogCollector:
+ """
+ Output collector.
+ """
+
+ INFO = "info"
+ WARNING = "warning"
+ ERROR = "error"
+
+ class MessagesList(list):
+ def append(self, obj):
+ list.append(
+ self,
+ "{} - {}".format(
+ datetime.datetime.utcnow().strftime("%T.%f")[:-3], obj
+ ),
+ )
+
+ __call__ = append
+
+ def __init__(self):
+ self.messages = {
+ self.INFO: self.MessagesList(),
+ self.WARNING: self.MessagesList(),
+ self.ERROR: self.MessagesList(),
+ }
+
+ def msg(self, message, *args, **kwargs):
+ title = kwargs.get("title")
+ if title:
+ message = "{}: {}".format(title, message)
+ self.messages[self.INFO](message)
+
+ def info(self, message, *args, **kwargs):
+ self.msg(message)
+
+ def warning(self, message, *args, **kwargs):
+ self.messages[self.WARNING](message)
+
+ def error(self, message, *args, **kwargs):
+ self.messages[self.ERROR](message)
+
+ def put(self, message, *args, **kwargs):
+ self.messages[self.INFO](message)
+
+ def highlight(self, message, *values, **kwargs):
+ self.msg(message.format(*values))
+
+
+class SaltSupportModule(SaltSupport):
+ """
+ Salt Support module class.
+ """
+
+ def __init__(self):
+ """
+ Constructor
+ """
+ self.config = self.setup_config()
+
+ def setup_config(self):
+ """
+ Return current configuration
+ :return:
+ """
+ return __opts__
+
+ def _get_archive_name(self, archname=None):
+ """
+ Create default archive name.
+
+ :return:
+ """
+ archname = re.sub("[^a-z0-9]", "", (archname or "").lower()) or "support"
+ for grain in ["fqdn", "host", "localhost", "nodename"]:
+ host = __grains__.get(grain)
+ if host:
+ break
+ if not host:
+ host = "localhost"
+
+ return os.path.join(
+ tempfile.gettempdir(),
+ "{hostname}-{archname}-{date}-{time}.bz2".format(
+ archname=archname,
+ hostname=host,
+ date=time.strftime("%Y%m%d"),
+ time=time.strftime("%H%M%S"),
+ ),
+ )
+
+ @salt.utils.decorators.external
+ def profiles(self):
+ """
+ Get list of profiles.
+
+ :return:
+ """
+ return {
+ "standard": salt.cli.support.get_profiles(self.config),
+ "custom": [],
+ }
+
+ @salt.utils.decorators.external
+ def archives(self):
+ """
+ Get list of existing archives.
+ :return:
+ """
+ arc_files = []
+ tmpdir = tempfile.gettempdir()
+ for filename in os.listdir(tmpdir):
+ mtc = re.match(r"\w+-\w+-\d+-\d+\.bz2", filename)
+ if mtc and len(filename) == mtc.span()[-1]:
+ arc_files.append(os.path.join(tmpdir, filename))
+
+ return arc_files
+
+ @salt.utils.decorators.external
+ def last_archive(self):
+ """
+ Get the last available archive
+ :return:
+ """
+ archives = {}
+ for archive in self.archives():
+ archives[int(archive.split(".")[0].split("-")[-1])] = archive
+
+ return archives and archives[max(archives)] or None
+
+ @salt.utils.decorators.external
+ def delete_archives(self, *archives):
+ """
+ Delete archives
+ :return:
+ """
+ # Remove paths
+ _archives = []
+ for archive in archives:
+ _archives.append(os.path.basename(archive))
+ archives = _archives[:]
+
+ ret = {"files": {}, "errors": {}}
+ for archive in self.archives():
+ arc_dir = os.path.dirname(archive)
+ archive = os.path.basename(archive)
+ if archives and archive in archives or not archives:
+ archive = os.path.join(arc_dir, archive)
+ try:
+ os.unlink(archive)
+ ret["files"][archive] = "removed"
+ except Exception as err:
+ ret["errors"][archive] = str(err)
+ ret["files"][archive] = "left"
+
+ return ret
+
+ def format_sync_stats(self, cnt):
+ """
+ Format stats of the sync output.
+
+ :param cnt:
+ :return:
+ """
+ stats = salt.utils.odict.OrderedDict()
+ if cnt.get("retcode") == salt.defaults.exitcodes.EX_OK:
+ for line in cnt.get("stdout", "").split(os.linesep):
+ line = line.split(": ")
+ if len(line) == 2:
+ stats[line[0].lower().replace(" ", "_")] = line[1]
+ cnt["transfer"] = stats
+ del cnt["stdout"]
+
+ # Remove empty
+ empty_sections = []
+ for section in cnt:
+ if not cnt[section] and section != "retcode":
+ empty_sections.append(section)
+ for section in empty_sections:
+ del cnt[section]
+
+ return cnt
+
+ @salt.utils.decorators.depends("rsync")
+ @salt.utils.decorators.external
+ def sync(self, group, name=None, host=None, location=None, move=False, all=False):
+ """
+ Sync the latest archive to the host on given location.
+
+ CLI Example:
+
+ .. code-block:: bash
+
+ salt '*' support.sync group=test
+ salt '*' support.sync group=test name=/tmp/myspecial-12345-67890.bz2
+ salt '*' support.sync group=test name=/tmp/myspecial-12345-67890.bz2 host=allmystuff.lan
+ salt '*' support.sync group=test name=/tmp/myspecial-12345-67890.bz2 host=allmystuff.lan location=/opt/
+
+ :param group: name of the local directory to which sync is going to put the result files
+ :param name: name of the archive. Latest, if not specified.
+ :param host: name of the destination host for rsync. Default is master, if not specified.
+ :param location: local destination directory, default temporary if not specified
+ :param move: move archive file[s]. Default is False.
+ :param all: work with all available archives. Default is False (i.e. latest available)
+
+ :return:
+ """
+ tfh, tfn = tempfile.mkstemp()
+ processed_archives = []
+ src_uri = uri = None
+
+ last_arc = self.last_archive()
+ if name:
+ archives = [name]
+ elif all:
+ archives = self.archives()
+ elif last_arc:
+ archives = [last_arc]
+ else:
+ archives = []
+
+ for name in archives:
+ err = None
+ if not name:
+ err = "No support archive has been defined."
+ elif not os.path.exists(name):
+ err = 'Support archive "{}" was not found'.format(name)
+ if err is not None:
+ log.error(err)
+ raise salt.exceptions.SaltInvocationError(err)
+
+ if not uri:
+ src_uri = os.path.dirname(name)
+ uri = "{host}:{loc}".format(
+ host=host or __opts__["master"],
+ loc=os.path.join(location or tempfile.gettempdir(), group),
+ )
+
+ os.write(tfh, salt.utils.stringutils.to_bytes(os.path.basename(name)))
+ os.write(tfh, salt.utils.stringutils.to_bytes(os.linesep))
+ processed_archives.append(name)
+ log.debug("Syncing {filename} to {uri}".format(filename=name, uri=uri))
+ os.close(tfh)
+
+ if not processed_archives:
+ raise salt.exceptions.SaltInvocationError("No archives found to transfer.")
+
+ ret = __salt__["rsync.rsync"](
+ src=src_uri,
+ dst=uri,
+ additional_opts=["--stats", "--files-from={}".format(tfn)],
+ )
+ ret["files"] = {}
+ for name in processed_archives:
+ if move:
+ salt.utils.dictupdate.update(ret, self.delete_archives(name))
+ log.debug("Deleting {filename}".format(filename=name))
+ ret["files"][name] = "moved"
+ else:
+ ret["files"][name] = "copied"
+
+ try:
+ os.unlink(tfn)
+ except OSError as err:
+ log.error(
+ "Cannot remove temporary rsync file {fn}: {err}".format(fn=tfn, err=err)
+ )
+
+ return self.format_sync_stats(ret)
+
+ @salt.utils.decorators.external
+ def run(self, profile="default", pillar=None, archive=None, output="nested"):
+ """
+ Run Salt Support on the minion.
+
+ profile
+ Set available profile name. Default is "default".
+
+ pillar
+ Set available profile from the pillars.
+
+ archive
+ Override archive name. Default is "support". This results to "hostname-support-YYYYMMDD-hhmmss.bz2".
+
+ output
+ Change the default outputter. Default is "nested".
+
+ CLI Example:
+
+ .. code-block:: bash
+
+ salt '*' support.run
+ salt '*' support.run profile=network
+ salt '*' support.run pillar=something_special
+ """
+
+ class outputswitch:
+ """
+ Output switcher on context
+ """
+
+ def __init__(self, output_device):
+ self._tmp_out = output_device
+ self._orig_out = None
+
+ def __enter__(self):
+ self._orig_out = salt.cli.support.intfunc.out
+ salt.cli.support.intfunc.out = self._tmp_out
+
+ def __exit__(self, *args):
+ salt.cli.support.intfunc.out = self._orig_out
+
+ self.out = LogCollector()
+ with outputswitch(self.out):
+ self.collector = SupportDataCollector(
+ archive or self._get_archive_name(archname=archive), output
+ )
+ self.collector.out = self.out
+ self.collector.open()
+ self.collect_local_data(
+ profile=profile, profile_source=__pillar__.get(pillar)
+ )
+ self.collect_internal_data()
+ self.collector.close()
+
+ return {"archive": self.collector.archive_path, "messages": self.out.messages}
+
+
+def __virtual__():
+ """
+ Set method references as module functions aliases
+ :return:
+ """
+ support = SaltSupportModule()
+
+ def _set_function(obj):
+ """
+ Create a Salt function for the SaltSupport class.
+ """
+
+ def _cmd(*args, **kwargs):
+ """
+ Call support method as a function from the Salt.
+ """
+ _kwargs = {}
+ for kw in kwargs:
+ if not kw.startswith("__"):
+ _kwargs[kw] = kwargs[kw]
+ return obj(*args, **_kwargs)
+
+ _cmd.__doc__ = obj.__doc__
+ return _cmd
+
+ for m_name in dir(support):
+ obj = getattr(support, m_name)
+ if getattr(obj, "external", False):
+ setattr(sys.modules[__name__], m_name, _set_function(obj))
+
+ return __virtualname__
diff --git a/salt/scripts.py b/salt/scripts.py
index 07393373c9..16b032af2e 100644
--- a/salt/scripts.py
+++ b/salt/scripts.py
@@ -622,3 +622,18 @@ def salt_pip():
] + _pip_args(sys.argv[1:], extras)
ret = subprocess.run(command, shell=False, check=False, env=env)
sys.exit(ret.returncode)
+
+
+def salt_support():
+ """
+ Run Salt Support that collects system data, logs etc for debug and support purposes.
+ :return:
+ """
+
+ import salt.cli.support.collector
+
+ if "" in sys.path:
+ sys.path.remove("")
+ client = salt.cli.support.collector.SaltSupport()
+ _install_signal_handlers(client)
+ client.run()
diff --git a/salt/state.py b/salt/state.py
index 868be2749e..8352a8defc 100644
--- a/salt/state.py
+++ b/salt/state.py
@@ -1671,7 +1671,9 @@ class State:
names = []
if state.startswith("__"):
continue
- chunk = {"state": state, "name": name}
+ chunk = OrderedDict()
+ chunk["state"] = state
+ chunk["name"] = name
if orchestration_jid is not None:
chunk["__orchestration_jid__"] = orchestration_jid
if "__sls__" in body:
@@ -2382,9 +2384,16 @@ class State:
else:
self.format_slots(cdata)
with salt.utils.files.set_umask(low.get("__umask__")):
- ret = self.states[cdata["full"]](
- *cdata["args"], **cdata["kwargs"]
- )
+ if cdata["full"].split(".")[-1] == "__call__":
+ # __call__ requires OrderedDict to preserve state order
+ # kwargs are also invalid overall
+ ret = self.states[cdata["full"]](
+ cdata["args"], module=None, state=cdata["kwargs"]
+ )
+ else:
+ ret = self.states[cdata["full"]](
+ *cdata["args"], **cdata["kwargs"]
+ )
self.states.inject_globals = {}
if "check_cmd" in low:
state_check_cmd = "{0[state]}.mod_run_check_cmd".format(low)
@@ -3489,10 +3498,31 @@ class State:
running.update(errors)
return running
+ def inject_default_call(self, high):
+ """
+ Sets .call function to a state, if not there.
+
+ :param high:
+ :return:
+ """
+ for chunk in high:
+ state = high[chunk]
+ for state_ref in state:
+ needs_default = True
+ for argset in state[state_ref]:
+ if isinstance(argset, str):
+ needs_default = False
+ break
+ if needs_default:
+ order = state[state_ref].pop(-1)
+ state[state_ref].append("__call__")
+ state[state_ref].append(order)
+
def call_high(self, high, orchestration_jid=None):
"""
Process a high data call and ensure the defined states.
"""
+ self.inject_default_call(high)
errors = []
# If there is extension data reconcile it
high, ext_errors = self.reconcile_extend(high)
diff --git a/salt/states/saltsupport.py b/salt/states/saltsupport.py
new file mode 100644
index 0000000000..fb0c9e0372
--- /dev/null
+++ b/salt/states/saltsupport.py
@@ -0,0 +1,225 @@
+#
+# Author: Bo Maryniuk <bo@suse.de>
+#
+# Copyright 2018 SUSE LLC
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+r"""
+:codeauthor: :email:`Bo Maryniuk <bo@suse.de>`
+
+Execution of Salt Support from within states
+============================================
+
+State to collect support data from the systems:
+
+.. code-block:: yaml
+
+ examine_my_systems:
+ support.taken:
+ - profile: default
+
+ support.collected:
+ - group: somewhere
+ - move: true
+
+"""
+import logging
+import os
+import tempfile
+
+import salt.exceptions
+
+# Import salt modules
+import salt.fileclient
+import salt.utils.decorators.path
+import salt.utils.odict
+
+log = logging.getLogger(__name__)
+__virtualname__ = "support"
+
+
+class SaltSupportState:
+ """
+ Salt-support.
+ """
+
+ EXPORTED = ["collected", "taken"]
+
+ def get_kwargs(self, data):
+ kwargs = {}
+ for keyset in data:
+ kwargs.update(keyset)
+
+ return kwargs
+
+ def __call__(self, state):
+ """
+ Call support.
+
+ :param args:
+ :param kwargs:
+ :return:
+ """
+ ret = {
+ "name": state.pop("name"),
+ "changes": {},
+ "result": True,
+ "comment": "",
+ }
+
+ out = {}
+ functions = ["Functions:"]
+ try:
+ for ref_func, ref_kwargs in state.items():
+ if ref_func not in self.EXPORTED:
+ raise salt.exceptions.SaltInvocationError(
+ "Function {} is not found".format(ref_func)
+ )
+ out[ref_func] = getattr(self, ref_func)(**self.get_kwargs(ref_kwargs))
+ functions.append(" - {}".format(ref_func))
+ ret["comment"] = "\n".join(functions)
+ except Exception as ex:
+ ret["comment"] = str(ex)
+ ret["result"] = False
+ ret["changes"] = out
+
+ return ret
+
+ def check_destination(self, location, group):
+ """
+ Check destination for the archives.
+ :return:
+ """
+ # Pre-create destination, since rsync will
+ # put one file named as group
+ try:
+ destination = os.path.join(location, group)
+ if os.path.exists(destination) and not os.path.isdir(destination):
+ raise salt.exceptions.SaltException(
+ 'Destination "{}" should be directory!'.format(destination)
+ )
+ if not os.path.exists(destination):
+ os.makedirs(destination)
+ log.debug("Created destination directory for archives: %s", destination)
+ else:
+ log.debug(
+ "Archives destination directory %s already exists", destination
+ )
+ except OSError as err:
+ log.error(err)
+
+ def collected(
+ self, group, filename=None, host=None, location=None, move=True, all=True
+ ):
+ """
+ Sync archives to a central place.
+
+ :param name:
+ :param group:
+ :param filename:
+ :param host:
+ :param location:
+ :param move:
+ :param all:
+ :return:
+ """
+ ret = {
+ "name": "support.collected",
+ "changes": {},
+ "result": True,
+ "comment": "",
+ }
+ location = location or tempfile.gettempdir()
+ self.check_destination(location, group)
+ ret["changes"] = __salt__["support.sync"](
+ group, name=filename, host=host, location=location, move=move, all=all
+ )
+
+ return ret
+
+ def taken(self, profile="default", pillar=None, archive=None, output="nested"):
+ """
+ Takes minion support config data.
+
+ :param profile:
+ :param pillar:
+ :param archive:
+ :param output:
+ :return:
+ """
+ ret = {
+ "name": "support.taken",
+ "changes": {},
+ "result": True,
+ }
+
+ result = __salt__["support.run"](
+ profile=profile, pillar=pillar, archive=archive, output=output
+ )
+ if result.get("archive"):
+ ret[
+ "comment"
+ ] = "Information about this system has been saved to {} file.".format(
+ result["archive"]
+ )
+ ret["changes"]["archive"] = result["archive"]
+ ret["changes"]["messages"] = {}
+ for key in ["info", "error", "warning"]:
+ if result.get("messages", {}).get(key):
+ ret["changes"]["messages"][key] = result["messages"][key]
+ else:
+ ret["comment"] = ""
+
+ return ret
+
+
+_support_state = SaltSupportState()
+
+
+def __call__(*args, **kwargs):
+ """
+ SLS single-ID syntax processing.
+
+ module:
+ This module reference, equals to sys.modules[__name__]
+
+ state:
+ Compiled state in preserved order. The function supposed to look
+ at first level array of functions.
+
+ :param cdata:
+ :param kwargs:
+ :return:
+ """
+ return _support_state(kwargs.get("state", {}))
+
+
+def taken(name, profile="default", pillar=None, archive=None, output="nested"):
+ return _support_state.taken(
+ profile=profile, pillar=pillar, archive=archive, output=output
+ )
+
+
+def collected(
+ name, group, filename=None, host=None, location=None, move=True, all=True
+):
+ return _support_state.collected(
+ group=group, filename=filename, host=host, location=location, move=move, all=all
+ )
+
+
+def __virtual__():
+ """
+ Salt Support state
+ """
+ return __virtualname__
diff --git a/salt/utils/args.py b/salt/utils/args.py
index 536aea3816..04a8a14054 100644
--- a/salt/utils/args.py
+++ b/salt/utils/args.py
@@ -15,6 +15,7 @@ import salt.utils.jid
import salt.utils.versions
import salt.utils.yaml
from salt.exceptions import SaltInvocationError
+from salt.utils.odict import OrderedDict
log = logging.getLogger(__name__)
@@ -399,7 +400,7 @@ def format_call(
ret = initial_ret is not None and initial_ret or {}
ret["args"] = []
- ret["kwargs"] = {}
+ ret["kwargs"] = OrderedDict()
aspec = get_function_argspec(fun, is_class_method=is_class_method)
diff --git a/salt/utils/decorators/__init__.py b/salt/utils/decorators/__init__.py
index 1f62d5f3d6..1906cc2ecc 100644
--- a/salt/utils/decorators/__init__.py
+++ b/salt/utils/decorators/__init__.py
@@ -866,3 +866,27 @@ def ensure_unicode_args(function):
return function(*args, **kwargs)
return wrapped
+
+
+def external(func):
+ """
+ Mark function as external.
+
+ :param func:
+ :return:
+ """
+
+ def f(*args, **kwargs):
+ """
+ Stub.
+
+ :param args:
+ :param kwargs:
+ :return:
+ """
+ return func(*args, **kwargs)
+
+ f.external = True
+ f.__doc__ = func.__doc__
+
+ return f
diff --git a/salt/utils/parsers.py b/salt/utils/parsers.py
index 911b2cbb04..dc125de7d7 100644
--- a/salt/utils/parsers.py
+++ b/salt/utils/parsers.py
@@ -17,6 +17,7 @@ import optparse
import os
import signal
import sys
+import tempfile
import traceback
import types
from functools import partial
@@ -31,6 +32,7 @@ import salt.utils.args
import salt.utils.data
import salt.utils.files
import salt.utils.jid
+import salt.utils.network
import salt.utils.platform
import salt.utils.process
import salt.utils.stringutils
@@ -2026,6 +2028,118 @@ class SyndicOptionParser(
return opts
+class SaltSupportOptionParser(
+ OptionParser,
+ ConfigDirMixIn,
+ MergeConfigMixIn,
+ LogLevelMixIn,
+ TimeoutMixIn,
+ metaclass=OptionParserMeta,
+):
+ default_timeout = 5
+ description = "Salt Support is a program to collect all support data: logs, system configuration etc."
+ usage = "%prog [options] '<target>' <function> [arguments]"
+ # ConfigDirMixIn config filename attribute
+ _config_filename_ = "master"
+
+ # LogLevelMixIn attributes
+ _default_logging_level_ = config.DEFAULT_MASTER_OPTS["log_level"]
+ _default_logging_logfile_ = config.DEFAULT_MASTER_OPTS["log_file"]
+
+ def _mixin_setup(self):
+ self.add_option(
+ "-P",
+ "--show-profiles",
+ default=False,
+ action="store_true",
+ dest="support_profile_list",
+ help="Show available profiles",
+ )
+ self.add_option(
+ "-p",
+ "--profile",
+ default="",
+ dest="support_profile",
+ help='Specify support profile or comma-separated profiles, e.g.: "salt,network"',
+ )
+ support_archive = "{t}/{h}-support.tar.bz2".format(
+ t=tempfile.gettempdir(), h=salt.utils.network.get_fqhostname()
+ )
+ self.add_option(
+ "-a",
+ "--archive",
+ default=support_archive,
+ dest="support_archive",
+ help=(
+ "Specify name of the resulting support archive. "
+ 'Default is "{f}".'.format(f=support_archive)
+ ),
+ )
+ self.add_option(
+ "-u",
+ "--unit",
+ default="",
+ dest="support_unit",
+ help='Specify examined unit (default "master").',
+ )
+ self.add_option(
+ "-U",
+ "--show-units",
+ default=False,
+ action="store_true",
+ dest="support_show_units",
+ help="Show available units",
+ )
+ self.add_option(
+ "-f",
+ "--force",
+ default=False,
+ action="store_true",
+ dest="support_archive_force_overwrite",
+ help="Force overwrite existing archive, if exists",
+ )
+ self.add_option(
+ "-o",
+ "--out",
+ default="null",
+ dest="support_output_format",
+ help=(
+ "Set the default output using the specified outputter, "
+ 'unless profile does not overrides this. Default: "yaml".'
+ ),
+ )
+
+ def find_existing_configs(self, default):
+ """
+ Find configuration files on the system.
+ :return:
+ """
+ configs = []
+ for cfg in [default, self._config_filename_, "minion", "proxy", "cloud", "spm"]:
+ if not cfg:
+ continue
+ config_path = self.get_config_file_path(cfg)
+ if os.path.exists(config_path):
+ configs.append(cfg)
+
+ if default and default not in configs:
+ raise SystemExit("Unknown configuration unit: {}".format(default))
+
+ return configs
+
+ def setup_config(self, cfg=None):
+ """
+ Open suitable config file.
+ :return:
+ """
+ _opts, _args = optparse.OptionParser.parse_args(self)
+ configs = self.find_existing_configs(_opts.support_unit)
+ if configs and cfg not in configs:
+ cfg = configs[0]
+
+ return config.master_config(self.get_config_file_path(cfg))
+
+
class SaltCMDOptionParser(
OptionParser,
ConfigDirMixIn,
diff --git a/scripts/salt-support b/scripts/salt-support
new file mode 100755
index 0000000000..4e0e79f3ea
--- /dev/null
+++ b/scripts/salt-support
@@ -0,0 +1,11 @@
+#!/usr/bin/env python
+"""
+Salt support is to collect logs,
+debug data and system information
+for support purposes.
+"""
+
+from salt.scripts import salt_support
+
+if __name__ == "__main__":
+ salt_support()
diff --git a/setup.py b/setup.py
index 931ed40a51..e60f1b7085 100755
--- a/setup.py
+++ b/setup.py
@@ -1061,6 +1061,7 @@ class SaltDistribution(distutils.dist.Distribution):
"scripts/salt-minion",
"scripts/salt-proxy",
"scripts/salt-run",
+ "scripts/salt-support",
"scripts/salt-ssh",
"scripts/salt-syndic",
"scripts/spm",
@@ -1109,6 +1110,7 @@ class SaltDistribution(distutils.dist.Distribution):
"salt-master = salt.scripts:salt_master",
"salt-minion = salt.scripts:salt_minion",
"salt-run = salt.scripts:salt_run",
+ "salt-support = salt.scripts:salt_support",
"salt-ssh = salt.scripts:salt_ssh",
"salt-syndic = salt.scripts:salt_syndic",
"spm = salt.scripts:salt_spm",
diff --git a/tests/pytests/unit/cli/test_support.py b/tests/pytests/unit/cli/test_support.py
new file mode 100644
index 0000000000..dc0e99bb3d
--- /dev/null
+++ b/tests/pytests/unit/cli/test_support.py
@@ -0,0 +1,553 @@
+"""
+ :codeauthor: Bo Maryniuk <bo@suse.de>
+"""
+
+
+import os
+
+import jinja2
+import salt.cli.support.collector
+import salt.exceptions
+import salt.utils.files
+import yaml
+from salt.cli.support.collector import SaltSupport, SupportDataCollector
+from salt.cli.support.console import IndentOutput
+from salt.utils.color import get_colors
+from salt.utils.stringutils import to_bytes
+from tests.support.mock import NO_MOCK, NO_MOCK_REASON, MagicMock, patch
+from tests.support.unit import TestCase, skipIf
+
+try:
+ import pytest
+except ImportError:
+ pytest = None
+
+
+@skipIf(not bool(pytest), "Pytest needs to be installed")
+@skipIf(NO_MOCK, NO_MOCK_REASON)
+class SaltSupportIndentOutputTestCase(TestCase):
+ """
+ Unit Tests for the salt-support indent output.
+ """
+
+ def setUp(self):
+ """
+ Setup test
+ :return:
+ """
+
+ self.message = "Stubborn processes on dumb terminal"
+ self.device = MagicMock()
+ self.iout = IndentOutput(device=self.device)
+ self.colors = get_colors()
+
+ def tearDown(self):
+ """
+ Remove instances after test run
+ :return:
+ """
+ del self.message
+ del self.device
+ del self.iout
+ del self.colors
+
+ def test_standard_output(self):
+ """
+ Test console standard output.
+ """
+ self.iout.put(self.message)
+ assert self.device.write.called
+ assert self.device.write.call_count == 5
+ for idx, data in enumerate(
+ ["", str(self.colors["CYAN"]), self.message, str(self.colors["ENDC"]), "\n"]
+ ):
+ assert self.device.write.call_args_list[idx][0][0] == data
+
+ def test_indent_output(self):
+ """
+ Test indent distance.
+ :return:
+ """
+ self.iout.put(self.message, indent=10)
+ for idx, data in enumerate(
+ [
+ " " * 10,
+ str(self.colors["CYAN"]),
+ self.message,
+ str(self.colors["ENDC"]),
+ "\n",
+ ]
+ ):
+ assert self.device.write.call_args_list[idx][0][0] == data
+
+ def test_color_config(self):
+ """
+ Test color config changes on each ident.
+ :return:
+ """
+
+ conf = {0: "MAGENTA", 2: "RED", 4: "WHITE", 6: "YELLOW"}
+ self.iout = IndentOutput(conf=conf, device=self.device)
+ for indent in sorted(list(conf)):
+ self.iout.put(self.message, indent=indent)
+
+ step = 1
+ for ident_key in sorted(list(conf)):
+ assert str(self.device.write.call_args_list[step][0][0]) == str(
+ self.colors[conf[ident_key]]
+ )
+ step += 5
+
+
+@skipIf(not bool(pytest), "Pytest needs to be installed")
+@skipIf(NO_MOCK, NO_MOCK_REASON)
+class SaltSupportCollectorTestCase(TestCase):
+ """
+ Collector tests.
+ """
+
+ def setUp(self):
+ """
+ Setup the test case
+ :return:
+ """
+ self.archive_path = "/highway/to/hell"
+ self.output_device = MagicMock()
+ self.collector = SupportDataCollector(self.archive_path, self.output_device)
+
+ def tearDown(self):
+ """
+ Tear down the test case elements
+ :return:
+ """
+ del self.collector
+ del self.archive_path
+ del self.output_device
+
+ @patch("salt.cli.support.collector.tarfile.TarFile", MagicMock())
+ def test_archive_open(self):
+ """
+ Test archive is opened.
+
+ :return:
+ """
+ self.collector.open()
+ assert self.collector.archive_path == self.archive_path
+ with pytest.raises(salt.exceptions.SaltException) as err:
+ self.collector.open()
+ assert "Archive already opened" in str(err)
+
+ @patch("salt.cli.support.collector.tarfile.TarFile", MagicMock())
+ def test_archive_close(self):
+ """
+ Test archive is opened.
+
+ :return:
+ """
+ self.collector.open()
+ self.collector._flush_content = lambda: None
+ self.collector.close()
+ assert self.collector.archive_path == self.archive_path
+ with pytest.raises(salt.exceptions.SaltException) as err:
+ self.collector.close()
+ assert "Archive already closed" in str(err)
+
+ def test_archive_addwrite(self):
+ """
+ Test add to the archive a section and write to it.
+
+ :return:
+ """
+ archive = MagicMock()
+ with patch("salt.cli.support.collector.tarfile.TarFile", archive):
+ self.collector.open()
+ self.collector.add("foo")
+ self.collector.write(title="title", data="data", output="null")
+ self.collector._flush_content()
+
+ assert archive.bz2open().addfile.call_args[1]["fileobj"].read() == to_bytes(
+ "title\n-----\n\nraw-content: data\n\n\n\n"
+ )
+
+ @patch("salt.utils.files.fopen", MagicMock(return_value="path=/dev/null"))
+ def test_archive_addlink(self):
+ """
+ Test add to the archive a section and link an external file or directory to it.
+
+ :return:
+ """
+ archive = MagicMock()
+ with patch("salt.cli.support.collector.tarfile.TarFile", archive):
+ self.collector.open()
+ self.collector.add("foo")
+ self.collector.link(title="Backup Path", path="/path/to/backup.config")
+ self.collector._flush_content()
+
+ assert archive.bz2open().addfile.call_count == 1
+ assert archive.bz2open().addfile.call_args[1]["fileobj"].read() == to_bytes(
+ "Backup Path\n-----------\n\npath=/dev/null\n\n\n"
+ )
+
+ @patch("salt.utils.files.fopen", MagicMock(return_value="path=/dev/null"))
+ def test_archive_discard_section(self):
+ """
+ Test discard a section from the archive.
+
+ :return:
+ """
+ archive = MagicMock()
+ with patch("salt.cli.support.collector.tarfile.TarFile", archive):
+ self.collector.open()
+ self.collector.add("solar-interference")
+ self.collector.link(
+ title="Thermal anomaly", path="/path/to/another/great.config"
+ )
+ self.collector.add("foo")
+ self.collector.link(title="Backup Path", path="/path/to/backup.config")
+ self.collector._flush_content()
+ assert archive.bz2open().addfile.call_count == 2
+ assert archive.bz2open().addfile.mock_calls[0][2][
+ "fileobj"
+ ].read() == to_bytes(
+ "Thermal anomaly\n---------------\n\npath=/dev/null\n\n\n"
+ )
+ self.collector.close()
+
+ archive = MagicMock()
+ with patch("salt.cli.support.collector.tarfile.TarFile", archive):
+ self.collector.open()
+ self.collector.add("solar-interference")
+ self.collector.link(
+ title="Thermal anomaly", path="/path/to/another/great.config"
+ )
+ self.collector.discard_current()
+ self.collector.add("foo")
+ self.collector.link(title="Backup Path", path="/path/to/backup.config")
+ self.collector._flush_content()
+ assert archive.bz2open().addfile.call_count == 2
+ assert archive.bz2open().addfile.mock_calls[0][2][
+ "fileobj"
+ ].read() == to_bytes("Backup Path\n-----------\n\npath=/dev/null\n\n\n")
+ self.collector.close()
+
+
+@skipIf(not bool(pytest), "Pytest needs to be installed")
+@skipIf(NO_MOCK, NO_MOCK_REASON)
+class SaltSupportRunnerTestCase(TestCase):
+ """
+ Test runner class.
+ """
+
+ def setUp(self):
+ """
+ Set up test suite.
+ :return:
+ """
+ self.archive_path = "/dev/null"
+ self.output_device = MagicMock()
+ self.runner = SaltSupport()
+ self.runner.collector = SupportDataCollector(
+ self.archive_path, self.output_device
+ )
+
+ def tearDown(self):
+ """
+ Tear down.
+
+ :return:
+ """
+ del self.archive_path
+ del self.output_device
+ del self.runner
+
+ def test_function_config(self):
+ """
+ Test function config formation.
+
+ :return:
+ """
+ self.runner.config = {}
+ msg = "Electromagnetic energy loss"
+ assert self.runner._setup_fun_config({"description": msg}) == {
+ "print_metadata": False,
+ "file_client": "local",
+ "fun": "",
+ "kwarg": {},
+ "description": msg,
+ "cache_jobs": False,
+ "arg": [],
+ }
+
+ def test_local_caller(self):
+ """
+ Test local caller.
+
+ :return:
+ """
+ msg = "Because of network lag due to too many people playing deathmatch"
+ caller = MagicMock()
+ caller().call = MagicMock(return_value=msg)
+
+ self.runner._get_caller = caller
+ self.runner.out = MagicMock()
+ assert self.runner._local_call({}) == msg
+
+ caller().call = MagicMock(side_effect=SystemExit)
+ assert self.runner._local_call({}) == "Data is not available at this moment"
+
+ err_msg = "The UPS doesn't have a battery backup."
+ caller().call = MagicMock(side_effect=Exception(err_msg))
+ assert (
+ self.runner._local_call({})
+ == "Unhandled exception occurred: The UPS doesn't have a battery backup."
+ )
+
+ def test_local_runner(self):
+ """
+ Test local runner.
+
+ :return:
+ """
+ msg = "Big to little endian conversion error"
+ runner = MagicMock()
+ runner().run = MagicMock(return_value=msg)
+
+ self.runner._get_runner = runner
+ self.runner.out = MagicMock()
+ assert self.runner._local_run({}) == msg
+
+ runner().run = MagicMock(side_effect=SystemExit)
+ assert self.runner._local_run({}) == "Runner is not available at this moment"
+
+ err_msg = "Trojan horse ran out of hay"
+ runner().run = MagicMock(side_effect=Exception(err_msg))
+ assert (
+ self.runner._local_run({})
+ == "Unhandled exception occurred: Trojan horse ran out of hay"
+ )
+
+ @patch("salt.cli.support.intfunc", MagicMock(spec=[]))
+ def test_internal_function_call_stub(self):
+ """
+ Test missing internal function call is handled accordingly.
+
+ :return:
+ """
+ self.runner.out = MagicMock()
+ out = self.runner._internal_function_call(
+ {"fun": "everythingisawesome", "arg": [], "kwargs": {}}
+ )
+ assert out == "Function everythingisawesome is not available"
+
+ def test_internal_function_call(self):
+ """
+ Test missing internal function call is handled accordingly.
+
+ :return:
+ """
+ msg = "Internet outage"
+ intfunc = MagicMock()
+ intfunc.everythingisawesome = MagicMock(return_value=msg)
+ self.runner.out = MagicMock()
+ with patch("salt.cli.support.intfunc", intfunc):
+ out = self.runner._internal_function_call(
+ {"fun": "everythingisawesome", "arg": [], "kwargs": {}}
+ )
+ assert out == msg
+
+ def test_get_action(self):
+ """
+ Test action meta gets parsed.
+
+ :return:
+ """
+ action_meta = {
+ "run:jobs.list_jobs_filter": {"info": "List jobs filter", "args": [1]}
+ }
+ assert self.runner._get_action(action_meta) == (
+ "List jobs filter",
+ None,
+ {"fun": "run:jobs.list_jobs_filter", "kwargs": {}, "arg": [1]},
+ )
+ action_meta = {
+ "user.info": {"info": 'Information about "usbmux"', "args": ["usbmux"]}
+ }
+ assert self.runner._get_action(action_meta) == (
+ 'Information about "usbmux"',
+ None,
+ {"fun": "user.info", "kwargs": {}, "arg": ["usbmux"]},
+ )
+
+ def test_extract_return(self):
+ """
+ Test extract return from the output.
+
+ :return:
+ """
+ out = {"key": "value"}
+ assert self.runner._extract_return(out) == out
+ assert self.runner._extract_return({"return": out}) == out
+
+ def test_get_action_type(self):
+ """
+ Test action meta determines action type.
+
+ :return:
+ """
+ action_meta = {
+ "run:jobs.list_jobs_filter": {"info": "List jobs filter", "args": [1]}
+ }
+ assert self.runner._get_action_type(action_meta) == "run"
+
+ action_meta = {
+ "user.info": {"info": 'Information about "usbmux"', "args": ["usbmux"]}
+ }
+ assert self.runner._get_action_type(action_meta) == "call"
+
+ @patch("os.path.exists", MagicMock(return_value=True))
+ def test_cleanup(self):
+ """
+ Test cleanup routine.
+
+ :return:
+ """
+ arch = "/tmp/killme.zip"
+ unlink = MagicMock()
+ with patch("os.unlink", unlink):
+ self.runner.config = {"support_archive": arch}
+ self.runner.out = MagicMock()
+ self.runner._cleanup()
+
+ assert (
+ self.runner.out.warning.call_args[0][0]
+ == "Terminated earlier, cleaning up"
+ )
+ unlink.assert_called_once_with(arch)
+
+ @patch("os.path.exists", MagicMock(return_value=True))
+ def test_check_existing_archive(self):
+ """
+ Test check existing archive.
+
+ :return:
+ """
+ arch = "/tmp/endothermal-recalibration.zip"
+ unlink = MagicMock()
+ with patch("os.unlink", unlink), patch(
+ "os.path.exists", MagicMock(return_value=False)
+ ):
+ self.runner.config = {
+ "support_archive": "",
+ "support_archive_force_overwrite": True,
+ }
+ self.runner.out = MagicMock()
+ assert self.runner._check_existing_archive()
+ assert self.runner.out.warning.call_count == 0
+
+ with patch("os.unlink", unlink):
+ self.runner.config = {
+ "support_archive": arch,
+ "support_archive_force_overwrite": False,
+ }
+ self.runner.out = MagicMock()
+ assert not self.runner._check_existing_archive()
+ assert self.runner.out.warning.call_args[0][
+ 0
+ ] == "File {} already exists.".format(arch)
+
+ with patch("os.unlink", unlink):
+ self.runner.config = {
+ "support_archive": arch,
+ "support_archive_force_overwrite": True,
+ }
+ self.runner.out = MagicMock()
+ assert self.runner._check_existing_archive()
+ assert self.runner.out.warning.call_args[0][
+ 0
+ ] == "Overwriting existing archive: {}".format(arch)
+
+
+@skipIf(not bool(pytest), "Pytest needs to be installed")
+@skipIf(NO_MOCK, NO_MOCK_REASON)
+class ProfileIntegrityTestCase(TestCase):
+ """
+ Default profile integrity
+ """
+
+ def setUp(self):
+ """
+ Set up test suite.
+
+ :return:
+ """
+ self.profiles = {}
+ profiles = os.path.join(
+ os.path.dirname(salt.cli.support.collector.__file__), "profiles"
+ )
+ for profile in os.listdir(profiles):
+ self.profiles[profile.split(".")[0]] = os.path.join(profiles, profile)
+
+ def tearDown(self):
+ """
+ Tear down test suite.
+
+ :return:
+ """
+ del self.profiles
+
+ def _render_template_to_yaml(self, name, *args, **kwargs):
+ """
+ Get template referene for rendering.
+ :return:
+ """
+ with salt.utils.files.fopen(self.profiles[name]) as t_fh:
+ template = t_fh.read()
+ return yaml.load(
+ jinja2.Environment().from_string(template).render(*args, **kwargs)
+ )
+
+ def test_non_template_profiles_parseable(self):
+ """
+ Test shipped default profile is YAML parse-able.
+
+ :return:
+ """
+ for t_name in ["default", "jobs-active", "jobs-last", "network", "postgres"]:
+ with salt.utils.files.fopen(self.profiles[t_name]) as ref:
+ try:
+ yaml.load(ref)
+ parsed = True
+ except Exception:
+ parsed = False
+ assert parsed
+
+ def test_users_template_profile(self):
+ """
+ Test users template profile.
+
+ :return:
+ """
+ users_data = self._render_template_to_yaml(
+ "users", salt=MagicMock(return_value=["pokemon"])
+ )
+ assert len(users_data["all-users"]) == 5
+ for user_data in users_data["all-users"]:
+ for tgt in ["user.list_groups", "shadow.info", "cron.raw_cron"]:
+ if tgt in user_data:
+ assert user_data[tgt]["args"] == ["pokemon"]
+
+ def test_jobs_trace_template_profile(self):
+ """
+ Test jobs-trace template profile.
+
+ :return:
+ """
+ jobs_trace = self._render_template_to_yaml(
+ "jobs-trace", runners=MagicMock(return_value=["0000"])
+ )
+ assert len(jobs_trace["jobs-details"]) == 1
+ assert (
+ jobs_trace["jobs-details"][0]["run:jobs.list_job"]["info"]
+ == "Details on JID 0000"
+ )
+ assert jobs_trace["jobs-details"][0]["run:jobs.list_job"]["args"] == [0]
diff --git a/tests/unit/modules/test_saltsupport.py b/tests/unit/modules/test_saltsupport.py
new file mode 100644
index 0000000000..f9ce7be29a
--- /dev/null
+++ b/tests/unit/modules/test_saltsupport.py
@@ -0,0 +1,496 @@
+"""
+ :codeauthor: Bo Maryniuk <bo@suse.de>
+"""
+
+
+import datetime
+
+import salt.exceptions
+from salt.modules import saltsupport
+from tests.support.mixins import LoaderModuleMockMixin
+from tests.support.mock import NO_MOCK, NO_MOCK_REASON, MagicMock, patch
+from tests.support.unit import TestCase, skipIf
+
+try:
+ import pytest
+except ImportError:
+ pytest = None
+
+
+@skipIf(not bool(pytest), "Pytest required")
+@skipIf(NO_MOCK, NO_MOCK_REASON)
+class SaltSupportModuleTestCase(TestCase, LoaderModuleMockMixin):
+ """
+ Test cases for salt.modules.support::SaltSupportModule
+ """
+
+ def setup_loader_modules(self):
+ return {saltsupport: {}}
+
+ @patch("tempfile.gettempdir", MagicMock(return_value="/mnt/storage"))
+ @patch("salt.modules.saltsupport.__grains__", {"fqdn": "c-3po"})
+ @patch("time.strftime", MagicMock(return_value="000"))
+ def test_get_archive_name(self):
+ """
+ Test archive name construction.
+
+ :return:
+ """
+ support = saltsupport.SaltSupportModule()
+ assert support._get_archive_name() == "/mnt/storage/c-3po-support-000-000.bz2"
+
+ @patch("tempfile.gettempdir", MagicMock(return_value="/mnt/storage"))
+ @patch("salt.modules.saltsupport.__grains__", {"fqdn": "c-3po"})
+ @patch("time.strftime", MagicMock(return_value="000"))
+ def test_get_custom_archive_name(self):
+ """
+ Test get custom archive name.
+
+ :return:
+ """
+ support = saltsupport.SaltSupportModule()
+ temp_name = support._get_archive_name(archname="Darth Wader")
+ assert temp_name == "/mnt/storage/c-3po-darthwader-000-000.bz2"
+ temp_name = support._get_archive_name(archname="Яйця з сіллю")
+ assert temp_name == "/mnt/storage/c-3po-support-000-000.bz2"
+ temp_name = support._get_archive_name(archname="!@#$%^&*()Fillip J. Fry")
+ assert temp_name == "/mnt/storage/c-3po-fillipjfry-000-000.bz2"
+
+ @patch(
+ "salt.cli.support.get_profiles",
+ MagicMock(return_value={"message": "Feature was not beta tested"}),
+ )
+ def test_profiles_format(self):
+ """
+ Test profiles format.
+
+ :return:
+ """
+ support = saltsupport.SaltSupportModule()
+ profiles = support.profiles()
+ assert "custom" in profiles
+ assert "standard" in profiles
+ assert "message" in profiles["standard"]
+ assert profiles["custom"] == []
+ assert profiles["standard"]["message"] == "Feature was not beta tested"
+
+ @patch("tempfile.gettempdir", MagicMock(return_value="/mnt/storage"))
+ @patch(
+ "os.listdir",
+ MagicMock(
+ return_value=[
+ "one-support-000-000.bz2",
+ "two-support-111-111.bz2",
+ "trash.bz2",
+ "hostname-000-000.bz2",
+ "three-support-wrong222-222.bz2",
+ "000-support-000-000.bz2",
+ ]
+ ),
+ )
+ def test_get_existing_archives(self):
+ """
+ Get list of existing archives.
+
+ :return:
+ """
+ support = saltsupport.SaltSupportModule()
+ out = support.archives()
+ assert len(out) == 3
+ for name in [
+ "/mnt/storage/one-support-000-000.bz2",
+ "/mnt/storage/two-support-111-111.bz2",
+ "/mnt/storage/000-support-000-000.bz2",
+ ]:
+ assert name in out
+
+ def test_last_archive(self):
+ """
+ Get last archive name
+ :return:
+ """
+ support = saltsupport.SaltSupportModule()
+ support.archives = MagicMock(
+ return_value=[
+ "/mnt/storage/one-support-000-000.bz2",
+ "/mnt/storage/two-support-111-111.bz2",
+ "/mnt/storage/three-support-222-222.bz2",
+ ]
+ )
+ assert support.last_archive() == "/mnt/storage/three-support-222-222.bz2"
+
+ @patch("os.unlink", MagicMock(return_value=True))
+ def test_delete_all_archives_success(self):
+ """
+ Test delete archives
+ :return:
+ """
+ support = saltsupport.SaltSupportModule()
+ support.archives = MagicMock(
+ return_value=[
+ "/mnt/storage/one-support-000-000.bz2",
+ "/mnt/storage/two-support-111-111.bz2",
+ "/mnt/storage/three-support-222-222.bz2",
+ ]
+ )
+ ret = support.delete_archives()
+ assert "files" in ret
+ assert "errors" in ret
+ assert not bool(ret["errors"])
+ assert bool(ret["files"])
+ assert isinstance(ret["errors"], dict)
+ assert isinstance(ret["files"], dict)
+
+ for arc in support.archives():
+ assert ret["files"][arc] == "removed"
+
+ @patch(
+ "os.unlink",
+ MagicMock(
+ return_value=False,
+ side_effect=[
+ OSError("Decreasing electron flux"),
+ OSError("Solar flares interference"),
+ None,
+ ],
+ ),
+ )
+ def test_delete_all_archives_failure(self):
+ """
+ Test delete archives failure
+ :return:
+ """
+ support = saltsupport.SaltSupportModule()
+ support.archives = MagicMock(
+ return_value=[
+ "/mnt/storage/one-support-000-000.bz2",
+ "/mnt/storage/two-support-111-111.bz2",
+ "/mnt/storage/three-support-222-222.bz2",
+ ]
+ )
+ ret = support.delete_archives()
+ assert "files" in ret
+ assert "errors" in ret
+ assert bool(ret["errors"])
+ assert bool(ret["files"])
+ assert isinstance(ret["errors"], dict)
+ assert isinstance(ret["files"], dict)
+
+ assert ret["files"]["/mnt/storage/three-support-222-222.bz2"] == "removed"
+ assert ret["files"]["/mnt/storage/one-support-000-000.bz2"] == "left"
+ assert ret["files"]["/mnt/storage/two-support-111-111.bz2"] == "left"
+
+ assert len(ret["errors"]) == 2
+ assert (
+ ret["errors"]["/mnt/storage/one-support-000-000.bz2"]
+ == "Decreasing electron flux"
+ )
+ assert (
+ ret["errors"]["/mnt/storage/two-support-111-111.bz2"]
+ == "Solar flares interference"
+ )
+
+ def test_format_sync_stats(self):
+ """
+ Test format rsync stats for preserving ordering of the keys
+
+ :return:
+ """
+ support = saltsupport.SaltSupportModule()
+ stats = """
+robot: Bender
+cute: Leela
+weird: Zoidberg
+professor: Farnsworth
+ """
+ f_stats = support.format_sync_stats({"retcode": 0, "stdout": stats})
+ assert list(f_stats["transfer"].keys()) == [
+ "robot",
+ "cute",
+ "weird",
+ "professor",
+ ]
+ assert list(f_stats["transfer"].values()) == [
+ "Bender",
+ "Leela",
+ "Zoidberg",
+ "Farnsworth",
+ ]
+
+ @patch("tempfile.mkstemp", MagicMock(return_value=(0, "dummy")))
+ @patch("os.close", MagicMock())
+ def test_sync_no_archives_failure(self):
+ """
+ Test sync failed when no archives specified.
+
+ :return:
+ """
+ support = saltsupport.SaltSupportModule()
+ support.archives = MagicMock(return_value=[])
+
+ with pytest.raises(salt.exceptions.SaltInvocationError) as err:
+ support.sync("group-name")
+ assert "No archives found to transfer" in str(err)
+
+ @patch("tempfile.mkstemp", MagicMock(return_value=(0, "dummy")))
+ @patch("os.path.exists", MagicMock(return_value=False))
+ def test_sync_last_picked_archive_not_found_failure(self):
+ """
+ Test sync failed when archive was not found (last picked)
+
+ :return:
+ """
+ support = saltsupport.SaltSupportModule()
+ support.archives = MagicMock(
+ return_value=[
+ "/mnt/storage/one-support-000-000.bz2",
+ "/mnt/storage/two-support-111-111.bz2",
+ "/mnt/storage/three-support-222-222.bz2",
+ ]
+ )
+
+ with pytest.raises(salt.exceptions.SaltInvocationError) as err:
+ support.sync("group-name")
+ assert (
+ ' Support archive "/mnt/storage/three-support-222-222.bz2" was not found'
+ in str(err)
+ )
+
+ @patch("tempfile.mkstemp", MagicMock(return_value=(0, "dummy")))
+ @patch("os.path.exists", MagicMock(return_value=False))
+ def test_sync_specified_archive_not_found_failure(self):
+ """
+ Test sync failed when archive was not found (last picked)
+
+ :return:
+ """
+ support = saltsupport.SaltSupportModule()
+ support.archives = MagicMock(
+ return_value=[
+ "/mnt/storage/one-support-000-000.bz2",
+ "/mnt/storage/two-support-111-111.bz2",
+ "/mnt/storage/three-support-222-222.bz2",
+ ]
+ )
+
+ with pytest.raises(salt.exceptions.SaltInvocationError) as err:
+ support.sync("group-name", name="lost.bz2")
+ assert ' Support archive "lost.bz2" was not found' in str(err)
+
+ @patch("tempfile.mkstemp", MagicMock(return_value=(0, "dummy")))
+ @patch("os.path.exists", MagicMock(return_value=False))
+ @patch("os.close", MagicMock())
+ def test_sync_no_archive_to_transfer_failure(self):
+ """
+ Test sync failed when no archive was found to transfer
+
+ :return:
+ """
+ support = saltsupport.SaltSupportModule()
+ support.archives = MagicMock(return_value=[])
+ with pytest.raises(salt.exceptions.SaltInvocationError) as err:
+ support.sync("group-name", all=True)
+ assert "No archives found to transfer" in str(err)
+
+ @patch("tempfile.mkstemp", MagicMock(return_value=(0, "dummy")))
+ @patch("os.path.exists", MagicMock(return_value=True))
+ @patch("os.close", MagicMock())
+ @patch("os.write", MagicMock())
+ @patch("os.unlink", MagicMock())
+ @patch(
+ "salt.modules.saltsupport.__salt__", {"rsync.rsync": MagicMock(return_value={})}
+ )
+ def test_sync_archives(self):
+ """
+ Test sync archives
+ :return:
+ """
+ support = saltsupport.SaltSupportModule()
+ support.archives = MagicMock(
+ return_value=[
+ "/mnt/storage/one-support-000-000.bz2",
+ "/mnt/storage/two-support-111-111.bz2",
+ "/mnt/storage/three-support-222-222.bz2",
+ ]
+ )
+ out = support.sync("group-name", host="buzz", all=True, move=False)
+ assert "files" in out
+ for arc_name in out["files"]:
+ assert out["files"][arc_name] == "copied"
+ assert saltsupport.os.unlink.call_count == 1
+ assert saltsupport.os.unlink.call_args_list[0][0][0] == "dummy"
+ calls = []
+ for call in saltsupport.os.write.call_args_list:
+ assert len(call) == 2
+ calls.append(call[0])
+ assert calls == [
+ (0, b"one-support-000-000.bz2"),
+ (0, b"\n"),
+ (0, b"two-support-111-111.bz2"),
+ (0, b"\n"),
+ (0, b"three-support-222-222.bz2"),
+ (0, b"\n"),
+ ]
+
+ @patch("salt.modules.saltsupport.__pillar__", {})
+ @patch("salt.modules.saltsupport.SupportDataCollector", MagicMock())
+ def test_run_support(self):
+ """
+ Test run support
+ :return:
+ """
+ saltsupport.SupportDataCollector(None, None).archive_path = "dummy"
+ support = saltsupport.SaltSupportModule()
+ support.collect_internal_data = MagicMock()
+ support.collect_local_data = MagicMock()
+ out = support.run()
+
+ for section in ["messages", "archive"]:
+ assert section in out
+ assert out["archive"] == "dummy"
+ for section in ["warning", "error", "info"]:
+ assert section in out["messages"]
+ ld_call = support.collect_local_data.call_args_list[0][1]
+ assert "profile" in ld_call
+ assert ld_call["profile"] == "default"
+ assert "profile_source" in ld_call
+ assert ld_call["profile_source"] is None
+ assert support.collector.open.call_count == 1
+ assert support.collector.close.call_count == 1
+ assert support.collect_internal_data.call_count == 1
+
+
+@skipIf(not bool(pytest), "Pytest required")
+@skipIf(NO_MOCK, NO_MOCK_REASON)
+class LogCollectorTestCase(TestCase, LoaderModuleMockMixin):
+ """
+ Test cases for salt.modules.support::LogCollector
+ """
+
+ def setup_loader_modules(self):
+ return {saltsupport: {}}
+
+ def test_msg(self):
+ """
+ Test message to the log collector.
+
+ :return:
+ """
+ utcmock = MagicMock()
+ utcmock.utcnow = MagicMock(return_value=datetime.datetime.utcfromtimestamp(0))
+ with patch("datetime.datetime", utcmock):
+ msg = "Upgrading /dev/null device"
+ out = saltsupport.LogCollector()
+ out.msg(msg, title="Here")
+ assert saltsupport.LogCollector.INFO in out.messages
+ assert (
+ type(out.messages[saltsupport.LogCollector.INFO])
+ == saltsupport.LogCollector.MessagesList
+ )
+ assert out.messages[saltsupport.LogCollector.INFO] == [
+ "00:00:00.000 - {}: {}".format("Here", msg)
+ ]
+
+ def test_info_message(self):
+ """
+ Test info message to the log collector.
+
+ :return:
+ """
+ utcmock = MagicMock()
+ utcmock.utcnow = MagicMock(return_value=datetime.datetime.utcfromtimestamp(0))
+ with patch("datetime.datetime", utcmock):
+ msg = "SIMM crosstalk during tectonic stress"
+ out = saltsupport.LogCollector()
+ out.info(msg)
+ assert saltsupport.LogCollector.INFO in out.messages
+ assert (
+ type(out.messages[saltsupport.LogCollector.INFO])
+ == saltsupport.LogCollector.MessagesList
+ )
+ assert out.messages[saltsupport.LogCollector.INFO] == [
+ "00:00:00.000 - {}".format(msg)
+ ]
+
+ def test_put_message(self):
+ """
+ Test put message to the log collector.
+
+ :return:
+ """
+ utcmock = MagicMock()
+ utcmock.utcnow = MagicMock(return_value=datetime.datetime.utcfromtimestamp(0))
+ with patch("datetime.datetime", utcmock):
+ msg = "Webmaster kidnapped by evil cult"
+ out = saltsupport.LogCollector()
+ out.put(msg)
+ assert saltsupport.LogCollector.INFO in out.messages
+ assert (
+ type(out.messages[saltsupport.LogCollector.INFO])
+ == saltsupport.LogCollector.MessagesList
+ )
+ assert out.messages[saltsupport.LogCollector.INFO] == [
+ "00:00:00.000 - {}".format(msg)
+ ]
+
+ def test_warning_message(self):
+ """
+ Test warning message to the log collector.
+
+ :return:
+ """
+ utcmock = MagicMock()
+ utcmock.utcnow = MagicMock(return_value=datetime.datetime.utcfromtimestamp(0))
+ with patch("datetime.datetime", utcmock):
+ msg = "Your e-mail is now being delivered by USPS"
+ out = saltsupport.LogCollector()
+ out.warning(msg)
+ assert saltsupport.LogCollector.WARNING in out.messages
+ assert (
+ type(out.messages[saltsupport.LogCollector.WARNING])
+ == saltsupport.LogCollector.MessagesList
+ )
+ assert out.messages[saltsupport.LogCollector.WARNING] == [
+ "00:00:00.000 - {}".format(msg)
+ ]
+
+ def test_error_message(self):
+ """
+ Test error message to the log collector.
+
+ :return:
+ """
+ utcmock = MagicMock()
+ utcmock.utcnow = MagicMock(return_value=datetime.datetime.utcfromtimestamp(0))
+ with patch("datetime.datetime", utcmock):
+ msg = "Learning curve appears to be fractal"
+ out = saltsupport.LogCollector()
+ out.error(msg)
+ assert saltsupport.LogCollector.ERROR in out.messages
+ assert (
+ type(out.messages[saltsupport.LogCollector.ERROR])
+ == saltsupport.LogCollector.MessagesList
+ )
+ assert out.messages[saltsupport.LogCollector.ERROR] == [
+ "00:00:00.000 - {}".format(msg)
+ ]
+
+ def test_hl_message(self):
+ """
+ Test highlighter message to the log collector.
+
+ :return:
+ """
+ utcmock = MagicMock()
+ utcmock.utcnow = MagicMock(return_value=datetime.datetime.utcfromtimestamp(0))
+ with patch("datetime.datetime", utcmock):
+ out = saltsupport.LogCollector()
+ out.highlight("The {} TTYs became {} TTYs and vice versa", "real", "pseudo")
+ assert saltsupport.LogCollector.INFO in out.messages
+ assert (
+ type(out.messages[saltsupport.LogCollector.INFO])
+ == saltsupport.LogCollector.MessagesList
+ )
+ assert out.messages[saltsupport.LogCollector.INFO] == [
+ "00:00:00.000 - The real TTYs became " "pseudo TTYs and vice versa"
+ ]
--
2.39.2
Reference in New Issue View Git Blame Copy Permalink