yudaike/openSUSE-release-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #2738 from coolo/add_test_docu

Browse Source 
Refresh the documentation on how to add tests
This commit is contained in:
Stephan Kulow 2022-03-07 14:28:53 +01:00 committed by GitHub
commit a7a1aa83dc
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 59 additions and 138 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

61
README.md
View File

@ -78,9 +78,9 @@ Then you can use the new `local` alias to access this new instance.
osc -A local api /about
Some tests will attempt to run against the local OBS, but not all.
Some tests will attempt to run against the local OBS, but not all. It's still recommended to run this through docker-compose (see below)
nosetests
pytest tests/*.py
## Running Continuous Integration
@ -130,3 +130,60 @@ To debug problems in the test suite or in the code, place a `breakpoint()` call
You can access your testing OBS instance at `http://0.0.0.0:3000` and log in using "Admin" as username and "opensuse" as password. To prevent the data being removed while you are inspecting the OBS instance, you can put a call to the `breakpoint()` function.
Finally, if you miss anything for debugging, you can use `zypper` to install it.
### Adding tests
Testing the release tools isn't quite trivial as a lot of these tools rely on running openSUSE infrastructure. Some of the workflows we replay (not mock) in above described docker-compose setup. So each test will setup the required projects and e.g. staging workflows in a local containerized OBS installation and then do its assertions. If you want to add coverage, best check existing unit tests in tests/*.py. A generic test case looks similiar to this:
``` {.python title="Basic Test Example" }
class TestExample(unittest.TestCase):
def test_basic(self):
# Keep the workflow in local scope so that ending the test case will destroy it.
# Destroying the workflow will also delete all created projects and packages. The
# created workflow has a target project, but most of the test assets need to be created
# as needed
wf = OBSLocal.FactoryWorkflow()
staging = wf.create_staging('A', freeze=True)
wf.create_submit_request('devel:wine', 'wine')
ret = SelectCommand(wf.api, staging.name).perform(['wine'])
self.assertEqual(True, ret)
```
To ease having many such tests, we also have the `OBSLocal` class, which moves the creation of the workflow into `setUp` and the destruction in `tearDwon` functions of pytest. The principle stays the same though.
``` {.python title="OBSLocal Usage"}
class TestExampleWithOBS(OBSLocal.TestCase):
"""
Tests for various api calls to ensure we return expected content
"""
def setUp(self):
super(TestExampleWithOBS, self).setUp()
self.wf = OBSLocal.FactoryWorkflow()
self.wf.setup_rings()
self.staging_b = self.wf.create_staging('B')
def tearDown(self):
del self.wf
super(TestExampleWithOBS, self).tearDown()
def test_list_projects(self):
"""
List projects and their content
"""
staging_a = self.wf.create_staging('A')
# Prepare expected results
data = [staging_a.name, self.staging_b.name]
# Compare the results
self.assertEqual(data, self.wf.api.get_staging_projects())
```
Note that we have some (older) test cases using httpretty, but those are very special cases and require you a lot of extra mocking as you can't mix httpretty and testing against the minimal OBS. So every extra
call that osc libraries or our code do, will require changes in your test case. It can still be a viable option, especially if more than OBS is involved.
The method that you can combine with `OBSLocal` though is using MagicMock. This class is used to mock individual functions. So splitting the code to use helper functions to retrieve information and then
mocking this inside the test case can be a good alternative to mocking the complete HTTP traffic.

136
docs/testing.asciidoc
View File

@ -1,136 +0,0 @@
Testing
=======
Dependencies
------------
Test suite is using +nose+, +httpretty+ and +mock+ for testing. You need these
three python modules installed to run tests. In openSUSE, you can do it using
the following command as a root:
--------------------------------------------------------------------------------
zypper in python-nose python-httpretty python-mock
--------------------------------------------------------------------------------
Running tests
-------------
To run the tests, you need to be in the topmost directory of your checkout and
run the following command there:
--------------------------------------------------------------------------------
nosetests
--------------------------------------------------------------------------------
Structure of the suite
----------------------
Each object contains functions for the individual tests so split them per
area of interest.
In directory fixtures there are resulting xml files obtained from the OBS with
osc api calls.
Writing tests
-------------
There are a few nice building stones available to implement test.
OBS class
~~~~~~~~~
+OBS+ class provides simulation of OBS including keeping internal states. It
supports only a limited number of commands, but that can be extended.
Extending OBS class
^^^^^^^^^^^^^^^^^^^
You can extend the OBS mockup class creating new method, and
decorating it with one of the @GET, @PUT, @POST or @DELETE. The
parameter of the decorator is the PATH of the URL or a regular
expression that matches one of the possible paths.
If the new response can be implemented as a simple fixture, you can
create the file in the +fixtures/+ directory in a place compatible in
the expected path.
Because we are decorating methods, we can maintain an internal status
inside the OBS mock-up instance.
example
^^^^^^^
First we create a template for our response. We will be using pythons string
Template class therefore XML has some values replaced with +${variable}+ and we
will assign those later.
.Template
[source,xml]
--------------------------------------------------------------------------------
<request id="${id}">
<action type="submit">
<source project="home:Admin" package="${package}" rev="59f0f46262d7b57b9cdc720c06d5e317"/>
<target project="openSUSE:Factory" package="${package}"/>
</action>
<state name="${request}" who="Admin" when="2014-02-17T12:38:52">
<comment>...</comment>
</state>
<review state="${review}" when="2014-02-17T12:34:10" who="${who}" by_${by}="${by_who}">
<comment>...</comment>
</review>
<description>test</description>
</request>
--------------------------------------------------------------------------------
We can also define helpful local data structure representing actual state of OBS
[source,python]
--------------------------------------------------------------------------------
# Initial request data
self.requests = {
'123': {
'request': 'new',
'review': 'accepted',
'who': 'Admin',
'by': 'group',
'id': '123',
'by_who': 'opensuse-review-team',
'package': 'gcc',
},
'321': {
'request': 'review',
'review': 'new',
'who': 'Admin',
'by': 'group',
'id': '321',
'by_who': 'factory-staging',
'package': 'puppet',
},
}
--------------------------------------------------------------------------------
And the most important part is implementing OBS behaviour.
[source,python]
--------------------------------------------------------------------------------
@GET(re.compile(r'/request/\d+'))
def request(self, request, uri, headers):
"""Return a request XML description."""
request_id = re.search(r'(\d+)', uri).group(1)
response = (404, headers, '<result>Not found</result>')
try:
template = string.Template(self._fixture(uri))
response = (200, headers, template.substitute(self.requests[request_id]))
except Exception as e:
if DEBUG:
print uri, e
if DEBUG:
print 'REQUEST', uri, response
return response
--------------------------------------------------------------------------------
The method +request+ will be called when a request to /request/NUMBER
is made. The previous code will load the XML template and replace
variables with the request dictionary content.