Testing
=======

Dependencies
------------

Test suite is using +nose+, +httpretty+ and +mock+ for testing. You need these
three python modules installed to run tests. In openSUSE, you can do it using
the following command as a root:

--------------------------------------------------------------------------------
zypper in python-nose python-httpretty python-mock
--------------------------------------------------------------------------------

Running tests
-------------

To run the tests, you need to be in the topmost directory of your checkout and
run the following command there:

--------------------------------------------------------------------------------
nosetests
--------------------------------------------------------------------------------

Structure of the suite
----------------------

Each object is containing functions for the individual tests so split them per
area of interest.

In directory fixtures there are resulting xml files obtained from the OBS with
osc api calls.

Writing tests
-------------

There are few nice building stones available to implement test.

+responses+ dictionary
~~~~~~~~~~~~~~~~~~~~~~

It contains dictionaries for 'GET', 'PUT', 'POST' and 'ALL'. All of them
correspond to the method used by program. 'ALL' is just a shortcut that covers
all methods and it is called only if no specific handler is found.

Dictionaries associated with each method contains as a key relative URL to the
top of the server and as data one of the following:

* path to the file in fixtures directory with XML to return
* XML itself starting with +<?xml+
* function taking three arguments - 'requst', 'uri' and 'headers'
* list which is combination of any of the above

If handling of +responses+ dictionary is enabled, URL is find in dictionary and
appropriate action is taken. If there is XML or path to the XML, XML is
returned directly. In case of function, function gets called and is expected to
return string that will be passed back. If there is array, first item is used
as described above and removed from the array.

+register_pretty+ function
~~~~~~~~~~~~~~~~~~~~~~~~~~~

This function should be called before you start running any functions you want
to test. It registers handler for 'localhost' requests to be
+_pretty_callback+. This handler in short does interpretation of +responses+
dictionary as described in previous section and if no handler is found, it
raises exception providing URL that program tried to access together with
method used.

example
~~~~~~~

First get template for XML using pythons string Template class. XML has some
values replaced with +${variable}+ and we will assign those later.

.Template
[source,xml]
--------------------------------------------------------------------------------
<?xml version="1.0"?>
<request id="123">
  <action type="submit">
    <source project="home:Admin" package="puppet" rev="59f0f46262d7b57b9cdc720c06d5e317"/>
    <target project="openSUSE:Factory" package="puppet"/>
  </action>
  <state name="${request}" who="Admin" when="2014-02-17T12:38:52">
    <comment>Passed staging project "openSUSE:Factory:Staging:test"</comment>
  </state>
  <review state="${review}" when="2014-02-17T12:34:10" who="Admin" by_project="${review_project}">
    <comment>Passed staging project "openSUSE:Factory:Staging:test"</comment>
  </review>
  <description>test</description>
</request>
--------------------------------------------------------------------------------

[source,python]
--------------------------------------------------------------------------------
# Getting template in test
tmpl = Template(self._get_fixture_content('request_review.xml'))
--------------------------------------------------------------------------------

Now prepare some default starting values
[source,python]
--------------------------------------------------------------------------------
# clear responses dictionary(just in case)
self._clear_responses()
# set default response
self.responses['GET']['/request/123'] =  tmpl.substitute(request='new', review='accepted', review_project="openSUSE:Factory")
# register everything
self._register_pretty()
--------------------------------------------------------------------------------

Now the interesting part. Let's define a function, that would interpret 'POST'
method, extract data from command and will actually change the response next
'GET' method will give. We can change the responses dictionary anytime, so we
can use this to keep state once we get a command to do something.

[source,python]
--------------------------------------------------------------------------------
# Create function making transitions between states
def review_change(responses, request, uri):
   if request.querystring.has_key(u'cmd') and request.querystring[u'cmd'] == [u'addreview']:
       responses['GET']['/request/123']  = tmpl.substitute(request='review', review='new', review_project=request.querystring[u'by_project'][0])
   if request.querystring.has_key(u'cmd') and request.querystring[u'cmd'] == [u'changereviewstate']:
       responses['GET']['/request/123']  = tmpl.substitute(request='new', review=request.querystring[u'newstate'][0], review_project=request.querystring[u'by_project'][0])
   return responses['GET']['/request/123']

# Register it
self.responses['ALL']['/request/123'] = review_change
--------------------------------------------------------------------------------

So whenever somebody sends 'addreview' command, XML that 'GET' on request
provides will be changed to reflect newly added review. And whenever somebody
sends 'changereviewstate', review will be closed with appropriate state.

So we have a simple testing framework with multiple states that reacts to the
API calls from functions we are testing following the behaviour we specified.
So tests itself can be simple.

[source,python]
--------------------------------------------------------------------------------
# Add first review, there is none at beginning
api.add_review('123', 'openSUSE:Factory:Staging:A')
self.assertEqual(httpretty.last_request().method, 'POST')
self.assertEqual(httpretty.last_request().querystring[u'cmd'], [u'addreview'])
# Try to add it again, should do anything
api.add_review('123', 'openSUSE:Factory:Staging:A')
self.assertEqual(httpretty.last_request().method, 'GET')
# Accept review
api.set_review('123', 'openSUSE:Factory:Staging:A')
self.assertEqual(httpretty.last_request().method, 'POST')
self.assertEqual(httpretty.last_request().querystring[u'cmd'], [u'changereviewstate'])
# Try to accept it again should do anything
api.set_review('123', 'openSUSE:Factory:Staging:A')
self.assertEqual(httpretty.last_request().method, 'GET')
# But we should be able to create a new one
api.add_review('123', 'openSUSE:Factory:Staging:A')
self.assertEqual(httpretty.last_request().method, 'POST')
self.assertEqual(httpretty.last_request().querystring[u'cmd'], [u'addreview'])
--------------------------------------------------------------------------------