Functional Verification Test Plan for openCryptoki

Version 0.2

08/15/2005
 
 
 

Owner: Michael A. Halcrow
mhalcrow@us.ibm.com
(512) 838-8096
11501 Burnet Rd Austin, TX 78758

Document Control

Document Change Control

Initial Release: 0.1
Review Frequency: At each major revision
Final Page Indicator: "End of Document;" statement
Retention: Valid until superseded by a new version/level.

Reviewers/Approvers

<List names of approver(s) and reviewer(s) – indicate next to name approver or reviewer>

 Tom Lendacky – Reviewer

Emily Ratliff - Reviewer

Change Summary

<List reviews of this document : include review date, version reviewed, new version(if needed),  reviewer(s),  approver(s)>

Table of Contents

I. Introduction. 4

A. References/Related Documents. 4

B. LDP Items

C. Hardware. 5

D. Firmware. 5

E. Limitations. 5

F. General 5

G. Past History. 5

II. Test Plan Overview.. 5

A. Additional Program Products. 5

B. Test Approach and Methodology. 6

C. System Operation. 6

D. Performance. 6

E. Standards Compliance. 7

F. Stress. 7

G. Regression. 7

H. Ship Test 7

I. Installation Documentation. 7

J. Installation/Configuration Test 7

K. Reliability, Availability, and Serviceability. 7

L. Usability. 7

IV. Quality Goals. 8

A. Goals. 8

B. Measurements. 8

V. Status Information. 8

VI. Testcase Descriptions. 9

A. Naming Conventions. 9

B. Testcase Location. 9

C. Testcases description. 9

VII. Functional Coverage Matrix. 9

VIII. Approval Criteria. 10

End of Document 10

I. Introduction

A. References/Related Documents

<List any documents or references to LDP entries  covered in this plan OR used in developing this plan>

B. LDP Items

<List LDP entries  covered in this plan : include LDP  number, one-liner description, product/package name that will include the LDP entry, and  targeted release>

1.      End-Use Impact

<Identify/List any  end-user impacts/benefits of this feature/LDP item(s)? (i.e. performance,  new function allowing end-user to...,  change in behavior of an existing function allowing end-user to ... )>

Allows users to access cryptographic hardware through a PKCS#11 interface.

2.      Files (Design/Implementation Details – New Section)

<Identify/List any  files or code  impacted by OR are new for this feature/LDP item(s)? (i.e.  list files, or directories)>

/usr/lib/opencryptoki/

/var/lib/opencryptoki/

/etc/pkcs11

3.      Enablement

<Identify how this feature/LDP item(s) is enabled. Is it automatically enabled/default turned on?  If not, how would an end-user enable/"turn on" this feature/LDP item?>

openCryptoki is enabled by executing an initialization script and then running /etc/init.d/pkcsslotd. Applications link against libopencryptoki.a and make various calls through the library.

4.      Parameters (Design/Implementation Details – New Section)

<Are there any  parameters that can be passed to any files or commands in conjunction with this feature/LDP item?  If so, please list all parameters and for each parameter (or provide access to a man page/help)>

See opencryptoki/doc/openCryptoki-HOWTO.pdf in the source base for documentation on openCryptoki application parameters.

5.      Bugs/Defects

<Identify how l issues/bugs/defects be tracked? (i.e. Notes DB, Bugzilla, Bugzilla Family, Component, etc.) List components.>

 Bugs are tracked via the Sourceforge bugzilla: http://sourceforge.net/tracker/?group_id=128009&atid=710344

6.      Targeted Code Completion

<Identify targeted code completion date.>

 05/31/2005

C. Hardware

<Identify/List supported hardware architectures/platforms for this feature/LDP item? (i.e.  common/architecture neutral, xSeries, pSeries, zSeries, iSeries, Power5 only, etc..)>

i386, ppc, ppc64, s390, s390x.

D. Firmware

<Identify/List supported/required firmware for this feature/LDP item?>

N/A

E. Limitations

< List any known limitations or restrictions of this feature.>

N/A

F. General

<Identify/List any other “general” dependencies not covered above that are required to support this feature/LDP item.>

Some hardware accelerators will be required to test specific openCryptoki STDLL's or OpenSSL will be required to test the software STDLL. For the ICA s390 token, VICOM emulation of certain instructions (e.g., SHA-256 or AES) will need to be enabled.

G. Past History

<If available, describe any past history relating to LDP items and/or components: customer problems, error prone areas, and any strengths/weaknesses of previous testing.>

Weaknesses in testing: although testcases exist, some of them may be token specific and therefor require updating. There is work currently in plan for 2005 to resolve this issue.

II. Test Plan Overview

<Describe test goals, objectives, level of testing and scope of this plan in relation to the LDP item(s) covered.>

The goals of the current tests available are to test the PKCS#11 API and also the functionality of specific tokens (STDLL files).

A. Additional Program Products

<Identify/List software/products required to perform the tests covered in this plan – be sure to list the product/package that includes the LDP item(s)>

B. Test Approach and Methodology

<Document the test approach and methodology to be used.>

Manually, by running individual test cases included in the openCryptoki tarball.

C. System Operation

<Document verification methods used for hardware and software configurations/combinations.>

Assume RHEL4+/s390/s390x or SLES9+/i386/ppc/ppc64/s390/s390x

rpm -q openCryptoki succeeds

rpm -q openCryptoki-32/64bit succeeds

D. Performance

< If applicable, document verification methods used to determine performance equal to or better than existing configurations.>

N/A

E. Standards Compliance

<If applicable, identify applicable test suites (SBLIM, GNU Automake, etc) to be run to verify standards compliance.>

N/A

F. Stress

<If applicable, describe stress testing to be done on the product to verify robustness during high system and possibly network usage.  Include target length of test and expected/acceptable breaking point>

N/A

G. Regression

<Identify/List a set of tests from current and proposed set of testcases to be used during regression testing.>

The following directories under testcases/ contain tests that should be run during regression testing:

• speed/

• driver/

• oc-digest/

• v2.11/

H. Ship Test

<Identify/List a set of tests from current and proposed set of testcases to be used during ship/final testing.>

The following directories under testcases/ contain tests that should be run prior to shipping:

• speed/

• driver/

• oc-digest/

• v2.11/

I. Installation Documentation

<If applicable, describe how installation INSTRUCTIONS/DOCUMENTATION  will be verified for the product/package containing the LDP item(s) covered in this plan.   These instructions may be contained in README files shipped with the software.>

The instructions for installing the package are in the README and INSTALL files contained within the package tarball.

J. Installation/Configuration Test

<If applicable, describe the various configurations/combinations to be used during the installation and configuration verification tasks of  LDP item(s) covered in this plan.>

N/A

K. Reliability, Availability, and Serviceability

<If applicable, describe the RAS goals of the LDP item(s) covered in this plan and how these will be verified.>

N/A

L. Usability

<If applicable, describe how usability of the LDP item(s) covered in this plan will be verified.>

N/A

IV. Quality Goals

A. Goals

<Identify the quality goals of this plan.>

1. Runs stably under load (multiple applications concurrently making PKCS#11 calls through the openCryptoki library).

2. Provides PKCS#11 interface to an application.

B. Measurements

<What measurement methods will be used to track goals?>.

Correct operation is measured via the tests found in the testcases/ directory.

V. Status Information

<The following information will need to be collected and stored on a regular basis until the execution of this  plan is completed. Identify here the location of this stored information (could  be tracked by project management) and how frequently it will be updated.

 NOTE: Some testcases my be logged by hours of successful test execution – which is ok.>

SUMMARY:

DETAILS: <List uncompleted work/testcases first>

VI. Testcase Descriptions

A. Naming Conventions

<If applicable, describe any name conventions used for the testcases.>

N/A

B. Testcase Location

<Indicate the location/storage of these test cases.>

The tests are included in the package tarball under the testcases/ directory.

C. Testcases description

VII. Functional Coverage Matrix

This table describes the functional coverage of the test suite(s). For each new or modified testcase, it shows the associated list of assertions, whether or not the test case is automated, and whether or not the test case is suitable for a lasting regression test suite.

VIII. Approval Criteria

<Explicitly identify the approval criteria for the test case execution results.>

FV exit criteria:

• 100% of planned test cases written

• 100% of planned test cases executed

• Resolution of all Sev 1 Bugs found executing this plan

End of Document