Notes

Outline

Testing Security Functions Role of the Security Function Testing Aspects of the System Security Realization ATE_COV: The High Level (Security) Design ATE_DPT: The Low Level (Security) Design ATE_FUN: TSF Internals ATE_IND: The Implementation Representation Security Function Testing In EROS

Role of Security Function Testing Show That TOE Functional Requirements Are Satisfied Emphasis Is On Confirming That TOE Operates According To Its Specification Positive testing to show presence of required behaviors Negative testing to show absence of prohibited  behaviors Can be Applied Recursively to TOE Internal Structure

Points to Remember Penetration Testing Covered Under Vulnerability Assessment Class Functional Testing Limitations Only shows behavior for tested configurations Cannot be exhaustive for complex systems Cannot show absence of unspecified functions Main Value of Functional Testing Is to Show That There Is At Least One TOE Configuration That Meets ST SFRs

ATE_COV – Test Coverage Definition Aspects Components Evidence Of Coverage Analysis of Coverage Rigorous Analysis of Coverage

Definition Coverage Is A Measure Of The Extent To Which The Test Cases Demonstrate That The Specified TSF Are Present And Operate As Specified i.e., a measure of how many of the TSF interfaces specified in the Functional Specification are actually exercised by tests

Aspects of the Family Correspondence Completeness

Correspondence Provide An Analysis that Shows or Demonstrates Which TSF Interface(s) Each Test Is Exercising “Show” Only Requires An Assertion That A Test Exercises An Interface “Demonstrate” Requires An Explanation Of How The Test Exercises The Interface(s)

Completeness Provide An Analysis That Demonstrates That The Full Set Of Tests Exercises The Full Range Of TSF Interfaces Rigorous Demonstration Requires That It Be Shown That The Testing Of Each Interface Be Complete (i.e., exhaustive) Potentially a flawed requirement, since it is generally acknowledged that complex system interfaces cannot be exhaustively tested

Evidence Of Coverage “The developer shall provide evidence of the test coverage.” “The evidence of the test coverage shall show the correspondence between the tests identified in the test documentation and the TSF as described in the functional specification.”

Analysis of Coverage The developer shall provide an analysis of the test coverage. The analysis of the test coverage shall demonstrate the correspondence between the tests identified in the test documentation and the TSF as described in the functional specification. The analysis of the test coverage shall demonstrate that the correspondence between the TSF as described in the functional specification and the tests identified in the test documentation is complete.

Rigorous Analysis of Coverage The analysis of the test coverage shall rigorously demonstrate that all external interfaces of the TSF identified in the functional specification have been completely tested.

ATE_DPT – Test Depth Definition Aspects Components Testing: High-Level Design Testing: Low-Level Design Testing: Implementation Representation

Definition Depth Is The Level Of Detail To Which TSF Is Tested Keyed to the ADV representations Objective Is To Catch Both Subtle Errors That Are Only Visible At Internal Interfaces And Insertion Of Malicious Code

Aspect of the Family Granularity Of Testing Essentially specifies the resolution of the white/grey box testing that most be done

Testing: High-Level Design The developer shall provide the analysis of the depth of testing. The depth analysis shall demonstrate that the tests identified in the test documentation are sufficient to demonstrate that the TSF operates in accordance with its high-level design.

Testing: Low-Level Design The depth analysis shall demonstrate that the tests identified in the test documentation are sufficient to demonstrate that the TSF operates in accordance with its high-level design and low-level design.

Testing: Implementation The depth analysis shall demonstrate that the tests identified in the test documentation are sufficient to demonstrate that the TSF operates in accordance with its high-level design, low-level design and implementation representation.

ATE_FUN – Functional Testing Definition Aspects Components Functional Testing Ordered Functional Testing

Definition Functional Testing Is Performed By The Developer To Establish That The TSF Exhibit The Properties Necessary To Satisfy The Functional Requirements Of The PP/ST The Chosen ATE_COV and ATE_DPT Components Will Determine The Amount Of Evidence Required By ATE_FUN

Aspects of the Family Plans Procedures Expected Results Actual Results Ordering Dependencies

Plans Focus Of The Test Plans Is On The Security Functions Being Tested Essentially rationalize each test The plan for each test describes its goal(s) in terms of the security functions it will exercise

Procedures The Focus Of The Test Procedures Is On The Mechanics Of Executing The Tests Each Test Procedure Includes: Description of how to run the test Description of the system security state assumed by the test Description of other tests that must be executed first to bring system to the required security state (i.e., test ordering dependencies)

Expected Results Actually An Aspect Of The Test Procedures That Identifies The Output Predicted By The Specification From Which The Test Is Derived

Actual Results The Log Of The Results Obtained When The Developer Actually Runs The Test And A Demonstration That Those Results Are Consistent With The Expected Results And Therefore With The Specified Security Behavior Of The TSF

Ordering Dependencies An Analysis That Rationalizes The Test Ordering Dependencies Is Useful In Determining Adequacy Of Testing, Since It Might Allow Discovery Of Boundary Conditions That Can Be Concealed By Careful Ordering Of The Tests

Functional Testing The developer shall test the TSF and document the results. The developer shall provide test documentation. The test documentation shall consist of test plans, test procedure descriptions, expected test results and actual test results. The test plans shall identify the security functions to be tested and describe the goal of the tests to be performed. The test procedure descriptions shall identify the tests to be performed and describe the scenarios for testing each security function. These scenarios shall include any ordering dependencies on the results of other tests. The expected test results shall show the anticipated outputs from a successful execution of the tests. The test results from the developer execution of the tests shall demonstrate that each tested security function behaved as specified.

Ordered Functional Testing The test documentation shall include an analysis of the test procedure ordering dependencies.