Kurt Nørmark ©
Department of Computer Science, Aalborg University, Denmark

Abstract Previous lecture Next lecture Index References Contents

This lecture is about test of object-oriented programs.

Testing according to Glen Myers book "The art of Software Testing"

Program testing is the process of executing a program with the intent of finding errors A good test is one that has a high probability of finding an error Program testing cannot show the absence of errors It can only show if errors are present It is possible to write the tests before the program Test driven development

Program testing is very resource and time consuming It is not unusual to spend 40% of the total project efforts on testing

Overview of Program Testing Slide Annotated slide Contents Index References Textbook

Types of test White box test Black box test Levels of test Unit test Integration test System test Repetition of test Regression test

Testability Slide Annotated slide Contents Index References Textbook

Testability refers to the software qualities that affect our ability to reveal errors

Observability The outcome of a test execution must be approachable and visible Controllability The program input and state must be controllable prior to test execution Decomposability A program must be broken into parts which can be tested individually Understandability The specification of the program - what is the correct program behaviour - must be available

Design for Testability

Test Utopia Slide Annotated slide Contents Index References Textbook

In the ideal world, all possible paths through a programs should be tested

The concept total test: A total test is a combinatorial complete test of all possible paths through a program

Total test All possible combinatoric combinations of commands and expressions should be executed relative to the control structures in a program Even in small programs, the number of paths in a total test is very large In many programs, there is no upper limit on the number of paths in a total test

In the real world, we need to select a subset of the paths in a total test which most likely will reveal possible program errors

From Program Test to Program Proof Slide Annotated slide Contents Index References Textbook

A mathematical program proof is an alternative to program testing

Program proofs Prove that the postcondition of the program is true, given that The precondition of the program is true The program terminates Relies on A decoration of the program with mathematical assertions which reflect the intended result of the program relative to the initial program assumptions Proof rules for all constructs of the program

Manually performed program proofs are likely to have more errors in the proof than in the program Therefore only automatic program proofs will improve our confidence

White box testing uses the control structures of a program unit to derive test cases The aim of white box testing is to guarantee that all commands and expressions in a given program are executed at least once

Reference Contrast to black box testing

Wish list to white box testing: All independent paths of a program unit has been executed at least once All logical branches have been exercised All loops and datastructures have been exercised at their boundaries

Reference Software Engineering - A Practitioner's Approach: Roger S. Pressman, McGraw-Hill, 1992

Basis Path Testing Slide Annotated slide Contents Index References Textbook

Basis path testing is a white box testing technique that helps select a minimal set of paths through a program unit that covers all statements and conditions

The concept path: A path through a program unit is a sequence of commands and conditions that starts at program entry and ends at program exit
The concept independent path: An independent path is path that adds at least one new command or condition relative to already identified independent paths

Overall approach Draw a flow chart of the program unit Abstract the flow chart to a flow graph Find the cyclomatic complexity (say n) - a test metric Identify n test cases that follow each of the independent paths

Cyclomatic Complexity - flow chart Slide Annotated slide Contents Index References Textbook

Program: A method P - emphasizing the program flow inside P.

public static void P(){ // Entry while(A){ X; if (B){ if(C) Y; else Z; // p } else{ V; W; } // q } // Exit: r }

Figure. The flow chart for the program unit P

Cyclomatic Complexity - flow graph Slide Annotated slide Contents Index References Textbook

A slight abstraction of the flow chart leads to a so-called flow graph

Figure. The accompanying flow chart for the program unit P

Table.

A, r A, X, B, C, Y, p, q, A, r A, X, B, C, Z, p, q, A, r A, X, B, V, W, q, A, r

Cyclomatic Complexity - Metric and Test Cases Slide Annotated slide Contents Index References Textbook

Calculation of the test metric and finding test cases

The cyclomatic complexity of the program unit P is determined Can be done the flow graph - by use of simple graph theory concepts The number of regions of the flow graph   or The number of predicate notes + 1 Construct a test case for each independent path The test case should follow the control-flow of the path

Exercise 14.2. Cyclomatic complexity of GCD

Find the cyclomatic complexity of Euclid's, gcd, function in this C program. If possible, find test cases that allow you to test each independent path of the gcd function. Independent paths are defined here in course material. How many test cases do you actually need to cover all source lines of the gcd function?

Black box testing uses the interface of a program unit to derive test cases The aim of black box testing is to demonstrate that the program unit produces correct output on suitable input - relative to the functional requirements

Reference Contrast to white box testing

Wish list of black box testing Locate functional errors Do we get the expected result on given inputs to a method? Locate interface errors Are data passed correctly to and from methods? Locate efficiency errors Is the method fast enough?

Input to a Black Box Test Slide Annotated slide Contents Index References Textbook

It is not realistic - from a combinatoric point of view - to test a program unit on all possible inputs Therefore, the choice of input values to black box testing is an important concern

The concept equivalence partitioning: An equivalence partitioning splits a set of input values in a (small) number of classes. It is hypothesized that it is sufficient to test the program unit on a single representative from each class

Include boundary representatives Such as empty collections and full collections Include one or more representative for invalid input Typecheck and use of preconditions may eliminate this need

Example of Equivalence Partitioning (1) Slide Annotated slide Contents Index References Textbook

Program: The method signature SwapElements.

// Exchange element i and j in table public void SwapElements<T>(T[] table, int i, int j){ ... }

Equivalence partitions - all combinations of table table is empty, table is singular, table with two or more elements i and j One of i and j are outside the bounds of table i inside and j outside, i outside and j inside, both are outside Both i and j are inside the bounds of table i < j, i > j, i = j

Use of a stronger precondition limits the number of test cases

Example of Equivalence Partitioning (2) Slide Annotated slide Contents Index References Textbook

Program: The method signature FindMaxIndex.

// Find the largest element in table in between the indexes from and to. // Return the index of this element. // Precondition: 0 <= from <= to <= table.Length-1 public int FindMaxIndex<T>(T[] table, int from, int to) where T: IComparable<T> { ... }

Equivalence partitions - all combinations of table table is empty/singular, largest element is first in table, largest element is last in table, largest element is in mid of table, all elements in table are equal from and to from and to are located at the boundaries of table from = to from = to - 1 from and to are not close to each other

Regression testing Slide Annotated slide Contents Index References Textbook

The purpose of regression test is to find regression errors A regression error is an error in a previously correct program, which recently has been modified A regression error is an error which not used to be there

After modification of part P in a program Q Test that the part P works correctly Test that the overall program Q has not been affected by the modification

Execution of regression test Very time consuming if it is left as a manual activity Automation is crucial Re-run all tests upon each modification   and/or Re-run all test every night

Which kind of program units are tested? What is the smallest testing unit? The program units for which there exists at least one test case

Possible units Packages (namespaces, assemblies) Types (classes, structs) Members of types: Procedures, functions and methods Single commands and expressions

In an object-oriented program, the test units are the individual, public operations in each class

Unit Testing Slide Annotated slide Contents Index References Textbook

Unit testing aims to ensure that each individual part of a program works correct in isolation Unit testing should assure correct behaviour of the given unit before it is integrated with other units Unit testing is black box testing

A Unit Test example in C# (1) Slide Annotated slide Contents Index References Textbook

Unit test of the class BankAccount

Reference Attributes

Program: The class BankAccount.

using System; public class BankAccount { private double interestRate; private string owner; private double balance; public BankAccount(string owner): this(owner, 0.0, 0.0) { } public BankAccount(string owner, double balance): this(owner, balance, 0.0) { } public BankAccount(string owner, double balance, double interestRate) { this.interestRate = interestRate; this.owner = owner; this.balance = balance; } public double Balance { get{ return balance; } } public double InterestRate { get{ return interestRate; } } public void Withdraw (double amount) { balance -= amount; } public void Deposit (double amount) { balance += amount; } public void AddInterests() { balance = balance + balance * interestRate; } public override string ToString() { return owner + "'s account holds " + + balance + " kroner"; } }

Program: The class BankAccountTest.

using System; using NUnit.Framework; [TestFixture] public class BankAccountTest{ BankAccount ba1, ba2, ba3; const double tol = 0.000001; [SetUp] public void Init(){ ba1 = new BankAccount("Peter"); ba2 = new BankAccount("Jens", 1000.0); ba3 = new BankAccount("Martin", 2000.0, 0.03); } [Test] public void InitTest(){ Assert.AreEqual(0.0, ba1.Balance, tol, "ba1 init"); Assert.AreEqual(1000.0, ba2.Balance, tol, "ba2 init balance"); Assert.AreEqual(2000.0, ba3.Balance, tol, "ba3 init balance"); Assert.AreEqual(0.0, ba1.InterestRate, tol, "ba1 interest rate"); Assert.AreEqual(0.0, ba2.InterestRate, tol, "ba2 interest rate"); Assert.AreEqual(0.03, ba3.InterestRate, tol, "ba3 interest rate"); } [Test] public void DepositTest(){ ba1.Deposit(100); ba2.Deposit(100); ba3.Deposit(100); Assert.AreEqual(100.0, ba1.Balance, tol, "ba1 deposit"); Assert.AreEqual(1100.0, ba2.Balance, tol, "ba2 deposit"); Assert.AreEqual(2100.0, ba3.Balance, tol, "ba3 deposit"); } [Test] public void WithdrawTest(){ ba1.Withdraw(100.0); ba2.Withdraw(100.0); ba3.Withdraw(100.0); Assert.AreEqual(-100.0, ba1.Balance, tol, "ba1 withdraw"); Assert.AreEqual(900.0, ba2.Balance, tol, "ba2 withdraw"); Assert.AreEqual(1900.0, ba3.Balance, tol, "ba3 withdraw"); } [Test] public void AddInterestsTest(){ ba1.AddInterests(); ba2.AddInterests(); ba3.AddInterests(); Assert.AreEqual(0.0, ba1.Balance, tol, "ba1 add interest"); Assert.AreEqual(1000.0, ba2.Balance, tol, "ba2 add interest"); Assert.AreEqual(2060.0, ba3.Balance, tol, "ba3 add interest"); } }

Program: Compilation and execution.

csc /t:library bank-account.cs csc /r:bank-account.dll /r:nunit.framework.dll /t:library bank-account-test.cs Execute bank-account-test.dll with NUnit test runner

Figure. A screenshot of NUnit - all tests are successful

A Unit Test example in C# (2) Slide Annotated slide Contents Index References Textbook

Unit test of an erroneous version of class BankAccount

Program: The class BankAccount - with an error.

using System; public class BankAccount { private double interestRate; private string owner; private double balance; public BankAccount(string owner): this(owner, 0.0, 0.0) { } /* Wrong initialization of interest rate */ public BankAccount(string owner, double balance): this(owner, balance, 0.01) { // Should have been 0.0 } public BankAccount(string owner, double balance, double interestRate) { this.interestRate = interestRate; this.owner = owner; this.balance = balance; } public double Balance { get{ return balance; } } public double InterestRate { get{ return interestRate; } } public void Withdraw (double amount) { balance -= amount; } public void Deposit (double amount) { balance += amount; } public void AddInterests() { balance = balance + balance * interestRate; } public override string ToString() { return owner + "'s account holds " + + balance + " kroner"; } }

Program: The class BankAccountTest - same as before.

using System; using NUnit.Framework; [TestFixture] public class BankAccountTest{ BankAccount ba1, ba2, ba3; const double tol = 0.000001; [SetUp] public void Init(){ ba1 = new BankAccount("Peter"); ba2 = new BankAccount("Jens", 1000.0); ba3 = new BankAccount("Martin", 2000.0, 0.03); } [Test] public void InitTest(){ Assert.AreEqual(0.0, ba1.Balance, tol, "ba1 init "); Assert.AreEqual(1000.0, ba2.Balance, tol, "ba2 init balance"); Assert.AreEqual(2000.0, ba3.Balance, tol, "ba3 init balance"); Assert.AreEqual(0.0, ba1.InterestRate, tol, "ba1 interest rate"); Assert.AreEqual(0.0, ba2.InterestRate, tol, "ba2 interest rate"); Assert.AreEqual(0.03, ba3.InterestRate, tol, "ba3 interest rate"); } [Test] public void DepositTest(){ ba1.Deposit(100); ba2.Deposit(100); ba3.Deposit(100); Assert.AreEqual(100.0, ba1.Balance, tol, "ba1 deposit"); Assert.AreEqual(1100.0, ba2.Balance, tol, "ba2 deposit"); Assert.AreEqual(2100.0, ba3.Balance, tol, "ba3 deposit"); } [Test] public void WithdrawTest(){ ba1.Withdraw(100.0); ba2.Withdraw(100.0); ba3.Withdraw(100.0); Assert.AreEqual(-100.0, ba1.Balance, tol, "ba1 withdraw"); Assert.AreEqual(900.0, ba2.Balance, tol, "ba2 withdraw"); Assert.AreEqual(1900.0, ba3.Balance, tol, "ba3 withdraw"); } [Test] public void AddInterestsTest(){ ba1.AddInterests(); ba2.AddInterests(); ba3.AddInterests(); Assert.AreEqual(0.0, ba1.Balance, tol, "ba1 add interest"); Assert.AreEqual(1000.0, ba2.Balance, tol, "ba2 add interest"); Assert.AreEqual(2060.0, ba3.Balance, tol, "ba3 add interest"); } }

Figure. A screenshot of NUnit - some tests fail

NUnit for C# Slide Annotated slide Contents Index References Textbook

NUnit is a unit testing framework for C#

NUnit characteristics Pieces of test programs are marked by means of particular attributes Correctness is stated by use of assertions: Methods with boolean result Nunit provides a large amount of different assertions The NUnit tools are able to execute the marked test in one or more assemblies In this context, an assembly is typically a dll file Nunit tools A Console Runner A GUI Runner

Exercise 14.4. Install Nunit

Download and install the latest stable version Nunit from www.nunit.org on your computer. If you have not already done so, consult the basic information about NUnit on the accompanying slide. More specifically, for Windows users, goto the NUnit download page and download the most recent stable version of NUNIT for .NET 2.0. As of February 2010 this is the NUnit-2.5.3.9345.msi file (a Windows installer file). You can also get this file directly from here. Mono users on Linux probably already have an installation of NUnit. According to the NUnit documentation, Mono 1.0 through Mono 1.9 include NUnit 2.2. Try calling the NUnit console runner named nunit-console from your shell. Please be aware of the version you are running. The newest stable version of NUnit is 2.5.3 (as of February 2010). Mono users are recommended to use version 2.4.8, however. See also the release notes. Next, consult the NUnit documentation. Pay attention to the menu to the right. In particular the CORE FEATURES documentation of Assertions and Attributes.

Exercise 14.4. Give Nunit a Try

This exercise is guided tour in using NUnit (version 2.5.3) together with Visual C# 2008 Express on Windows. The exercise will help set up the BankAccount test, which we have seen on an earlier slide page in this material. If you use C# via another IDE, or on a non-Windows platform, you should not follow this guide. We will assume that you already have installed Nunit and that you use Visual C# 2008 Express. Start Visual C# 2008 Express. Use File > New Project... and make a Class Library project. In the Solution Explorer (usually at the far right of you screen) right click References, and do Add Reference.... Select and add nunit.framework (version 2.5.3) from the list. (The list may be long, so be careful to select the right entry). Use Project > Add Class... to add the BankAccount class. (Do copy and paste from the BankAccount class of this material). Paste the BankAccountTest class instead of Class1, which was made automatically by Visual C# 2008 Express. Now build your library application: Use F6 as usual. "Build succeeded" is expected. Do File > Save All. Notice the Location of you project. On a piece of paper (or in a text editor) notice the full file path to your Visual C# 2008 project. You can also get information about your project locations via the C# 2008 Express menu entry Tools > Options..., 'Projects and Solutions'. Start NUnit 2.5.3 - for instance via the Icon on your desktop or via the Windows start menu. In NUnit use Tools -> Settings... and check the 'Enable Visual Studio Support' in the window which appears. You find the check box in IDE Support > Visual Studio. (This is most likely the default setting in NUNIT 2.5.3). In NUnit use File > Open Project, and select the dll named after your project. The file is most likely located in the Bin/Debug branch of your project directory (as you should have noticed above). Activate the Run button in NUnit. You should see 'green light'. Introduce an run-time error in BankAccount class. Is the error revealed in unit test? If you managed to get to here you should be able to use NUnit on your own stuff. Congratulations.

NUnit Attributes Slide Annotated slide Contents Index References Textbook

Reference Attributes

[TestFixture] Marks a class that contains tests [Test] Marks a method in a test class. The method becomes a test case. [SetUp] Marks a method which is executed just before each test method. At most one such method per test class [TearDown] Marks a method which is executed after each test method. At most one such method Is not executed if a test fails, of if it throws an exception [TestFixtureSetUp] Marks a method which is executed once before execution of any test method. At most one such method Executed once, before the execution of the first test method in this fixture [TestFixtureTearDown] Marks a method which is executed once after all tests are completed. Executed once, after the execution of the last test method in this fixture At most one such method [ExpectedException(ExceptionType)] Marks a test which is expected to lead to a given Exceptiontype [Category("name")] Categorizes a given test fixture or test Individual categories can be included or excluded in a test-run [Ignore] Marks a test method or a test class which (temporarily) should not be executed

Consult the NUnit documentation for the full and exact set of test-related attributes

NUnit Assertions Slide Annotated slide Contents Index References Textbook

Assert.AreEqual(expected, actual) 18 overloads. With/without message, tolerance, object params Assert.AreNotEqual(expected, actual) 18 overloads. With/without: message Assert.AreSame(expected, actual)            references to same object? 3 overloads. With/without: message, object params Assert.NotSame(expected, actual) 3 overloads. With/without: message, object params Assert.Contains(object, anIList) 3 overloads. With/without: message, object params Assert.Greater(arg1, arg2) 18 overloads. With/without: message, object params Assert.Less(arg1, arg2) 18 overloads. With/without: message, object params Assert.IsInstanceOfType(Type, object) 3 overloads. With/without: message, object params Assert.IsNotInstanceOfType(Type, object) 3 overloads. With/without: message, object params Assert.IsAssignableFrom(Type, object) 3 overloads. With/without: message, object params Assert.IsNotAssignableFrom(Type, object) 3 overloads. With/without: message, object params Assert.IsTrue(aBool) 3 overloads. With/without: message, object params Assert.IsFalse(aBool) 3 overloads. With/without: message, object params Assert.IsNaN(aDouble) 3 overloads. With/without: message, object params Assert.IsEmpty(aString) 3 overloads. With/without: message, object params Assert.IsEmpty(anICollection) 3 overloads. With/without: message, object params Assert.IsNotEmpty(aString) 3 overloads. With/without: message, object params Assert.IsNotEmpty(anICollection) 3 overloads. With/without: message, object params Assert.Fail() 3 overloads. With/without: message, object params StringAssert.Contains(expectedString, actualString) 3 overloads. With/without: message, object params StringAssert.StartsWith(expectedString, actualString) 3 overloads. With/without: message, object params StringAssert.EndsWith(expectedString, actualString) 3 overloads. With/without: message, object params StringAssert.AreEqualIgnoringCase(expectedString, actualString) 3 overloads. With/without: message, object params

Consult the NUnit documentation for the full and exact set of Assertions

Unit Test Concepts Slide Annotated slide Contents Index References Textbook

Understanding the common concepts of Unit testing

Assertion A boolean function which compares some value or state with its expected result "An atomic test" - true means pass, false means fail Test or Test case Execution of a piece of code followed by activation of one or more assertions Test suite An aggregation of test cases Test fixture A fixed state used as a baseline for running a set of tests [Wikipedia] Used to ensure that each test is executed in a fixed and well-defined state, such that a test is repeatable

Test cases and Test suites may be organized as a Composite

Reference The Composite design pattern

Another Unit Test example in C# Slide Annotated slide Contents Index References Textbook

Unit test of the class Set<T> from the lecture about generics

Program: The class Set<T>.

using System; using System.Collections.Generic; using System.Collections; public class Set<T> { /* Invariant: At most one occurrence of an element in the array store Consequtive storing from low end. */ private int capacity; private static int DefaultCapacity = 3; private T[] store; private int next; public Set(int capacity){ this.capacity = capacity; store = new T[capacity]; next = 0; } public Set(): this(DefaultCapacity){ } public Set(T[] elements): this(elements.Length){ foreach(T el in elements) this.Insert(el); } // Copy constructor public Set(Set<T> s): this(s.capacity){ foreach(T el in s) this.Insert(el); } public bool Member(T element){ for(int idx = 0; idx < next; idx++) if (element.Equals(store[idx])) return true; return false; } public void Insert(T element){ if (!this.Member(element)){ if (this.Full){ Console.WriteLine("[Resize to {0}]", capacity * 2); Array.Resize<T>(ref store, capacity * 2); capacity = capacity * 2; } store[next] = element; next++; } } public void Delete(T element){ bool found = false; int foundIdx = 0; for(int idx = 0; !found && (idx < next); idx++){ if (element.Equals(store[idx])){ found = true; foundIdx = idx; } } if (found){ // shift remaining elements left for(int idx = foundIdx+1; idx < next; idx++) store[idx-1] = store[idx]; store[next-1] = default(T); next--; } } public int Count{ get{ return next; } } // Is this set a subset of other public bool Subset(Set<T> other){ foreach(T e in this) if (!other.Member(e)) return false; return true; } public Set<T> Intersection(Set<T> other){ Set<T> res = new Set<T>(this.Count); foreach(T e in this) if (other.Member(e)) res.Insert(e); return res; } public Set<T> Union(Set<T> other){ Set<T> res = new Set<T>(this.Count + other.Count); foreach(T e in this) res.Insert(e); foreach(T e in other) res.Insert(e); return res; } // Subtract other elements from this set. public Set<T> Diff(Set<T> other){ Set<T> res = new Set<T>(this); foreach(T e in other) res.Delete(e); return res; } public override string ToString(){ string elRes = ""; for(int idx = 0; idx < next; idx++) if (store[idx] != null) elRes += " " + store[idx]; return "{" + elRes + " "+ "}" + this.Count; } private class SetEnumerator: IEnumerator<T>{ private readonly Set<T> set; private int idx; public SetEnumerator (Set<T> s){ this.set = s; idx = -1; // position enumerator outside range } public T Current{ get { return set.store[idx]; } } Object IEnumerator.Current{ get { return set.store[idx]; } } public bool MoveNext(){ if (idx < set.next - 1){ idx++; return true; } else return false; } public void Reset(){ idx = -1; } public void Dispose(){ } } public IEnumerator<T> GetEnumerator (){ return new SetEnumerator(this); } private bool Full{ get{ return next == capacity; } } }

Program: The class SetTest.

using System; using NUnit.Framework; [TestFixture] public class SetTest{ Set<int> set1, set2, set3, set4; [SetUp] public void Init(){ set1 = new Set<int>(); set2 = new Set<int>(10); set3 = new Set<int>(new int[]{1,2,3}); set4 = new Set<int>(set3); } [Test] public void MemberTest(){ Assert.IsFalse(set1.Member(1), "Member test 1"); Assert.IsFalse(set2.Member(1), "Member test 2"); Assert.IsTrue(set3.Member(1), "Member test 3"); Assert.IsTrue(set4.Member(1), "Member test 4"); Assert.IsFalse(set3.Member(4), "Member test 5"); Assert.IsFalse(set4.Member(4), "Member test 6"); } [Test] public void InsertNonMember(){ set1.Insert(1); Assert.AreEqual(set1.Count, 1, "Insert test 1"); Assert.IsTrue(set1.Member(1), "Insert test 2"); set3.Insert(4); Assert.AreEqual(set3.Count, 4, "Insert test 3"); set4.Insert(4); Assert.AreEqual(set4.Count, 4, "Insert test 4"); } [Test] public void InsertMember(){ set1.Insert(1); set1.Insert(1); Assert.AreEqual(set1.Count, 1, "Insert test 1"); Assert.IsTrue(set1.Member(1), "Insert test 2"); set3.Insert(3); Assert.AreEqual(set3.Count, 3, "Insert test 3"); set4.Insert(3); Assert.AreEqual(set4.Count, 3, "Insert test 4"); } [Test] public void DeleteNonMember(){ set1.Delete(1); Assert.AreEqual(set1.Count, 0, "Delete test 1"); set3.Delete(4); Assert.AreEqual(set3.Count, 3, "Delete test 2"); } [Test] public void DeleteMember(){ set3.Delete(1); Assert.AreEqual(set3.Count, 2, "Delete test 1"); Assert.IsFalse(set3.Member(1), "Delete test 2"); } // Delete member twice [Test] public void DeleteMemberTwice(){ set3.Delete(1); set3.Delete(1); Assert.AreEqual(set3.Count, 2, "Delete test 1"); Assert.IsFalse(set3.Member(1), "Delete test 2"); } [Test] public void MultipleDelete(){ set3.Delete(1); set3.Delete(2); set3.Delete(3); Assert.AreEqual(set3.Count, 0, "Delete test 1"); Assert.IsFalse(set3.Member(1), "Delete test 2"); Assert.IsFalse(set3.Member(2), "Delete test 3"); Assert.IsFalse(set3.Member(3), "Delete test 4"); } }

Exercise 14.5. Test of class Set

In a previous exercise we have implemented the operations intersection, union, and set difference in class Set<T>. In continuation of class SetTest, perform unit tests of the intersection, union, and set difference operations. The natural starting point is your solution to the previous exercise. You can also chose to test my solution.

Unit test of struct Interval Slide Annotated slide Contents Index References

Struct Interval was primarily used to introduce overloaded operators In this lecture we will unit test struct Interval

Program: A slightly revised version of the Interval struct - with an error.

/* There is a deliberately placed error in this program */ using System; using System.Collections; /* An interval with orientation */ public class IntervalAccessException: ApplicationException{ } public struct Interval{ private readonly int from, to; private bool empty; public Interval(int from, int to){ this.empty = false; this.from = from; this.to = to; } /* Constructs an empty interval. This cannot be done elegantly by a constructor, because parameterless constructors cannot be used for structs */ public static Interval MakeEmptyInterval (){ Interval res = new Interval(); res.empty = true; return res; } /* Assumed as a precondition that the interval is non-empty */ public int From{ get {if (this.Empty) throw new IntervalAccessException(); else return from;} } /* Assumed as a precondition that the interval is non-empty */ public int To{ get {if (this.Empty) throw new IntervalAccessException(); else return to;} } public bool Empty{ get {return empty;} } /* Return the orientation of the interval. Returns -1 in case from is larger than to, 1 if from is small than to, and 0 is the interval is empty or singular. */ private int Orientation { get{ int res; if (empty) res = 0; else if (from < to) res = 1; else if (to < from) res = -1; else res = 0; return res; } } public int Length{ get {return empty? 0: Math.Abs(to - from) + 1;} } /* Return element i of the interval, relative to its From edge. Like for arrays, the indexing is zero-bound */ public int this[int i]{ get {if (empty) throw new IndexOutOfRangeException("Error"); else if (from <= to){ if (i >= 0 && i <= Math.Abs(from-to)) return from + i; else throw new IndexOutOfRangeException("Error"); } else if (from > to){ if (i >= 0 && i <= Math.Abs(from-to)) return from - i; else throw new IndexOutOfRangeException("Error"); } else throw new Exception("Should not happen"); } } /* The current interval determines the orientation of the resulting interval */ public Interval OverlapWith (Interval other){ // Enforce positively oriented intervals: Interval thisPI = (this.Orientation < 0) ? !this : this, otherPI = (other.Orientation < 0) ? !other : other, res; /* In the if-else chain we work with positively oriented intervals In such intervals From <= To: */ if (thisPI.Empty || otherPI.Empty) // both empty res = MakeEmptyInterval(); else if (thisPI.From > otherPI.To || thisPI.To < otherPI.From) // disjoint res = MakeEmptyInterval(); else if (thisPI.From < otherPI.From && otherPI.To < thisPI.To) // other inside this res = thisPI; else if (otherPI.From <= thisPI.From && thisPI.To <= otherPI.To) // this inside other res = otherPI; else if (thisPI.From <= otherPI.From && otherPI.From <= thisPI.To) // this overlaps left border of other res = new Interval(otherPI.From, thisPI.To); else if (otherPI.From <= thisPI.From && thisPI.From <= otherPI.To) // other overlaps left border of this res = new Interval(thisPI.From, otherPI.To); else throw new Exception("Should not happen"); // Reintroduce orientation: return (this.Orientation < 0) ? !res : res; } public static Interval operator +(Interval i, int j){ return i.empty ? MakeEmptyInterval() : new Interval(i.From + j, i.To + j); } public static Interval operator +(int j, Interval i){ return i.Empty ? MakeEmptyInterval() : new Interval(i.From + j, i.To + j); } public static Interval operator >>(Interval i, int j){ return i.Empty ? MakeEmptyInterval() : new Interval(i.From, i.To + j); } public static Interval operator <<(Interval i, int j){ return i.Empty ? MakeEmptyInterval() : new Interval(i.From + j, i.To); } public static Interval operator *(Interval i, int j){ return i.Empty ? MakeEmptyInterval() : new Interval(i.From * j, i.To * j); } public static Interval operator *(int j, Interval i){ return i.Empty ? MakeEmptyInterval() : new Interval(i.From * j, i.To * j); } public static Interval operator -(Interval i, int j){ return i.Empty ? MakeEmptyInterval() : new Interval(i.From - j, i.To - j); } public static Interval operator !(Interval i){ return i.Empty ? MakeEmptyInterval() : new Interval(i.To, i.From); } public static explicit operator int[] (Interval i){ int[] res = new int[i.Length]; for (int j = 0; j < i.Length; j++) res[j] = i[j]; return res; } private class IntervalEnumerator: IEnumerator{ private readonly Interval interval; private int idx; public IntervalEnumerator (Interval i){ this.interval = i; idx = -1; // position enumerator outside range } public Object Current{ get {return (interval.From < interval.To) ? interval.From + idx : interval.From - idx;} } public bool MoveNext (){ if (interval.Empty) return false; else if ( idx < Math.Abs(interval.To - interval.From)) {idx++; return true;} else {return false;} } public void Reset(){ idx = -1; } } public IEnumerator GetEnumerator (){ return new IntervalEnumerator(this); } }

Program: An initial test suite of struct Interval.

using System; using NUnit.Framework; [TestFixture] public class IntervalTest{ Interval iv_1, iv_2, iv_3, iv_4; [SetUp] public void Init(){ iv_1 = new Interval(3,8); iv_2 = new Interval(8,3); iv_3 = new Interval(3,3); iv_4 = Interval.MakeEmptyInterval(); } [Test] public void FromTest1(){ Assert.AreEqual(3, iv_1.From, "Fromtest: Positive orientation"); } [Test] public void FromTest2(){ Assert.AreEqual(8, iv_2.From, "Fromtest: Negative orientation"); } [Test] public void FromTest3(){ Assert.AreEqual(3, iv_3.From, "Fromtest: Singleton"); } [Test, ExpectedException("IntervalAccessException")] public void FromTestEmpty(){ int res = iv_4.From; } [Test] public void ToTest1(){ Assert.AreEqual(8, iv_1.To, "ToTest: Positive orientation"); } [Test] public void ToTest2(){ Assert.AreEqual(3, iv_2.To, "ToTest: Negative orientation"); } [Test] public void ToTest3(){ Assert.AreEqual(3, iv_3.To, "ToTest: Singleton"); } [Test, ExpectedException("IntervalAccessException")] public void ToTestEmpty(){ int res = iv_4.To; } [Test] public void LengthTest1(){ Assert.AreEqual(6, iv_1.Length, "Length positive orientation"); } [Test] public void LengthTest2(){ Assert.AreEqual(6, iv_2.Length, "Length negative orientation"); } [Test] public void LengthTest3(){ Assert.AreEqual(1, iv_3.Length, "Length singleton"); } [Test] public void LengthTest4(){ Assert.AreEqual(0, iv_4.Length, "Length empty"); } [Test] public void EmptyTest1(){ Assert.IsFalse(iv_1.Empty, "Empty positive orientation"); } [Test] public void EmptyTest2(){ Assert.IsFalse(iv_2.Empty, "Empty negative orientation"); } [Test] public void EmptyTest3(){ Assert.IsFalse(iv_3.Empty, "Empty singleton"); } [Test] public void EmptyTest4(){ Assert.IsTrue(iv_4.Empty, "Empty - Empty"); } [Test] public void PlusTest1(){ Interval iv = new Interval(3,5); Assert.AreEqual((iv+3).From, 6, "Plus - Interval + integer - From limit"); } [Test] public void PlusTest2(){ Interval iv = new Interval(3,5); Assert.AreEqual((iv+3).To, 8, "Plus - Interval + integer - From limit"); } [Test] public void PlusTest3(){ Interval iv = new Interval(3,5); Assert.AreEqual((3+iv).From, 6, "Plus - integer + Interval - From limit"); } [Test] public void PlusTest4(){ Interval iv = new Interval(3,5); Assert.AreEqual((3+iv).To, 8, "Plus - integer + Interval - To limit"); } }

Exercise 14.6. Unit test of struct Interval

You are given a version of struct Interval, which we have worked with in an earlier exercise. You are also given a partial test suite of struct Interval. In the given Interval test suite all tests should be successful. (You may have to adjust the full qualified names of exceptions in the ExpectedException attributes, however). Please check for yourself that all tests pass successfully. Add a test of the indexer and a test of the OverlapWith method. Both are members of type Interval, as given above. In the version of struct Interval from above there is at least one error in either the indexer or the OverlapWith method. You should be able to reveal the error from your tests! Please correct the error when you have found it. Carry out a regression test. There are several overloaded operators in struct Interval. I have only made a test of the + operator. If time allows, add some additional tests of the non-tested Interval operators. Finally, discuss the expected coverage of my Interval test cases together with your additional Interval test cases.

Test Scaffolding Slide Annotated slide Contents Index References Textbook

The test scaffolding denotes the auxilliary programs and classes that allow us to test a given program unit

Test scaffolding

The test units depends on a number of other units These units are established as stubs possibly in the form of mockups The test cases are executed by means of driver program

It appears attractive to test the classes bottom up in order to avoid excessive use of stubs

The Background and Context of Unit Testing Slide Annotated slide Contents Index References Textbook

Unit testing was was popularized for Java by tool called JUnit Kent Beck and Erich Gamma are the originators of unit testing

JUnit Java classes and interfaces for organization and execution of test methods Explicit or implicit activation of test methods More recently being based on annotations in Java, like the attributes in C# as supported by NUnit.

Unit testing is a cornerstone in Extreme Programming

Test of Object-oriented programs Slide Annotated slide Contents Index References Textbook

What are the basic challenges of testing object-oriented programs?

Button-up OOP development Makes it natural to test the classes bottom-up Bottom-up testing of classes relieves the need for stubs Information hiding Makes it difficult/impossible to directly observe the effect of operations on an object Dynamic binding and polymorphism May contribute with a lot additional independent paths It is difficult to identify these paths on a static basis Complicates white box testing considerably

Test Driven Development Slide Annotated slide Contents Index References Textbook

Test a little. Code a little.

Test driven development Start by writing the test cases Initially, all test cases are likely to fail The test cases serve both as program specification and documentation Implement the program The implementation is done when all tests succeed Refactor the program to improve its quality Do full regression test in between each modification

Unit Test Recommendation Slide Annotated slide Contents Index References Textbook

Test case independence Each test should examine a single program aspect - in isolation from other aspects Test simplicity Test code should be simple Each test should only activate a single assertion Contrary to most examples that have been shown earlier in this lecture Because the first assertion that fails prevents execution of additional assertions Testing of test code is simply too much Test coverage Each public operation in a class should have at least one associated test Typically a number of tests per operations Indeed a very loose meaning of coverage! Regression test For each modification - at a reasonable grain size - re-execute all tests

Test Recommendations in your Project Slide Annotated slide Contents Index References

Recommendations for program test in your project

Use of NUnit Use unit testing for a subset of the non-trivial classes in the project Get experience with test-driven development on selected classes Test a little. Code a little. ... Only relevant for classes which have not yet been implemented

Collected references Contents Index

Contrast to black box testing Software Engineering - A Practitioner's Approach: Roger S. Pressman, McGraw-Hill, 1992 Contrast to white box testing Attributes The Composite design pattern

Chapter 14: Test of Object-oriented Programs
Course home     Author home     About producing this web     Previous lecture (top)     Next lecture (top)     Previous lecture (bund)     Next lecture (bund)     
Generated: February 7, 2011, 12:22:50