Unit Test of Object-oriented Programs

Previous theme in this lecture -- Keyboard shortcut: 'p'

Next slide in this lecture -- Keyboard shortcut: 'n'

Test of Object-oriented Programs

A complete PDF version of the text book is now available. The PDF version is an almost complete subset of the HTML version (where only a few, long program listings have been removed). See here.

55. Unit Test of Object-oriented Programs

55.1 Test Units 55.8 Unit Test Concepts
55.2 Unit Testing 55.9 Another Unit Test example in C#
55.3 A Unit Test example in C# (1) 55.10 Test Scaffolding
55.4 A Unit Test example in C# (2) 55.11 The Background and Context of Unit Testing
55.5 NUnit for C# 55.12 Test Driven Development
55.6 NUnit Attributes 55.13 Unit Test Recommendation
55.7 NUnit Assertions 55.14 Test of Object-oriented programs

55.1. Test Units
Contents Up Previous Next Slide Annotated slide Aggregated slides Subject index Program index Exercise index

Which kind of program units are tested?

What is the smallest testing unit?

The program units for which there exists at least one test case

Possible units
Packages (namespaces, assemblies)
Types (classes, structs)
Members of types: Procedures, functions and methods
Single commands and expressions

In an object-oriented program, the test units are the individual, public operations in each class

55.2. Unit Testing
Contents Up Previous Next Slide Annotated slide Aggregated slides Subject index Program index Exercise index

Unit testing aims to ensure that each individual part of a program works correct in isolation

Unit testing should assure correct behaviour of the given unit before it is integrated with other units

Unit testing is black box testing

55.3. A Unit Test example in C# (1)
Contents Up Previous Next Slide Annotated slide Aggregated slides Subject index Program index Exercise index

Unit test of the class BankAccount

using System;

public class BankAccount {

   private double interestRate;
   private string owner;
   private double balance;

   public BankAccount(string owner): 
     this(owner, 0.0, 0.0) {
   }

   public BankAccount(string owner, double balance): 
     this(owner, balance, 0.0) {
   }

   public BankAccount(string owner, double balance, double interestRate) {
      this.interestRate = interestRate;
      this.owner = owner; 
      this.balance = balance;
   }

   public double Balance {
     get{
      return balance;
     }
   }

   public double InterestRate {
     get{
      return interestRate;
     }
   }

   public void Withdraw (double amount) {
      balance -= amount;
   }

   public void Deposit (double amount) {
      balance += amount;
   }

   public void AddInterests() {
      balance = balance + balance * interestRate;
   }    

   public override string ToString() {
      return owner + "'s account holds " +
            + balance + " kroner";
   }
}

Program 55.1 The class BankAccount.

using System;
using NUnit.Framework;

[TestFixture]
public class BankAccountTest{

  BankAccount ba1, ba2, ba3;
  const double tol = 0.000001;

  [SetUp] 
  public void Init(){
    ba1 = new BankAccount("Peter");
    ba2 = new BankAccount("Jens", 1000.0);
    ba3 = new BankAccount("Martin", 2000.0, 0.03);
  }

  [Test]
  public void InitTest(){
    Assert.AreEqual(0.0, ba1.Balance, tol, "ba1 init");
    Assert.AreEqual(1000.0, ba2.Balance, tol, "ba2 init balance");
    Assert.AreEqual(2000.0, ba3.Balance, tol, "ba3 init balance");

    Assert.AreEqual(0.0, ba1.InterestRate, tol, "ba1 interest rate");
    Assert.AreEqual(0.0, ba2.InterestRate, tol, "ba2 interest rate");
    Assert.AreEqual(0.03, ba3.InterestRate, tol, "ba3 interest rate");
  }    
  
  [Test]
  public void DepositTest(){
    ba1.Deposit(100); ba2.Deposit(100); ba3.Deposit(100);
    Assert.AreEqual(100.0, ba1.Balance, tol, "ba1 deposit");
    Assert.AreEqual(1100.0, ba2.Balance, tol, "ba2 deposit");
    Assert.AreEqual(2100.0, ba3.Balance, tol, "ba3 deposit");
  }

  [Test]
  public void WithdrawTest(){
    ba1.Withdraw(100.0);  ba2.Withdraw(100.0);  ba3.Withdraw(100.0);
    Assert.AreEqual(-100.0, ba1.Balance, tol, "ba1 withdraw");
    Assert.AreEqual(900.0, ba2.Balance, tol, "ba2 withdraw");
    Assert.AreEqual(1900.0, ba3.Balance, tol, "ba3 withdraw");
  }

  [Test]
  public void AddInterestsTest(){
    ba1.AddInterests();  ba2.AddInterests();  ba3.AddInterests();
    Assert.AreEqual(0.0, ba1.Balance, tol, "ba1 add interest");
    Assert.AreEqual(1000.0, ba2.Balance, tol, "ba2 add interest");
    Assert.AreEqual(2060.0, ba3.Balance, tol, "ba3 add interest");
  }

}

Program 55.2 The class BankAccountTest.

Figure 55.1 A screenshot of NUnit - all tests are successful

55.4. A Unit Test example in C# (2)
Contents Up Previous Next Slide Annotated slide Aggregated slides Subject index Program index Exercise index

Unit test of an erroneous version of class BankAccount

using System;

public class BankAccount {

   private double interestRate;
   private string owner;
   private double balance;

   public BankAccount(string owner): 
     this(owner, 0.0, 0.0) {
   }

   /* Wrong initialization of interest rate */
   public BankAccount(string owner, double balance): 
     this(owner, balance, 0.01) {           // Should have been 0.0
   }

   public BankAccount(string owner, double balance, double interestRate) {
      this.interestRate = interestRate;
      this.owner = owner; 
      this.balance = balance;
   }

   public double Balance {
     get{
      return balance;
     }
   }

   public double InterestRate {
     get{
      return interestRate;
     }
   }

   public void Withdraw (double amount) {
      balance -= amount;
   }

   public void Deposit (double amount) {
      balance += amount;
   }

   public void AddInterests() {
      balance = balance + balance * interestRate;
   }    

   public override string ToString() {
      return owner + "'s account holds " +
            + balance + " kroner";
   }
}

Program 55.3 The class BankAccount - with an error.

using System;
using NUnit.Framework;

[TestFixture]
public class BankAccountTest{

  BankAccount ba1, ba2, ba3;
  const double tol = 0.000001;

  [SetUp] 
  public void Init(){
    ba1 = new BankAccount("Peter");
    ba2 = new BankAccount("Jens", 1000.0);
    ba3 = new BankAccount("Martin", 2000.0, 0.03);
  }

  [Test]
  public void InitTest(){
    Assert.AreEqual(0.0, ba1.Balance, tol, "ba1 init ");
    Assert.AreEqual(1000.0, ba2.Balance, tol, "ba2 init balance");
    Assert.AreEqual(2000.0, ba3.Balance, tol, "ba3 init balance");

    Assert.AreEqual(0.0, ba1.InterestRate, tol, "ba1 interest rate");
    Assert.AreEqual(0.0, ba2.InterestRate, tol, "ba2 interest rate");
    Assert.AreEqual(0.03, ba3.InterestRate, tol, "ba3 interest rate");
  }    
  
  [Test]
  public void DepositTest(){
    ba1.Deposit(100); ba2.Deposit(100); ba3.Deposit(100);
    Assert.AreEqual(100.0, ba1.Balance, tol, "ba1 deposit");
    Assert.AreEqual(1100.0, ba2.Balance, tol, "ba2 deposit");
    Assert.AreEqual(2100.0, ba3.Balance, tol, "ba3 deposit");
  }

  [Test]
  public void WithdrawTest(){
    ba1.Withdraw(100.0);  ba2.Withdraw(100.0);  ba3.Withdraw(100.0);
    Assert.AreEqual(-100.0, ba1.Balance, tol, "ba1 withdraw");
    Assert.AreEqual(900.0, ba2.Balance, tol, "ba2 withdraw");
    Assert.AreEqual(1900.0, ba3.Balance, tol, "ba3 withdraw");
  }

  [Test]
  public void AddInterestsTest(){
    ba1.AddInterests();  ba2.AddInterests();  ba3.AddInterests();
    Assert.AreEqual(0.0, ba1.Balance, tol, "ba1 add interest");
    Assert.AreEqual(1000.0, ba2.Balance, tol, "ba2 add interest");
    Assert.AreEqual(2060.0, ba3.Balance, tol, "ba3 add interest");
  }

}

Program 55.4 The class BankAccountTest - same as before.

Figure 55.2 A screenshot of NUnit - some tests fail

55.5. NUnit for C#
Contents Up Previous Next Slide Annotated slide Aggregated slides Subject index Program index Exercise index

NUnit is a unit testing framework for C#

NUnit characteristics
Pieces of test programs are marked by means of particular attributes
Correctness is stated by use of assertions: Methods with boolean result
Nunit provides a large amount of different assertions
The NUnit tools are able to execute the marked test in one or more assemblies
In this context, an assembly is typically a dll file
Nunit tools
A Console Runner
A GUI Runner

Exercise 14.2. Install Nunit

Download and install the latest stable version Nunit from www.nunit.org on your computer. If you have not already done so, consult the basic information about NUnit on the accompanying slide.

More specifically, for Windows users, goto the NUnit download page and download the most recent stable version of NUNIT for .NET 2.0. As of February 2010 this is the NUnit-2.5.3.9345.msi file (a Windows installer file). You can also get this file directly from here.

Mono users on Linux probably already have an installation of NUnit. According to the NUnit documentation, Mono 1.0 through Mono 1.9 include NUnit 2.2. Try calling the NUnit console runner named nunit-console from your shell. Please be aware of the version you are running. The newest stable version of NUnit is 2.5.3 (as of February 2010). Mono users are recommended to use version 2.4.8, however. See also the release notes.

Next, consult the NUnit documentation. Pay attention to the menu to the right. In particular the CORE FEATURES documentation of Assertions and Attributes.

There is no solution to this exercise

55.6. NUnit Attributes
Contents Up Previous Next Slide Annotated slide Aggregated slides Subject index Program index Exercise index

[TestFixture]
Marks a class that contains tests
[Test]
Marks a method in a test class. The method becomes a test case.
[SetUp]
Marks a method which is executed just before each test method.
At most one such method per test class
[TearDown]
Marks a method which is executed after each test method.
At most one such method
Is not executed if a test fails, of if it throws an exception
[TestFixtureSetUp]
Marks a method which is executed once before execution of any test method.
At most one such method
Executed once, before the execution of the first test method in this fixture
[TestFixtureTearDown]
Marks a method which is executed once after all tests are completed.
Executed once, after the execution of the last test method in this fixture
At most one such method
[ExpectedException(ExceptionType)]
Marks a test which is expected to lead to a given Exceptiontype
[Category("name")]
Categorizes a given test fixture or test
Individual categories can be included or excluded in a test-run
[Ignore]
Marks a test method or a test class which (temporarily) should not be executed

55.7. NUnit Assertions
Contents Up Previous Next Slide Annotated slide Aggregated slides Subject index Program index Exercise index

Assert.AreEqual(expected, actual)
18 overloads. With/without message, tolerance, object params
Assert.AreNotEqual(expected, actual)
18 overloads. With/without: message
Assert.AreSame(expected, actual) references to same object?
3 overloads. With/without: message, object params
Assert.NotSame(expected, actual)
3 overloads. With/without: message, object params
Assert.Contains(object, anIList)
3 overloads. With/without: message, object params
Assert.Greater(arg1, arg2)
18 overloads. With/without: message, object params
Assert.Less(arg1, arg2)
18 overloads. With/without: message, object params
Assert.IsInstanceOfType(Type, object)
3 overloads. With/without: message, object params
Assert.IsNotInstanceOfType(Type, object)
3 overloads. With/without: message, object params
Assert.IsAssignableFrom(Type, object)
3 overloads. With/without: message, object params
Assert.IsNotAssignableFrom(Type, object)
3 overloads. With/without: message, object params
Assert.IsTrue(aBool)
3 overloads. With/without: message, object params
Assert.IsFalse(aBool)
3 overloads. With/without: message, object params
Assert.IsNaN(aDouble)
3 overloads. With/without: message, object params
Assert.IsEmpty(aString)
3 overloads. With/without: message, object params
Assert.IsEmpty(anICollection)
3 overloads. With/without: message, object params
Assert.IsNotEmpty(aString)
3 overloads. With/without: message, object params
Assert.IsNotEmpty(anICollection)
3 overloads. With/without: message, object params
Assert.Fail()
3 overloads. With/without: message, object params
StringAssert.Contains(expectedString, actualString)
3 overloads. With/without: message, object params
StringAssert.StartsWith(expectedString, actualString)
3 overloads. With/without: message, object params
StringAssert.EndsWith(expectedString, actualString)
3 overloads. With/without: message, object params
StringAssert.AreEqualIgnoringCase(expectedString, actualString)
3 overloads. With/without: message, object params

55.8. Unit Test Concepts
Contents Up Previous Next Slide Annotated slide Aggregated slides Subject index Program index Exercise index

Understanding the common concepts of Unit testing

Assertion
A boolean function which compares some value or state with its expected result
"An atomic test" - true means pass, false means fail
Test or Test case
Execution of a piece of code followed by activation of one or more assertions
Test suite
An aggregation of test cases
Test fixture
A fixed state used as a baseline for running a set of tests [Wikipedia]
Used to ensure that each test is executed in a fixed and well-defined state, such that a test is repeatable

Test cases and Test suites may be organized as a Composite

55.9. Another Unit Test example in C#
Contents Up Previous Next Slide Annotated slide Aggregated slides Subject index Program index Exercise index

Unit test of the class Set<T> from the lecture about generics

using System;
using System.Collections.Generic;
using System.Collections;

public class Set<T> {
 
  /* Invariant: At most one occurrence of an element in the array store 
     Consequtive storing from low end.
  */

  private int capacity;
  private static int DefaultCapacity = 3;
  private T[] store;
  private int next;

  public Set(int capacity){
    this.capacity = capacity;
    store = new T[capacity];
    next = 0;
  }

  public Set(): this(DefaultCapacity){
  }

  public Set(T[] elements): this(elements.Length){
    foreach(T el in elements) this.Insert(el);
  }

  // Copy constructor
  public Set(Set<T> s): this(s.capacity){
    foreach(T el in s) this.Insert(el);
  }

  public bool Member(T element){
    for(int idx = 0; idx < next; idx++)
      if (element.Equals(store[idx]))
        return true;
    return false;
  }

  public void Insert(T element){
    if (!this.Member(element)){
      if (this.Full){
        Console.WriteLine("[Resize to {0}]", capacity * 2);
        Array.Resize<T>(ref store, capacity * 2);
        capacity = capacity * 2;
      }
      store[next] = element;
      next++;
    }
  }

  public void Delete(T element){
    bool found = false;
    int  foundIdx = 0;
    for(int idx = 0; !found && (idx < next); idx++){
      if (element.Equals(store[idx])){
         found = true;
         foundIdx = idx;
      }
    }
    if (found){   // shift remaining elements left
      for(int idx = foundIdx+1; idx < next; idx++)
        store[idx-1] = store[idx];
      store[next-1] = default(T);
      next--;
    }
  }

  public int Count{
    get{
      return next;
    }
  }

  // Is this set a subset of other
  public bool Subset(Set<T> other){
    foreach(T e in this)
      if (!other.Member(e))
         return false;
    return true;
  }
     
  public Set<T> Intersection(Set<T> other){
    Set<T> res = new Set<T>(this.Count);
    foreach(T e in this)
      if (other.Member(e))
          res.Insert(e);
    return res;
  }

  public Set<T> Union(Set<T> other){
    Set<T> res = new Set<T>(this.Count + other.Count);
    foreach(T e in this) res.Insert(e);
    foreach(T e in other) res.Insert(e);
    return res;
  }

  // Subtract other elements from this set.
  public Set<T> Diff(Set<T> other){
    Set<T> res = new Set<T>(this);
    foreach(T e in other) res.Delete(e);
    return res;
  }

  public override string ToString(){
    string elRes = "";
    for(int idx = 0; idx < next; idx++) 
      if (store[idx] != null)
        elRes += " " + store[idx];
    return "{" + elRes + " "+ "}" + this.Count;
  }
  
  private class SetEnumerator: IEnumerator<T>{
 
    private readonly Set<T> set; 
    private int idx;

    public SetEnumerator (Set<T> s){
      this.set = s;
      idx = -1;   // position enumerator outside range
    }
 
    public T Current{ 
      get {
       return set.store[idx];
      }
    }

    Object IEnumerator.Current{ 
      get {
       return set.store[idx];
      }
    }

    public bool MoveNext(){
      if (idx < set.next - 1){
        idx++; 
        return true;
      }
      else
         return false;
    }

    public void Reset(){
      idx = -1;         
    }

    public void Dispose(){
    }

  }    
    
  public IEnumerator<T> GetEnumerator (){
    return new SetEnumerator(this);
  }

  private bool Full{
    get{
      return next == capacity;
    }
  }

}

Program 55.5 The class Set<T>.

using System;
using NUnit.Framework;

[TestFixture]
public class SetTest{

  Set<int> set1, set2, set3, set4;

  [SetUp] 
  public void Init(){
    set1 = new Set<int>();
    set2 = new Set<int>(10);
    set3 = new Set<int>(new int[]{1,2,3});
    set4 = new Set<int>(set3);
  }

  [Test]
  public void MemberTest(){
    Assert.IsFalse(set1.Member(1), "Member test 1");
    Assert.IsFalse(set2.Member(1), "Member test 2");
    Assert.IsTrue(set3.Member(1), "Member test 3");
    Assert.IsTrue(set4.Member(1), "Member test 4");
    Assert.IsFalse(set3.Member(4), "Member test 5");
    Assert.IsFalse(set4.Member(4), "Member test 6");
  }    

  [Test]
  public void InsertNonMember(){
    set1.Insert(1); 
    Assert.AreEqual(set1.Count, 1,  "Insert test 1");
    Assert.IsTrue(set1.Member(1), "Insert test 2");

    set3.Insert(4);
    Assert.AreEqual(set3.Count, 4,  "Insert test 3");

    set4.Insert(4);
    Assert.AreEqual(set4.Count, 4,  "Insert test 4");
  }  


  [Test]
  public void InsertMember(){
    set1.Insert(1);  set1.Insert(1);
    Assert.AreEqual(set1.Count, 1,  "Insert test 1");
    Assert.IsTrue(set1.Member(1), "Insert test 2");

    set3.Insert(3);
    Assert.AreEqual(set3.Count, 3,  "Insert test 3");

    set4.Insert(3);
    Assert.AreEqual(set4.Count, 3,  "Insert test 4");
  }  

  [Test]
  public void DeleteNonMember(){
    set1.Delete(1); 
    Assert.AreEqual(set1.Count, 0,  "Delete test 1");

    set3.Delete(4); 
    Assert.AreEqual(set3.Count, 3,  "Delete test 2");
  }

  [Test]
  public void DeleteMember(){
    set3.Delete(1); 
    Assert.AreEqual(set3.Count, 2,  "Delete test 1");
    Assert.IsFalse(set3.Member(1),  "Delete test 2");
  }

  // Delete member twice
  [Test]
  public void DeleteMemberTwice(){
    set3.Delete(1);  set3.Delete(1); 
    Assert.AreEqual(set3.Count, 2,  "Delete test 1");
    Assert.IsFalse(set3.Member(1),  "Delete test 2");
  }

  [Test]
  public void MultipleDelete(){
    set3.Delete(1);  set3.Delete(2); set3.Delete(3); 
    Assert.AreEqual(set3.Count, 0,  "Delete test 1");
    Assert.IsFalse(set3.Member(1),  "Delete test 2");
    Assert.IsFalse(set3.Member(2),  "Delete test 3");
    Assert.IsFalse(set3.Member(3),  "Delete test 4");
  }


}

Program 55.6 The class SetTest.

Exercise 14.3. Test of class Set

In a previous exercise we have implemented the operations intersection, union, and set difference in class Set<T>.

In continuation of class SetTest, perform unit tests of the intersection, union, and set difference operations.

The natural starting point is your solution to the previous exercise. You can also chose to test my solution.

There is no solution to this exercise

55.10. Test Scaffolding
Contents Up Previous Next Slide Annotated slide Aggregated slides Subject index Program index Exercise index

The test scaffolding denotes the auxilliary programs and classes that allow us to test a given program unit

Figure 55.3 Test scaffolding

The test units depends on a number of other units
These units are established as stubs possibly in the form of mockups
The test cases are executed by means of driver program

It appears attractive to test the classes bottom up in order to avoid excessive use of stubs

55.11. The Background and Context of Unit Testing
Contents Up Previous Next Slide Annotated slide Aggregated slides Subject index Program index Exercise index

Unit testing was was popularized for Java by tool called JUnit

Kent Beck and Erich Gamma are the originators of unit testing

JUnit
Java classes and interfaces for organization and execution of test methods
Explicit or implicit activation of test methods
More recently being based on annotations in Java, like the attributes in C# as supported by NUnit.

Unit testing is a cornerstone in Extreme Programming

55.12. Test Driven Development
Contents Up Previous Next Slide Annotated slide Aggregated slides Subject index Program index Exercise index

Test a little. Code a little.

Test driven development
Start by writing the test cases
Initially, all test cases are likely to fail
The test cases serve both as program specification and documentation
Implement the program
The implementation is done when all tests succeed
Refactor the program to improve its quality
Do full regression test in between each modification

55.13. Unit Test Recommendation
Contents Up Previous Next Slide Annotated slide Aggregated slides Subject index Program index Exercise index

Test case independence
Each test should examine a single program aspect - in isolation from other aspects
Test simplicity
Test code should be simple
Each test should only activate a single assertion
Contrary to most examples that have been shown earlier in this lecture
Because the first assertion that fails prevents execution of additional assertions
Testing of test code is simply too much
Test coverage
Each public operation in a class should have at least one associated test
Typically a number of tests per operations
Indeed a very loose meaning of coverage!
Regression test
For each modification - at a reasonable grain size - re-execute all tests

55.14. Test of Object-oriented programs
Contents Up Previous Next Slide Annotated slide Aggregated slides Subject index Program index Exercise index

What are the basic challenges of testing object-oriented programs?

Button-up OOP development
Makes it natural to test the classes bottom-up
Bottom-up testing of classes relieves the need for stubs
Information hiding
Makes it difficult/impossible to directly observe the effect of operations on an object
Dynamic binding and polymorphism
May contribute with a lot additional independent paths
It is difficult to identify these paths on a static basis
Complicates white box testing considerably

Generated: Monday February 7, 2011, 12:23:25