Developers have written small test program since the beginning of programming. But the term unit test implies a formal unit test framework and not just some adhoc test programs.
This more formalized form for unit test have been used for 20 years by now. The first widely used unit test framework JUnit (for Java) date back to the late 1990's.
Today use of unit test frameworks is a generally accepted standard for all professional software development no matter what programming language.
Running unit tests are usually done automatically by the build scripts. The build scipts check out from source control, build the software and run the unit tests.
And if a CI (Continious Integration) tool is used then build and therefor unit tests are run all the time.
The benefits from using unit tests are huge:
It is so important for initial development that a development methodology TDD (Test Driven Development) has been created where the unit tests are driving/documenting the actual software design.
A key principle is to write the unit tests before the actual code. That is the best way to ensure that you test what the code should be doing and not test what the code is actual doing.
For bug fixing that means start by creating a unit test that triggers the bug and causes the unit test to fail. And first after that do the fix and verify that th eunit test now succeed.
It is important to keep that unit test around after the fix by adding it to the unit tests run regularly. Because experience show that bugs tend to be occasionally reintroduced - if the bug was easy to make once then it is rather likely that a new implementation will make the same mistake.
I strongly agree with the approach of using unit tests and writing the tests before the code to be tested.
To demonstrate unit tests I will use a little trivial integer vector class with a simple bug in.
Do not focus on whether the code make sense or not. It is just a demo.
package unittest;
public class IntVector {
private int[] v;
public IntVector(int n) {
v = new int[n];
}
public IntVector(int[] v) {
this.v = v;
}
public IntVector add(int k) {
int[] res = new int[v.length];
for (int i = 0; i < v.length; i++) res[i] = v[i] + k;
return new IntVector(res);
}
public IntVector sub(int k) {
int[] res = new int[v.length];
for (int i = 0; i < v.length; i++) res[i] = v[i] - k;
return new IntVector(res);
}
public IntVector mul(int k) {
int[] res = new int[v.length];
for (int i = 0; i < v.length; i++) res[i] = v[i] * k;
return new IntVector(res);
}
public IntVector div(int k) {
int[] res = new int[v.length];
for (int i = 0; i < v.length; i++) res[i] = v[i] / k;
return new IntVector(res);
}
public IntVector mod(int k) {
int[] res = new int[v.length];
for (int i = 0; i < v.length; i++) res[i] = v[i] - (v[i] / k);
return new IntVector(res);
}
/*
public MathVector mod(int k) {
int[] res = new int[v.length];
for (int i = 0; i < v.length; i++) res[i] = v[i] - (v[i] / k) * k;
return new IntVector(res);
}
*/
public int size() {
return v.length;
}
public int[] getV() {
return v;
}
}
using System;
namespace UnitTest
{
public class IntVector {
private int[] v;
public IntVector(int n)
{
v = new int[n];
}
public IntVector(int[] v)
{
this.v = v;
}
public IntVector Add(int k)
{
int[] res = new int[v.Length];
for (int i = 0; i < v.Length; i++) res[i] = v[i] + k;
return new IntVector(res);
}
public IntVector Sub(int k)
{
int[] res = new int[v.Length];
for (int i = 0; i < v.Length; i++) res[i] = v[i] - k;
return new IntVector(res);
}
public IntVector Mul(int k)
{
int[] res = new int[v.Length];
for (int i = 0; i < v.Length; i++) res[i] = v[i] * k;
return new IntVector(res);
}
public IntVector Div(int k)
{
int[] res = new int[v.Length];
for (int i = 0; i < v.Length; i++) res[i] = v[i] / k;
return new IntVector(res);
}
public IntVector Mod(int k)
{
int[] res = new int[v.Length];
for (int i = 0; i < v.Length; i++) res[i] = v[i] - (v[i] / k);
return new IntVector(res);
}
/*
public MathVector Mod(int k)
{
int[] res = new int[v.Length];
for (int i = 0; i < v.Length; i++) res[i] = v[i] - (v[i] / k) * k;
return new IntVector(res);
}
*/
public int Count {
get { return v.Length; }
}
public int[] Raw {
get { return v; }
}
}
}
Imports System
Namespace UnitTest
Public Class IntVector
Private v As Integer()
Public Sub New(n As Integer)
v = New Integer(n - 1) {}
End Sub
Public Sub New(v As Integer())
Me.v = v
End Sub
Public Function Add(k As Integer) As IntVector
Dim res As Integer() = New Integer(v.Length - 1) {}
For i As Integer = 0 To v.Length - 1
res(i) = v(i) + k
Next
Return New IntVector(res)
End Function
Public Function Subtract(k As Integer) As IntVector
Dim res As Integer() = New Integer(v.Length - 1) {}
For i As Integer = 0 To v.Length - 1
res(i) = v(i) - k
Next
Return New IntVector(res)
End Function
Public Function Multiply(k As Integer) As IntVector
Dim res As Integer() = New Integer(v.Length - 1) {}
For i As Integer = 0 To v.Length - 1
res(i) = v(i) * k
Next
Return New IntVector(res)
End Function
Public Function Divide(k As Integer) As IntVector
Dim res As Integer() = New Integer(v.Length - 1) {}
For i As Integer = 0 To v.Length - 1
res(i) = v(i) \ k
Next
Return New IntVector(res)
End Function
Public Function Modulus(k As Integer) As IntVector
Dim res As Integer() = New Integer(v.Length - 1) {}
For i As Integer = 0 To v.Length - 1
res(i) = v(i) - (v(i) \ k)
Next
Return New IntVector(res)
End Function
'Public Function Modulus(k As Integer) As IntVector
' Dim res As Integer() = New Integer(v.Length - 1) {}
' For i As Integer = 0 To v.Length - 1
' res(i) = v(i) - (v(i) \ k) * k
' Next
' Return New IntVector(res)
'End Function
Public ReadOnly Property Count() As Integer
Get
Return v.Length
End Get
End Property
Public ReadOnly Property Raw() As Integer()
Get
Return v
End Get
End Property
End Class
End Namespace
<?php
namespace UnitTest;
class IntVector {
private $v;
public function __construct($arg) {
if(is_int($arg)) {
$this->v = array_fill(0, $arg, 0);
} else if(is_array($arg) && is_int($arg[0])) {
$this->v = $arg;
} else {
throw new \Exception('Bad argument to IntVector ctor');
}
}
public function add($k) {
$res = array();
for ($i = 0; $i < count($this->v); $i++) $res[$i] = $this->v[$i] + $k;
return new IntVector($res);
}
public function sub($k) {
$res = array();
for ($i = 0; $i < count($this->v); $i++) $res[$i] = $this->v[$i] - $k;
return new IntVector($res);
}
public function mul($k) {
$res = array();
for ($i = 0; $i < count($this->v); $i++) $res[$i] = $this->v[$i] * $k;
return new IntVector($res);
}
public function div($k) {
$res = array();
for ($i = 0; $i < count($this->v); $i++) $res[$i] = (int)($this->v[$i] / $k);
return new IntVector($res);
}
public function mod($k) {
$res = array();
for ($i = 0; $i < count($this->v); $i++) $res[$i] = $this->v[$i] - (int)($this->v[$i] / $k);
return new IntVector($res);
}
/*
public function mod($k) {
$res = array();
for ($i = 0; $i < count($this->v); $i++) $res[$i] = $this->v[$i] - (int)($this->v[$i] / $k) * $k;
return new IntVector($res);
}
*/
public function size() {
return count($this->v);
}
public function getV() {
return $this->v;
}
}
?>
(the code may look a bit more Java'ish than PHP'ish but ignore that)
class IntVector(object):
def __init__(self, arg):
if isinstance(arg,list):
self.v = arg
else:
self.v = [0 for _ in range(arg)]
def add(self, k):
return IntVector(list(map(lambda i: i + k, self.v)))
def sub(self, k):
return IntVector(list(map(lambda i: i - k, self.v)))
def mul(self, k):
return IntVector(list(map(lambda i: i * k, self.v)))
def div(self, k):
return IntVector(list(map(lambda i: i / k, self.v)))
def mod(self, k):
return IntVector(list(map(lambda i: i - (i / k), self.v)))
# def mod(self, k):
# return IntVector(list(map(lambda i: i - (i / k) * k, self.v)))
For Python 3.x change / to //.
unit IntVectorUnit;
interface
uses
Classes;
type
ArrayOfInteger = array of integer;
IntVector = class(TObject)
constructor Create(n : integer);
constructor Create(v : ArrayOfInteger);
function OpAdd(k : integer) : IntVector;
function OpSub(k : integer) : IntVector;
function OpMul(k : integer) : IntVector;
function OpDiv(k : integer) : IntVector;
function OpMod(k : integer) : IntVector;
function Size : integer;
function GetV : ArrayOfInteger;
private
_v : array of integer;
end;
implementation
constructor IntVector.Create(n : integer);
begin
SetLength(_v, n);
end;
constructor IntVector.Create(v : ArrayOfInteger);
var
i : integer;
begin
SetLength(_v, Length(v));
for i := Low(_v) to High(_v) do begin
_v[i] := v[i];
end;
end;
function IntVector.OpAdd(k : integer) : IntVector;
var
res : ArrayOfInteger;
i : integer;
begin
SetLength(res, Length(_v));
for i := Low(_v) to High(_v) do begin
res[i] := _v[i] + k;
end;
OpAdd := IntVector.Create(res);
end;
function IntVector.OpSub(k : integer) : IntVector;
var
res : ArrayOfInteger;
i : integer;
begin
SetLength(res, Length(_v));
for i := Low(_v) to High(_v) do begin
res[i] := _v[i] - k;
end;
OpSub := IntVector.Create(res);
end;
function IntVector.OpMul(k : integer) : IntVector;
var
res : ArrayOfInteger;
i : integer;
begin
SetLength(res, Length(_v));
for i := Low(_v) to High(_v) do begin
res[i] := _v[i] * k;
end;
OpMul := IntVector.Create(res);
end;
function IntVector.OpDiv(k : integer) : IntVector;
var
res : ArrayOfInteger;
i : integer;
begin
SetLength(res, Length(_v));
for i := Low(_v) to High(_v) do begin
res[i] := _v[i] div k;
end;
OpDiv := IntVector.Create(res);
end;
function IntVector.OpMod(k : integer) : IntVector;
var
res : ArrayOfInteger;
i : integer;
begin
SetLength(res, Length(_v));
for i := Low(_v) to High(_v) do begin
res[i] := _v[i] - (_v[i] div k);
end;
OpMod := IntVector.Create(res);
end;
(*
function IntVector.OpMod(k : integer) : IntVector;
var
res : ArrayOfInteger;
i : integer;
begin
SetLength(res, Length(_v));
for i := Low(_v) to High(_v) do begin
res[i] := _v[i] - (_v[i] div k) * k;
end;
OpMod := IntVector.Create(res);
end;
*)
function IntVector.Size : integer;
begin
Size := Length(_v);
end;
function IntVector.GetV : ArrayOfInteger;
begin
GetV := _v;
end;
end.
Operator methods are prefixed with "Op" because div and mod are keywords in Pascal.
Now we will look at testing this code using some unit testing frameworks.
These unit testing frameworks are often collectively known as xUnit.
Unit test:
package unittest.junit3;
import junit.framework.TestCase;
import unittest.IntVector;
public class TestIntVector extends TestCase {
public TestIntVector(String s) {
super(s);
}
protected void setUp() {
}
protected void tearDown() {
}
public void testAdd() {
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.add(777);
assertEquals("add size", iv.size(), iv2.size());
for(int i = 0; i < iv2.size(); i++) {
assertEquals("add " + i, iv.getV()[i] + 777, iv2.getV()[i]);
}
}
public void testSub() {
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.sub(777);
assertEquals("sub size", iv.size(), iv2.size());
for(int i = 0; i < iv2.size(); i++) {
assertEquals("sub " + i, iv.getV()[i] - 777, iv2.getV()[i]);
}
}
public void testMul() {
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.mul(777);
assertEquals("mul size", iv.size(), iv2.size());
for(int i = 0; i < iv2.size(); i++) {
assertEquals("mul " + i, iv.getV()[i] * 777, iv2.getV()[i]);
}
}
public void testDiv() {
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.div(777);
assertEquals("div size", iv.size(), iv2.size());
for(int i = 0; i < iv2.size(); i++) {
assertEquals("div " + i, iv.getV()[i] / 777, iv2.getV()[i]);
}
}
public void testMod() {
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.mod(777);
assertEquals("mod size", iv.size(), iv2.size());
for(int i = 0; i < iv2.size(); i++) {
assertEquals("mod " + i, iv.getV()[i] % 777, iv2.getV()[i]);
}
}
}
Suite of unit tests:
package unittest.junit3;
import junit.framework.Test;
import junit.framework.TestCase;
import junit.framework.TestSuite;
public class AllTests extends TestCase {
public AllTests(String name) {
super(name);
}
public static Test suite() {
TestSuite test = new TestSuite();
test.addTestSuite(TestIntVector.class);
/*
test.addTestSuite(Abc.class);
test.addTestSuite(Xyz.class);
*/
return test;
}
}
The rules are:
The test/test suite can be run directly from the IDE (Eclipse/NetBeans/IntelliJ IDEA).
It can be run from the command line:
java -cp junit.jar;myprojectbindir junit.textui.TestRunner unittest.junit3.AllTests
Output:
.....F
Time: 0.002
There was 1 failure:
1) testMod(unittest.junit3.TestIntVector)junit.framework.AssertionFailedError: mod 0 expected:<-1> but was:<-998713>
at unittest.junit3.TestIntVector.testMod(TestIntVector.java:57)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
FAILURES!!!
Tests run: 5, Failures: 1, Errors: 0
It can be run via ant. build.xml snippet:
<junit fork="yes">
<formatter type="plain" usefile="false"/>
<classpath path="junit.jar;myprojectbindir"/>
<test name="unittest.junit3.AllTests"/>
</junit>
Unit test:
package unittest.junit4;
import static org.junit.Assert.assertEquals;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import unittest.IntVector;
public class TestIntVector {
@Before
public void setUp() {
}
@After
public void tearDown() {
}
@Test
public void testAdd() {
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.add(777);
assertEquals("add size", iv.size(), iv2.size());
for(int i = 0; i < iv2.size(); i++) {
assertEquals("add " + i, iv.getV()[i] + 777, iv2.getV()[i]);
}
}
@Test
public void testSub() {
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.sub(777);
assertEquals("sub size", iv.size(), iv2.size());
for(int i = 0; i < iv2.size(); i++) {
assertEquals("sub " + i, iv.getV()[i] - 777, iv2.getV()[i]);
}
}
@Test
public void testMul() {
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.mul(777);
assertEquals("mul size", iv.size(), iv2.size());
for(int i = 0; i < iv2.size(); i++) {
assertEquals("mul " + i, iv.getV()[i] * 777, iv2.getV()[i]);
}
}
@Test
public void testDiv() {
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.div(777);
assertEquals("div size", iv.size(), iv2.size());
for(int i = 0; i < iv2.size(); i++) {
assertEquals("div " + i, iv.getV()[i] / 777, iv2.getV()[i]);
}
}
@Test
public void testMod() {
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.mod(777);
assertEquals("mod size", iv.size(), iv2.size());
for(int i = 0; i < iv2.size(); i++) {
assertEquals("mod " + i, iv.getV()[i] % 777, iv2.getV()[i]);
}
}
}
Suite of unit tests:
package unittest.junit4;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;
@RunWith(Suite.class)
@SuiteClasses({TestIntVector.class/*,Abc.class,Xyz.class*/})
public class AllTests {
}
The rules are:
The test/test suite can be run directly from the IDE (Eclipse/NetBeans/IntelliJ IDEA).
It can be run from the command line:
java -cp junit.jar;myprojectbindir org.junit.runner.JUnitCore unittest.junit4.AllTests
Output:
.....E
Time: 0.008
There was 1 failure:
1) testMod(unittest.junit4.TestIntVector)
java.lang.AssertionError: mod 0 expected:<-1> but was:<-998713>
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.failNotEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:126)
at org.junit.Assert.assertEquals(Assert.java:470)
at unittest.junit4.TestIntVector.testMod(TestIntVector.java:65)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
...
It can be run via ant. build.xml snippet:
<junit fork="yes">
<formatter type="plain" usefile="false"/>
<classpath path="junit.jar;myprojectbindir"/>
<test name="unittest.junit4.AllTests"/>
</junit>
using System;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using UnitTest;
namespace UnitTest.MS
{
[TestClass]
public class TestIntVector
{
[ClassInitialize]
public static void SetUp(TestContext context)
{
}
[ClassCleanup]
public static void TearDown()
{
}
[TestMethod]
public void TestAdd()
{
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.Add(777);
Assert.AreEqual(iv.Count, iv2.Count, "add size");
for(int i = 0; i < iv2.Count; i++)
{
Assert.AreEqual(iv.Raw[i] + 777, iv2.Raw[i], "add " + i);
}
}
[TestMethod]
public void TestSub()
{
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.Sub(777);
Assert.AreEqual(iv.Count, iv2.Count, "sub size");
for(int i = 0; i < iv2.Count; i++)
{
Assert.AreEqual(iv.Raw[i] - 777, iv2.Raw[i], "sub " + i);
}
}
[TestMethod]
public void TestMul()
{
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.Mul(777);
Assert.AreEqual(iv.Count, iv2.Count, "mul size");
for(int i = 0; i < iv2.Count; i++)
{
Assert.AreEqual(iv.Raw[i] * 777, iv2.Raw[i], "mul " + i);
}
}
[TestMethod]
public void TestDiv()
{
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.Div(777);
Assert.AreEqual(iv.Count, iv2.Count, "div size");
for(int i = 0; i < iv2.Count; i++)
{
Assert.AreEqual(iv.Raw[i] / 777, iv2.Raw[i], "div " + i);
}
}
[TestMethod]
public void TestMod()
{
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.Mod(777);
Assert.AreEqual(iv.Count, iv2.Count, "mod size");
for(int i = 0; i < iv2.Count; i++)
{
Assert.AreEqual(iv.Raw[i] % 777, iv2.Raw[i], "mod " + i);
}
}
}
}
The rules are:
The tests can be run directly from the IDE (Visual Studio).
It can be run from the command line:
mstest /testcontainer:UnitTest.MS.dll /resultsfile:z.z
Output:
Loading UnitTest.MS.dll ... Starting execution... Results Top Level Tests ------- --------------- Passed UnitTest.MS.TestIntVector.TestAdd Passed UnitTest.MS.TestIntVector.TestDiv Failed UnitTest.MS.TestIntVector.TestMod Passed UnitTest.MS.TestIntVector.TestMul Passed UnitTest.MS.TestIntVector.TestSub 4/5 test(s) Passed, 1 Failed Summary ------- Test Run Failed. Passed 4 Failed 1 --------- Total 5 Results file: z.z Test Settings: Default Test Settings
using System;
using NUnit.Framework;
using UnitTest;
namespace UnitTest.NUnit
{
[TestFixture]
public class TestIntVector
{
[SetUp]
public static void SetUp()
{
}
[TearDown]
public static void TearDown()
{
}
[Test]
public void TestAdd()
{
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.Add(777);
Assert.AreEqual(iv.Count, iv2.Count, "add size");
for(int i = 0; i < iv2.Count; i++)
{
Assert.AreEqual(iv.Raw[i] + 777, iv2.Raw[i], "add " + i);
}
}
[Test]
public void TestSub()
{
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.Sub(777);
Assert.AreEqual(iv.Count, iv2.Count, "sub size");
for(int i = 0; i < iv2.Count; i++)
{
Assert.AreEqual(iv.Raw[i] - 777, iv2.Raw[i], "sub " + i);
}
}
[Test]
public void TestMul()
{
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.Mul(777);
Assert.AreEqual(iv.Count, iv2.Count, "mul size");
for(int i = 0; i < iv2.Count; i++)
{
Assert.AreEqual(iv.Raw[i] * 777, iv2.Raw[i], "mul " + i);
}
}
[Test]
public void TestDiv()
{
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.Div(777);
Assert.AreEqual(iv.Count, iv2.Count, "div size");
for(int i = 0; i < iv2.Count; i++)
{
Assert.AreEqual(iv.Raw[i] / 777, iv2.Raw[i], "div " + i);
}
}
[Test]
public void TestMod()
{
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.Mod(777);
Assert.AreEqual(iv.Count, iv2.Count, "mod size");
for(int i = 0; i < iv2.Count; i++)
{
Assert.AreEqual(iv.Raw[i] % 777, iv2.Raw[i], "mod " + i);
}
}
}
}
The rules are:
It can be run from the command line:
nunit-console-x86 UnitTest.NUnit.dll
Output:
ProcessModel: Default DomainUsage: Single
Execution Runtime: Default
...F..
Tests run: 5, Errors: 0, Failures: 1, Inconclusive: 0, Time: 0.0690039 seconds
Not run: 0, Invalid: 0, Ignored: 0, Skipped: 0
Errors and Failures:
1) Test Failure : UnitTest.NUnit.TestIntVector.TestMod
mod 0
Expected: -1
But was: -998713
at UnitTest.NUnit.TestIntVector.TestMod() in \UnitTest\UnitTest.NUnit\UnitTestNUnit.cs:line 77
It can be run via NAnt. Build file snippet:
<nunit2>
<formatter type="Plain"/>
<test>
<assemblies>
<include name="UnitTest.NUnit.dll"/>
</assemblies>
</test>
</nunit2>
using System;
using Xunit;
using UnitTest;
namespace UnitTest.Xunit
{
public class TestIntVector : IDisposable
{
public TestIntVector()
{
}
public void Dispose()
{
}
[Fact]
public void TestAdd()
{
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.Add(777);
Assert.Equal(iv.Count, iv2.Count);
for(int i = 0; i < iv2.Count; i++)
{
Assert.Equal(iv.Raw[i] + 777, iv2.Raw[i]);
}
}
[Fact]
public void TestSub()
{
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.Sub(777);
Assert.Equal(iv.Count, iv2.Count);
for(int i = 0; i < iv2.Count; i++)
{
Assert.Equal(iv.Raw[i] - 777, iv2.Raw[i]);
}
}
[Fact]
public void TestMul()
{
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.Mul(777);
Assert.Equal(iv.Count, iv2.Count);
for(int i = 0; i < iv2.Count; i++)
{
Assert.Equal(iv.Raw[i] * 777, iv2.Raw[i]);
}
}
[Fact]
public void TestDiv()
{
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.Div(777);
Assert.Equal(iv.Count, iv2.Count);
for(int i = 0; i < iv2.Count; i++)
{
Assert.Equal(iv.Raw[i] / 777, iv2.Raw[i]);
}
}
[Fact]
public void TestMod()
{
int[] v = { -1000000, -1, 0, 1, 1000000 };
IntVector iv = new IntVector(v);
IntVector iv2 = iv.Mod(777);
Assert.Equal(iv.Count, iv2.Count);
for(int i = 0; i < iv2.Count; i++)
{
Assert.Equal(iv.Raw[i] % 777, iv2.Raw[i]);
}
}
}
}
The rules are:
It can be run from the command line:
xunit.console.exe UnitTest.Xunit.dll
Output:
xUnit.net Console Runner v2.4.2+f110e5bee5 (64-bit Desktop .NET 4.5.2, runtime:
4.0.30319.42000)
Discovering: UnitTest.Xunit
Discovered: UnitTest.Xunit
Starting: UnitTest.Xunit
UnitTest.Xunit.TestIntVector.TestMod [FAIL]
Assert.Equal() Failure
Expected: -1
Actual: -998713
Stack Trace:
at UnitTest.Xunit.TestIntVector.TestMod()
Finished: UnitTest.Xunit
=== TEST EXECUTION SUMMARY ===
UnitTest.Xunit Total: 5, Errors: 0, Failed: 1, Skipped: 0, Time: 0.296s
Imports System
Imports Microsoft.VisualStudio.TestTools.UnitTesting
Imports UnitTest
Namespace UnitTest.MS
<TestClass> _
Public Class TestIntVector
<ClassInitialize> _
Public Shared Sub SetUp(context As TestContext)
End Sub
<ClassCleanup> _
Public Shared Sub TearDown()
End Sub
<TestMethod> _
Public Sub TestAdd()
Dim v As Integer() = {-1000000, -1, 0, 1, 1000000}
Dim iv As New IntVector(v)
Dim iv2 As IntVector = iv.Add(777)
Assert.AreEqual(iv.Count, iv2.Count, "add size")
For i As Integer = 0 To iv2.Count - 1
Assert.AreEqual(iv.Raw(i) + 777, iv2.Raw(i), "add " & i)
Next
End Sub
<TestMethod> _
Public Sub TestSub()
Dim v As Integer() = {-1000000, -1, 0, 1, 1000000}
Dim iv As New IntVector(v)
Dim iv2 As IntVector = iv.Subtract(777)
Assert.AreEqual(iv.Count, iv2.Count, "sub size")
For i As Integer = 0 To iv2.Count - 1
Assert.AreEqual(iv.Raw(i) - 777, iv2.Raw(i), "sub " & i)
Next
End Sub
<TestMethod> _
Public Sub TestMul()
Dim v As Integer() = {-1000000, -1, 0, 1, 1000000}
Dim iv As New IntVector(v)
Dim iv2 As IntVector = iv.Multiply(777)
Assert.AreEqual(iv.Count, iv2.Count, "mul size")
For i As Integer = 0 To iv2.Count - 1
Assert.AreEqual(iv.Raw(i) * 777, iv2.Raw(i), "mul " & i)
Next
End Sub
<TestMethod> _
Public Sub TestDiv()
Dim v As Integer() = {-1000000, -1, 0, 1, 1000000}
Dim iv As New IntVector(v)
Dim iv2 As IntVector = iv.Divide(777)
Assert.AreEqual(iv.Count, iv2.Count, "div size")
For i As Integer = 0 To iv2.Count - 1
Assert.AreEqual(iv.Raw(i) \ 777, iv2.Raw(i), "div " & i)
Next
End Sub
<TestMethod> _
Public Sub TestModulus()
Dim v As Integer() = {-1000000, -1, 0, 1, 1000000}
Dim iv As New IntVector(v)
Dim iv2 As IntVector = iv.Modulus(777)
Assert.AreEqual(iv.Count, iv2.Count, "mod size")
For i As Integer = 0 To iv2.Count - 1
Assert.AreEqual(iv.Raw(i) Mod 777, iv2.Raw(i), "mod " & i)
Next
End Sub
End Class
End Namespace
The rules are:
The tests can be run directly from the IDE (Visual Studio).
It can be run from the command line:
mstest /testcontainer:UnitTest.MS.dll /resultsfile:z.z
Output:
Loading UnitTest.MS.dll ... Starting execution... Results Top Level Tests ------- --------------- Passed UnitTest.MS.TestIntVector.TestAdd Passed UnitTest.MS.TestIntVector.TestDivide Failed UnitTest.MS.TestIntVector.TestModulus Passed UnitTest.MS.TestIntVector.TestMultiply Passed UnitTest.MS.TestIntVector.TestSubtract 4/5 test(s) Passed, 1 Failed Summary ------- Test Run Failed. Passed 4 Failed 1 --------- Total 5 Results file: z.z Test Settings: Default Test Settings
Imports System
Imports NUnit.Framework
Imports UnitTest
Namespace UnitTest.NUnit
<TestFixture> _
Public Class TestIntVector
<SetUp> _
Public Shared Sub SetUp()
End Sub
<TearDown> _
Public Shared Sub TearDown()
End Sub
<Test> _
Public Sub TestAdd()
Dim v As Integer() = {-1000000, -1, 0, 1, 1000000}
Dim iv As New IntVector(v)
Dim iv2 As IntVector = iv.Add(777)
Assert.AreEqual(iv.Count, iv2.Count, "add size")
For i As Integer = 0 To iv2.Count - 1
Assert.AreEqual(iv.Raw(i) + 777, iv2.Raw(i), "add " & i)
Next
End Sub
<Test> _
Public Sub TestSub()
Dim v As Integer() = {-1000000, -1, 0, 1, 1000000}
Dim iv As New IntVector(v)
Dim iv2 As IntVector = iv.Subtract(777)
Assert.AreEqual(iv.Count, iv2.Count, "sub size")
For i As Integer = 0 To iv2.Count - 1
Assert.AreEqual(iv.Raw(i) - 777, iv2.Raw(i), "sub " & i)
Next
End Sub
<Test> _
Public Sub TestMul()
Dim v As Integer() = {-1000000, -1, 0, 1, 1000000}
Dim iv As New IntVector(v)
Dim iv2 As IntVector = iv.Multiply(777)
Assert.AreEqual(iv.Count, iv2.Count, "mul size")
For i As Integer = 0 To iv2.Count - 1
Assert.AreEqual(iv.Raw(i) * 777, iv2.Raw(i), "mul " & i)
Next
End Sub
<Test> _
Public Sub TestDiv()
Dim v As Integer() = {-1000000, -1, 0, 1, 1000000}
Dim iv As New IntVector(v)
Dim iv2 As IntVector = iv.Divide(777)
Assert.AreEqual(iv.Count, iv2.Count, "div size")
For i As Integer = 0 To iv2.Count - 1
Assert.AreEqual(iv.Raw(i) \ 777, iv2.Raw(i), "div " & i)
Next
End Sub
<Test> _
Public Sub TestModulus()
Dim v As Integer() = {-1000000, -1, 0, 1, 1000000}
Dim iv As New IntVector(v)
Dim iv2 As IntVector = iv.Modulus(777)
Assert.AreEqual(iv.Count, iv2.Count, "mod size")
For i As Integer = 0 To iv2.Count - 1
Assert.AreEqual(iv.Raw(i) Mod 777, iv2.Raw(i), "mod " & i)
Next
End Sub
End Class
End Namespace
The rules are:
It can be run from the command line:
nunit-console-x86 UnitTest.NUnit.dll
Output:
ProcessModel: Default DomainUsage: Single
Execution Runtime: Default
...F..
Tests run: 5, Errors: 0, Failures: 1, Inconclusive: 0, Time: 0.0690039 seconds
Not run: 0, Invalid: 0, Ignored: 0, Skipped: 0
Errors and Failures:
1) Test Failure : UnitTest.NUnit.TestIntVector.TestModulus
mod 0
Expected: -1
But was: -998713
at UnitTest.NUnit.TestIntVector.TestModulus() in \UnitTest\UnitTest.NUnit\UnitTestNUnit.vb:line 63
It can be run via NAnt. Build file snippet:
<nunit2>
<formatter type="Plain"/>
<test>
<assemblies>
<include name="UnitTest.NUnit.dll"/>
</assemblies>
</test>
</nunit2>
<?php
use PHPUnit\Framework\TestCase;
require 'IntVector.php';
use UnitTest\IntVector;
class IntVectorTest extends TestCase {
protected function setUp() {
}
protected function tearDown() {
}
public function testAdd() {
$v = array( -1000000, -1, 0, 1, 1000000 );
$iv = new IntVector($v);
$iv2 = $iv->add(777);
$this->assertEquals($iv->size(), $iv2->size(), "add size");
for($i = 0; $i < $iv2->size(); $i++) {
$this->assertEquals($iv->getV()[$i] + 777, $iv2->getV()[$i], "add " + $i);
}
}
public function testSub() {
$v = array( -1000000, -1, 0, 1, 1000000 );
$iv = new IntVector($v);
$iv2 = $iv->sub(777);
$this->assertEquals($iv->size(), $iv2->size(), "sub size");
for($i = 0; $i < $iv2->size(); $i++) {
$this->assertEquals($iv->getV()[$i] - 777, $iv2->getV()[$i], "sub " + $i);
}
}
public function testMul() {
$v = array( -1000000, -1, 0, 1, 1000000 );
$iv = new IntVector($v);
$iv2 = $iv->mul(777);
$this->assertEquals($iv->size(), $iv2->size(), "mul size");
for($i = 0; $i < $iv2->size(); $i++) {
$this->assertEquals($iv->getV()[$i] * 777, $iv2->getV()[$i], "mul " + $i);
}
}
public function testDiv() {
$v = array( -1000000, -1, 0, 1, 1000000 );
$iv = new IntVector($v);
$iv2 = $iv->div(777);
$this->assertEquals($iv->size(), $iv2->size(), "div size");
for($i = 0; $i < $iv2->size(); $i++) {
$this->assertEquals((int)($iv->getV()[$i] / 777), $iv2->getV()[$i], "div " + $i);
}
}
public function testMod() {
$v = array( -1000000, -1, 0, 1, 1000000 );
$iv = new IntVector($v);
$iv2 = $iv->mod(777);
$this->assertEquals($iv->size(), $iv2->size(), "mod size");
for($i = 0; $i < $iv2->size(); $i++) {
$this->assertEquals($iv->getV()[$i] % 777, $iv2->getV()[$i], "mod " + $i);
}
}
}
?>
The rules are:
It can be run from the command line:
phpunit .
Output:
....F 5 / 5 (100%) Time: 324 ms, Memory: 8.50MB There was 1 failure: 1) IntVectorTest::testMod 0 Failed asserting that -998713 matches expected -1. IntVectorTest.php:55 FAILURES! Tests: 5, Assertions: 26, Failures: 1.
Unit test:
import unittest
from IntVector import IntVector
class TestIntVector(unittest.TestCase):
def setUp(self):
pass
def tearDown(self):
pass
def test_add(self):
v = [ -1000000, -1, 0, 1, 1000000 ]
iv = IntVector(v)
iv2 = iv.add(777)
self.assertEqual(len(iv.v), len(iv2.v), 'add size')
for i in range(len(iv.v)):
self.assertEqual(iv.v[i] + 777, iv2.v[i], 'add ' + str(i))
def test_sub(self):
v = [ -1000000, -1, 0, 1, 1000000 ]
iv = IntVector(v)
iv2 = iv.sub(777)
self.assertEqual(len(iv.v), len(iv2.v), 'add size')
for i in range(len(iv.v)):
self.assertEqual(iv.v[i] - 777, iv2.v[i], 'sub ' + str(i))
def test_mul(self):
v = [ -1000000, -1, 0, 1, 1000000 ]
iv = IntVector(v)
iv2 = iv.mul(777)
self.assertEqual(len(iv.v), len(iv2.v), 'add size')
for i in range(len(iv.v)):
self.assertEqual(iv.v[i] * 777, iv2.v[i], 'mul ' + str(i))
def test_div(self):
v = [ -1000000, -1, 0, 1, 1000000 ]
iv = IntVector(v)
iv2 = iv.div(777)
self.assertEqual(len(iv.v), len(iv2.v), 'add size')
for i in range(len(iv.v)):
self.assertEqual(iv.v[i] / 777, iv2.v[i], 'div ' + str(i))
def test_mod(self):
v = [ -1000000, -1, 0, 1, 1000000 ]
iv = IntVector(v)
iv2 = iv.mod(777)
self.assertEqual(len(iv.v), len(iv2.v), 'add size')
for i in range(len(iv.v)):
self.assertEqual(iv.v[i] % 777, iv2.v[i], 'mod ' + str(i))
unittest.main(verbosity=2)
For Python 3.x change / to //.
The rules are:
The test is a standalone program that can be run as any other Python program.
Output:
test_add (__main__.TestIntVector) ... ok
test_div (__main__.TestIntVector) ... ok
test_mod (__main__.TestIntVector) ... FAIL
test_mul (__main__.TestIntVector) ... ok
test_sub (__main__.TestIntVector) ... ok
======================================================================
FAIL: test_mod (__main__.TestIntVector)
----------------------------------------------------------------------
Traceback (most recent call last):
File "TestIntVector.py", line 44, in test_mod
self.assertEqual(iv.v[i] % 777, iv2.v[i], 'mod ' + str(i))
AssertionError: mod 0
----------------------------------------------------------------------
Ran 5 tests in 0.000s
FAILED (failures=1)
Delphi DUnit2 is ported to Lazarus as FPTest. I have only tested with Lazarus and FPTest.
Unit test:
program TestDemo;
uses
Classes,
SysUtils,
IntVectorUnit,
TestFramework,
TextTestRunner;
type
TestIntVector = class(TTestCase)
procedure TestAdd;
procedure TestSub;
procedure TestMul;
procedure TestDiv;
procedure TestMod;
protected
procedure SetUp; override;
procedure Teardown; override;
end;
procedure TestIntVector.SetUp;
begin
end;
procedure TestIntVector.Teardown;
begin
end;
procedure TestIntVector.TestAdd;
var
iv, iv2 : IntVector;
v, v2 : ArrayOfInteger;
i : integer;
begin
SetLength(v, 5);
v[0] := -1000000;
v[1] := -1;
v[2] := 0;
v[3] := 1;
v[4] := 1000000;
iv := IntVector.Create(v);
iv2 := iv.OpAdd(777);
CheckEquals(iv.Size, iv2.Size, 'add size');
v2 := iv2.GetV;
for i := 0 to 4 do begin
CheckEquals(v[i] + 777, v2[i], 'add ' + IntToStr(v[i]));
end;
end;
procedure TestIntVector.TestSub;
var
iv, iv2 : IntVector;
v, v2 : ArrayOfInteger;
i : integer;
begin
SetLength(v, 5);
v[0] := -1000000;
v[1] := -1;
v[2] := 0;
v[3] := 1;
v[4] := 1000000;
iv := IntVector.Create(v);
iv2 := iv.OpSub(777);
CheckEquals(iv.Size, iv2.Size, 'sub size');
v2 := iv2.GetV;
for i := 0 to 4 do begin
CheckEquals(v[i] - 777, v2[i], 'sub ' + IntToStr(v[i]));
end;
end;
procedure TestIntVector.TestMul;
var
iv, iv2 : IntVector;
v, v2 : ArrayOfInteger;
i : integer;
begin
SetLength(v, 5);
v[0] := -1000000;
v[1] := -1;
v[2] := 0;
v[3] := 1;
v[4] := 1000000;
iv := IntVector.Create(v);
iv2 := iv.OpMul(777);
CheckEquals(iv.Size, iv2.Size, 'mul size');
v2 := iv2.GetV;
for i := 0 to 4 do begin
CheckEquals(v[i] * 777, v2[i], 'mul ' + IntToStr(v[i]));
end;
end;
procedure TestIntVector.TestDiv;
var
iv, iv2 : IntVector;
v, v2 : ArrayOfInteger;
i : integer;
begin
SetLength(v, 5);
v[0] := -1000000;
v[1] := -1;
v[2] := 0;
v[3] := 1;
v[4] := 1000000;
iv := IntVector.Create(v);
iv2 := iv.OpDiv(777);
CheckEquals(iv.Size, iv2.Size, 'div size');
v2 := iv2.GetV;
for i := 0 to 4 do begin
CheckEquals(v[i] div 777, v2[i], 'div ' + IntToStr(v[i]));
end;
end;
procedure TestIntVector.TestMod;
var
iv, iv2 : IntVector;
v, v2 : ArrayOfInteger;
i : integer;
begin
SetLength(v, 5);
v[0] := -1000000;
v[1] := -1;
v[2] := 0;
v[3] := 1;
v[4] := 1000000;
iv := IntVector.Create(v);
iv2 := iv.OpMod(777);
CheckEquals(iv.Size, iv2.Size, 'mod size');
v2 := iv2.GetV;
for i := 0 to 4 do begin
CheckEquals(v[i] mod 777, v2[i], 'mod ' + IntToStr(v[i]));
end;
end;
begin
TestFramework.RegisterTest(TestIntVector.Suite);
RunRegisteredTests;
end.
Output:
FPTest / Testing
.....F
Time: 00:00:00.000
Test Results:
Run: 5
Errors: 0
Failures: 1
Warnings: 0
There was 1 failure:
1) TestDemo.exe.TestIntVector.TestMod: ETestFailure
at $0000000100002017 line 143 of TestDemo.pas
"mod -1000000"
Expected:
"-1"
But was:
"-998713"
The code coverage percentage is the percentage of code being executed when all unit tests are run.
It is usually recommended to achieve 100% code coverage.
I fully agree with that recommendation.
Note that 100% code coverage does not guarantee that "everything" is tested. A method with 2 if statemments has 4 different flows through it, so to just test all possible flows then 4 unit tests would be required. Also some bugs will only be triggered by some test values that may not have been tests by unit tests.
It is beneficial to test methods with multiple values. Preferably including potential critical values. Critical values can be zero, very large value, very small value, very long string, empty string, null (if allowed).
There are two schools of thougth:
There are also two schools of thougth for unit testing multi layered applications.
The "purists" want to unit test only one layer at a time and do that by mocking lower layers. They consider a unit test that hits more than one layer to be an automated integration test and not a unit test.
Visual:
The "pragmatists" are OK with hitting more than one layer in a unit test as long as the lower levels has already passed their own unit tests.
Visual:
I am a pragmatist. In theory I agree with the purist view, but in practice I think the risk of the mockup having different behavior than what it is mocking is way more important.
| Version | Date | Description |
|---|---|---|
| 1.0 | October 9th 2016 | Initial version based on multiple old articles on Eksperten.dk |
| 1.1 | November 4th 2018 | Add Python builtin |
| 1.2 | April 21st 2019 | Add Delphi/Lazarus DUnit2/FPTest |
| 1.3 | January 30th 2023 | Add C# XUnit |
See list of all articles here
Please send comments to Arne Vajhøj
I strongly believe in the second approach. Because: