Many Test Driven Developers (TDDers) are familiar with the
Fit and FitNesse tools for testing. These tools bring a lot of value to the testing process, particularly in the ability to push testing to earlier in the development cycle. In particular, tools like FIT and FitNesse have the ability to move testing into the requirements elaboration stage before coding even begins. These tools are also able to put the test definitions into the hands of the business users, analysts and testers where they may more rightfully belong.
Fit and FitNesse have a bit of a learning curve and require a bit of infrastructure to setup. These are not huge problems and they add great value, don’t get me wrong, however Visual Studio also has the ability to perform “Fit-like” testing. The features that provide these capabilities in Visual Studio are part of the unit testing framework and are nowhere near as extensive as those provided by FitNesse. However, if you are looking to get something that provides the ability for an analyst or tester or business user to create test data as requirements that can be used by Visual Studio keep reading.
As part of the unit testing framework that Microsoft introduced with Visual Studio Team System 2005 Developer Edition and available in Visual Studio 2008 Professional Edition there is the ability to define data driven unit tests. These tests can leverage either a SQL Mobile database, Excel (csv file) or an xml file as the source of the test data. The easiest option is generally the Excel spreadsheet as this is the most familiar to all parties involved.
To begin we will create the ubiquitous Add method that is used in unit test examples.
public static int Add(int a, int b)
{
return a + b;
}
A unit test of this method looks like this:
[TestMethod()]
public void AddTest()
{
int a = 1;
int b = 2;
int expected = 3;
int actual;
actual = MyMath.Add(a, b);
Assert.AreEqual(expected, actual);
}
Writing this unit test is all well and good and validates one set of inputs but as we know, with real world code the methods that we test are never as trivial as my Add method and more variations of the input values need to be tested. In particular we need to ensure that boundary conditions are tested, values such as very large, very small and both positive and negative. A common approach to this is to copy and paste the unit test and change the values, however the data driven approach makes this much easier.
We will next convert the existing test into a data drive test.
Start by creating an Excel spreadsheet and placing it in the test project folder, save it in csv format. I named mine MathTestData.csv. I then created some test data as shown below:
Open the Test View window (Test -> Windows -> Test View) and select AddTest from the test list. With the AddTest selected open the properties window for the test. You should see something like the screenshot below.
By selecting the button with the ellipses in the Data Connection String input field you will be presented with the following window:
Select the CSV File option and click the next button, browse to where you saved the csv file and select it. Visual Studio reads the contents of the file and should present you with a view into the data.
As you can see it took the first row from my file and interpreted this as column names. Clicking Finish should result in a prompt to add the file into your project and add it as a deployment item. Your unit test is now decorated with two more attributes, one describing the data source for your test data and another indicating that you csv file should be a deployment item.
[
DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV",
"DataDirectory\\MathTestData.csv",
"MathTestData#csv",
DataAccessMethod.Sequential),
DeploymentItem("MathTestProject\\MathTestData.csv"),
TestMethod()
]
We can now alter our unit test to read values from the data source using the TestContext property
public void AddTest()
{
int a = int.Parse(TestContext.DataRow["a"].ToString());
int b = int.Parse(TestContext.DataRow["b"].ToString());
int expected = int.Parse(TestContext.DataRow["sum"].ToString());
int actual;
actual = MyMath.Add(a, b);
Assert.AreEqual(expected, actual);
}
When running the unit test now Visual Studio will iterate over each row in the data source executing the test for each. The test results will indicate success only if every row in the test data results in a passing test. If any row fails then the test is considered a failed test. For my simple example I added a row to my test data that supplies 3 for the value of “a”, -1 for the value of “b” and 6 for the “sum”. Obviously this will result in a failed test. Not because the code is wrong but rather the test data is faulty.
You will see that the Results indicates that 2/4 tests passed. This count is a bit misleading as there are three rows in my test data. The framework considers the one failing row as one failure and the test as a whole as the other failure. Double clicking on the failed test will open the test results window giving you a view into the results of each run.
Visual Studio automatically executes the rows in the test data file so that anytime more rows are added more tests are executed. Although this is a trivial example hopefully it demonstrates the potential. It is easy enough to represent property values of complex objects in the csv file as well as have multiple results being tested. The developer needs only to map the individual values into the individual properties of the objects under consideration. For example a row in the test data could contain as many columns as are required to represent a purchase order as well as the resulting values from submitting a purchase order. The potential is huge.
The real benefit of this is the ease in which a tester, business user or analyst can add and modify this test data using Excel. In fact, having the test data created in advance of the code for both the tests and the application will provide increased clarity and purpose for the developer moving towards a richer test first approach to application development.
In our approaches the test team has been crafting most of these spreadsheets and utilizing Visual Studio to check them in and out of source control to ensure that they are part of the automated test cycle. To support analysts and business users a method needs to be crafted that provides an easier means of accessing the file rather than source control while also ensuring that any changes are pushed to the project based instance of the file. Integration with SharePoint is the first option that springs to mind such that any changes to a SharePoint copy of the document is pushed to the source controlled copy using a plug-in. Sounds like a good CodePlex project….
Using this approach provides the very base level of functionality that FitNesse provides. As your team becomes increasingly comfortable with these techniques and realizes the value of pushing testing earlier and earlier in the cycle I strongly urge you to take a close look at FitNesse and the increased functionality that it provides. The learning curve is steeper but the capabilities far exceed what is available out of the box with Visual Studio.
