TestNG Parameterization with CSV Files by Keys

TestNG is a great tool for running tests and has some advantages over JUnit and other commonly used unit test frameworks.  I personally like being able to configure test suites easily with an XML file and have the suite definition separate from the tests themselves.  I also like the Maven plugin, which combined with the XML suite definition, allows for a lot of flexibility in running combinations of tests and configurations via Maven.  Conversely, one thing that is not very built-out about the framework is how test parameterization works (specifically that DataProviders are only supported as Object arrays). 

I can see why they did it that way.  It is easy to create an Object array in your test class as all of the examples I have seen describe, but what if you want to use the same data in multiple places?  You either have to use inheritance or write redundant data into two separate tests, right? What if you already have external data; do you need to write your own custom adapter? Why do your tests and test data have to be so coupled?

Ideally, you should just be able to manage your test data in something lightweight like a CSV file, reference it in your test, and retrieve the test data values by the heading name in the file, right?  Despite how common I thought this request would be (seeming as most every test tool I have used has supported parameterization via a CSV file), after doing some searching, I found no one explaining how to do this with TestNG.  As a result, I did it myself, and I am posting this article describing how I created an object that maps test data in CSV files to something TestNG can parameterize to save anyone from having to do this from scratch again.

Let’s start with the object that reads the CSV files, holds the data, and spits out the Object arrays (as that is the only thing that TestNG can read).  Really all it does is maintain a list of HashMaps (one for every test data row, storing the data for each particular test execution with the key for each parameter being the heading in the test data file and the value being the value for that particular test data row; I used a HashMap so that you can get the values by the name of the column in the test data file).   The object also has some additional logic to deal with the case where you want to add to multiple data files of varying lengths by repeating the values of the shorter file so all values in each test file are tested:

public class TestDataHolder {
	private static final long serialVersionUID = 1L;
	private ArrayList<HashMap<String,String>> dataRows;
	private int numberOfDataSources;

	public TestDataHolder(){
		dataRows = new ArrayList<HashMap<String,String>>();
		numberOfDataSources = 0;
	}

	public void addDataLocation(String fileLocation) throws FileNotFoundException {
		addDataLocation(new FileReader(fileLocation));
	}

	public void addDataLocation(Reader csvReader){
		try{
			final CsvReader cs = new CsvReader(csvReader);
			final String [] headings = cs.readLine();
			
			int rowNumber = 0;
			int rowsToUpdateWithWrappingValues = 0;
			String [] nextLine;
		    
			while ((nextLine = cs.readLine()) != null) {
		    	if (dataRows.size() <= rowNumber){
		    		dataRows.add(rowNumber,new HashMap<String,String>());
		    		//case that this data file has more rows than the datastore
		    		rowsToUpdateWithWrappingValues++;
		    	}
		    	for(int i = 0; i < nextLine.length; i++){
		    		dataRows.get(rowNumber).put(headings[i], nextLine[i]);
		    	}
		    	dataRows.get(rowNumber).put("rownumber", Integer.toString(rowNumber));
		    	rowNumber++;
		    }
			if(rowNumber > 0){ this.numberOfDataSources++; }
		    
		    //case that the data file has less rows than the datastore
		    if(rowsToUpdateWithWrappingValues == 0 && rowNumber < this.dataRows.size()){
		    	rowsToUpdateWithWrappingValues = this.dataRows.size() - rowNumber;
		    }
    		
		    //to account for data files with different lengths, if a file is 
    		//added with more rows, the data attributes will recycle for data
    		//files with less elements
		    if(this.numberOfDataSources > 1 && rowsToUpdateWithWrappingValues > 0){
		    	populateValuesForNulls(rowsToUpdateWithWrappingValues);
		    }
		}catch(IOException ie){
			System.out.println("A data file couldn't be loaded");
		}
	}

	public Object[][] getAllDataRows(){
		Object[][] returnObject = new Object[this.dataRows.size()][1];
		for(int itemIndex = 0; itemIndex < this.dataRows.size(); itemIndex++){
			returnObject[itemIndex][0] = this.dataRows.get(itemIndex);
		}
		return returnObject;
	}

	private void populateValuesForNulls(int rowsToUpdateWithWrappingValues){
		final int resultsBeforeUpdate = (this.dataRows.size() - rowsToUpdateWithWrappingValues);
		//get first list item's key set
		final Set keySet = this.dataRows.get(0).keySet();
		for(int itemNeedsUpdate = 0; itemNeedsUpdate < rowsToUpdateWithWrappingValues;
    	itemNeedsUpdate++){
			int itemCurrentlyChecking = (resultsBeforeUpdate+itemNeedsUpdate);
			// iterate over the keys, check if they are in the current item, if not add them
			for(String currKey : keySet){
				if(!(this.dataRows.get(itemCurrentlyChecking).containsKey(currKey))){
					final int itemToUse = (itemNeedsUpdate % resultsBeforeUpdate);
					final String itemToUseValue = this.dataRows.get(itemToUse).get(currKey);
					this.dataRows.get(itemCurrentlyChecking).put(currKey,itemToUseValue);
				}
			}
	    }
	}

In order to utilize this object, you simply instantiate one in your test, call the addDataLocation method with each of the data files you want to use, and create a DataProvider that calls the getAllDataRows in the TestDataHolder.  Then you are free to retrieve test data values for each parameterized execution of your test by the parameter name via get method of HashMap (as TestDataHolder conveniently returns a HashMap for each parameterized execution of a test instead of each object individually).  Here is an example:

import com.jeffmw.test.common.resources.TestDataHolder;
public class ComprehensiveCamcorderSearchTest{
	private TestDataHolder testData;

	public ComprehensiveCamcorderSearchTest() throws IOException{
		Resource r = new ClassPathResource("CamcorderQueries.csv");
		this.addDataFile(new InputStreamReader(r.getInputStream()));
	}

 	public void addDataFile(Reader thisReader){
	   if(this.testData== null){
		   this.testData = new TestDataHolder();
	   }
		this.testData.addDataLocation(thisReader);
   	}

   	@DataProvider(name="iterativeDataProvider")
   	public Object[][] getTestData(){
	   return testData.getAllDataRows();
   	}

	@Test(dataProvider = "iterativeDataProvider")
	public void ComprehensiveHomePageSearchResultTest(HashMap<String,String> parameterValues){
		final String query = parameterValues.get("Query");
		final String expectedId = parameterValues.get("ExpectedId");
		final String category = parameterValues.get("Category");

		this.TransitionHomeToResultsSearchBox(query, category);
		this.AssertResultsGreaterThanZero(query);
		this.AssertResultsContainItem(query, expectedId);
	}
}

Where the test data file looks like:

“Category”,”Query”,”ExpectedId”
“Camcorders”,”Sony Handycam DCR-DVD650″,”15051028″
…

Once you’ve created this test, you are free to manipulate the test data file all you want.  You can add data, add more columns, use it in other tests without having to worry about the method signature of the test conflicting (as the only parameter for these test will always be a HashMap<String,String> instead of having a parameter for each variable).  This satisfies all of the requirements to decouple the test data from the test, be able to store test data in a CSV, and to retrieve the value for each parameter by the column name in the test data file itself.

Tags: Automation, CSV, Parameterization, Testing, TestNG