Creating HTML reports for your Selenium tests using ExtentReports

This post was published on February 5, 2015

In a comment on a previous post I wrote on creating custom HTML reports for Selenium tests, Anshoo Arora of Relevant Codes pointed me to ExtentReports, his own solution for generating HTML reports. This week, I have been playing around with it for a while and I must say I’m pretty impressed by its ease of use and the way the reports it generates turn out. Let’s have a look.

Note: this post is based on ExtentReports version 1.4 and therefore might not reflect any changes made in more recent versions.

Downloading and installation
Installation of ExtentReports is a breeze. Just download the latest .jar from the website, add it as a dependency to your Java project and you’re all set.

An example test and report
To illustrate the workings of ExtentReports, we’ll use two simple login tests that are executed against the ParaBank demo application on the Parasoft website. Both tests perform a login action, where the first will be successful and the second one will be unsuccessful due to invalid credentials. We’ll have ExtentReports generate a HTML report that contain the appropriate test results.

public class ExtentReportsDemo {

	static final ExtentReports extent = ExtentReports.get(ExtentReportsDemo.class);
	static WebDriver driver = new FirefoxDriver();

	static String reportLocation = "C:\\Tools\\reports\\";
	static String imageLocation = "images/";
	
	public static void main (String args[]) {

		extent.init(reportLocation + "ScreenshotReport.html", true, DisplayOrder.BY_OLDEST_TO_LATEST, GridType.STANDARD);

        extent.config().documentTitle("Sample ExtentReports report");
        
		extent.config().reportHeadline("Test report for ParaBank login tests generated using <b>ExtentReports</b>");

		extent.startTest("TC01.1","This test is a positive login test for ParaBank");
		runPositiveTest();
		extent.endTest();

		extent.startTest("TC01.2","This is a negative login test for ParaBank (contains a defect)");
		runNegativeTest();
		extent.endTest();

		driver.quit();
	}

	public static void runPositiveTest() {

		driver.get("http://parabank.parasoft.com");

		checkTitle(driver,"ParaBank | Welcome | Online Banking");

		driver.findElement(By.name("username")).sendKeys("john");
		driver.findElement(By.name("password")).sendKeys("demo");
		driver.findElement(By.xpath("//input[@value='Log In']")).click();

		checkTitle(driver,"ParaBank | Accounts Overview");

		driver.findElement(By.partialLinkText("Log Out")).click();
	}

	public static void runNegativeTest() {

		driver.get("http://parabank.parasoft.com");

		checkTitle(driver,"ParaBank | Welcome | Online Banking");

		driver.findElement(By.name("username")).sendKeys("john");
		driver.findElement(By.name("password")).sendKeys("incorrectpassword");
		driver.findElement(By.xpath("//input[@value='Log In']")).click();

		checkErrorMessage(driver,"Incorrect credentials");
	}

	public static void checkTitle(WebDriver driver, String expectedTitle) {

		if(driver.getTitle().equals(expectedTitle)) {
			extent.log(LogStatus.PASS, "Check page title", "Page title is " + expectedTitle);
		} else {
			extent.log(LogStatus.FAIL, "Check page title", "Incorrect login page title " + driver.getTitle());
		}
	}

	public static void checkErrorMessage(WebDriver driver, String expectedMessage) {

		String errorMessage = driver.findElement(By.className("error")).getText();

		if(errorMessage.equals(expectedMessage)) {
			extent.log(LogStatus.PASS, "Check error message", "Error message is " + expectedMessage);
		} else {
			extent.log(LogStatus.FAIL, "Check error message","View details below:",createScreenshot(driver));
		}
	}
}

As you can see, using ExtentReports is very straightforward. All we need to do is initialize the report by specifying:

• The location where the report needs to be stored
• If any existing reports with the same name should be overwritten
• Whether the tests in the report should appear in chronological (DisplayOrder.BY_OLDEST_TO_LATEST) or antichronological order
• Whether the grid type used for the results should be standard (GridType.STANDARD) or using the Masonry framework (GridType.MASONRY)

With .config().documentTitle(“Your report title”) you can set the HTML document title for your report.

We can also specify a custom header using .config().reportHeadline(“place header here”). This header can also contain HTML code, which gives you a wide array of options for creating a nice looking header for your report.

For every test, we start a test-specific report using .startTest(), and we end recording reporting events using .endTest(). .startTest() takes two parameters, the first is the test name or identifier, the second is a description that is added to the report as we will see. .endTest() ends the report activities for the test that has been started earlier. As of version 1.0, explicitly ending a test using .endTest() is no longer required when you want to start a new test.

Logging actual events is straightforward as well using the .log() method. Logging events can be created on six different levels (their use as presented here are suggestions only):

• PASS for checks that are executed with a positive result
• ERROR for logging any (non-fatal) error that occurs during test execution
• WARNING for logging warnings
• FAIL for checks that are executed with a negative result
• INFO for logging additional info relevant to the test (or for debugging purposes)
• FATAL for logging fatal errors occurring during test execution

Since we can include HTML markup everywhere in the report, we can also add screenshots wherever we want. To do this, all we need to do is to add a slightly modified version of the screenshot method I introduced here:

public static String createScreenshot(WebDriver driver) {

	UUID uuid = UUID.randomUUID();

	// generate screenshot as a file object
	File scrFile = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
	try {
		// copy file object to designated location
		FileUtils.copyFile(scrFile, new File(reportLocation + imageLocation + uuid + ".png"));
	} catch (IOException e) {
		System.out.println("Error while generating screenshot:\n" + e.toString());
	}
	return imageLocation + uuid + ".png";
}

Note that I’ve predefined the report folder (an absolute path) and image subfolder (a subdirectory of the report location) somewhere else in the code. This method is called in the example above when we log an error and want to have a screenshot added to the report.

The result can be viewed here (the link to the screenshot is in the results for the second test). The same report using the Masonry framework can be viewed here. You’re free to play around with the various buttons to filter, expand and collapse test results and change the overall layout of the report a bit to have a look at the various display options.

More configuration options
Apart from the options already used in the examples above, ExtentReports offers a couple of other useful features. If you really want to give your HTML reports a customized look, you can apply your own specific CSS simply by specifying it during report initialization as follows:

extent.config().addCustomStylesheet("C:\\path\\to\\yourcustomcss.css");

Alternatively, you can use inline style definitions as follows:

String style = "p{font-size:20px;}";
extent.config().addCustomStyles(style);

You can also add your own JavaScript code to your report:

extent.config().insertJS("$('.test').click(function(){ alert('test clicked'); });");

You can also disable the footer that is displayed at the bottom of the page:

extent.config().removeExtentFooter();

You can also include a screenshot in the report without having to log a test step:

extent.attachScreenshot(createScreenshot(driver),"This is a sample screenshot without an accompanying log step");

Finally, you can exchange the standard icons for any other icon included in the FontAwesome library:

extent.config().statusIcon(LogStatus.PASS, "check-circle");

Since version 1.4, a lot of new features have been added. For the complete documentation please review the usage documentation at http://relevantcodes.com/extentreports-documentation/.

Further notes
As you have seen in this post, ExtentReports is a very useful library for quick generation of great looking HTML reports. Although ExtentReports is labeled as a solution for generating HTML reports for Selenium tests, it’s a generic solution that can be applied easily to other test frameworks, as long as they’re built in Java.

Thanks again to Anshoo Arora for pointing me to ExtentReports.

"