Selenium and what it is not

This post was published on June 22, 2016

Please note: this post is in no way meant as a rant towards some of my fellow testers. I’ve only written this to try and clear up some of the misconceptions with regards to Selenium WebDriver that I see popping up on various message boards and social media channels.

Selenium is hot in the test automation world. This probably isn’t news to you. If it is, you’re welcome, please remember you read it here first! In all seriousness, Selenium has been widely used for the past couple of years, and I don’t see its popularity fading any time soon. Due to its popularity, Selenium has been the go-to solution for everybody looking to write automated checks for browser-based applications, even though sometimes it might not be the actual best answer to the question at hand. In the past couple of years I have seen a lot of funny, strange and downright stupid questions being asked with regards to Selenium.

In this post, I would like to look at Selenium from a slightly different perspective by answering the question ‘what is Selenium NOT?’. All of the examples in the remainder of this blog post are based on actual questions or anecdotes I have read. I’m not going to link to them, since this post is not about blaming and shaming, it’s about educating those in need of some education.

Selenium is not a test tool
You read that right, Selenium is not a test tool. A lot of people (including a staggeringly large percentage of recruiters) get this wrong, unfortunately. To quote the Selenium web site:

Selenium automates browsers. That’s it! What you do with that power is entirely up to you.

My personal number one reason why Selenium is not a test tool: it does not provide a mechanism to perform assertions. For that, you’ll have to combine Selenium with JUnit or TestNG (for Java), NUnit (for C#) or any other actual (unit) testing framework. Selenium only performs predefined actions on a web site or other application running in a browser. It does this fairly well, but that doesn’t make it a test tool.

Off-topic: the same applies to tools that support Behaviour Driven Development, such as Cucumber and SpecFlow. Those aren’t test tools either. They can be used as an assistant in writing automated tests (checks), but that isn’t the same thing.

Selenium is not a tool to be used in API testing
I’ve seen this one come by a number of times, even as recently as a few days ago:

How can I perform tests on APIs using Selenium?

Again, Selenium automates browsers, so it operates on the user interface level of an application. Actions performed on a user interface might invoke API calls. Selenium is completely oblivious to this API interaction, however. There’s no way to have Selenium interact with APIs directly. If you want to perform automated checks on the API level, please use a tool that is specifically created for these types of checks, such as REST Assured. It might be wise to repeatedly ask yourself ‘Am I actually testing the user interface, or am I merely testing my application THROUGH that user interface?‘.

Selenium is not a performance test tool
Seeing how easy it is to write tests using Selenium (I’m not saying it’s easy to write GOOD tests, though), a lot of people seem to think that these tests can easily be leveraged to execute load and performance tests as well, for example using Selenium Grid. This is a pretty bad idea, though. Selenium Grid is a means to execute your functional tests in parallel, thereby shortening the execution time. It is not meant to be (ab)used as a performance testing platform, if only for these two simple reasons:

• It doesn’t scale very well, so you’ll probably have a hard time generating anything but trivial loads
• Selenium does not offer a mechanism to measure response times (at least not without taking into account the time needed at the client side to render a page or specific elements), and Selenium Grid isn’t able to gather and aggregate these response times for each individual note and present them in a concise and useful manner.

Both of these are essential if you’re serious about your performance testing, so please use a tool that is specifically written for that purpose, such as JMeter.

Selenium can’t handle your desktop applications
I feel that I’m starting to repeat myself here: Selenium automates browsers. Anything outside the scope of that browser cannot be handled by Selenium. This means that Selenium also can’t handle alerts and dialogs that are native to the operating system, such as the Windows file upload/download dialogs. To handle these, either use a tool such as AutoIt (if you’re on Windows) or, and this approach is much preferred, bypass the user interface altogether and use a solution such as the one presented in this blog post.

I sincerely hope this clears up some of the misconceptions around Selenium. If you have any other examples, feel free to post a comment or send me an email.

"