Why there’s no such thing as codeless automation
This post was published on September 27, 2017In today’s blog post – which, again, is really nothing more than a thinly veiled rant – I’d like to cover something that’s been covered before, just not by me: codeless test automation and why I think there isn’t and should not be such a thing.
I’ve seen numerous ‘solution’ vendors advertise their products as ‘codeless’, implying that everybody in the team will be able to create, run and maintain automated tests, without having to, well, write code. I’ve got a number of problems with selling test automation in this way.
It’s not codeless. It’s hiding code.
The first gripe I have with ‘codeless’ automation is a semantic one. These solutions aren’t codeless at all. They simply hide the code that runs the test from plain sight. There are no monkeys in the solution that magically execute the instructions that make up a test. No, those instructions are translated into actual code by the solution, then executed. As a user of such a solution, you’re still coding (i.e., writing instructions in a manner that can be interpreted by a machine), just in a different syntax. That’s not codeless.
While it might be empowering, it’s also limiting.
Sure, using codeless tools might potentially lead to more people contributing to writing automated tests (although from my experience, that’s hardly how it’s going to be in the end). The downside is: it’s also limiting the power of the automated tests. As I said above, the ‘codeless’ solution is usually nothing more than an abstraction layer on top of the test automation code. And with abstraction comes loss of detail. In this case, this might be loss of access to features of the underlying code. For example, if you’re using a codeless abstraction on top of Selenium, you might lose access to specific waiting, synchronization or error handling mechanisms (which are among the exact things that makes Selenium so powerful).
It might also be loss of access to logging, debugging or other types of root cause analysis tools, which in turn leads to shallower feedback in case something goes wrong. While the solution might show you that something has gone wrong, it loses detail on where things went wrong and what caused the failure. Not something I like.
Finally, it might also limit access to hooks in the application, or limit you to a specific type of automated tests. If such a solution makes it potentially easier to write automated tests on the user interface level, for example, there’s significant risk that all tests will be written at that level, even though that might not be the most efficient approach in the first place. If all you’ve got is a hammer…
It’s doing nothing for the hard problems in creating maintainable automation.
Let’s face it: while writing code might seem hard to people that haven’t done it before, it actually isn’t that difficult once you’ve had a couple of basic programming classes, or followed a course or two on Codecademy. What is hard is writing good, readable, maintainable code. Applying SOLID and DRY principles. Structuring your tests. Testing the right thing at the right level. Creating a solid test data and test environment strategy. Those things are hard. And codeless test automation does nothing for those problems. As I tried to make clear in the previous paragraphs, it’ll often make it even harder to solve those problems effectively.
I’m all for creating solutions that make it easier to write, run and maintain automation. I hate people selling solutions as something they’re not. Codeless test automation is not going to solve your test automation problems. People that know
- how to decide what good automation is
- how to write that automation, and
- how to pick the tools that will help them achieve the goals of the team and organization
will.
"