There is a place for Selenium and test tools of it’s like but not to the extent that I see many companies using it. It can be made relatively maintenance free with a bit of finessing and product and development teams that think about testing continuously. Of course, this is usually not the case and release after release, I see more and more time being devoted to just getting the test cases to work. You see, with an agile environment comes A/B testing, rapid iteration and many changes to the GUI.
Test case maintenance should never take so much time that new test case creation suffers. This is an overarching problem that I find with web-based functional testing. Since this is the area that’s constantly changing – visually, not functionally – teams should automate at this layer sparingly. I’m a big fan of back-end automation and what little bit of front-end automation I endorse, usually eliminates the browser all together.
Essentially, you can get the best bang for your front-end functional automation by simply POSTing/GETing forms via an xUnit test framework without worrying about your test cases due to GUI changes. There’s no reason why a functional test case should fail because a button moved on a page or an image is taking too long to load – not that the latter is unimportant but a different test case verifying that functionality should fail instead.
I demonstrated this kind of testing at past companies and recently at Chegg. Even though my script hadn’t been used in weeks, the only maintenance needed was a change to a POST url and the addition of another parameter (both of which would have also failed with Selenium, btw). The script is not part of the regular regression suite simply because it’s used as a helper script to speed up tedium and thus was not used for a while.
To be clear, it’s a slow process to get started automating in this way but the extra amount of time spent building the test cases FAR out-weigh the time saved in the long run. To illustrate this, the above script was created because the Selenium automation engineer refused to automate the functionality; those web pages constantly changed and would require maintenance every week. Besides, once you start building up a library of HTML parsers and POST/GET URLs, the time to create new test cases speeds up.
Here’s an example of how simple a test case can become after the necessary libraries are built (example in Python, of course):
#open a new liquidation cart
url = cfg.get('urls', 'openCart') % wimsBaseUrl
elements = {'universal_input':cartName}
result = CL.postHtml(url, **elements)
myTag = CL.getTagData(result, 'input', **myArgs)
The “url” is the url to POST the parameters to, “elements” is a Dictionary of the parameters being POSTed, “result” is the HTML response after the POST and “myTag” is an example of a function used to parse the HTML (check out BeautifulSoup for Python).
In conclusion, slow and steady wins the race.
