Writing Selenium 2 Acceptance Tests Using Behaviour Driven Vocabulary

Our project is using Selenium 2 to automate Acceptance Tests on our web app. We try to write the tests browser and implementation agnostic. This because we want to use the same test code to run the tests with Internet Explorer, Firefox and Chrome and parts of our application are written in JQuery and other parts are written using Wicket and we want the tests to work even if we change the implementation of some part for any reason.

We tried first to write the actual tests with JUnit but somehow the TDD vocabulary of JUnit was not satisfactory when we had been using the JDave BDD framework for our BDD specifications.

In JDave the specifications for one type are put to one class using the naming convention TypeSpec:



Then you create a context representing some state of an object of the type being specified as an inner class and write specifications to that:



And in natural language you read the specifications like this:

Shopping cart specification:
Empty shopping cart:
- is empty
...

We decided that in behaviorish acceptance test language the context is the state of the browser and we wanted to then write something like in the above example. So we created a class called WebSpecification. It also has an inner class WebSpecificationContext that is extended to create the contexts:



The "specification class" is the web browser which is represented in Selenium 2 by the interface WebDriver. The getWebDriver()-method is used to get instance of the currently active driver:



The driver is set by our version of the JDaveRunner, Selenium2JDaveRunner, which sets the driver class based on system property and sets it to the specification before running it:

Setting up Selenium 2.0 test engine with Hudson and Maven

In our project we as a team wanted to implement a strict ATDD practice where we write functional test cases for features before implementing them. We also wanted to write some of these tests as automated browser based tests and decided to use Selenium 2.0 (currently in alpha).

Our Continuous Integration server is Hudson but it is running in Linux so it cannot run the Internet Explorer driver. So we decided to set up another instance of Hudson running in a Windows box to run the tests with IE, Firefox and Chrome. After a very easy Hudson installation (Java WebStart + setting it to run as a windows service from the Hudson web gui with one click!), lengthy maven configuration and exploiting one windows vulnerability we now have a setup that runs the tests in three browsers and publishes the results to our main Hudson CI server.

Here is a short summary of the Hudson + maven setup we have:

• Hudson has separate jobs for each browser. The webdriver class name is passed in MAVEN_OPTS using -D to our custom test runner class that runs the functional tests with the specified driver.
• Maven has a separate profile called functional-test used to run the tests.
• Functional tests are run in the integration-test-phase. Integration tests are not ran in functional test profile.
• Jetty-plugin is started in the pre-integration-test phase of the maven build to run our application.
• Hudson runs jobs one at a time because we have not yet (Note to self: //TODO) configured different Jetty ports for different jobs.
• Maven arguments to run the individual jobs and to build the correct reports and to fail if tests fail (not default in failsafe...): "-P functional-test clean integration-test surefire-report:report failsafe:verify".

Below some relevant excerpts from our pom.xml.

First the profile for functional tests. We have naming convention for them "*FunctionalTest". It does not overlap our unit tests because they are named in BDD convention "*Spec" (we use JDave instead of JUnit). Integration tests are named "*IntegrationTest".



Integration test plugin failsafe config in build-section of the pom:

Selenium 2.0, Chrome, Hudson, Maven and Windows

I finally got our functional test server to run our functional tests also with the ChromeDriver of Selenium 2.o (WebDriver). Our functional test server is a Windows machine because it is required to run Internet Explorer and our Selenium 2.0 tests with it. This has caused quite a many problems and the last one was that our Hudson continuous integration server did not want to run the Chrome tests.

After a lot of blood, sweat and tears I figured out the problem. The problem was that the Chrome browser seems to install only to the user that run the installation program. The binary just does not start if you run it with another account. The Hudson CI is ran as a Windows service and Windows services are ran with a special system account. You can change the account to another one with the Windows admin tools but then the service cannot interact with the desktop. Like start a browser. Which is quite a showstopper when trying to do browser based testing.

So I would just have to install the Chrome browser using the system account. Easier said than done: one cannot log in as the system account. Some very dark windows magic: I started cmd.exe and then Firefox with system account using the "at" command, downloaded the Chrome installation program with the Firefox instance just started and then installed another instance of Chrome as system user. I also copied the installation from the C:\Documents and Settings\blahblahblah to C:\GoogleSys to have it in a directory without a space in the name - a major menace when passing paths in environment variables like MAVEN_OPTS.

This just so worked and now we have a Hudson running a maven build running Selenium 2.0 functional tests with Internet Explorer, Firefox and Chrome.