Saturday, October 30, 2010

How to run selenium tests as part of maven build phase

Recently I came across a very simple requirement.  Our project used Maven as the build tool and we wanted to run Selenium tests as part of the build.  The idea was, QA's write the automated selenium tests, these should run as part of the integration-test build phase

There are multiple advantages of doing this:
  • We could run these Selenium tests on our Continuous Integration server (Hudson in our case) as part of build process.  This means, if because of some check-in, a selenium tests fails, the build fails and hence someone has to fix it!
  • Developers and QA's can easily run these functional tests whenever they want.  For e.g. if developers want to make sure that the code they are about to checkin does not break any existing functionality they can run the functional tests before checkin with just one command.
So how do they do it?

Lets try and reverse engineer the problem.  To run selenium tests we need to do the following things
  1. Compile the app
  2. Run unit-tests
  3. Create a war file of the application
  4. Run an tomcat server (or any other servlet container you are using)
  5. Deploy the war file created in step 3 on the server started in step 4
  6. Start the selenium server
  7. Fire the selenium tests
Looks like a lot of work to me!  But don't worry we will see in a moment that its not as difficult as it sounds.

Lets do this step-by-step:

Lets break down the problem.  If you look at the above mentioned 7 steps.  We can classify them in four broad categories
  1. Compile the application and package it as a war file. (steps 1-3)
  2. Run the tomcat server (steps 4-5)
  3. Run the selenium server (step 6)
  4. Run the selenium tests (steps 7-8)   
Problem looks simpler now?  Lets try and slove the above four problems separately first.  Later  we will see how we can solve all of them using just one command.

Solving the step 1:

Assuming that you already have the pom.xml setup to properly compile and run unit-tests.  We need to add the configuration in pom.xml to package the application as war file.  This is what we have to do

Find the tag called <packaging> in pom.xml and set the value of the tag to war instead of jar.

Then, we need to tell maven about how the war file will be generated.  This is done using the maven-war-plugin.  The sample configuration looks like this:

Of course for all this to work you should have a proper directory structure and web.xml.  But I will assume that you already have that in place.

At this point if you run the following command:

You will get a war file ready to be deployed in any servlet container.

Next, Solving the step 2:

To run an embedded tomcat server we have to add the following configuration to pom.xml

Now, when we run the following command, we will have a running tomcat server with our application deployed on it.

Next, Solving the step 3:

To start the selenium server from command prompt we need to do two things.
  • Add the dependency of selenium drivers to the app.  This is required so that we could use selenium api to programmatically write selenium testsThis is done using the following code:

  • Add the selenium plug-in so that we can start the selenium server from the command prompt.  This is done using the following code:

After this we can start the selenium server using the following command:

Next, Solving the step 4:

Now, we are ready to run the tests simply run the tests using the following command:

We have our selenium tests running as part of our maven build!  To do this we have to fire three commands from three command prompts/terminal windows.  One running tomcat, another running selenium server and the third running the tests.

All this is good but we don't want three command prompts running.  We just want one command that does it all!

And this is exactly what we are going to see next!

So as you already know that Maven is build around phases, we are going to run our functional tests (selenium tests) in the integration-test phase.

To run the selenium tests in the integration-test phase we have to do the following:
  1. Start the tomcat server
  2. Deploy the war
  3. Start the selenium server
  4. Fire the function tests
Here's how we do steps 1 and 2:

Above configuration means that following things in plain english:

  • Start the tomcat6 server
  • Deploy the app during the pre-integration-test phase.  
  • Stop the server during the post-integration-test phase.
You will have to define an environment variable CATALINA_HOME, which points to the tomcat server installation directory on your machine.

Next up, Step 3 do this:

Above configuration states the following:
  • Start the selenium server during the pre-integration-test.
  • Stop the server during the post-integration-test.
Next, Step 4: 

To run the functional tests during the integration-test phase, do the following (I promise this is the last bit of XML that you see on this post!)

This configuration does the following:
  • Tells maven-surefire-plugin to run the unit tests during the test phase but exclude the classes in the functional folder.
  • Run the test class in the functional folder during the integration-test phase.
And that's it my friend!  Now when you run the following command:

  1. Application will compile
  2. Unit tests will run
  3. War file will be created
  4. Tomcat server will be started 
  5. war file will be deployed on the tomcat started in step 4
  6. Selenium server will be started
  7. Functional tests will be fired
  8. And we will live happily ever after!
Have some Fun!