Programming  /  Testing January 18, 2010

Web testing with Selenium and xUnit.net

I’ve been playing around with the web testing tool Selenium lately. At first I used Selenium IDE which is a Firefox add-on that is used to record tests. The tests can then be replayed in Selenium IDE or exported to code in a number of languages, including C#. As I wanted to create a suite of automated Selenium tests for this site I choose to do just that, export my tests to C#.

Once I had the Selenium tests exported I faced three problems. First of all the tests where written for NUnit while my current testing framework of choice is xUnit.net so I had to rewrite them. I also had to figure out how to run them, preferably in a way that would require as little effort as possible. That is I wanted running the tests to be a very automated process. Finally I also quickly realized that Selenium IDE didn’t always generate tests that would be easy to maintain, which I would have to fix.

One of my tests tested that the comment functionality worked correctly. In this post I’ll describe how I created that test and dealt with the previously mentioned problems.

Prerequisites

In order to create and run the tests in Visual Studio there are a few things we need to download and install:

  • Download xUnit.net
  • Create a Visual Studio project of type Class Library and add a reference to xunit.dll
  • Download and install a test runner that supports xUnit.net such as TestDriven.Net 
  • Enable TestDriven.Net for xUnit.net by running xunit.installer.exe
  • Download Selenium IDE and Selenium RC
  • Add a reference to ThoughtWorks.Selenium.Core.dll (shipped with Selenium RC)
  • Download and install Java which is required for running Selenium

The test description

The test for the comment functionality can be divided into three steps:

  1. Locate a blog post to add the comment to.
  2. Post a comment by entering values in the form fields and clicking on the submit button
  3. Verify that the comment is displayed in the list of comments

Starting Selenium Server from the test

In order to run Selenium tests without using Selenium IDE we need to run Selenium Server which is shipped with Selenium RC. Since it’s a Java program we run it by typing java –jar selenium-server.jar in a console window. However, doing so each time we want to run tests is time consuming and will in the end lead to that the tests aren’t run as often as they would be if running them would be a fully automated process.

Therefore I created a simple utility class for starting and stopping Selenium Server that my tests could use in their constructors and Dispose methods (SetUp and Teardown methods in NUnit).

using System.Diagnostics;

namespace JoelAbrahamsson.Web.Tests.UITests
{
    public class SeleniumServer
    {
        private static Process process;

        public static void Start()
        {
            if (IsStarted)
                return;

            IsStarted = true;

            process = new Process();
            process.StartInfo.FileName = JavaPath;
            process.StartInfo.Arguments = JavaArguments;
            process.StartInfo.CreateNoWindow = true;
            process.Start();
        }

        public static void Stop()
        {
            process.Kill();

            IsStarted = false;
        }

        public static bool IsStarted { get; set; }

        private static string JavaPath
        {
            get
            {
                return @"c:\Program Files (x86)\Java\jre6\bin\java.exe";
            }
        }

        private static string JavaArguments
        {
            get
            {
                return @"-jar c:\selenium-server-1.0.1\selenium-server.jar";
            }
        }
    }
}

The class have two static methods, Start() and Stop() and I guess what they do is pretty self-explanatory. The path to Java and the path to selenium-server.jar is hardcoded in the JavaPath and JavaArguments properties. This is sufficient for me at the moment but should of course be configurable in a multi developer project.

Creating the test

Once I had the starting and stopping of Selenium Server dealt with I started creating the actual test. I begun by navigating to the start page of my site and firing up Selenium IDE  from the tools menu in Firefox. There I clicked the record button and went back to Firefox, leaving the Selenium IDE window open.

verifyTextPresent In Firefox I clicked on the link to the latest blog post, scrolled down to the comment form and posted a new comment. Then I right clicked on the text in the newly posted comment and choose the verify TextPresent alternative.

Back in Selenium IDE I choose to view the test source as C# code by selecting Options –> Format –> C# – Selenium RC. The generated code looked like this (using statements and namespace omitted for brevity):

[TestFixture]
public class Untitled
{
    private ISelenium selenium;
    private StringBuilder verificationErrors;
    
    [SetUp]
    public void SetupTest()
    {
        selenium = new DefaultSelenium("localhost", 4444, "*chrome", "http://change-this-to-the-site-you-are-testing/");
        selenium.Start();
        verificationErrors = new StringBuilder();
    }
    
    [TearDown]
    public void TeardownTest()
    {
        try
        {
            selenium.Stop();
        }
        catch (Exception)
        {
            // Ignore errors if unable to close the browser
        }
        Assert.AreEqual("", verificationErrors.ToString());
    }
    
    [Test]
    public void TheUntitledTest()
    {
        selenium.Open("/");
        selenium.Click("link=The headline of a blog post");
        selenium.WaitForPageToLoad("30000");
        selenium.Type("authorName", "Joel");
        selenium.Type("authorEmail", "test@test.com");
        selenium.Type("authorUrl", "http://www.test.com/");
        selenium.Type("comment", "This is a test comment.");
        selenium.Click("subscribeToComments");
        selenium.Click("//input[@value='Add Comment']");
        selenium.WaitForPageToLoad("30000");
        try
        {
            Assert.IsTrue(selenium.IsTextPresent("This is a test comment."));
        }
        catch (AssertionException e)
        {
            verificationErrors.Append(e.Message);
        }
    }
}

Converting the test to xUnit.net and some clean up

As xUnit doesn’t have SetUp or Teardown methods but instead works with constructors (instead of SetUp) and IDisposable (instead of Teardown) I turned the SetupTest() method into a constructor and renamed the TeardownTest() method into Dispose. I also made the class implement IDisposable.

Furthermore I replaced the Assert.IsTrue call with a call to xUnits Assert.True() method, removed the TestFixture attribute and renamed the Test attribute to Fact.

Having done that the code would compile but it would crash in the constructor as Selenium Server wasn’t started. So, I introduced a call to SeleniumServer.Start() in the constructor. Finally I did some cleaning up. I gave the class and test method more descriptive names. I also removed the try-catch in the test and the StringBuilder as I wanted the test to halt immediately if the assertion failed.

Also, note that the generated test instantiates an instance of DefaultSelenium with an URL to the site you are testing. For some reason the generated test calls the constructor with a default value “http://change-this-to-the-site-you-are-testing/”, which I of course changed to the URL of my (local testing) site.

public class CommentTests : IDisposable
{
    private ISelenium selenium;

    public CommentTests()
    {
        SeleniumServer.Start();

        selenium = new DefaultSelenium("localhost", 4444, "*chrome", "http://local.joelabrahamsson.com/");
        selenium.Start();
    }

    public void Dispose()
    {
        selenium.Stop();

        SeleniumServer.Stop();
    }

    [Fact]
    public void ValidCommentInptut_NewCommentPosted_CommentIsDisplayed()
    {
        selenium.Open("/");
        selenium.Click("link=The headline of a blog post");
        selenium.WaitForPageToLoad("30000");
        selenium.Type("authorName", "Joel");
        selenium.Type("authorName", "Joel");
        selenium.Type("authorEmail", "test@test.com");
        selenium.Type("authorUrl", "http://www.test.com/");
        selenium.Type("comment", "This is a test comment.");
        selenium.Click("subscribeToComments");
        selenium.Click("//input[@value='Add Comment']");
        selenium.WaitForPageToLoad("30000");

        Assert.True(selenium.IsTextPresent("This is a test comment."));
    }
}

Refining the test

While the above test passed it has a couple of serious flaws. First of all, it relies on there being a link to a blog post on the first page on the site with the text “The headline of a blog post”. When I created the test with Selenium IDE that was the latest blog post but in a couple of months that post will probably not be listed on the start page and the test will fail. One solution to this would be to create a post that the test always will use and rewrite the test to start from that post’s URL.

I however choose a different solution and rewrote the test to always click on the link to the latest post. To do that I rewrote the first call to selenium.Click() to use a CSS selector, one type of what Selenium calls Element Locators, to identify the link to the latest post, that is the first a-tag descendant of the element with ID entryTeaserList. Apart from CSS selectors Selenium also supports XPATH, javascript and a few others. For more on Element Locators check out the Selenium API documentation.

[Fact]
public void ValidCommentInptut_NewCommentPosted_CommentIsDisplayed()
{
    //Open the first page
    selenium.Open("/");

    //Click on the link to the latest post
    selenium.Click("css=#entryTeaserList a:first-child");
    selenium.WaitForPageToLoad("30000");

    //Post a new comment
    selenium.Type("authorName", "Joel");
    selenium.Type("authorName", "Joel");
    selenium.Type("authorEmail", "test@test.com");
    selenium.Type("authorUrl", "http://www.test.com/");
    selenium.Type("comment", "This is a test comment.");
    selenium.Click("subscribeToComments");
    selenium.Click("//input[@value='Add Comment']");
    selenium.WaitForPageToLoad("30000");

    Assert.True(selenium.IsTextPresent("This is a test comment."));
}

Another flaw with the test was that it used the same test values for each test run. Unless I also implemented some clean up to remove the posted comment after each test run the test would always pass as the text it would be looking for would be present, even if the comment wasn’t successfully saved, as it would have been posted by previous test runs.

While removing the comment after each test run is probably a good idea (I’ll do that later…) I didn’t want my test to rely on the successful cleanup in previous test runs. So, I instead modified the test to post a comment with a random text.

[Fact]
public void ValidCommentInptut_NewCommentPosted_CommentIsDisplayed()
{
    string comment = Guid.NewGuid().ToString();

    //Open the first page
    selenium.Open("/");

    //Click on the link to the latest post
    selenium.Click("css=#entryTeaserList a:first-child");
    selenium.WaitForPageToLoad("30000");

    //Post a new comment
    selenium.Type("authorName", "Joel");
    selenium.Type("authorName", "Joel");
    selenium.Type("authorEmail", "test@test.com");
    selenium.Type("authorUrl", "http://www.test.com/");
    selenium.Type("comment", comment);
    selenium.Click("subscribeToComments");
    selenium.Click("//input[@value='Add Comment']");
    selenium.WaitForPageToLoad("30000");

    Assert.True(selenium.IsTextPresent(comment), "The comment text was not displayed");
}

Last but not least I also wanted to test that the author’s name was displayed as a link to the authors URL.

[Fact]
public void ValidCommentInptut_NewCommentPosted_CommentIsDisplayed()
{
    string authorName = Guid.NewGuid().ToString();
    string authorUrl = "http://" + Guid.NewGuid();
    string comment = Guid.NewGuid().ToString();

    //Open the first page
    selenium.Open("/");

    //Click on the link to the latest post
    selenium.Click("css=#entryTeaserList a:first-child");
    selenium.WaitForPageToLoad("30000");

    //Post a new comment
    selenium.Type("authorName", authorName);
    selenium.Type("authorEmail", "test@test.com");
    selenium.Type("authorUrl", authorUrl);
    selenium.Type("comment", comment);
    selenium.Click("subscribeToComments");
    selenium.Click("//input[@value='Add Comment']");
    selenium.WaitForPageToLoad("30000");

    Assert.True(selenium.IsElementPresent("link=" + authorName), "No link with the commenters name found");
    Assert.True(selenium.IsElementPresent(string.Format("css=a[href=\"{0}\"]", authorUrl)), "No link to the commenters url found");
    Assert.True(selenium.IsTextPresent(comment), "The comment text was not displayed");
}

Conclusion and source code

I have really enjoyed playing around with Selenium and I’m quite sure that I will start using it at work as well. I’m also quite happy with how  my tests work and how I handle starting Selenium Server and I hope that this post will prove useful to others as well.

You can download the source code for my project with the comment test in it here.

Any ideas about how the code can be improved or any other feedback would be very appreciated!

PS. For updates about new posts, sites I find useful and the occasional rant you can follow me on Twitter. You are also most welcome to subscribe to the RSS-feed.

Joel Abrahamsson

Joel Abrahamsson

I'm a passionate web developer and systems architect living in Stockholm, Sweden. I work as CTO for a large media site and enjoy developing with all technologies, especially .NET, Node.js, and ElasticSearch. Read more

Comments

comments powered by Disqus

More about Testing