Regression Testing for Styling and Layout of Web Applications

Regression testing for styling and layout of web applications

Actually, one of the hidden gems of Selenium is that you can use it to take screen captures of the browser. Then using a technique as described in Find differences between images using C#, you can highlight the differences between previous snapshots.

This example shows how to grab a screenshot of the homepage and then create a difference image. With a little extra tinkering, you can use the same technique to count the pixels that are different.

[Test]
public void CompareHomePageToPrevious()
{
    string fileName = Path.Combine(Environment.CurrentDirectory, "homepage.bmp");
    var selenium = new DefaultSelenium("localhost", 4444, "*chrome", "http://mywebsite");
    selenium.Start();
    selenium.Open("/");
    selenium.CaptureEntirePageScreenshot(fileName, "");

    // Load current and previous captures
    var current  = new Bitmap(filename);
    var previous = new Bitmap(_previousHomepageImage);

    // Find all pixels that are the same and make them pink
    Bitmap diff = ImageTool.GetDifferenceImage(current,previous,Color.Pink);
    diff.MakeTransparent(Color.Pink);

    // Save the image for viewing
    // or count the number of different
}

Selenium is a really interesting choice because it's cross-platform and cross-browser, meaning that you can capture screens of different browsers. You can use this technique to compare snapshots between builds or to compare between browsers.

Layout Testing - Web Testing

This is a vague and broad question but I'll attempt to answer it:

Yes, I have used this. My thoughts are better expressed by Michael Tamm and Co. I have bug out a couple of videos that were helpful when I implemented Fighting-Layout-Bugs around five weeks ago:

http://www.infoq.com/presentations/Fighting-Layout-Bugs

http://www.testingtv.com/2010/01/18/fighting-layout-bugs/

Who writes the automated UI tests? Developers or Testers?

Ideally it should really be QA who end up writing the tests. The problem with using a programmatic solution is the learning curve involved in getting the QA people up to speed with using the tool. Developers can certainly help with this learning curve and help the process by mentoring, but it still takes time and is a drag on development.

The alternative is to use a simple GUI tool which backs a language (and data scripts) and enables QA to build scripts visually, delving into the finer details of the language only when really necessary - development can also get involved here also.

The most successful attempts I've seen have definitely been with the latter, but setting this up is the hard part. Selenium has worked well for simple web applications and simple threads through the application. JMeter also (for scripted web conversations for web services) has worked well... Another option which is that of in house built test harness - a simple tool over the top of a scripting language (Groovy, Python, Ruby) that allows QA to put test data into the application either via a GUI or via data files. The data files can be simple properties files, or in more complex cases structured (something like YAML or even Excel) data files. That way they can build the basic smoke tests to start, and later expand that into various scenario driven tests.

Finally... I think rich client apps are way more difficult to test in this way, but it depends on the nature of the language and the tools available to you...

Is testing web app with images comparision a good approach?

There are solutions for this kind of testing, for example, there is Applitools.

It is based on taking a baseline screenshot, and then the subsequent screenshots take the diff from the original image. There are 4 different comparison levels:

1. Exact (MatchLevel.EXACT) - pixel-to-pixel comparison
2. Strict (MatchLevel.STRICT) - Strict compares everything including content (text), fonts, layout, colors and position of each of the elements but knows to ignore rendering changes that are not visible to the human
3. Content (MatchLevel.CONTENT) - Content works in a similar way to Strict except for the fact that it ignores colors
4. Layout (MatchLevel.LAYOUT) - compares the layouts (i.e. structure) of the baseline and actual images. It ignores the content, color and other style changes between the pages.

A big advantage, in my opinion, is that this kind of testing can catch unexpected bugs (visual or otherwise) and in one go (you don't need to write multiple assertions, you just compare screenshots). You can write scripts with less code, as well.

Possible downsides are: you cannot handle dynamic content, sometimes it is impossible to take a screenshot and, since there are screenshots, test execution (working with img files) can be longer.

NOTE: I'm not involved with Applitools, but they have a site with many tutorial courses.

---

Related Topics