<< Blog

How Image Comparison Works in Website Testing

Sunday September 7, 2014

In a recent software testing scenario, I almost completed the testing cycle for one of the complex projects on the floor. Much to my annoyance, the developer made an eleventh hour change in one of the features, and guess what? The changed feature required to pass the visual comparison test. I needed to confirm that the image in question was similar to the baseline image in all aspects like shape, size, texture, color and other contents that may show pixel variances in different browsers and devices.

I was sure that a long and tedious image comparison test was waiting for me because identifying image differences at pixel level requires you to adopt a holistic approach and attention to minute detailing. Initially, it was difficult to identify the difference, but running a few tests pinpointed the dissimilarities. Let me take you through the image comparison testing environment.

The Idea of Image Comparison is Simple

In my case, I used TestingWhiz, a futuristic test automation software, for web UI comparison. Its web UI test automation solution does exactly what I wanted. The tool is capable of segmenting the image and analyzing each and every segment for pixel differences.

Considering the capabilities of the tool, I developed a plan that would compare two images and generate a different image if there was any difference.

I captured the images from the specified path in the test command which got resized to 600×600. I performed image to image comparison. You can also perform image to URL and URL to URL image comparison. Here is how I got the result on testing the images at pixel level.

Pixel Difference

I compared the source image with the targeted image. I got the result in the form of a third image that showed pixel difference in green. I could identify characters of the images that were incorrectly programmed to prompt necessary changes. But prior developing a test plan and executing it, I considered key factors for getting the desired comparison test.

As a tester, if you are not aware of these influencers, the result may surprise you. So consider the following factors that affect the image comparison test.

Factors to Consider Before Performing Image Comparison Test

Pixel Tolerance

Before performing the test, I set the tolerance level as 10%. The test considered the tolerance value and gave the outcome accordingly. The difference was less than the defined tolerance level. As a result, the image was considered identical.

Image Capturing Conditions

Image capturing condition is critical to visual comparison. To start with, in different conditions (from different browsers). The images looked identical, but on running the test, the comparison failed as the pixel difference exceeded the specified tolerance level. So it is important to capture the image to be compared in the same condition as the baseline image has been captured.

Format of Image

TestingWhiz supports png, jpg, bmp and gif image file formats. While testing, I ensured that the format of the base image and the image to be compared were same. Had the format been different, the images were likely to be reported dissimilar.

User Interface (UI) Theme

Windows offer the luxury to set UI themes the way we want. So it was necessary for me to ensure that the images captured were from a similar UI theme. If the images were captured from different UI themes, the color schemes would have changed at the time of image capture, resulting in unequal images.

In addition to these factors, you must also consider image size, font smoothing, capture tool and screen resolutions for getting identical results.


There are different techniques, factors and parameters that I needed to consider before comparing images. TestingWhiz helped me identify differences in images at pixel level and complete my task successfully.

The last minute entry of the features were thoroughly tested and now they are similar and consistent across different browser versions and devices.

To experience Image Comparison in Website Testing, Download TestingWhiz now!


TestingWhiz is committed to providing innovative, automated software testing solution to global enterprises and software companies for their web, mobile and cloud applications.


Cygnet Infotech LLC,
125 Village Boulevard,
Suite 315, Princeton,
NJ 08540, USA