Selenium RWD help file and manual


Table of contents

  1. Description
  2. Abilities and limitations
  3. Licensing
  4. Installation instructions - getting going
  5. Architecture and component description
  6. Tweaking the settings
  7. How to set up and run a test
  8. Getting the most of the utility
  9. Troubleshooting hints
  10. Support information

Brief description

This is the help file of the SeleniumRWD. The utility is meant to help with testing Responsive Web Design.

The tool works by looping through a number of web pages in different web browsers, and in different window resolutions. While doing this a number of tests may be performed.

Browser and OS support

The Selenium RWD utilizes Selenium for driving the web pages. That means the following web browsers are supported:

  • Internet Explorer
  • Chrome
  • Firefox
  • Safari

There are possibilities to extend this framework to be used with Robotium or the Android SDK drivers. These are not (yet) incorporated in this utility.

The use of Selenium also mean you could use this framework on any Java enabled environment, but that you might expect to encounter file path issues since it has yet only been used on Windows.

Support for different resolutions

Although there are many ways for a web page to be responsive, one of the more common ones are to use the resolution width as a divider for content and its layout. This framework support any browser width it can natively do.


Installation instructions

The tool has a few pre-requisites to work. It needs:

  • Java runtime environment (JRE)
  • Compatible web browser(s) installed (see above for compliance list)
  • Selenium drivers for the web browsers that are intended for testing. An option is to use a configured Stand-alone-Selenium-server, but that will require some tweaking.
The procedure below states how to achieve this.

The following procedures must be performed in order to use this utility at all.

  1. Acquire SeleniumRWD
    Download the SeleniumRWD from here and place the SeleniumRWD.jar file in an appropriate empty folder in your file system. Although an empty folder is not a nessecity for the tool to work, it's good to have an empty folder since there will be more files and folders generated in this folder.
  2. Verify local Java JRE
    Make sure Java is installed on the machine you intend to use this utility on. The easiest way to make sure you do have Java Runtime Environment ready (installed and in within the reach of your path variable) is to open a prompt and run the command "java -version". If you don't have Java installed you can download it from https://java.com/.
  3. Get web browsers
    Web browsers are not included in this package, so you will need to install the ones you would like testing to occur on separately. Bare in mind that test execution time increase with the number of web browsers tested.
  4. Get updated Selenium drivers to support your browsers
    If you intend to run tests on other web browsers than Firefox you'll need to download the appropriate Selenium Drivers for those browsers. The drivers can be found on the http://www.seleniumhq.org website. Make note of in what folder the drivers are placed.
  5. Perform first run for default file creation
    Run the SeleniumRWD by double clicking it. Not much will happen at first run, but the default properties files are created in the same file folder as the SeleniumRWD.jre resides.
  6. Get settings right
    Edit the properties file in any text editor (e.g. vi, Notepad). Pay special attention to the google account details if you intend to have test result emails sent, and the file locations (especially the resultRepoFilePath. If you intend to use any other browser than Firefox for testing you also need to tend to the pathToSeleniumDriverExecutables property).
  7. Get email recipient list altered
    Edit the reportEmailRecipientList.txt which is the file with the list of email recipients to receive test results email.
  8. Prepare the Test Setup
    Get a list of urls to test. Get a list of resolutions to test. Get a list of browsers to test on. Place them all in the Test Setup.


The SeleniumRWD parts

This utility consist of a few parts, as described by the following model.



JAR file


=>
use
Properties file
=>
point out
Email recipient file
=>
use
Report styling
information file

Jar file

The Jar file is a file archive containing the Java files and needed libraries.
Wherever this file is placed also is the application working directory where the properties file, email recipient list, report style settings are also placed.

For the inner workings of the code the code itself, and its Javadocs is the best available description.

The help file (this file)

This file is created at first run, but since it also i s mean of getting the product installed, it could be distributed on its own as well.

The result and image repository

This is a file system folder where the screenshots are saved as well as results files.

One of the files holds a copy of the latest result file. This is used for comparison of results to enable the functionality of only sending email if test results has changed since last run.

The control files

To make this utility as versatile towards the stakeholders needs as possible some parameters has been broken out to a properties file. The settings in this file control the behaviour in areas like logging, what to verify, file locations and so forth.

The first time the jar file is being executed default files for properties, report styling and email recipient list is generated in the working directory of the Jar file. They are named as follows:

  • SeleniumRWD.properties
  • SeleniumRWD.reportCSSstyle
  • reportEmailRecipientList.txt

The properties file

The parameters inside the properties file can be altered at will. If you for any reason want to resort to default settings, just delete the file and a new one with safe settings will be generated for you.

The following table describes the meaning and consequences of the different settings in the file:

Setting nameDescriptionComment
Verification behaviour
compareResultWithLatestThere is a possibility to have different subject lines in the test result email depending on wheter the test result of the current test differs from the latest run before this run. This setting is used to enable this differentiation.Boolean value ("true" or "false"). The test result must be exactly the same as last time. It's the html file that is being compared. The slightest date change in a file name for an image will make it detect a change and behave accordingly.
checkForOverlappingElementsThis settings enable the check for if the elements that are stated as test objects for a web page in the test setup are overlapping anywhere (not one completely within another, that is the nested objects check in this context. If overlapping elements are found, culprit elements are highlighted with a thin yellow frame and a screenshot is taken.Boolean value ("true" or "false")
checkForNestedElementsSimilar to the setting for checking for overlapping elements on a page, but this one looks for objects where one object is fully inside another. Since that is a normal behaviour in a web page DOM, this only works with explicitly stated elements. If nesting is found, culprit elements are highlighted with a thin yellow frame and a screenshot is taken.Boolean value
performHeuristicCheckingThe heuristic scan looks at all elements on a web page to see if any of them interfere with each other. For performance it recursively checks childitems of an element. Interfering objecs are highlighted and screenshots are taken of the area with interferance.Boolean value
checkIfWholeElementsIsInBrowserWindowWhen scaling down a page width some objects might become larger than the screen width so they don't fit in the browser window. This setting enable a check for this type of behaviour and list the pages (and browsers/resolutions) where this occurs.Boolean valus
Logging behaviour
captureScreenshotsIf this is set to true a screenshot is saved for every page (and browser/resolution) opened. The image is saved in the result repository and named by date, time, browser, url, and resolution.Boolean value
openReportWhenDoneThis settings makes sure the report open in the default web browser after test completion. You want to turn this off when the tools is used in a build mechanism.Boolean value
debuglevelThis setting controls the detail level of informaiton in the Java std out according to the following scale:
0=none but installation notes
1=essential
2=warnings
3=debug
4=absurd
Integer value
File management and paths
resultRepoFilePathThis is the path used for saving images and test results. If this is stated in UNC format to a share the test report links to images and other files will work from any computer that is logged in with access to the share where these files reside.
The repository also might contain a file holding the last result, for comparison purposes.
String value. Make sure you use double "\" (as "\\") to bypass Java limitations in file path managements and escape characters.
nameForLatestResultFileThis is the name of a file in the file repository. This file is used to store the results from last run to enable comparison with latest run, to enable reporting functionality only if changes since last run occurs.String value. Avoid blanks and special characters for file system compatibility.
pathToSeleniumDriverExecutablesThe Selenium Drivers to manipulate specific browsers (other than Firefox) is looked for in this path. You need these to test on other browsers than Firefox.String value. Remember to use double back-slashes (for Java escape characters compliance)
urlToReportTopLogoYou can easily customize the top logotype of the html report by editing this property.String value stating a url to an image of decent size
Email settings
googleUserNameFor sending emails the framework utilizes the Google mail API. Hence an account is needed for sending mails. This settings states what username that account has.String value
googlePasswordPassword corresponding to the Google account.String value. Spacial characters might need to be escaped with a "\" sign.
subjectLineWhenSameAsLatestIf you want to alter what email subject line is used when no change in test results has occured since last test run, this is where you do that.String value
subjectLineWhenNotSameAsLatestSame as the previous one, but opposite: When changes has occured.String value
fromAddressThe automatic emails sent will look like they came from this account.String value. No validation currently, so you might get strange errors if malformed.
reportEmailRecipientListFile path to the email recipient list, which is a file with a new email address on each line. All of the email recipients will receive a result email when those are sent.String value. Escape characters will be needed for rhe "\" sign.
sendEmailEvenWhenUnchangedTestResultThis setting exist to avoid spamming of redundant informaiton by enabling the tool to only send emails when the test results differ from last run.Boolean value
sendResultEmailEnabling or disabling emailing functionality all together.Boolean value


Setting up and running a test

Running the test

Running the test is as easy as invoking the SeleniumRWD.jar file (=double clicking the file, or kicking it from a command prompt). If you want to start it from a command prompt or build server you might want to use a syntax like:

java -jar SeleniumRWD.jar

The memory utilization should not be extreme, but if you notice that garbage collections occur frequently or that it runs slow you might want to increase the available memory size for the java VM.


Licensing

This tool utilizes a few other libraries that has been released under different licenses. The following licensing apply:

Component / libraryLicense that apply
SeleniumRWD (this assembly)Don't sell this utility for money while parts of the original code still exist. You may however alter code, and sell services arount the tool.
Selenium (org.openqa.selenium)Apache licence 2.0
JavaGNU GPL (General Public License)
Apache commons collection (org.apache.commons)Apache licence 2.0
JavaxGNU GPL (General Public License)
Google email ("gmail") APIGoogle user agreement apply


Getting the most of the utility

Don't forget to tweak the behaviour of the tool to your liking by altering the properties file, the style info and the test setup. Info on each of the settings in these components can be found in this file.

Test times can be veeeeeery looooooong. Some things to note in this department are:

  • At the time of writing this Chrome is about three times faster than Firefox, that still is faster than Internet Explorer.
  • The Chrome driver currently doesn't take full screenshots. This is an annoying limitation.
  • The Internet Explorer driver hasn't been updated to enable test execution on later versions of Internet Explorer yet.
  • Heuristic scan takes a few minutes per web page and per resolution. You might want to avoid this by using the method of manual xPath identification of relevant elements on the web page.
  • It's quite easy to alter the look and feel of the report, but that require som knowledge of the CSS style attribute. There are plenty of instructions and samples on the Internet.

Troubleshooting

A lot of things can go wrong when trying to use the same code in many different contexts. Some that has been thought of, that still could pose a problem are:

  • File system rights: The tool need read, write (including creating files and folders) and execute rights.
  • Access problems from the report due to repository folder not being on a network enabled share (UNC naming convention, CIFS/SAMBA access).
  • No friendly url:s on the site (single-frame site) that need other web navigation method to get to a page in a testable state.
  • Character sets on different operating systems, that messes things up both executionwice and reportingwice
  • Frequent similar mails sent making the email sending address banned, or making result reports end up in spam folder in email program.
  • If no Internet access is at hand, fonts may be altered from original intention and a few images might get missing. This may lead to odd(er) look and feel of the reports and help file due to layout issues. Of course; if your web site under test is reached over the Internet, that will not be reachable either.
  • There is a file item called backlog.txt that resides inside the Jar-file. If this file is opened (by Winzip, Winrar, 7-zip or similar tool) you can see a list of things still not taken care of. Be aware that it's a mix of bugs, wild ideas, things to consider implementing, and good intentions.


Support

This framework is a hobby project of one single person. Support levels are based on this fact. If you want to get in touch with the author, send an email to kejsardamberg@hotmail.com. Please also do this for suggestions, and to communicate your own improvements.