Selenium RWD help file and manual
Table of contents
- Abilities and limitations
- Installation instructions - getting going
- Architecture and component description
- Tweaking the settings
- How to set up and run a test
- Getting the most of the utility
- Troubleshooting hints
- Support information
This is the help file of the SeleniumRWD. The utility is meant to help with testing Responsive Web Design.
The tool works by looping through a number of web pages in different web browsers, and in different window resolutions. While doing this a number of tests may be performed.
Browser and OS support
The Selenium RWD utilizes Selenium for driving the web pages. That means the following web browsers are supported:
- Internet Explorer
There are possibilities to extend this framework to be used with Robotium or the Android SDK drivers. These are not (yet) incorporated in this utility.
The use of Selenium also mean you could use this framework on any Java enabled environment, but that you might expect to encounter file path issues since it has yet only been used on Windows.
Support for different resolutions
Although there are many ways for a web page to be responsive, one of the more common ones are to use the resolution width as a divider for content and its layout. This framework support any browser width it can natively do.
The tool has a few pre-requisites to work. It needs:
The procedure below states how to achieve this.
- Java runtime environment (JRE)
- Compatible web browser(s) installed (see above for compliance list)
- Selenium drivers for the web browsers that are intended for testing. An option is to use a configured Stand-alone-Selenium-server, but that will require some tweaking.
The following procedures must be performed in order to use this utility at all.
- Acquire SeleniumRWD
Download the SeleniumRWD from here and place the SeleniumRWD.jar file in an appropriate empty folder in your file system. Although an empty folder is not a nessecity for the tool to work, it's good to have an empty folder since there will be more files and folders generated in this folder.
- Verify local Java JRE
Make sure Java is installed on the machine you intend to use this utility on. The easiest way to make sure you do have Java Runtime Environment ready (installed and in within the reach of your path variable) is to open a prompt and run the command "java -version". If you don't have Java installed you can download it from https://java.com/.
- Get web browsers
Web browsers are not included in this package, so you will need to install the ones you would like testing to occur on separately. Bare in mind that test execution time increase with the number of web browsers tested.
- Get updated Selenium drivers to support your browsers
If you intend to run tests on other web browsers than Firefox you'll need to download the appropriate Selenium Drivers for those browsers. The drivers can be found on the http://www.seleniumhq.org website. Make note of in what folder the drivers are placed.
- Perform first run for default file creation
Run the SeleniumRWD by double clicking it. Not much will happen at first run, but the default properties files are created in the same file folder as the SeleniumRWD.jre resides.
- Get settings right
Edit the properties file in any text editor (e.g. vi, Notepad). Pay special attention to the google account details if you intend to have test result emails sent, and the file locations (especially the resultRepoFilePath. If you intend to use any other browser than Firefox for testing you also need to tend to the pathToSeleniumDriverExecutables property).
- Get email recipient list altered
Edit the reportEmailRecipientList.txt which is the file with the list of email recipients to receive test results email.
- Prepare the Test Setup
Get a list of urls to test. Get a list of resolutions to test. Get a list of browsers to test on. Place them all in the Test Setup.
The SeleniumRWD parts
This utility consist of a few parts, as described by the following model.
The Jar file is a file archive containing the Java files and needed libraries.
Wherever this file is placed also is the application working directory where the properties file, email recipient list, report style settings are also placed.
For the inner workings of the code the code itself, and its Javadocs is the best available description.
The help file (this file)
This file is created at first run, but since it also i s mean of getting the product installed, it could be distributed on its own as well.
The result and image repository
This is a file system folder where the screenshots are saved as well as results files.
One of the files holds a copy of the latest result file. This is used for comparison of results to enable the functionality of only sending email if test results has changed since last run.
The control files
To make this utility as versatile towards the stakeholders needs as possible some parameters has been broken out to a properties file. The settings in this file control the behaviour in areas like logging, what to verify, file locations and so forth.
The first time the jar file is being executed default files for properties, report styling and email recipient list is generated in the working directory of the Jar file. They are named as follows:
The properties file
The parameters inside the properties file can be altered at will. If you for any reason want to resort to default settings, just delete the file and a new one with safe settings will be generated for you.
The following table describes the meaning and consequences of the different settings in the file:
|compareResultWithLatest||There is a possibility to have different subject lines in the test result email depending on wheter the test result of the current test differs from the latest run before this run. This setting is used to enable this differentiation.||Boolean value ("true" or "false"). The test result must be exactly the same as last time. It's the html file that is being compared. The slightest date change in a file name for an image will make it detect a change and behave accordingly.
|checkForOverlappingElements||This settings enable the check for if the elements that are stated as test objects for a web page in the test setup are overlapping anywhere (not one completely within another, that is the nested objects check in this context. If overlapping elements are found, culprit elements are highlighted with a thin yellow frame and a screenshot is taken.||Boolean value ("true" or "false")
|checkForNestedElements||Similar to the setting for checking for overlapping elements on a page, but this one looks for objects where one object is fully inside another. Since that is a normal behaviour in a web page DOM, this only works with explicitly stated elements. If nesting is found, culprit elements are highlighted with a thin yellow frame and a screenshot is taken.||Boolean value
|performHeuristicChecking||The heuristic scan looks at all elements on a web page to see if any of them interfere with each other. For performance it recursively checks childitems of an element. Interfering objecs are highlighted and screenshots are taken of the area with interferance.||Boolean value
|checkIfWholeElementsIsInBrowserWindow||When scaling down a page width some objects might become larger than the screen width so they don't fit in the browser window. This setting enable a check for this type of behaviour and list the pages (and browsers/resolutions) where this occurs.||Boolean valus
|captureScreenshots||If this is set to true a screenshot is saved for every page (and browser/resolution) opened. The image is saved in the result repository and named by date, time, browser, url, and resolution.||Boolean value
|openReportWhenDone||This settings makes sure the report open in the default web browser after test completion. You want to turn this off when the tools is used in a build mechanism.||Boolean value
|debuglevel||This setting controls the detail level of informaiton in the Java std out according to the following scale:|
0=none but installation notes
|File management and paths
|resultRepoFilePath||This is the path used for saving images and test results. If this is stated in UNC format to a share the test report links to images and other files will work from any computer that is logged in with access to the share where these files reside.|
The repository also might contain a file holding the last result, for comparison purposes.
|String value. Make sure you use double "\" (as "\\") to bypass Java limitations in file path managements and escape characters.
|nameForLatestResultFile||This is the name of a file in the file repository. This file is used to store the results from last run to enable comparison with latest run, to enable reporting functionality only if changes since last run occurs.||String value. Avoid blanks and special characters for file system compatibility.
|pathToSeleniumDriverExecutables||The Selenium Drivers to manipulate specific browsers (other than Firefox) is looked for in this path. You need these to test on other browsers than Firefox.||String value. Remember to use double back-slashes (for Java escape characters compliance)
|urlToReportTopLogo||You can easily customize the top logotype of the html report by editing this property.||String value stating a url to an image of decent size
|googleUserName||For sending emails the framework utilizes the Google mail API. Hence an account is needed for sending mails. This settings states what username that account has.||String value
|googlePassword||Password corresponding to the Google account.||String value. Spacial characters might need to be escaped with a "\" sign.
|subjectLineWhenSameAsLatest||If you want to alter what email subject line is used when no change in test results has occured since last test run, this is where you do that.||String value
|subjectLineWhenNotSameAsLatest||Same as the previous one, but opposite: When changes has occured.||String value
|fromAddress||The automatic emails sent will look like they came from this account.||String value. No validation currently, so you might get strange errors if malformed.
|reportEmailRecipientList||File path to the email recipient list, which is a file with a new email address on each line. All of the email recipients will receive a result email when those are sent.||String value. Escape characters will be needed for rhe "\" sign.
|sendEmailEvenWhenUnchangedTestResult||This setting exist to avoid spamming of redundant informaiton by enabling the tool to only send emails when the test results differ from last run.||Boolean value
|sendResultEmail||Enabling or disabling emailing functionality all together.||Boolean value
Setting up and running a test
Running the test
Running the test is as easy as invoking the SeleniumRWD.jar file (=double clicking the file, or kicking it from a command prompt). If you want to start it from a command prompt or build server you might want to use a syntax like:
java -jar SeleniumRWD.jar
The memory utilization should not be extreme, but if you notice that garbage collections occur frequently or that it runs slow you might want to increase the available memory size for the java VM.
This tool utilizes a few other libraries that has been released under different licenses. The following licensing apply:
Getting the most of the utility
Don't forget to tweak the behaviour of the tool to your liking by altering the properties file, the style info and the test setup. Info on each of the settings in these components can be found in this file.
Test times can be veeeeeery looooooong. Some things to note in this department are:
- At the time of writing this Chrome is about three times faster than Firefox, that still is faster than
- The Chrome driver currently doesn't take full screenshots. This is an annoying limitation.
- The Internet Explorer driver hasn't been updated to enable test execution on later versions of Internet Explorer yet.
- Heuristic scan takes a few minutes per web page and per resolution. You might want to avoid this by using the method of manual xPath identification of relevant elements on the web page.
- It's quite easy to alter the look and feel of the report, but that require som knowledge of the CSS style attribute. There are plenty of instructions and samples on the Internet.
A lot of things can go wrong when trying to use the same code in many different contexts. Some that has been thought of, that still could pose a problem are:
- File system rights: The tool need read, write (including creating files and folders) and execute rights.
- Access problems from the report due to repository folder not being on a network enabled share (UNC naming convention, CIFS/SAMBA access).
- No friendly url:s on the site (single-frame site) that need other web navigation method to get to a page in a testable state.
- Character sets on different operating systems, that messes things up both executionwice and reportingwice
- Frequent similar mails sent making the email sending address banned, or making result reports end up in spam folder in email program.
- If no Internet access is at hand, fonts may be altered from original intention and a few images might get missing. This may lead to odd(er) look and feel of the reports and help file due to layout issues. Of course; if your web site under test is reached over the Internet, that will not be reachable either.
- There is a file item called backlog.txt that resides inside the Jar-file. If this file is opened (by Winzip, Winrar, 7-zip or similar tool) you can see a list of things still not taken care of. Be aware that it's a mix of bugs, wild ideas, things to consider implementing, and good intentions.
This framework is a hobby project of one single person. Support levels are based on this fact. If you want to get in touch with the author, send an email to firstname.lastname@example.org. Please also do this for suggestions, and to communicate your own improvements.