Automate Acceptance Tests With Selenium

Automate acceptance tests with Selenium

1 of 10

http://www-128.ibm.com/developerworks/library/wa-selenium-ajax/

Automate acceptance tests with Selenium How to use the Selenium test tool for functional testing of a Ruby on Rails and Ajax application

Level: Intermediate Christian Hellsten ([email protected]), IT Specialist, IBM 20 Dec 2005 Acceptance, or functional, testing is designed to put manual tasks through their paces, but testing these tasks by hand can be time consuming and prone to human error. In this article, the author shows architects, developers, and testers how to use the Selenium testing tools to automate acceptance tests; automating the tests saves times and helps eliminate tester mistakes. You also are provided with an example of how to apply Selenium in a real-world project using Ruby on Rails and Ajax. Acceptance testing of Web applications usually involves manual tasks like opening a browser and performing actions described in a test case. Manually performed tasks are prone to operator error and are time consuming. It is, therefore, a good practice to automate these tasks whenever possible to remove the human factor. This is where test tools like Selenium come into the picture. Selenium helps you automate your acceptance tests and allows you to build better tested and, hence, more reliable and maintainable software. Acceptance testing, also known as black-box and functional testing, is a way of testing and verifying that an application works according to functional, non-functional, and other important stakeholder requirements. Acceptance tests complement unit and integration tests, which are usually written using xUnit frameworks. Acceptance tests can also be written using a programming language, but Selenium and similar tools like Fitnesse also support tests written using a tool-specific document format. Acceptance tests differ from unit and integration tests in the following ways: The application is tested as a whole, end-to-end entity, not just one class or a set of classes as it is with unit and integration tests. Acceptance tests are performed against the user interface using, for example, a browser against a Web application interface. The person writing the test cases does not necessarily know about the internals of the application, hence the designation of black-box testing. Non-technical users can write acceptance tests.

A bit of background Before going further into Selenium, I'd like to offer some background on the following three topics, since they are relevant to this article even though they're not the central theme: Continuous integration Ajax Ruby/Ruby on Rails

Continuous integration Continuous integration aims at automating the build and test processes so that they run automatically once or many times a day instead of, say, manually once a month. The biggest benefit of using continuous integration is that code changes are integrated automatically on a regular basis. And if the system is broken and does not build successfully, continuous integration tools like Apache Continuum and Luntbuild can automatically notify the team by sending an e-mail (see Resources).

Ajax Ajax stands for Asynchronous JavaScript and XML; it is a newly coined term for relatively old technology. The main idea behind Ajax is that the Web application responds more quickly to user actions because only parts of the page need to be updated instead of the whole page.

2007-8-30 3:59

Automate acceptance tests with Selenium

2 of 10

http://www-128.ibm.com/developerworks/library/wa-selenium-ajax/

Ajax introduces more complexity to Web applications that is also reflected in testing. This is because Ajax, as the name implies, uses JavaScript and asynchronous HTTP requests to update the page contents. Each browser contains small differences in implementation compared to others. Selenium makes a perfect tool for testing and detecting these differences since it runs inside the most popular browsers.

Ruby/Ruby on Rails Ruby is an open-source interpreted scripting language for quick and easy object-oriented programming; it features a wide range of libraries and is simple, straight-forward, extensible, and portable. The language was created by Yukihiro "Matz" Matsumoto and was designed to make programmers happy by allowing them to concentrate more on the task at hand than syntax. Rails is a full-stack, open source Ruby Web framework created by David Heinemeier Hansson. Rails is designed to make writing real world applications less code-intensive and easier than J2EE and XML, for example. All the layers are built to work seamlessly together so you can use a single language to write everything from templates to control flow to business logic. Rails uses YAML instead of XML configuration files and reflection and runtime extensions in favor of annotations. And there is no compilation phase -- you make a change and watch it work.

What is Selenium? Selenium is an acceptance testing tool written by ThoughtWorks specifically for Web applications. The Selenium home page says the following is the biggest benefit of using Selenium when compared with other test tools: Selenium tests run directly in a browser, just as real users do. And they run in Internet Explorer, Mozilla, and Firefox on Windows, Linux, and Macintosh. No other test tool covers such a wide array of platforms. Note that IBM's Rational Functional Tester tools perform these tests as well, but at a much more complex and wide-reaching level, and come with IBM support. Selenium, on the other hand, prioritizes one task -- a Web application -- and can perform with more agility. Additional benefits of using Selenium and running tests inside a browser are many. Here are two main ones: You test the application from the end user's perspective by writing Selenium test scripts that mimic user actions. You can identify browser incompatibilities more easily by running the tests in different browsers. The core of Selenium, also referred to as the browser bot, is written in JavaScript. This allows test scripts to run inside supported browsers. The browser bot is responsible for executing commands received from test scripts which are written in either HTML using a table layout or a supported programming language. Selenium works with the following: Internet Explorer Mozilla Windows XP

6.0

Firefox

Safari

1.6+, 1.7+ 0.8+, 0.9+, 1.0

Red Hat Linux

1.6+, 1.7+ 0.8+, 0.9+, 1.0+

Mac OS X 10.3 not supported

1.6+, 1.7+ 0.8+, 0.9+, 1.0+ 1.3+

Selenium commands With Selenium commands, the script writer describes the actions that the browser bot is to perform in the

Selenium and the Rational Functional Tester Companies or organizations with bigger projects typically need a standard product with many features and reliable support. All projects within the organisation then

2007-8-30 3:59

Automate acceptance tests with Selenium

3 of 10

browser. You can split these commands into two groups -- actions and assertions:

http://www-128.ibm.com/developerworks/library/wa-selenium-ajax/

use this product. One solution is the Rational Functional Tester, rather than Selenium, which is purposely not standardized and is limited in its design to address much smaller tasks with ease.

Rational Functional Tester is used with such applications as .NET and Java GUIs, Actions simulate the Siebel, Terminal-based applications, and, of course, Web applications. The Tester user's interaction contains thousands of features not available in Selenium, including Test Manager with the Web and ScriptAssure. It's integrated with version control and has tools and components application. Clicking that extend its use to many areas, and is used by all projects within the same a button and filling organization. out a form are both examples of Get more specs on this advanced test automation at common user http://www3.software.ibm.com/ibmdl/pub/software/rational/web/datasheets/rft.pdf. actions that you can automate with Selenium commands. Assertions verify the expected outcome of a command. Common assertions include verifying that the page content or that the current location is correct. You can find a complete list of available commands on the Selenium Web site (see Resources).

Selenium modes You can use Selenium in two modes: test runner and driven. The two modes differ in complexity and the way they are written. Driven test scripts tend to be more complex to write since they are written in a programming language. The difference in complexity is minimal if you use a high-level dynamic programming language like Python or Ruby. The biggest difference between the two modes is that the tests are partly run from outside the browser if using driven scripts in which test runner scripts are run completely inside the browser. Both test runner and driven test cases can be integrated with continuous integration tools.

Test runner mode Selenium test runner scripts, also referred to as test cases, are written in HTML using a simple table layout shown in Listing 1. Listing 1. Selenium test case structure First command Target Value Second command Target Value

Test runner scripts are usually deployed on the same server as the application under test (AUT). This is because the browser bot uses JavaScripts to emulate user actions. These scripts run in a restricted sandbox environment. If you

2007-8-30 3:59

Automate acceptance tests with Selenium

4 of 10

http://www-128.ibm.com/developerworks/library/wa-selenium-ajax/

need to circumvent these restrictions, use a proxy. Test runner scripts use the same concepts of test suite and test case as xUnit frameworks. Both test cases and commands execute sequentially in the order they appear in test suites and test cases. In Listing 1: The first column contains the command or assertion. The second column contains the target for the command or assertion. Specify the target with one of the many supported element locators. Usually the ID or name of the element is used, but also XPath and DOM locators are supported. The third column contains the value which specifies parameters for commands or assertions. This can be the desired value for a text field when using the type command. Test runner scripts are easy to read and write even for non-technical people. The example in Listing 1 will, when opened in a browser, look like this table: First command

Target Value

Second command Target Value Next I'll describe how to write a simple but complete test case using both commands and assertions.

Example test case The test script in Listing 2 performs the following actions upon execution: 1. It opens the change address page by going to /change_address_form.html. 2. It types Betelgeuse state prison in the text box that has the ID address_field. 3. It clicks on the input field named Submit. Note this uses XPath to find the Submit button, which sends the form data to the server. 4. It verifies that the page contains the text Address change successful. Listing 2. Example of using commands and assertions in a test case open /change_address_form.html type address_field Betelgeuse state prison clickAndWait //input[@name='Submit'] verifyTextPresent Address change successful

Test suites Acquiring full test coverage of an application usually requires more than one test case. This is why Selenium uses the concept of test suites. Test suites are used to group a set of test cases with similar functionality together so that they can be run in sequential order. Test suites are written in the same way as the test cases, using simple HTML tables. The default test suite executed by Selenium is named TestSuite.html. Listing 3 shows a test suite that tests an application in the same way a typical user would. Note that test suites use a single-column table and that each row points to a file containing a test case.

2007-8-30 3:59

Automate acceptance tests with Selenium

5 of 10

http://www-128.ibm.com/developerworks/library/wa-selenium-ajax/

Listing 3. Test suite example Test suite for the whole application Access main page Login to application Change address Logout from application

Next I'll look at driven test scripts.

Driven mode Driven Selenium scripts are written in one of the supported programming languages -- currently Java, Ruby, and Python drivers are available. The scripts run in a separate process outside of the browser. The driver's task is to execute the test script and drive the browser by communicating with the browser bot which runs inside the browser. The communication between the driver and the browser bot uses a simple Selenium-specific wire language called Selenese. Driven scripts are more powerful and flexible than test runner scripts and you can integrate them with xUnit frameworks. The downside to driven scripts (when compared with test runner scripts) is that they are more complex to deploy and write. This is because the driver has to perform the following tasks: Start the server. Deploy application under test (AUT). Deploy test scripts. Start the browser. Send commands to the browser bot. Verify the result of commands executed by the browser bot. Driven scripts are also more dependant on the application runtime environment. For example, the Java driver uses an embedded Jetty or Tomcat instance to deploy the application under test. Currently, an effort is in progress to integrate Selenium into Ruby on Rails, but at the time of writing it was not yet released. Listing 4 is an excerpt from a driven test script that uses the Ruby driver. Note that I left out the steps for instantiating the server and browser and that the test script code is almost as simple as a test runner script. Listing 4. Example of using the Ruby driver . . puts selenium.open('/logout.html') puts selenium.verify_location('/index.html') . .

Real-life requirements

2007-8-30 3:59

Automate acceptance tests with Selenium

6 of 10

http://www-128.ibm.com/developerworks/library/wa-selenium-ajax/

In the next two major sections (Real-life requirements and Real-world use cases), I describe how to use Selenium in a real-world scenario and write Selenium test cases for a simple stock quotes viewer application using Ruby on Rails and a bit of Ajax technology. Although the application is written using Ruby on Rails, you can apply the example to any Web application since the test scripts are written in HTML for the test runner mode. The sample application will probably work with older or later versions, but it was tested with Ruby 1.8.3 and Ruby on Rails 0.14.2. Ruby is usually included in the distribution if you have Linux. Run ruby -v from a command prompt to check what version you have. You can find a Ruby distribution for most platforms at http://www.ruby-lang.org/. Your next step is to install Ruby on Rails through the RubyGems packaging system. To do this, simply execute gem install rails --include-dependencies. On some platforms you have to perform some extra steps, so check the Ruby on Rails Web site for more details. The current Selenium version available, at the time of writing, is 0.6. I've already integrated it into the sample application (see Download) by downloading the Selenium Core package from http://selenium.thoughtworks.com/ and copying the folder named selenium to the folder that serves static content. In Ruby on Rails applications, the name of this folder public. In a J2EE Web application, you put it in the root of the Web application or WAR archive file. Your last step is to download the example application; access the package from Download. Extract the application and open a command prompt. Then change directory to where you extracted the application. To start the application, run ruby script/server. You should see that Rails started without errors as shown in Figure 1. Figure 1. Running Ruby on Rails from command prompt

Real-world use cases In this section I'll list the use cases for the sample application. With these simplified use cases, you can write acceptance tests that simulate the steps performed by the user and verify that the result of these is as expected. The stock quotes application implements the following four use cases: Login View stocks View stock details Logout The code implementing these use cases is already written; you can find it in the app directory and the test cases in the public/selenium/tests folder.

Login use case Most people know how a login page works -- you enter your username and password and submit the data to the server. If the credentials are valid you are logged in and taken to the secured resource. In the sample application, this test case involves the following user actions and assertions which you have to translate into a Selenium test case: 1. Click login link. 2. Verify that the system requires that the user logs in. 3. Enter username.

2007-8-30 3:59

Automate acceptance tests with Selenium

7 of 10

http://www-128.ibm.com/developerworks/library/wa-selenium-ajax/

4. Enter password. 5. Press login button. 6. Verify that login was successful. Figure 2 shows the Selenium test case for these requirements. Note that I took the screenshot after running the test. The green rows indicate assertions that were successfully verified. Figure 2. Login and view stocks test case

View stocks use case The view stocks page displays a list of companies. The test case for this is so simple that it was included at the end of the previous test case. It verifies that the current location is /main/list_stocks and that the page contains the text Click on a company name to view details.

View stock details use case The view stock details use case is triggered on the view stocks page. A user's clicks on a company name triggers an Ajax request to the server. The response from the server contains the company details and will be inserted into the current page without a complete page reload. The test case for this use case performs the following user actions: 1. 2. 3. 4.

Click on company name Acme Oil. Verify that the company details are displayed on the page. Click on company name Acme Automotive. Verify that the company details are displayed on the page.

The request happens asynchronously because Ajax is used. This poses a different challenge than with normal Web applications where everything typically happens synchronously. You can test Ajax functionality in the same way as other functionality. The only exception is that you have to tell Selenium to pause and wait for the Ajax command to finish. For this, you will use the pause command to wait for the Ajax command to finish. You can also use the waitForValue and waitForCondition commands instead of pause as Joseph Moore points out in his recent blog post (see Resources). Figure 3 shows the requirements translated into a Selenium use case. Figure 3. View stock details test case

Note the pause commands: These are required to allow the asynchronous request to finish and update the page content. Without the 500 millisecond pause, the test fails (shown in Figure 4). Figure 4. Failed view stock details test case

2007-8-30 3:59

Automate acceptance tests with Selenium

8 of 10

http://www-128.ibm.com/developerworks/library/wa-selenium-ajax/

The pause command also tests the non-functional requirements for the Ajax functionality. 500 milliseconds is a good value for the pause command since Ajax requests should execute and finish quickly. You can try to remove the pause commands and see what happens. If the tests fail on your machine, try to increase the value to 1000 milliseconds.

Logout use case The logout use case is easy to implement and is summarized by the following two steps: 1. Click logout link. 2. Verify that logout was successful. Figure 5 shows the final test case. Figure 5. Logout test case

All tests are added to the default test suite which is shown to the left of Figure 6. Figure 6. Test suite for sample application

Executing the test suite The last thing you do is execute the whole test suite in both Mozilla Firefox and Microsoft Internet Explorer. To do this, open http://localhost:3000/selenium/TestRunner.html in the browser and click on the All button shown in Figure 6. Tests cases and assertions that fail are colored red, but in your case they should all complete successfully in both browsers (also shown in Figure 6). Note that I used Mozilla Firefox 1.0.7 and Internet Explorer 6.0. You can also walk or step through the test suite which means Selenium will execute the test suite slowly enough for you to see each step of the test suite as it executes in the browser.

Conclusion Selenium is a useful and important addition to the toolbox of software engineers, designers, and testers. Together

2007-8-30 3:59

Automate acceptance tests with Selenium

9 of 10

http://www-128.ibm.com/developerworks/library/wa-selenium-ajax/

with a continuous integration tool, it allows teams to automate acceptance tests and build better software as they find bugs easier, earlier, and more often. An added benefit of Selenium is that it saves time and allows the team to concentrate on more valuable activities by freeing developers and testers from manual tasks which can and should be automated.

Download Description

Name

Sample application

wa-selenium-ajax-example.zip

Size

Download method 286KB HTTP

Information about download methods

Resources Learn Review a complete list of available Selenium commands. See Wikipedia's definition of acceptance tests. Discover how a Selenium integration into Ruby on Rails looks in this brainstorming session. See Wikipedia's definition of Ajax. Try Selenium's waitForValue and waitForCondition commands -- Joseph Moore's alternative to the pause command. Dig into these Ajax resources on developerworks: Build apps with Asynchronous JavaScript with XML, or Ajax (November 2005) demonstrates how to construct real-time-validation-enabled Web applications with Ajax. Ajax for Java developers: Build dynamic Java applications (September 2005) is a guide to application development using Ajax. Ajax and scripting Web services with E4X (April 2005) is a primer for Ajax. Ajax for Java developers: Ajax with Direct Web Remoting (November 2005) demonstrates how to automate the heavy-lifting of Ajax. Review the archived Diagnosing Java code series (developerWorks, 2001) includes automating testing tasks as a matter of best programming and debugging practices. Explore all Ruby and Ruby on Rails articles at developerWorks, including these gems: The tutorial Sockets programming in Ruby teaches Ruby basics plus the most important classes to develop sockets-based networking applications using the Ruby language (October 2005). Fast-track your Web apps with Ruby on Rails (June 2005) describes a framework for rapid development using an MVC pattern. Ruby on Rails and J2EE: Is there room for both? (July 2005) compares Ruby on Rails and J2EE. Check out all the Python articles at developerWorks. Get the white paper on Advancing Toward Test Automation through Effective Manual Testing as well as the specifications for the Rational Functional Tester.

Get products and technologies Get details on the Selenium tool, built by ThoughtWorks in accordance to Agile development methods for

2007-8-30 3:59

Automate acceptance tests with Selenium

10 of 10

http://www-128.ibm.com/developerworks/library/wa-selenium-ajax/

enterprise software development. Visit the Ruby on Rails homepage and the Ruby language homepage for everything you need to get up to speed with these technologies. Check out Continuum, a continuous integration server for building Java-based projects. Explore Luntbuild, a powerful build automation and management tool designed to automate builds and act as a build management system. Try Fitnesse for an acceptance testing framework. Get the Mozilla Firefox browser. Get Selenium Recorder, a Firefox extension that allows you to record Selenium test scripts. Try out the Rational Functional Tester with this free 30-day trial download. Discuss Participate in the discussion forum.

About the author Christian Hellsten is currently an IT Specialist with IBM Business Consulting Services. He has more than six years of experience in the IT industry working on large-scale e-business projects. His area of expertise is in Java, J2EE and EAI architecture, design and development. He also enjoys working with more dynamic, productive, and fun languages and frameworks like Python, Ruby, and Ruby on Rails.

2007-8-30 3:59