Monday 19 November 2012

21st Century Integration Testing

Everything starts here




So what is it?

A combination of OpenSource projects, bundled with a user-friendly interface, to bring your integration tests in to the 21st Century.

As shown on the screenshot above, the framework will give you access to the following:
  • Run Chart - historic availability of MyApp1
  • Blame Chart - test(s) that have failed
  • Frontends, Service and Backends - gives us a simple view of where the fault lies, and more importantly, which functionality has been impacted

What are the key benefits?

  • Increased availability
  • Improved productivity
  • Reduced maintenance costs

Who will benefit?



How does it work?

It's quite simple, we take our existing (or new) integration tests, add them to this framework and navigate to the monitor page. Sound simple? That's because it is!

Time for a test drive?

Would you buy a car without test driving it? Even though nothing here is for sale, time still costs money, and for anyone to invest time and effort, it has to be worth it.
In order to show the power of "21st Century Integration Testing", a bundle with both the server and test cases is provided below. It enables you to test run it before deciding on setting up your own solution.

Please make sure you have the following products installed:

Java - http://www.java.com/getjava/

Firefox - http://www.mozilla.org/en-US/firefox/new/

The provided package is a fully functional three layered Bottom Up Testing, including SoapUI and Selenium tests. All code used to make this package, can be found on GitHub.
  1. Download https://github.com/downloads/dunse/try-it-out/21st%20Century%20Integration%20Testing.zip
  2. Unpack and run 1_startServer.bat (or 1_startServer.sh if you run *NIX)
    This should automatically start the browser, if not you can access the monitor page at: http://localhost:18080/ShowMyTests/
  3. Run a test: 2_runTestSuite.bat (2_runTestSuite.sh)
    Some tests are random, so run again to get different results...
    Cuanto (the backend) can be accessed by http://localhost:18080/cuanto/

How do I get started?

Step-by-step instructions on how to setup your own test server and add your own tests can be found here: http://dsysadm.blogspot.com.au/2012/11/getting-started-with-21st-century.html

You can now have your own system up and running in no time!


Not convinced?

Please leave a comment about your thoughts. Constructive citisism is always appreciated!

Getting started with 21st Century Integration Testing


Setup the server

Download and install the software listed below.
Plenty of useful tutorials exist on how to install Java, Tomcat and MySQL on every possible operating system, so please follow one of those. For Cuanto, please follow the instructions in the next two sections.

Installing Cuanto and MySQL JDBC Driver

(Source: INSTALL.html):

Creating the database

Create a database on your SQL server named "cuanto". Typically this is done with a command like "create database cuanto;" when logged onto your SQL server, although you may want to specify additional details depending on any additional database restrictions or options you want to enable.

Customizing cuanto-db.groovy

Edit cuanto-db.groovy with a text editor. You'll see a section like this:
production {
 dataSource {
  username = "my_sql_user"
  password = "my_sql_password"
  driverClassName = "com.mysql.jdbc.Driver"
  url = "jdbc:mysql://my_sql_server:3306/cuanto?autoreconnect=true"
 }
}
Edit the username, password, driverClassName and url to correspond to the correct values for your database credentials, JDBC driver and the JDBC URL for your SQL server. Make sure you edit the "production" section.

Deploying the application

Unzip the WAR into your application server's webapps directory into a subdirectory named "cuanto". Copy the cuanto-db.groovy you customized into the cuanto/WEB-INF/classes directory.


Installing ShowMyTests

Simply copy the downloaded .war file in to Tomcat's webapps directory, start Tomcat and navigate to http://<Tomcat_Server_IP>:<Tomcat_Server_Port>/ShowMyTests/.
For example, http://localhost:8080/ShowMyTests/
This should bring up a black screen with the friendly words "ERROR: No TestRuns found":


Server installation is now ready for use! Move on to checkout the framework and start running some tests....


Get the framework

Make sure your IDE have Maven Integration and supports Git.
Then clone the following Git repositories (Remember to tick the "Import all existing projects after clone finishes"):
git://github.com/dunse/ShowMyTests.git
git://github.com/dunse/my-integration-tests.git

While you wait, read through the next section if you are not familiar with Integration Testing fundamentals.


Integration Testing

As the name indicates, it is used to test integration between components. It does not matter if it is one service to another, or end-user to web-applications, they are all integrated and should be tested.

For the concept of 21st Century Integration Testing, we adopted the Bottom-Up Testing as it is the most suitable.

Bottom Up Testing is an approach to integrated testing where the lowest level components are tested first, then used to facilitate the testing of higher level components. The process is repeated until the component at the top of the hierarchy is tested.
All the bottom or low-level modules, procedures or functions are integrated and then tested. After the integration testing of lower level integrated modules, the next level of modules will be formed and can be used for integration testing. This approach is helpful only when all or most of the modules of the same development level are ready. This method also helps to determine the levels of software developed and makes it easier to report testing progress in the form of a percentage. (Source: Wikipedia)


It is important when creating tests for this purpose to approach them via a Top Down fashion, which means:
  1. Identify a User Action on the Frontend Layer, write a test for it and verify it.
  2. Move to the next Layer (Service Layer) and write tests for all calls produced by the User Action.
  3. Finally, move to the Backend Layer and write tests for all calls produced by the Service Layer for the given User Action.
Making the tests generic
Never, ever, ever have any environment specific details in the test itself. It is easy enough to create a property file and load it during runtime, so there is no reason not to.


Run the provided tests

In your IDE, create a new Run Configuration as:
  • Main class: org.testng.TestNG
  • Program arguments: ./src/main/resources/testng-MyApp1.xml
  • VM arguments:
    -Dcuanto.url=http://localhost:8080/cuanto
    -Dcuanto.projectkey=MyApp1
    -Dcuanto.testrun.create=true
    -Dcuanto.includeConfigDuration=false
    -DcrmId=399
If your Tomcat installation is not running on localhost:8080, update the cuanto.url accordingly.

Example screenshots using Eclipse:

  


When done, run the new configuration and switch back to ShowMyTests.
It should now display something simliar to this:



Add your own tests

The easiest way to get started is to reuse the existing classes:
  • OnlineFrontend - Frontend tests, could be login or a purchase using a browser
  • CRMService - Service (SOA) tests, normally WebServices
  • CRMBackend - Backend tests, such as databases or depending WebServices
Let's say we want to test a Login User Action.
First, we identify the flow through the application:

As shown in the above, we have four tests to write, starting with the bottom one moving up.

User DB

We create a new method called "userDB" in CRMBackend, which will check if the database if functional or not. Preferrably executing a query against the database.
@Test(groups = {"MyApp1", "userDB"})
public void userDB() {
  assertTrue("Here we can test if the DB works", true); // Replace this line with your own test
}

Important to note here, is the specified "groups". These will be used for two purposes:
  1. "MyApp1" - groups all tests which belongs to MyApp1. This is useful for reusing the same test code for multiple applications
  2. "userDB" - it is good practice to include the method in a group with the same name. You will see why when we move on to the Service layer

Security WS & User WS

In CRMService, add the following code:
@Test(groups = {"MyApp1", "authenticateUser"}, dependsOnGroups = {"userDB"})
public void authenticateUser() { // Part of Security WS
  assertTrue("Here we can test user authentication works", true); // Replace this line with your own test
}

@Test(groups = {"MyApp1", "retrieveUserDetails"}, dependsOnGroups = {"userDB"})
public void retrieveUserDetails() { // Part of User WS
  assertTrue("Here we can test if user details can be retrieved", true); // Replace this line with your own test
}

We use the same principle as for the User DB with one exception, "dependsOnGroups".
"dependsOnGroups" tells TestNG that both methods depend on the success of "userDB" and will not be executed if "userDB" fails.

This is a key feature which makes your tests raise above the rest.

Login

Final method :
@Test(groups = {"MyApp1", "login"}, dependsOnGroups = {"authenticateUser", "retrieveUserDetails"})
public void login() { // Part of User WS
  assertTrue("Use Selenium to test the login sequence and verify retrieved data", true); // Replace this line with your own test
}

Summary

As a result of the "dependsOnGroups", TestNG will manage the dependencies and execution order. If multiple methods don't specify "dependsOnGroups", they will be executed in no particular order.
If we map out the execution, we would end up with something like this:

As you can see, it follows the Bottom Up Testing technique to the letter.

If you now run "my-integration-tests" again, the new tests will show up on "ShowMyTests" screen (hidden since they would all pass by default). To see how the userDB affects the upstream functions, you can simply change the assertTrue() call by changing true to false and re-run the tests.



Congratulations! You have now created your very first tests as part of 21st Century Integration Testing!

But this is only the beginning. If you haven't already, replace the tests with your own tests and see it all come to life.


Some help along the way

Refactoring classes

The names of the classes probably won't make much sense for your application, so let's take a closer look at how we can change them.

ShowMyTests will put any tests run from a class' name ending with Frontend into the Frontends layer, ending with Service into Service layer and Backend into the Backends Layer. This means no change should be required for the monitor page, as long as you follow the given naming convention.

TestNG however, needs to know which classes to look for tests in. Inside /src/main/resources/testng-MyApp1.xml make sure your new classes are mentioned inside <classes></classes> and they will be included in the test run.

Making SoapUI tests smarter

SoapUI provides an interfact to override any custom properties defined inside its projects. For example, we pass in "-DcrmId=399" while running the tests above.
This crmId is then picked up in CRMService and passed to the SoapUI project as:
String[] props = {
  "crmId=" + Init.getProperty("crmId")
};
System.out.println("crmId=" + Init.getProperty("crmId"));
runner.setProjectProperties(props);

Inside SoapUI we have crmId declared as a Project Customer Property:



Which we use inside one of the Test Steps:



This makes it very easy to adjust any (for example) environment specific values while running the integration tests, without the need to duplicate the tests.

One custom property that normaly comes in handy is the "myEndpoint". This makes it possible to pick and choose which server you want to run the tests against.


Here is how it can be implemented:

1. In SoapUI, add the custom property (which also serves as default value when running the tests inside SoapUI):



2. In SoapUI, change the endpoint(s) to use this new custom property:



3. Inside CRMService, add line 27:
String[] props = {
  "crmId=" + Init.getProperty("crmId")
  "myEndpoint=" + Init.getProperty("myEndpoint")
};
System.out.println("crmId=" + Init.getProperty("crmId"));
runner.setProjectProperties(props);

Now the next time you run my-integration-tests, it is possible to run the Service tests against any server, using the "-DmyEndoint=<new_value>" argument.