Take your testing to the next level: How to be an A+ tester in a DevOps world

Chapters

• Introduction
• Take your testing to the next level
• Introduction to proactive monitoring
  • Set up an availability/uptime monitor
  • Set up a transaction monitor
  • Set up availability and performance alerts
  • Identify slow performing content using reports
• Reuse your API functional tests
  • Identify faulty component in API tests

Introduction

Depending on your application type, you will allocate different resources, time and weightage to these various types of testing. However with modern websites and web applications, built with multiple dynamic components and diverse use cases, it’s difficult to know when your job is done. Even after advanced testing efforts, we see applications getting overwhelmed and even failing in production due to numerous reasons.

On the other hand, Modern organizations have moved from serial development of monolithic applications, assembling and releasing once or twice a year, to an agile, DevOps and CI-CD motion, where applications are developed and deployed often and in smaller blocks. The pursuit of rapid delivery distributes the responsibility of ensuring performance across the board – developers are expected to write applications that are optimized for performance, testers (functional or non-functional) are expected to test applications for performance and quality across all unique use cases and operations are expected to ensures performance in production and feed the insights back to the development team.

We hear arguments about where does testing fit in DevOps. If you look closely, DevOps is all about continuous testing. By bringing testing into mainstream development process, DevOps avoids the problems of doing ‘one-big-testing-effort’ at the end of the cycle. The maturity of testing is the key differentiator of Agile and DevOps best practices.

Take your testing to the next level

You can set yourself apart from other testers by taking your testing efforts to the next level – ensuring that the website or the app runs flawlessly in production-like environment. Proactive monitoring allows you to test the entire website or web application at the browser level, where all the dynamic components come together. By testing the application on real browsers running on diverse ISPs and wireless networks, located at various geographical locations across the globe, you can emulate end user transactions and observe your applications behavior and performance. The data you gather during proactive monitoring can be used to identify and fix the performance issues, if any, before they impact your end user.

In simple terms, proactive monitoring lets you validate your testing efforts across realistic user scenarios and enables you to launch your application with confidence.

Let’s get started

Considering that you are new to performance monitoring, let’s choose a tool that has the fastest time to value and is easy to use. In this tutorial, I’m going to use AlertSite synthetic monitoring tool. You can get free AlertSite trial here.

After completing the form, you automatically get into the AlertSite SaaS product. AlertSite allows you to create web, mobile or API monitors. For the sake of simplicity of this tutorial, let’s create a web monitor. You can create a simple availability and uptime check or create an advanced ‘real-browser’ check to see performance and UX metrics such as page load and DOM. (Learn more about performance Metrics and why you need them)

Create a Real-Browser Monitor

The real browser monitor could be a single URL monitor or a transaction / multi-step monitor (like the login procedure or order checkout process). Let’s create them one by one.

On the Dashboard, click the big + symbol, then Show All Monitor Types.

The monitor will start collecting data shortly. You can see the run results on AlertSite’s dashboards.

You can add more intelligence to the monitors by selecting locations that you care about and configuring alerts per your requirements. Open the monitor settings. To do this On AlertSite’s Dashboard, click the monitor and then click  to open the monitor settings.

1. Click Locations.
2. Select the check box next to the mobile locations you want to use. Type att wireless or sprint mobile into the search box to find mobile locations quickly.

Multi-Step Transaction monitor

AlertSite eliminates the complex scripting or coding required for transaction monitoring with our proprietary web recorder Dejaclick. Dejaclick is a simple browser based plugin which enables you to point to a browser, record transaction and automatically convert that into an AlertSite monitor. You can install DéjàClick for Chrome and Firefox from here.

For the sake on demonstration, let’s record a transaction in Chrome.

Recording in Chrome:

After the local replay completes successfully, you can upload the transaction toAlertSite. To do this:

1. Click the  Remote Services button on the DéjàClick toolbar.

   

   You will see a confirmation that the transaction has been uploaded to yourAlertSiteaccount.

2. You will be redirected back to AlertSite, where you will see the monitor settings.

   

3. Set Run interval to how often you want to run the transaction. The default is 5 minutes.
4. Switch to the Locations tab, and select the locations from which you want the monitor to run.
5. Click  in the top right.

(For more information about using the DéjàClick recorder, see DéjàClickUser Guide.)

---

Download a Free 30 Day Trial: Try AlertSite Today  |  Visit AlertSite Now

---

What if something does go wrong?

AlertSitecan send automatic alerts whenever you monitors discover a problem. There are two types of alerts:

• Availability alerts – sent when a monitor detects an error (the monitor status turns red). For example, if your site does not respond within the timeout, returns incorrect content, or returns an HTTP error like 500 Internal Server Error. More details on the conditions that trigger an error are specified here: monitor settings.
• Performance alerts – sent when the response time exceeds the configured threshold. More details can be found here: Performance Alerts.

When a problem resolves, a monitor sends the “clear” notification. AlertSite makes sure that you receive alerts on the notification channel that is the most convenient for you:

How to identify slow performing content

Your operations colleagues do this day in and out; possibly at a much deeper level. Considering that we are just getting started here, let’s see how AlertSite makes it easy to identify slow performing content from your monitoring data. Along with the alerts, AlertSite also has a highly sophisticated reporting engine.

Start with running a detailed report by clicking on the Reports tab on the menu, and selecting report type (Detail) , appropriate monitor, locations, date range and report types. Then hit Run Report button. I viewed the report as HTML.

In this interactive report, you can see the line chart showing test performance from various locations. And the report further shows the actual DNS, First byte, DOM load and page load times. The report details give you further more actionable data at the IP and transaction level.

What if you tested APIs?

If you are an API tester using the most popular API testing tool SoapUI or ReadyAPI, AlertSite makes things even easier for you. You can either create API endpoint checksor simply upload your existing SoapUI or SoapUI NG Pro script to test the actual API functionality across AlertSite monitoring network.

A. Uploading From AlertSite UXM

First, export your ReadyAPI project to a ZIP file:

1. In ReadyAPI, select Project > Export from the main menu.
2. Specify the ZIP file to create.

Then, in AlertSite UXM:

1. On the Dashboard, click the big + symbol, then Show All Monitor Types.
2. Select API and click Upload and Monitor.

   

3. Select your AlertSite plan, and click Continue.
4. Browse for your zipped API project, and click Submit File.
5. Select your test case, and click Next.

   

6. Enter the monitor name and how often it should run, and click Create.

   

7. On the screen that follows, click Test on Demand. This will run the test from a remote location to ensure the monitor can work from arbitrary locations.
8. If the test fails, review the errors and fix your ReadyAPI test appropriately, then export and upload the project again.

Your monitor will start collecting data shortly, and you can see the results on the AlertSite dashboards.

B. Uploading From ReadyAPI (Using AlertSite Connector Plugin)

ReadyAPI v. 1.3 and later include the AlertSite Connector plugin. With this plugin, you can create API monitors in AlertSite directly from ReadyAPI:

1. In ReadyAPI, switch to SoapUI NG.
2. ClickMonitor APIs on the toolbar. This will open the AlertSite Connector.

   

3. Enter your AlertSite email and password, and click Connect.

   

You will see the Connected to AlertSite message on successful login.

Now you can create an API monitor from your test:

1. Click Add Monitors in the top left.

   

2. In the dialog that appears, configure your monitor:
   1. Select the Project,Test suite and Test case to run.
   2. Enter the Monitor name and select how often the test should run (every 1 to 120 minutes, depending on your plan).
   3. If you have several plans, select the plan to use for this monitor.
   4. Click Add.

   

You will see the API Status page with your monitor added. The monitor will start running shortly, and you will see its status.

Identifying faulty component

AlertSitestores SoapUI test logs to help troubleshoot API test failures. The logs contain full HTTP request and response data, assertion results, and any errors that occurred.

A SoapUI log is attached to email alerts if email recipients have the Attach server response to e-mail alerts option selected.

To view SoapUI logs inAlertSite, go to Monitors > Runs and select your monitor. Expand a failed run, then click Detail Report.

In the report that opens, click the  camera icon.

The next screen lets you access SoapUI logs.View Step Log opens the log of just that step, and View Run Log opens the log of the whole run, including all step logs.

Below is an example of a step log. You can see that the test failed, because the API response did not contain the expected value.