Extending loadUI - Part 3

This time around we’ll create a component that continuously adapts the generated load to achieve the highest transactions per second. We’ll build on the ground covered by the previous posts and now add display values, custom settings, reporting and a bunch of other stuff possible in a loadUI component.

Background


The component itself has the following rationale; for a long-running test we would like our load generator(s) to automatically adjust their rate based on the performance of the target service, which allows us to keep a steady “manageable” load over time. For example when running a test over the weekend we don’t want intermittent slow-downs in the target system (for example due to a backup job) to totally overload our servers (or maybe that is exactly what you want, in which case you can ignore this component altogether)

The component created here does just that: it analyzes the output of a connected runner component and adjusts the rate of its target generator accordingly; as long as improved TPS values are obtained from the runner the component will increase load periodically. When the TPS starts dropping the rate is decreased instead, thus (hopefully) settling at a steady-state around our servers’ optimum performance. Also, the component keeps an eye on the “Queued” property of the runner, which will always be used to decrease load if it has a value > 0.

Using the Rate Adapter


Let’s get right to using the component and we’ll dig into the code later; download the attached file (containing the component and its icon) and unzip it to you loaduiscript-components folder. Create a new project in loadUI with the following components:
  • A Fixed Rate generator for generating load
  • A runner for loading some kind of target system
  • Our “Rate Adapter” component (from the Misc group) that will “control” the rate of the generator based on the outputs of the runner (set its target to the created Fixed Rate component)


Connect these as follows:

As you can see the left and right outputs from the Web Runner are connected to the corresponding inputs on the Rate Adapter, and the Fixed Rate Generator has been selected as the target Generator.


Also add a Statistics Component that you connect to the output of the runner and a TableLog that you connect to the output of the Rate Adapter:

Now when you run the test the following will happen:

  1. The Generator will start at the “Start At” rate configured in the Rate Adapter component and generator the corresponding load with the connected runner.
  2. The Rate Adapter will continuously measure the achieved TPS of the runner and increase the rate of the specified Generator as long as the TPS of the runner does not decrease compared to the obtained TPS for the previous rate. Measurement is initially done in 10 second periods, so if your expected max rate is a high value you might have to wait quite a while (ie if you have a max rate around 1000 and you start at 1 with a step of 1, you will have to wait around 10000 seconds, roughly 2 hours and 45 minutes
  3. The Rate Adapter will decrease the rate of the specified Generator in one of two situations:
    • If the number of Queued requests in the runner is > 0 (which means that the target service is overloaded)
    • If the TPS is less than the TPS obtained at the previous rate
    • If this is the second time the TPS drops directly after a Rate decrease, the Rate Adapter will decrease the TPS again.
    • If this is the third time the TPS drops in a row, the Rate Adapter will actually increase the Rate since the drop of the TPS might be directly related to the drop in the rate (I encountered this situation when testing the component; if it wasn’t for this “check” the rate would continuously decrease back to 1 since the TPS would decrease for each time the Rate is decreased)


Eventually you will see the statistics component showing something like the following:


Here you can see an obtained steady-state situation: when reaching the rate the gives the best TPS, the Rate Adapter will continuously increase/decrease the rate around this value. If your system for some reason exhibits changes in performance (either better or worse), the Rate Adapter will “notice” this in line with the algorithm outlined above and eventually settle to a adapted steady-state.


A number of counters are displayed on the component during execution:

  • Rate shows the current Rate (same as the target Generator)
  • Avg TPS shows the TPS currently being measured
  • Last Avg TPS shows the TPS obtained at the previous Rate
  • Rolling Avg value shows the average TPS of the last 10 measurements
  • Std Dev shows the standard deviation of the last 10 TPS measurements; a low value indicates a “steady-state” situation
  • Std Dev % shows the standard deviation as a percentage of the Rolling Average


The standard deviation values will give you an indication on how “steady” the performance of the target service has been during the last 10 measurement periods.
The output of the Rate Adapter is connected to a TableLog which shows the continuous measurement results, for example:

The displayed columns are as follows:

  • Action shows one of “Increase”, “Decrease” and “None” to show how the target rate was changed after the current period
  • AvgTps shows the measured average TPS for the current period
  • Diff shows the difference to the previous measurement
  • Rate shows the rate used for the current period


Now let’s take a look under the hood of the component; open the downloaded groovy file in your favorite code or text editor and come along!

Component Overview


This component is a bit larger (almost 350 lines of conservatively commented code) than the previous ones as it adds a number of things we haven’t looked at before, including display counters, a compact-mode layout, reporting and a settings tab. Let’s walk through the highlights of the code to see how all this works:

The Component Header


The component header at the top of our file looks as follows:
/**
* Adapts the rate of a configured Generator for the optimum TPS output
*
* @name Rate Adapter
* @nonBlocking true
* @dependency commons-math:commons-math:1.2
*/

This is an optional header (our previous examples had none) which allows us to customize some basic aspects of the component using annotation-like parameters;
  • @name – sets the name of the component in the Component Toolbar
  • @nonBlocking – tells loadUI that messages coming on input Terminals will be handled synchronously by the component. If your component instead “blocks” execution (for example waiting for some external response) you can set this value to false which will result in loadUI dispatching each message in a dedicated thread.
  • @dependency – allows you to specify external dependencies via Groovys Grape extension; these will be downloaded and initialized dynamically when the component is first created (repeated use of the component will use a cached version of the specified library). You can specify as many dependencies as needed (we have a dependency on the commons math library which we use for standard deviation calculations).

Defining Properties and Displayed Strings


Following the standard imports, the component defines the inputs, outputs, properties and local variables used by the component;
  • The Inputs are for collecting and measuring results from the connected runner component
  • The output is for logging measured values for each measurement period (as displayed in the TableLog above)
  • Local variables related to measurements and state.

For displaying counters on the component, the DelayedFormattedString class is used, which periodically updates itself internally every 500ms (to avoid updating at every change of the displayed value). A corresponding field for each of the displayed values is created as follows:
def displayRate = new DelayedFormattedString( '%d', 500, value { currentRate } )
def displayTPS = new DelayedFormattedString( '%.2f', 500, value { currentAvgTps } )
etc..


The arguments to the constructor are a printf-like format, the update frequency and a closure returning the value to be displayed (this will be evaluated for every update). We will see further down how these are then displayed on the component.

Methods and Functionality


After the core definitions a number of methods are defined for different aspects of the components functionality. First up is updateTargets, a method for updating the list of target components displayed on the Rate Adapter itself; an event listener is registered that updates the list every time a component is added or removed from the canvas.
After this we add our first scheduled functionality; let’s have a look at it in more detail (scheduling is done with a standard java executor):
executor.scheduleAtFixedRate(
{
if( context.canvas.running )
{
currentCnt++
currentSum += requestCnt
currentAvgTps = currentSum / currentCnt
requestCnt = 0
}
updateLed()
}, 1, 1, TimeUnit.SECONDS )


Here we schedule a closure that every second updates the current TPS calculation by adding up the number of requests finished during the current measurement period (as received by the Message Handler described below) and dividing that number with the number of seconds measured (which gives the number of finished transactions per second).

The updateLed method called at the end updates the LED displayed on the top left of the component as follows (a bit further down):

updateLed = {
if (context.canvas.running && context.canvas.getComponentByLabel( target.value )?.getProperty( "rate" ) != null )
setActivityStrategy(ActivityStrategies.BLINKING)
else
setActivityStrategy(ActivityStrategies.OFF)
}


The setActivityStrategy method is used to set the led to blinking if the test is running and a valid target component has been selected, otherwise the LED is turned off.

The main onMessage handler follows; it is called every time messae is received via any of the components inputs;

onMessage = { incoming, outgoing, message ->
if( message["TimeTaken"] != null )
{
requestCnt++
}
def queued = message["Queued"]
if( queued != null )
{
queuedRequests = queued
}
}


The incoming message is checked for a “TimeTaken” property (to identify it as a result from a runner, but it could be any source component creating a corresponding message) which increases the corresponding request counter used in the method described above for calculating the TPS value.

If the incoming message instead contains a “Queued” message property (as output by runner components from their right output), this value is used to keep track of the number of currently queued requests (which will result in a rate decrease as described previously).

The “workhorse” of the component is the closure scheduled with the “startScheduler” method; it is run at the end of each measurement period and does the following:

  • Increases or decreases the Rate of the target component in line with the algorithm described above
  • Outputs a message on the output terminal with the last and current results and its actions
  • Calculates the current Rolling average and standard deviations
  • Resets temporary counters and reschedules itself (the reason this is not a fixed schedule is that the user can change the length of the period while the test is running)


Finally an event listener is added for handling project start and stop events; depending on different actions it starts/stops the scheduled tasks and sets corresponding counters.

(I’ve deliberately skipped a number of methods “in the middle”, their purpose and workings should be easy to understand from the code comments).

The Layouts


Visually the component looks as follows in standard mode:

Which in code is defined as

// main layout
layout
{
box( layout:'wrap 2, ins 0' )
{
property( property: target, label: "Generator", options: targets, widget: "comboBox", constraints: 'w 100!, spanx 2' )
property( property: start, label: "Start At", min:1 )
property( property: step, label: "Step", min:1 )
}
separator( vertical : true )
box( widget:'display', layout:'wrap 3, align right' )
{
node( label:'Rate', fString:displayRate, constraints:'w 60!' )
node( label:'Avg TPS', fString:displayTPS, constraints:'w 60!' )
node( label:'Avg Running', fString:displayLastTPS, constraints:'w 70!' )
node( label:'Rolling Avg', fString:displayRollingAvg, constraints:'w 60!' )
node( label:'Std Dev', fString:displayStdDev, constraints:'w 60!' )
node( label:'Std Dev (%)', fString:displayStdDevPercent, constraints:'w 60!' )
}
}


The Compact Layout looks as follows:

which is defined in the exact same way:

compactLayout {
box( widget:'display', layout:'wrap 3, align right' )
{
node( label:'Rate', fString:displayRate, constraints:'w 60!' )
node( label:'Avg TPS', fString:displayTPS, constraints:'w 60!' )
node( label:'Last Avg TPS', fString:displayLastTPS, constraints:'w 70!' )
node( label:'Rolling Avg', fString:displayRollingAvg, constraints:'w 60!' )
node( label:'Std Dev', fString:displayStdDev, constraints:'w 60!' )
node( label:'Std Dev (%)', fString:displayStdDevPercent, constraints:'w 60!' )
}
}

Notable for both the standard can compact layout are two things:
  • the use of the “widget: display” attribute for the box containing the counters; this tells loadUI to create a “counter-display” look for the containing box
  • the contained nodes which use the formatted strings we defined at the beginning of the component to display their content

Settings


The component adds two settings; one for controlling how long the measurement period before increasing/decreasing should be (“period”), and one for controlling how many measurements that should be used to calculate the Average TPS and Std Deviation. In the settings dialog they are available as:

And in code this is just a few lines:

settings( label: "Basic" ) {
property( property: period, label: "Measurement Period" )
property( property: rollingCnt, label: "Rolling Count" )
}

Reporting


Adding to the generated summary report is equally easy; add a generateSummary method that takes the current “chapter” as an argument and add the desired sections, tables and values. The Rate Adapter will do just that; add a section with a table displaying the final values shown on the component, the code for this is straight-forward;
generateSummary = { chapter ->
// create table and its columns
LTableModel table = new LTableModel(1, false);
table.addColumn( "Target Generator" )
table.addColumn( "Rate" );
table.addColumn( "Avg TPS" )
table.addColumn( "Last Avg TPS" )
table.addColumn( "Rolling Avg" );
table.addColumn( "Std Dev" )
table.addColumn( "Std Dev %" )

// add values
ArrayList values = new ArrayList();
values.add( target.value )
values.add(displayRate);
values.add( displayTPS )
values.add( displayLastTPS )
values.add( displayRollingAvg );
values.add( displayStdDev )
values.add( displayStdDevPercent )
table.addRow(values);

// add section to report
MutableSection sect = chapter.addSection(getLabel());
sect.addTable(getLabel(), table)
}


First we define the table and columns, then add its data and finally add a section to the report containing the table. When generating a summary report after running our test we get a nicely formatted section in it:

Cleanup


The onRelease() method at the end of the component is called when a component instance is destroyed, here you must be sure to
  • release any DelayedFormattedStrings you have created for your display
  • release any listeners you have registered on project components
  • stop any scheduled tasks


The component does all this in a straight forward manner.

The Nice Icon


Also we’ve added a nice icon for the component to the toolbar:

This is achieved by adding a png named as the component in the script-components folder (in our case RateAdapter.png).

Wrap up!


That’s it for this time; a lot of new stuff has been introduced:
  • Displaying formatted strings
  • Creating a compact display
  • Adding a Settings tab
  • Updating the LED for the component
  • Adding to the generated summary report
  • Adding a nice icon for the component
  • Project event listeners and code for manipulating other components’ properties


In future posts I’ll just continue show you how easy it is to put all this to use, next up is a custom runner for running arbitrary command-line commands and reporting their results. And if you have any other ideas or wishes, please don’t hesitate to let us know!


And be sure to check out the available resources on all the above:


Have even more fun!


/Ole

Download


And I almost forgot the component itself; here it is: RateAdapter.zip. Unzip this into your loaduiscript-components folder and you should be all set.


Close

By submitting this form, you agree to our
Terms of Use and Privacy Policy

Thanks for Subscribing

Keep an eye on your inbox for more great content.

Continue Reading

Add a little SmartBear to your life

Stay on top of your Software game with the latest developer tips, best practices and news, delivered straight to your inbox