Jetstress 2010 Test Result Evaluation

In this blog, I am going to talk about how to evaluate your JetStress test. You can find the instruction on how to install, configure, and run JetStree in this blog.

The following criterion should be evaluated for each test run.

1. DB IOPS Target: Is the Achieved Transactional I/O per Second in the test report higher than the Total Database Required IOPS / Server predicted in the Mailbox Role Calculator.
2. Is the I/O Database Reads Average Latency in the test report < 20ms?
3. Is the I/O Log Writes Average Latency in the test report < 10ms?

The test will show Failed if neither 2 or 3 is met. However, you will have to determine No.1 by comparing the Achieved Transactional I/O per Second in the test report with the Total Database Required IOPS / Server predicted in the Mailbox Role Calculator.

You can find Total Database Required IOPS under Role Requirement worksheet in the Mailbox Role Calculator.

This value is calculated by the Mailbox Role Calculator using multiple input factors. Below are some of the key factors that determine the value:

1. Total Number of Tier-X user Mailboxes / Environment
2. Total Send /Receive Capability / Mailbox / Day
3. Number of database / Server

So when you run JetStress you should generate enough load on the server so that the total IOPS achieved by the test is higher than what is predicted by the Mailbox Role Calculator. If the number is higher, the first criterion is than met. But it doesn't mean the test succeeds. You still have to make sure both No.2 and No.3 are met. As the Achieved Transactional I/O per Second increases,  I/O Database Reads Average Latency and I/O Log Writes Average Latency also increase.

You can use Thread Count to achieve the target Achieved Transactional I/O per Second. Basically the bigger Thread Count is, the higher the Achieved Transactional I/O per Second will be achieved.

The first time you runt the test, you would leave everything as default (Thread Count = 1), you then compare the value to determine whether you should increase Thread Count or not. For example, assume Total Database Required IOPS / Server predicted in the Mailbox Role Calculator is 500. Your first test result looks like below:

Achieved Transactional I/O per Second	500
I/O Database Reads Average Latency	8
I/O Log Writes Average Latency	0.2

The test result will show Succeeded. But you know it is not because it doesn't achieve the target value which is 700 predicted by the Mailbox Role Calculator. Next you want to increase the Threat Count to achieve more IOPS. Now the question is what Thread Count should I use? Below is a table taken from a single database running on a RAID10 LUN with 24x2.5" 10K SAS HDD's.

I would recommend you increase the Thread Count to bigger value to find out where the result should fail. For example, I increased the Thread Count to 5 and received the following result:

Achieved Transactional I/O per Second	800
I/O Database Reads Average Latency	26
I/O Log Writes Average Latency	4.7

Now I've achieved the target IOPS of 800 which is higher than 700 predicted by Mailbox Role Calculator. Unfortunately, the test failed at both I/O Database Reads Average Latency and I/O Log Writes Average Latency. So I know my existing storage solution will not support a total IOPS on any server that is over 800.

Next I dropped the Thread Count to 3 and my result reads as below:

Achieved Transactional I/O per Second	730
I/O Database Reads Average Latency	17
I/O Log Writes Average Latency	3.2

I am very happy with the result as it shows my storage will support IOPS/server up to 730 which is higher than what is predicted by the Mailbox Role Calculator, while I/O Database Reads Average Latency and I/O Log Writes Average Latency are also below minimum.
In a real world, you don't have to run multiple tests. As long as you get one result that meet all three criterion you are certain the storage meets the requirement. But if you want to find out the maximum it can support up to, you will have to get a failure test.

If it is not possible to achieve the right IOPS value by modifying the thread count it becomes necessary to modify the Sluggishsessions value within the JetStressConfig.xml file. The Sluggishsessions value adds a pause between each task. This allows a level of fine-tuning over the workload dispatched by JetStress. As Sluggishsessions is increased the achieved IOPS value decreases.  To change the value, open the JetStressConfig.xml file and look for the default configuration option
<SluggishSessions>1</SluggishSessions>

Modify the value, save the configuration file and then re-start JetStress.

Use the table below as a guide to determine what to do next after each test till you get a satisfying result.