3.8 KiB
Test Cases
Test Case 1: Message Queue Throughput Test
Description
This test measures the aggregate throughput of a MQ layer by using the oslo.messaging simulator tool. Either RabbitMQ, ActiveMQ, or ZeroMQ can be used as the MQ layer. Throughput is calculated as the sum over the MQ clients of the throughput for each client. For each test the number of clients/threads is configured to one of the specific values defined in the test case parameters section. The full set of tests will cover all the "Threads count" values shown, plus additional values as needed to quantify the dependence of MQ throughput on load, and to find the maximum throughput.
Parameters
| Parameter name | Value |
|---|---|
| oslo.messaging version | 2.5.0 |
| simulator.py version | 1.0 |
| Threads count | 50, 70, 100 |
Measurements
| Value | Measurment Units | Description |
|---|---|---|
| Throughput | msg/sec | Directly measured by simulator tool |
Result Type
| Result type | Measurement Units | Description |
|---|---|---|
| Throughput Value | msg/sec | Table of numerical values |
| Throughput Graph | msg/sec vs # of threads | Graph |
Additional Measurements
| Measurement | Units | Description |
|---|---|---|
| Variance | msg/sec | Throughput variance over time |
Test Case 2: OMGBenchmark Rally test
Description
OMGBenchmark is a rally plugin for benchmarking oslo.messaging. The plugin and installation instructions are available on github: https://github.com/Yulya/omgbenchmark
Parameters
| Parameter name | Rally name | Value |
|---|---|---|
| oslo.messaging version | 2.5.0 | |
| Number of iterations | times | 50, 100, 500 |
| Threads count | concurrency | 40, 70, 100 |
| Number of RPC servers | num_servers | 10 |
| Number of RPC clients | num_clients | 10 |
| Number of topics | num_topics | 5 |
| Number of messages per iteration | num_messages | 100 |
| Message size | msg_length_file | 900-12000 bytes |
Measurements
| Name | Measurement Units | Description |
|---|---|---|
| min | sec | Minimal execution time of one iteration |
| median | sec | Median execution time |
| 90%ile | sec | 90th percentile execution time |
| 95%ile | sec | 95th percentile execution time |
| max | sec | Maximal execution time of one iteration |
| avg | sec | Average execution time |
| success | none | Number of successfully finished iterations |
| count | none | Number of executed iterations |
Result Type
| Result type | Measurement Units | Description |
|---|---|---|
| Throughput Graph | msg size vs median | Graph |
| Concurrency Graph | concurrency vs median | Graph |