aboutsummaryrefslogtreecommitdiff
path: root/scripts/simplebench
AgeCommit message (Collapse)AuthorFilesLines
2023-09-08scripts/: spelling fixesMichael Tokarev1-1/+1
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
2022-04-21python: rename qemu.aqmp to qemu.qmpJohn Snow1-1/+1
Now that we are fully switched over to the new QMP library, move it back over the old namespace. This is being done primarily so that we may upload this package simply as "qemu.qmp" without introducing confusion over whether or not "aqmp" is a new protocol or not. The trade-off is increased confusion inside the QEMU developer tree. Sorry! Note: the 'private' member "_aqmp" in legacy.py also changes to "_qmp"; not out of necessity, but just to remove any traces of the "aqmp" name. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Beraldo Leal <bleal@redhat.com> Acked-by: Hanna Reitz <hreitz@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@openvz.org> Message-id: 20220330172812.3427355-8-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2022-04-21scripts/bench-block-job: switch to AQMPJohn Snow1-2/+1
For this commit, we only need to remove accommodations for the synchronous QMP library. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Beraldo Leal <bleal@redhat.com> Acked-by: Hanna Reitz <hreitz@redhat.com> Message-id: 20220321203315.909411-3-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2022-01-10simplebench: Fix Python syntax error (reported by LGTM)Stefan Weil1-1/+1
Fixes: b2fcb0c5754c2554b8406376e99a75e9e0a6b7bd Signed-off-by: Stefan Weil <sw@weilnetz.de> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20220107153019.504124-1-sw@weilnetz.de Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01iotests: Accommodate async QMP Exception classesJohn Snow1-1/+2
(But continue to support the old ones for now, too.) There are very few cases of any user of QEMUMachine or a subclass thereof relying on a QMP Exception type. If you'd like to check for yourself, you want to grep for all of the derivatives of QMPError, excluding 'AQMPError' and its derivatives. That'd be these: - QMPError - QMPConnectError - QMPCapabilitiesError - QMPTimeoutError - QMPProtocolError - QMPResponseError - QMPBadPortError Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20211026175612.4127598-5-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-15simplebench: add img_bench_templater.pyVladimir Sementsov-Ogievskiy2-0/+157
Add simple grammar-parsing template benchmark. New tool consume test template written in bash with some special grammar injections and produces multiple tests, run them and finally print a performance comparison table of different tests produced from one template. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20210824101517.59802-2-vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-05-04simplebench/bench-backup: add --drop-caches argumentVladimir Sementsov-Ogievskiy2-2/+15
Add an option to drop caches before each test run. It may probably improve reliability of results when testing in cached mode. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2021-05-04simplebench/bench-backup: add --count and --no-initial-runVladimir Sementsov-Ogievskiy1-1/+11
Add arguments to set number of test runs per table cell and to disable initial run that is not counted in results. It's convenient to set --count 1 --no-initial-run to fast run test onece, and to set --count to some large enough number for good precision of the results. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2021-05-04simplebench/bench-backup: support qcow2 source filesVladimir Sementsov-Ogievskiy2-1/+11
Add support for qcow2 source. New option says to use test-source.qcow2 instead of test-source. Of course, test-source.qcow2 should be precreated. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com>
2021-05-04simplebench/bench_block_job: handle error in BLOCK_JOB_COMPLETEDVladimir Sementsov-Ogievskiy1-0/+4
We should not report success if there is an error in final event. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com>
2021-05-04simplebench/bench-backup: add target-cache argumentVladimir Sementsov-Ogievskiy2-10/+33
Allow benchmark with different kinds of target cache. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com>
2021-05-04simplebench/bench-backup: add --compressed optionVladimir Sementsov-Ogievskiy2-16/+62
Allow bench compressed backup. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2021-05-04simplebench: bench_one(): support count=1Vladimir Sementsov-Ogievskiy1-1/+4
statistics.stdev raises if sequence length is less than two. Support that case by hand. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com>
2021-05-04simplebench: bench_one(): add slow_limit argumentVladimir Sementsov-Ogievskiy1-1/+13
Sometimes one of cells in a testing table runs too slow. And we really don't want to wait so long. Limit number of runs in this case. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2021-01-26simplebench: add bench-backup.pyVladimir Sementsov-Ogievskiy1-0/+167
Add script to benchmark new backup architecture. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20210116214705.822267-24-vsementsov@virtuozzo.com> [mreitz: s/not unsupported/not supported/] Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26simplebench: bench_block_job: add cmd_options argumentVladimir Sementsov-Ogievskiy2-5/+8
Add argument to allow additional block-job options. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20210116214705.822267-23-vsementsov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26simplebench/bench_block_job: use correct shebang line with python3Vladimir Sementsov-Ogievskiy1-1/+1
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20210116214705.822267-22-vsementsov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18scripts/simplebench: add bench_prealloc.pyVladimir Sementsov-Ogievskiy1-0/+132
Benchmark for new preallocate filter. Example usage: ./bench_prealloc.py ../../build/qemu-img \ ssd-ext4:/path/to/mount/point \ ssd-xfs:/path2 hdd-ext4:/path3 hdd-xfs:/path4 The benchmark shows performance improvement (or degradation) when use new preallocate filter with qcow2 image. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20201021145859.11201-22-vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18simplebench/results_to_text: make executableVladimir Sementsov-Ogievskiy1-0/+14
Make results_to_text a tool to dump results saved in JSON file. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20201021145859.11201-21-vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18simplebench/results_to_text: add difference line to the tableVladimir Sementsov-Ogievskiy1-7/+60
Performance improvements / degradations are usually discussed in percentage. Let's make the script calculate it for us. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20201021145859.11201-20-vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> [mreitz: 'seconds' instead of 'secs'] Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18simplebench/results_to_text: improve view of the tableVladimir Sementsov-Ogievskiy1-1/+12
Move to generic format for floats and percentage for error. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20201021145859.11201-19-vsementsov@virtuozzo.com> Acked-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18simplebench: move results_to_text() into separate fileVladimir Sementsov-Ogievskiy4-33/+52
Let's keep view part in separate: this way it's better to improve it in the following commits. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20201021145859.11201-18-vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18simplebench: rename ascii() to results_to_text()Vladimir Sementsov-Ogievskiy3-7/+7
Next patch will use utf8 plus-minus symbol, let's use more generic (and more readable) name. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20201021145859.11201-17-vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18scripts/simplebench: use standard deviation for +- errorVladimir Sementsov-Ogievskiy1-5/+6
Standard deviation is more usual to see after +- than current maximum of deviations. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20201021145859.11201-16-vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18scripts/simplebench: support iopsVladimir Sementsov-Ogievskiy1-10/+28
Support benchmarks returning not seconds but iops. We'll use it for further new test. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20201021145859.11201-15-vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18scripts/simplebench: fix grammar: s/successed/succeeded/Vladimir Sementsov-Ogievskiy1-6/+6
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20201021145859.11201-14-vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-18scripts/simplebench: compare write request performanceAndrey Shinkevich1-0/+170
The script 'bench_write_req.py' allows comparing performances of write request for two qemu-img binary files. An example with (qemu-img binary 1) and without (qemu-img binary 2) the applied patch "qcow2: skip writing zero buffers to empty COW areas" (git commit ID: c8bb23cbdbe32f5) has the following results: SSD: ---------------- ------------------- ------------------- <qemu-img binary 1> <qemu-img binary 2> <cluster front> 0.10 +- 0.00 8.16 +- 0.65 <cluster middle> 0.10 +- 0.00 7.37 +- 1.10 <cross cluster> 7.40 +- 1.08 21.97 +- 4.19 <cluster 64K> 2.14 +- 0.94 8.48 +- 1.66 ---------------- ------------------- ------------------- HDD: ---------------- ------------------- ------------------- <qemu-img binary 1> <qemu-img binary 2> <cluster front> 2.30 +- 0.01 6.19 +- 0.06 <cluster middle> 2.20 +- 0.09 6.20 +- 0.06 <cross cluster> 8.32 +- 0.16 8.26 +- 0.14 <cluster 64K> 8.20 +- 0.05 6.26 +- 0.10 ---------------- ------------------- ------------------- Suggested-by: Denis V. Lunev <den@openvz.org> Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <1594741846-475697-1-git-send-email-andrey.shinkevich@virtuozzo.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2020-03-17scripts/simplebench: add example usage of simplebenchVladimir Sementsov-Ogievskiy1-0/+80
This example may be used as a template for custom benchmark. It illustrates three things to prepare: - define bench_func - define test environments (columns) - define test cases (rows) And final call of simplebench API. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com> Message-Id: <20200228071914.11746-4-vsementsov@virtuozzo.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2020-03-17scripts/simplebench: add qemu/bench_block_job.pyVladimir Sementsov-Ogievskiy1-0/+119
Add block-job benchmarking helper functions. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com> Message-Id: <20200228071914.11746-3-vsementsov@virtuozzo.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2020-03-17scripts/simplebench: add simplebench.pyVladimir Sementsov-Ogievskiy1-0/+128
Add simple benchmark table creator. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com> Message-Id: <20200228071914.11746-2-vsementsov@virtuozzo.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
generated by cgit v1.1 at 2024-10-13 01:43:05 +0000