diff options
| author | John Levon <john.levon@nutanix.com> | 2021-05-10 13:12:19 +0100 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2021-05-10 13:12:19 +0100 |
| commit | e92b1f6b57d0131ec4aa581e57e10ab7d628a0cd (patch) | |
| tree | 81fbf790b7bdc19bbb646ff3a4cbba4671971789 /Makefile | |
| parent | b95c886ed23b4cc4c539030bf383b55aae8859a3 (diff) | |
| download | libvfio-user-e92b1f6b57d0131ec4aa581e57e10ab7d628a0cd.zip libvfio-user-e92b1f6b57d0131ec4aa581e57e10ab7d628a0cd.tar.gz libvfio-user-e92b1f6b57d0131ec4aa581e57e10ab7d628a0cd.tar.bz2 | |
start python-based testing framework (#447)
Trying to do our unit/functional testing with C is very tedious, and cmocka
especially is a continual pain point. This commit introduces a Python-based
testing infrastructure, and adds an initial set of functional tests for client
negotiation.
The tests work under Valgrind for leak/bad access detection of the C code, but
not under ASAN, which lacks any meaningful shared-library support.
We should be able to replace all of current C-based unit tests with this,
reverting samples/ back to demo code only.
Signed-off-by: John Levon <john.levon@nutanix.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Diffstat (limited to 'Makefile')
| -rw-r--r-- | Makefile | 53 |
1 files changed, 51 insertions, 2 deletions
@@ -54,13 +54,61 @@ BUILD_DIR = $(BUILD_DIR_BASE)/$(BUILD_TYPE) INSTALL_PREFIX ?= /usr/local -PHONY_TARGETS := all test pre-push realclean buildclean force_cmake tags +PHONY_TARGETS := all pytest pytest-valgrind test pre-push realclean buildclean force_cmake tags .PHONY: $(PHONY_TARGETS) all $(filter-out $(PHONY_TARGETS), $(MAKECMDGOALS)): $(BUILD_DIR)/Makefile +$(MAKE) -C $(BUILD_DIR) $@ -test: all +# +# NB: add --capture=no to get a C-level assert failure output. +# +PYTESTCMD = \ + $(shell which -a pytest-3 /bin/true 2>/dev/null | head -1) \ + -rP \ + --quiet + +PYTEST = \ + BUILD_TYPE=$(BUILD_TYPE) \ + $(PYTESTCMD) + +# +# In our tests, we make sure to destroy the ctx at the end of each test; this is +# enough for these settings to detect (most?) library leaks as "definite", +# without all the noise from the rest of the Python runtime. +# +# As running under valgrind is very slow, we don't run this unless requested. +# +PYTESTVALGRIND = \ + BUILD_TYPE=$(BUILD_TYPE) \ + PYTHONMALLOC=malloc \ + valgrind \ + --suppressions=$(CURDIR)/test/py/valgrind.supp \ + --quiet \ + --track-origins=yes \ + --errors-for-leak-kinds=definite \ + --show-leak-kinds=definite \ + --leak-check=full \ + --error-exitcode=1 \ + $(PYTESTCMD) + +ifdef WITH_ASAN + +pytest pytest-valgrind: + +else + +pytest: all + @echo "=== Running python tests ===" + $(PYTEST) + +pytest-valgrind: all + @echo "=== Running python tests with valgrind ===" + $(PYTESTVALGRIND) + +endif + +test: all pytest cd $(BUILD_DIR)/test; ctest --verbose pre-push: realclean @@ -71,6 +119,7 @@ pre-push: realclean make realclean make test CC=gcc BUILD_TYPE=rel make test CC=gcc + make pytest-valgrind realclean: rm -rf $(BUILD_DIR_BASE) |