aboutsummaryrefslogtreecommitdiff
path: root/lib/dma.c
AgeCommit message (Collapse)AuthorFilesLines
2022-05-30allow concurrent dirty bitmap get (#677)John Levon1-6/+28
Use atomic operations to allow concurrent bitmap updates with VFIO_IOMMU_DIRTY_PAGES_FLAG_GET_BITMAP operations. Dirtying clients can race against each other, so we must use atomic or when marking dirty: we do this byte-by-byte. When reading the dirty bitmap, we must be careful to not race and lose any set bits within the same byte. If we miss an update, we'll catch it the next time around, presuming that before the final pass we'll have quiesced all I/O. Signed-off-by: John Levon <john.levon@nutanix.com> Reviewed-by: Raphael Norwitz <raphael.norwitz@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2022-05-27re-work SGL API (#675)John Levon1-8/+4
Harmonize and rename the vfu_*sg() APIs to better reflect their functionality: in our case, there is no mapping happening as part of these calls, they are merely housekeeping for range splitting, dirty tracking, and so on. Signed-off-by: John Levon <john.levon@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2022-05-27remove maps list from DMA controller (#674)John Levon1-21/+0
->maps existed so that if a consumer does vfu_map_sg() and then we are asked to enable dirty page tracking, we won't mark those pages as dirty, and will hence potentially lose data. Now that we require quiesce and the use of either vfu_unmap_sg() or vfu_sg_mark_dirty(), there's no need to have this list any more. Signed-off-by: John Levon <john.levon@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2022-05-27remove refcnt from region (#673)John Levon1-2/+0
The reference count is unused, and not atomically handled, remove it. Signed-off-by: John Levon <john.levon@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2022-04-28lib: export dma_sg_size symbol in library (#664)Daniel Berrangé1-1/+1
The dma_sg_size() method is listed in libvfio-user.h but the symbol is marked private in the ELF library. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: John Levon <john.levon@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2022-03-07check for allowed operations in quiesce state (#647)Thanos Makatos1-0/+4
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com> Reviewed-by: John Levon <john.levon@nutanix.com>
2022-02-24additional logging around dirty pages (#650)Thanos Makatos1-0/+24
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com> Reviewed-by: John Levon <john.levon@nutanix.com>
2021-11-30introduce device quiesce callback (#609)Thanos Makatos1-15/+4
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com> Reviewed-by: John Leon <john.levon@nutanix.com>
2021-11-24verify region is mapped before acquiring dirty bitmap (#627)John Levon1-0/+12
DMA regions not mapped by the server are not dirty tracked (the client must track changes via handling VFIO_USER_DMA_WRITE), but we weren't correctly enforcing this, which could segfault when ->dirty_bitmap was NULL. Found via AFL++. Signed-off-by: John Levon <john.levon@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2021-06-10dma: Fix returned sg array (#564)Swapnil Ingle1-1/+1
_dma_addr_sg_split() is supposed to return back sg's if the requested dma addr spans across regions. Also adding unit tests to cover these case. Signed-off-by: Swapnil Ingle <swapnil.ingle@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2021-06-09clear dirty pages bitmap after getting dirty pages but keep mapped segments ↵Thanos Makatos1-4/+43
dirty (#551) Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com> Reviewed-by: John Levon <john.levon@nutanix.com>
2021-06-01limit max DMA region size (#545)John Levon1-2/+9
Since the dirty bitmap in message replies is allocated based upon the maximum size of an individual region, add a limit (somewhat arbitrarily 8TiB, which is a bitmap size of 256MiB). Add a couple of basic tests on the two DMA limits. Signed-off-by: John Levon <john.levon@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2021-06-01fixes for VFIO_USER_DIRTY_PAGES (#537)John Levon1-4/+4
- we should only accept one range, not multiple ones - clearly define and implement argsz behaviour - we need to check if migration is configured - add proper test coverage; move existing testing to python Signed-off-by: John Levon <john.levon@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2021-05-25log dirty pages for new DMA regions as well (#518)Thanos Makatos1-26/+48
New DMA regions don't get their pages tracked if dirty page logging has already been started, this patch fixes this bug. Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com> Reviewed-by: John Levon <john.levon@nutanix.com>
2021-05-20migration: various dirty page tracking fixes (#457)Thanos Makatos1-1/+5
- document how to use a vfio-user device with libvirt - document how to use SPDK's nvmf/vfio-user target with libvirt - replace vfio_bitmap with vfio_user_bitmap and vfio_iommu_type1_dirty_bitmap_get with vfio_user_bitmap_range - fix bug for calculating number of pages needed for dirty page bitmap - align number of bytes for dirty page bitmap to QWORD - add debug messages around dirty page tracking - only support flags=0 when doing DMA unmap - set device state to running after reset - allow region read/write even if device is in stopped state - allow transitioning from stopped/stop-and-copy state to running state - fix unit tests Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com> Reviewed-by: John Levon <john.levon@nutanix.com>
2021-05-14dma: Use correct len type (#479)Swapnil Ingle1-3/+4
* dma: Use correct len type vfio_iommu_type1_dirty_bitmap_get.size is of type __u64 dma_controller_dirty_page_get() receives it as int, instead it should be u64 Also added UT to test overflow of length passed to dma_controller_dirty_page_get Fixes: #477 Signed-off-by: Swapnil Ingle <swapnil.ingle@nutanix.com> Reviewed-by: John Levon <john.levon@nutanix.com>
2021-05-10fix dma unregister callback during region removal (#464)John Levon1-7/+14
There are two issues with the unregister callback: - we were requiring the callback to be set when removing a region, but it's only required if a consumer wants to map regions - when we removed all regions (for example, on a reset), we weren't triggering the callback Signed-off-by: John Levon <john.levon@nutanix.com> swapnil code review add assert Reviewed-by: Swapnil Ingle <swapnil.ingle@nutanix.com>
2021-04-13dma: use ERROR_INT()John Levon1-43/+35
The first in a series excising the use of the "return -errno" idiom. This is a non-standard usage, and in userspace, we have "errno" for delivering side-band error values. As there have been multiple bugs from not using standard error return methods like -1+errno or NULL+errno, let's do that. Signed-off-by: John Levon <john.levon@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2021-04-13correct DMA region add/remove returns (#427)John Levon1-13/+17
Code was expecting -errno style returns, but the DMA code didn't do this. Signed-off-by: John Levon <john.levon@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2021-04-07clean up newlines in logs (#423)John Levon1-10/+9
vfu_log() and err() should not take newlines. Signed-off-by: John Levon <john.levon@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2021-04-07mark vfu_log() with format attribute (#426)John Levon1-5/+5
Fix up all resulting fallout. Signed-off-by: John Levon <john.levon@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2021-04-06vfu_reset_ctx(): tear down DMA and IRQs (#418)John Levon1-1/+1
When we lose the client connection, the IRQ and DMA region state is no longer valid; clean them up. Signed-off-by: John Levon <john.levon@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2021-04-06dma_controller_remove_regions: reset region array (#413)John Levon1-0/+3
Prepare this function for re-usability by clearing the array after removal. Signed-off-by: John Levon <john.levon@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2021-03-31rework DMA callbacks (#396)John Levon1-137/+139
This fixes a number of issues with how DMA is handled, based on some changes by Thanos Makatos: - rename callbacks to register/unregister, as there is not necessarily any mapping - provide the (large) page-aligned mapped start and size, the page size used, as well as the protection flags: some API users need these - for convenience, provide the virtual address separately that corresponds to the mapped region - we should only require a DMA controller to use vfu_addr_to_sg(), not an unregister callback - the callbacks should return errno not -errno - region removal was incorrectly updating the region array - various other cleanups and clarifications Signed-off-by: John Levon <john.levon@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2021-03-25re-work unit test mocking (#400)John Levon1-19/+11
Instead of trying to use the linker's --wrap, which just led to more problems when we want to call the real function, we'll add two defines, MOCK_DEFINE() and MOCK_DECLARE(), that behave differently when building the unit tests, such that all wrapped functions are picked up from test/mocks.c instead, regardless of compilation unit. Signed-off-by: John Levon <john.levon@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2021-03-24_dma_addr_sg_split(): set errno when not found (#402)John Levon1-1/+2
Signed-off-by: John Levon <john.levon@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2021-03-23add -Wmissing-declarations (#399)John Levon1-4/+5
This is used by SPDK, and it's generally useful. This also uncovered some issues in the test mocking. Signed-off-by: John Levon <john.levon@nutanix.com> Reviewed-by: Swapnil Ingle <swapnil.ingle@nutanix.com>
2021-03-04don't munmap(2) unmapped DMA region (#384)Thanos Makatos1-21/+22
Plus always notify user when DMA region is removed. Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com> Reviewed-by: John Levon <john.levon@nutanix.com>
2021-02-18use sizeof() consistently (#351)John Levon1-1/+1
The most common way we have written this is as "sizeof()"; use this form consistently. Signed-off-by: John Levon <john.levon@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2021-01-21Misc fixes for DMA_MAP region prot (#233)swapnili1-2/+8
* Misc fixes for DMA_MAP region prot 1. Validate prot passed in vfu_addr_to_sg() 2. Let user know region prot via vfu_unmap_dma_cb_t Signed-off-by: Swapnil Ingle <swapnil.ingle@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2021-01-08Fix error path (#229)swapnili1-0/+1
Signed-off-by: Swapnil Ingle <swapnil.ingle@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2021-01-08Use prot flags sent by client to map dma regions (#227)swapnili1-4/+11
* Use prot flags sent by client to map dma regions Signed-off-by: Swapnil Ingle <swapnil.ingle@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2020-12-17add mappable bit in DMA segment (#215)Thanos Makatos1-7/+1
remove duplicate code for initializing DMA segment, mark DMA segment whether it's mappable, plus basic unit test for dma_addr_to_sg Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
2020-12-02use log level defines from syslogThanos Makatos1-11/+11
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
2020-11-27rename to libvfio-user (#128)John Levon1-16/+17
The muser name no longer reflects the implementation, and will just serve to confuse. Bite the bullet now, and rename ourselves to reflect the actual implementation. Signed-off-by: John Levon <john.levon@nutanix.com> Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com> Reviewed-by: Swapnil Ingle <swapnil.ingle@nutanix.com>
2020-11-27add unit tests for DMA regions without file descriptorThanos Makatos1-1/+14
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
2020-11-27allow DMA regions without file descriptorThanos Makatos1-24/+22
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
2020-11-27add assertThanos Makatos1-0/+2
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
2020-11-24Api refactoring (#115)swapnili1-2/+2
API refactoring
2020-11-23fix printf argsThanos Makatos1-5/+6
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
2020-11-23don't pass -1 to fstatThanos Makatos1-0/+10
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
2020-11-23don't omit argument from printfThanos Makatos1-1/+1
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
2020-11-11Merge branch 'vfio-user'Thanos Makatos1-38/+210
2020-10-29support for live migration region and dirty page loggingThanos Makatos1-1/+122
This patch adds support for the live migration region and dirty page logging following VFIO. Live migration is NOT yet functional as handling accesses to the migration region is not yet implemented. Currenty the live migration region is fixed at index 9 simply for simplifying the implementation. Dirty page logging is simplified by requiring IOVA ranges to match exactly the entire IOVA range. Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
2020-10-05dma: add api to validate dma regionSwapnil Ingle1-0/+17
Signed-off-by: Swapnil Ingle <swapnil.ingle@nutanix.com>
2020-09-25remove extraneous newlineThanos Makatos1-1/+1
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
2020-08-12dma: Cope with dma being NULL in destructorFelipe Franciosi1-0/+4
On certain error paths, @dma will be NULL. This makes the destructor cope with it. Signed-off-by: Felipe Franciosi <felipe@nutanix.com>
2020-07-08fix assertionThanos Makatos1-1/+1
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
2020-07-08dma: notify device of DMA removal eventsThanos Makatos1-26/+50
This patch introduces a new device callback that is called when a DMA region is removed. The device must provide this callback and when the callback returns all references to the DMA regions must have been removed. This new functionality is necessary since a DMA region can be removed at any time, so the device must be given a chance to drop all references and perform the necessary cleanup. This patch also stores lm_ctx_t inside dma_controller_t so that we don't have to pass lm_ctx_t when calling dma_XXX functions. Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
2020-06-24don't blindly trust region_hintThanos Makatos1-0/+4
Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
generated by cgit v1.1 at 2024-07-18 07:00:39 +0000