Registering memory#
Memory from xnvme_buf_alloc() is usable as command payload on every backend.
Memory the application allocated itself becomes usable once it is registered
with xnvme_buf_register(), and is given back with xnvme_buf_unregister().
What registering does differs per backend: the library may program an IOMMU on
the caller’s behalf, export the memory as a dma-buf, hand it to SPDK, or do
nothing at all.
size_t align;
xnvme_buf_get_alignment(dev, &align);
nbytes = (nbytes + align - 1) & ~(align - 1);
buf = aligned_alloc(align, nbytes);
xnvme_buf_register(dev, buf, nbytes);
/* ... commands on buf ... */
xnvme_buf_unregister(dev, buf);
The rules are the same on every backend that registers memory:
Each register is given back by one unregister.
Registering a range within a registered range is counted on it. Any other overlap fails with
-EBUSY.Registering memory from
xnvme_buf_alloc()does nothing.vaddrandnbytesare multiples of the alignment fromxnvme_buf_get_alignment().
Per backend#
Memory interface |
Takes |
Registering does |
Alignment |
|---|---|---|---|
|
Any host memory |
Nothing |
|
|
Any host memory |
Nothing |
|
|
Host and GPU memory |
Host: nothing. GPU: a dma-buf export, registered with io_uring |
Page |
|
Host memory |
Page |
|
|
2 MiB |
||
|
Any host memory |
2 MiB |
|
|
Any host memory |
Nothing |
|
|
Nothing, |
||
|
GPU memory |
GPU page |
|
|
Nothing, |
Alignment of the I/O path#
xnvme_buf_get_alignment() gives the larger of what registering needs and what
the I/O path needs of a payload. Where nothing is registered, only the I/O path
is left:
Buffered I/O, the OS copies the payload, so any alignment serves, and
sizeof(void *)is given.Direct I/O,
opts.direct, the OS refuses a payload that is not aligned to what the device or file system needs. On Linux, that isstx_dio_mem_alignfromstatx()withSTATX_DIOALIGN. A page satisfies it, so the page size is given.