Python

Bindings to the C API from Python is provided via an interface implemented using ctypes and distributed via pypi.

The Python interface is a work-in-progress, provided as a preview of what can be done when integrating with Python. See, section TODO for a list of known shortcomings and areas to improve. Your input on these are most welcome along with pull-requests.

ctypes “sugar”

The ctypes bindings provides access to the xNVMe C API via ctypes. If you have used ctypes before, then you can go right ahead and use these. The only thing added as an augmentation of what ctypes provides is an improvement of the Dynamic loader logic:

• library-loading is extended with use of pkg-config

• library-loading is done implicitly upon import

• library-loading can be checked via: - xnvme.ctypes_bindings.is_loaded, check the bool - xnvme.ctypes_bindings.guard_unloadable(), exit when not loaded

A convenient aspect of the bindings are that Python ctypes comes with CPython, that is, with the reference implementation of the Python intepreter. When using the ctypes bindings, the only other runtime dependency is pytest.

FAQ

Attribute Error

This:

xnvme.xnvme_dev_get_geo(dev).lba_nbytes

Gives the error:

AttributeError: 'LP_xnvme_geo' object has no attribute 'lba_nbytes'

One might expect that accessing the above would be possible. However, what the error-message conveys is that the above accesses a pointer, not the object, and thus, the pointer does not have the attribute lba_nbytes.

Thus, instead do:

xnvme.xnvme_dev_get_geo(dev).contents.lba_nbytes

This is the Ctypes approach to dereferencing pointers.

Install

Before installing, then ensure that the following system dependencies are met:

• Python 3.6+

• xNVMe installed on your system

• pkg-config installed and able to locate xNVMe on your system

See, the Building xNVMe on how to build and install xNVMe. With the above then you can go ahead and install the bindings using pip:

# install: for the current user
python3 -m pip install xnvme --user

For a description of building and installing the package from source, then see the Development section.

Note

The xNVMe Python package comes with pytests, thus, you can verify your installation by running: python3 -m pytest --pyargs xnvme.ctypes_mapping.

Usage

See the files in bin/*. At the time of writing this, there were three examples, two of which require no arguments, these are run like so:

# Enumerate devices on the system
./bin/xpy_enumerate

# Print library-information
./bin/xpy_libconf

The third opens and queries a device for information, and thus requires a device as well as permissions, on a Linux system, running it could look something like this:

sudo -E ./bin/xpy_dev_open --uri /dev/nvme0n1

Note

In the above example sudo -E is used. The sudo is there to ensure permission to /dev/nvme0n1, then -E is there such that the xNVMe package installed as --user is available.

Development

A couple of notes on the idiosyncrasies of the implementations.

ctypes

The bindings are generated using the ctypeslib2 tool, patched (injecting an extension to the library-loader), and then formatted with black.

The build-requirements are installable via requirements.txt:

python3 -m pip install -r requirements.txt --user

Furthermore, clang is needed on the system:

# Debian
sudo apt-get install libclang-dev

# Alpine
sudo apk add clang-libs

A Makefile is available for common tasks, run:

make help

To see what it provides / common-tasks during development. For example:

make build uninstall install test

The above make invocation will generate the ctypes-mapping via clang2py, then patch the mappings using the auxilary scripts aux/patch_ctypes_mapping.py, adjust the style according to the conventions of black, create a Python sdist package, install the package, and finally run the pytests.

CAVEAT: the mappings produced by clang2py aren’t stable. That is, the ordering in which classes are emitted can change from each invocation of the tool.

TODO

As mentioned earlier, then the Python language bindings are a work in progress, the following are mixture of notes for improvment along with things to be aware of when using the Python language bindings.

• Explore how to distribute the xNVMe source on pypi

  • Should provide the source-archive of xNVMe

  • Should provide means of building the library along with the Python package

  • Should provide a means of making the library available to the Python language bindings

  • See one approach to explore in the mention on mesonpep517

• Explore using mesonpep517 for the bindings

  • https://pypi.org/project/mesonpep517/

  • https://github.com/mesonbuild/meson/issues/7863

  • https://thiblahute.gitlab.io/mesonpep517/

• Re-consider the API-guard capi.guard_unloadable().

• The package-readme python/README.rst is lacking in proper description and pointers to information. This should be improved.

• testing The bindings have only been tested on Linux and macOS

  • Add testing on Windows

  • Add testing on FreeBSD

• RECONSIDER: The auto-generated ctypes-mapping has prefixes for e.g. union_ and struct_, the patcher removes these. This works for the xNVMe C API since there are no collisions, however, in the general case it would break. So, reconsider which is the preferable form for a “raw C API mapping”.

• ctypes_mapping: The bit-fields and nested structs have cumbersome accessors in Python, this could be improved by modifying the clang2py / ctypeslib2

• ctypes_mapping: The generated bindings are not stable, that is, the output emitted from clang2py changes order of the generated items. This would be nice to fix by submitting a PR to the ctypeslib2.