homi#

homi, Host-Orchestrated Multi-path I/O, opens a set of NVMe devices and holds them open until it is told to stop. It does no I/O of its own.

Its purpose is to provide the long-lived primary process for uPCIe (multi-process). The primary owns the DMA hugepages and the controller hardware state, and it must outlive every secondary attached to it. Running homi puts that responsibility in a process of its own, so the processes actually doing I/O can come and go freely as secondaries.

uPCIe (multi-process) is only available for the uPCIe backend and its GPU variants, and for the SPDK backend. homi is therefore useful where one of those is built: upcie, upcie-cuda and upcie-hip on Linux, and spdk on Linux and FreeBSD. On Windows it exits with -ENOTSUP, since no multi-process capable backend is available there. On other platforms it runs, but opening a device with a non-zero --homi-id fails with -ENOTSUP.

Usage: homi <command> [<args>]

Where <command> is one of:

  start            | Open the given devices and hold them open

See 'homi <command> --help' for the description of [<args>]

homi - Host-Orchestrated Multi-path I/O -- ver: {major: 0, minor: 7, patch: 5}

`start`: Hold devices open#

Usage: homi start [<uri>...] [<args>]

Open the given devices and hold them open
  
Positional arguments:

  [uri ...]                     ; Device URI e.g. '/dev/nvme0n1', '0000:01:00.1', '10.9.8.1:8888', '\\.\PhysicalDrive1'
  
Where <args> include:

  --homi-id NUM                 ; The HOMI identifier to share controllers under
  
With <args> for backend:

  [ --be STRING ]               ; xNVMe backend, e.g. 'spdk', 'libvfn', 'upcie'
  [ --host_heap_size NUM ]      ; For be=upcie, host DMA heap size in bytes
  [ --device_heap_size NUM ]    ; For be={upcie-cuda,upcie-hip}, GPU device heap size in bytes
  [ --help ]                    ; Show usage / help

See 'homi --help' for other commands

homi - Host-Orchestrated Multi-path I/O -- ver: {major: 0, minor: 7, patch: 5}

Opens every device given as a positional argument, then blocks until signalled.

Every process that should share these controllers must be started with the same id.

Roles are not assigned by the tool: whichever process claims a given --homi-id first becomes the primary (see Process model). Starting homi before any secondary is therefore what makes it the primary.

The heap is allocated once per process, on the first device open, so --host_heap_size covers every device homi holds rather than being per device.

Rather than the 1 GiB the backend would otherwise use, homi defaults to 16 MiB per device held. It needs only the admin queue and the sync queue pair that opening a device creates, and each of those carries a request pool costing 4 MiB, so 16 MiB per device is roughly double what is required. Every process in multi-process mode allocates a heap of its own, so a primary claiming the backend default would leave nothing in the hugepage pool for the secondaries it exists to serve:

homi start 0000:03:00.0 --be upcie --homi-id 1 --host_heap_size 134217728

If any device fails to open, homi closes the devices it has already opened and exits with an error, rather than holding a partial set open. Once all devices are open it reports that it has started, then waits for SIGINT (Ctrl+C) or SIGTERM. On either signal it closes all devices and exits.

Example: hold a single device open as the primary for --homi-id 1:

homi start 0000:03:00.0 --be upcie --homi-id 1

Example: hold several devices under the same --homi-id:

homi start 0000:03:00.0 0000:04:00.0 --be upcie --homi-id 1

While it runs, other processes attach as secondaries by passing the same identifier:

xnvme info 0000:03:00.0 --be upcie --homi-id 1

Backends#

homi works with any backend that advertises the multi-process capability, which is spdk, upcie, upcie-cuda and upcie-hip:

homi start 0000:03:00.0 --be spdk --homi-id 1

With upcie-cuda and upcie-hip, homi additionally caps the GPU device heap at 2 MiB rather than the backend default of 1 GiB. It never allocates data buffers, which is all the device heap is used for, so claiming the default would take VRAM away from the secondaries:

homi start 0000:03:00.0 --be upcie-cuda --homi-id 1

homi status is the exception: it reads uPCIe’s shared segment directly, so a primary started with --be spdk reads as absent rather than as running, and on a platform without uPCIe the subcommand refuses outright. The same applies to the systemd unit, whose readiness gate is that command.

homi#

start: Hold devices open#

Backends#

`start`: Hold devices open#