FAQ

Why does /dev/nvmeXnY persist or change controller names after hotplug or reboot?

Question

When an NVMe controller is hot-unplugged or rebooted, why does the block device (e.g., /dev/nvme3n1) still appear in nvme list even after controller nvme3 is removed? Why do controller numbers change across nvme list and nvme list-subsys?

Answer

This is expected behavior and by design in the Linux NVMe driver:

1. Controller Numbers Are Not Persistent: The Linux kernel does not guarantee that controller instances (nvme0, nvme1, etc.) or block device nodes (/dev/nvmeXnY) retain the same index across reboot or hot-unplug/re-plug (PCIe hotplug/DLPAR). When a controller is re-discovered, it is assigned the next available dynamic instance number (e.g., nvme3 becomes nvme4).

2. Multipath Namespace Heads Follow the Subsystem, Not the Controller: When NVMe multipath is active, block devices shown in nvme list (/dev/nvmeSnN and generic char devices /dev/ngSnN) are multipath namespace heads. They are indexed by the subsystem instance (nvme-subsysS), not by a single controller:

   • /dev/nvme3n1 belongs to nvme-subsys3.
   • /dev/nvme2n1 belongs to nvme-subsys2.

   If a namespace is shared across multiple controllers (paths) within a subsystem, removing one controller (e.g., nvme3) leaves other controllers active (e.g., nvme1). As long as at least one path to nvme-subsys3 remains live, the subsystem and its namespace heads remain intact and their device nodes do not disappear.

Best Practices

• Do not rely on persistent /dev/nvmeX or /dev/nvmeXnY names in scripts, automation, or test cases.
• Correlating Topology: Run nvme list -v or nvme list-subsys -v to view subsystems, controllers, and namespaces organized together hierarchically.
• Stable Device Addressing: Identify and mount devices using persistent identifiers such as:
  • Persistent symlinks under /dev/disk/by-id/ (e.g., nvme-eui.*, nvme-nvme.*)
  • Subsystem NQN (nvme-subsysX - NQN=...)