From Example 12-11 on page 427, the host adapter port  21:00:00:24:ff:7d:37:98  has been  zoned to two target ports on storage,  50:05:07:60:5e:8c:4b:53  and  50:05:07:60:5e:8c:4b:73 . The other host port also is detecting two other IBM FlashSystem  900 ports. Using the Fibre Channel switch management GUI, turning off and on ports causes pathing  loss from the server to the storage. The  dmesg  command can be used to show what occurs  when the server is disconnected from storage on a port, as shown in Example 12-12. Upon detecting a link down, there is a 30 seconds duration where the server awaits a  reconnect, followed by the multipathing device driver failing those paths. Eventually the ports  reported as blocked are removed, as shown in Example 12-13. When activating the switch port, the detection and recovery happen fairly rapidly. 428  Implementing IBM FlashSystem 900 Model AE3 ... [ 1162.268562] lpfc 0000:13:00.0: 0:1303 Link Up Event x3 received Data: x3 x0 x80  x0 x0 x0 0 [ 1162.435843] nvme nvme0: NVME-FC{0}: controller connect complete [ 1162.435893] nvme nvme0: NVME-FC{0}: new ctrl: NQN  "nqn.2014-08.org.nvmexpress.discovery" [ 1162.436642] nvme nvme0: Removing ctrl: NQN  "nqn.2014-08.org.nvmexpress.discovery" [ 1162.443044] nvme nvme1: NVME-FC{1}: controller connect complete [ 1162.443432] nvme nvme1: NVME-FC{1}: new ctrl: NQN  "nqn.2014-08.org.nvmexpress.discovery" [ 1162.444364] nvme nvme1: Removing ctrl: NQN  "nqn.2014-08.org.nvmexpress.discovery" [ 1162.464550] nvme nvme0: NVME-FC{0}: controller connect complete Example 12-13 Turning on an NVMe switch port messages ... [ 912.194492] lpfc 0000:13:00.0: 0:1305 Link Down Event x2 received Data: x2 x20  x800110 x0 x0 [ 912.194636] nvme nvme0: NVME-FC{0}: controller connectivity lost. Awaiting  Reconnect [ 912.194648] nvme nvme1: NVME-FC{1}: controller connectivity lost. Awaiting  Reconnect [ 927.180604] device-mapper: multipath: Failing path 259:0. [ 927.187216] device-mapper: multipath: Failing path 259:1. [ 942.221350] rport-4:0-1: blocked FC remote port time out: removing rport [ 942.221433] rport-4:0-0: blocked FC remote port time out: removing rport [ 972.204286] nvme nvme1: NVME-FC{1}: dev_loss_tmo (60) expired while waiting for  remoteport connectivity. [ 972.204298] nvme nvme0: NVME-FC{0}: dev_loss_tmo (60) expired while waiting for  remoteport connectivity. [ 972.204306] nvme nvme1: Removing ctrl: NQN  "nqn.2017-12.com.ibm:nvme:mt:9840:guid:5005076061D31310:cid:000002006F362044" [ 972.204322] nvme nvme0: Removing ctrl: NQN  "nqn.2017-12.com.ibm:nvme:mt:9840:guid:5005076061D31310:cid:000002006F362044" ... Example 12-12 Turning off an NVMe switch port messages 12.6.13 NVMe behavior during Link Down/Up