Storage area network Node canister FlashCopy considerations For IBM FS9100 node canister hardware, the possible cause of the 1920 errors might be from  a heavily loaded secondary or primary system. If this condition persists, a 1920 error might be  posted. Global Mirror needs to synchronize its I/O processing across all nodes in the system to  ensure data consistency. If any node is running out of CPU, it can affect all relationships. So  check the CPU cores usage statistic. If it looks higher when there is a performance problem,  then running out of CPU bandwidth might be causing the problem. Of course, CPU usage  goes up when the IOPS going through a node goes up, so if the workload increases, you  would expect to see CPU usage increase. If there is an increase in CPU usage on the secondary system but no increase in IOPS, and  volume write latency increases too, it is likely that the increase in CPU usage has caused the  increased volume write latency. In that case, try to work out what might have caused the  increase in CPU usage (for example, starting many FlashCopy mappings). Consider moving  that activity to a time with less workload. If there is an increase in both CPU usage and IOPS,  and the CPU usage is close to 100%, then that node might be overloaded. A  Port-to-local  node send queue time  value higher than 0.2 ms often denotes CPU cores overloading. In a primary system, if it is sufficiently busy, the write ordering detection in Global Mirror can  delay writes enough to reach a latency of  gmmaxhostdelay  and cause a 1920 error. Stopping  replication potentially lowers CPU usage, and also lowers the opportunities for each I/O to be  delayed by slow scheduling on a busy system. Solve overloaded nodes by upgrading them to newer, faster hardware if possible, or by  adding more I/O groups/control enclosures (or systems) to spread the workload over more  resources. Issues and congestions both in local and remote SANs can lead to 1920 errors. The  Port to  local node send response time  is the key statistic to investigate on. It captures the round-trip  time between nodes in the same system. Anything over 1.0 ms is surprisingly high, and will  cause high secondary volume write response time. Values greater than 1 ms on primary  system will cause an impact on write latency to Global Mirror primary volumes of 3 ms or  more.  If you have checked CPU utilization on all the nodes, and it has not gotten near 100%, a high  Port to local node send response time  means that there is fabric congestion or a slow-draining  Fibre Channel device.  A good indicator of SAN congestion is the  Zero Buffer Credit Percentage  on the port statistics  (see “Buffer credits” on page 186 for more information on Buffer Credit). If any port is seeing  over 10% zero buffer credits, that is definitely going to cause a problem for all I/O, not just  Global Mirror writes. Values from 1 - 10% are moderately high and might contribute to  performance issues. For both primary and secondary systems, congestion on the fabric from other slow-draining  devices becomes much less of an issue when only dedicated ports are used for node-to-node  traffic within the system. However, this only really becomes an option on systems with more  than four ports per node. Use port masking to segment your ports. Check that FlashCopy mappings are in the  prepared  state. Check whether the Global Mirror  target volumes are the sources of a FlashCopy mapping and whether that mapping was in the  prepared  state for an extended time. Chapter 6. Copy services  211