Sun Microsystems 3.2 manual

The order of write operations must be maintained within a group. Therefore, these out of order requests must be stored in memory on the secondary host until the missing request comes in and completes.

The secondary host can store up to the hard-coded limit of 64 requests per group. Exceeding 64 stored requests stalls the primary host from issuing any more requests. This hard limit is applied only to the number of possible outstanding requests, not the size of their payload. For example, if the I/O consists of 4 Kbyte writes with six groups, the total memory requirement could be 4Kbyte x 6 x 64 = 1536 Kbyte. However, with an I/O size of 1 Mbyte, this requirement could rise to 1 Mbyte x 6 x 64 = 384 Mbyte. You need to be aware of this requirement to avoid exhausting the memory of the secondary host.

The most common symptom of secondary memory exhaustion is the volume sets changing to logging mode. The RPC requests fail when memory is low. To avoid the problem, change the number of asynchronous threads for some of the groups from the default of two to one. This forces the earlier version 3.1 behavior with no extra memory requirement on the secondary host.

The need for extra memory is most common with large I/O on a high latency link. When selecting groups to use a single thread, your best choices are those groups with a good network connection. Such groups are most likely to fill the secondary queue and they derive the least benefit from multiple threads.

Users upgrading from Sun StorEdge Availability Suite 3.1 software must be aware that the default number of asynchronous threads is two, not one. Out-of-order requests can occur and extra memory might be required on the secondary host.

Chapter 2 Remote Mirror Software Troubleshooting Tips 17