IBM SG24-5131-00 manual Special RS/6000 SP Topics

With reference to Figure 17 above, imagine two nodes, Node X and Node Y, running the same application. The nodes are connected by the switch and have locally-attached disks. On Node X’s disk resides a volume group containing the raw logical volume lv_X. Similarly, Node Y has lv_Y. For the sake of illustration, let us suppose that lv_X and lv_Y together constitute an Oracle Parallel Server database to which the application on each node makes I/O requests.

The application on Node X requests a piece of data in the database. After the node’s Virtual Memory Manager (VMM) determines that the data is not in memory, it talks not to the regular Logical Volume Manager (LVM), but rather to the VSD device driver. The VSD device driver is loaded as a kernel extension. Thus VSDs configured in the SP are known to the appropriate nodes at the kernel level.

The VSD device driver can fetch the data from one of three places:

1.From the VSD cache, if the data is still there from previous requests. VSD cache is shared by all VSDs configured on a node. Data is stored in 4KB blocks, a size optimized for Oracle Parallel Server. If your I/O patterns involve I/O operations larger than 4KB, we recommend disabling VSD cache, because its management becomes counterproductive.

2.From lv_X, in which case the VSD device driver exploits Node X’s normal LVM and Disk Device Driver (Disk DD) pathway to fetch the data.

3.From lv_Y, in which case the VSD device driver issues the request through the IP and Network Device Driver (Net DD) pathway to access Node Y. For performance, VSD uses its own stripped-down IP protocol. Once the request is passed up through Node Y’s Net DD and IP layers, Node Y’s VSD device driver accesses the data either from VSD cache or from lv_Y.

The VSD server node uses the buddy buffer to temporarily store data for I/O operations originating at a client node, and to handle requests that are greater than the IP message size. In contrast to the data in the cache buffer, the data in a buddy buffer is purged immediately after the I/O operation completes. Buddy buffers are used only when a shortage in the switch buffer pool occurs, or, on certain networks with small IP message sizes (for example, Ethernet). The maximum and minimum size for the buddy buffer must be defined when the VSD is created. For best performance, you must ensure that your buddy buffer limits accommodate your I/O transaction sizes to minimize the packetizing workload of the VSD protocol. Buddy buffers are discussed in detail in IBM Parallel System Support Programs for AIX Managing Shared Disks, SA22-7279.

Special RS/6000 SP Topics 191