Escali 4.4 manual 3.2 DET, Using detctl, Using detstat

Section: 2.2 SMC network devices

2.2.3.2 DET

Scali has developed a device called Direct Ethernet Transport (DET) to improve Ethernet performance. This device that bypasses the TCP/IP stack and uses raw Ethernet frames for sending messages. These devices are bondable over multiple Ethernets.

The /opt/scali/sbin/detctl command provides a means of creating and deleting DET devices. /opt/scali/bin/detstat can be used to obtain statistics on the devices.

2.2.3.3 Using detctl

detctl has the following syntax :

detctl -a[-q] [-c] <hca index> <device1> <device2> ...

-d[-q] <hca index> -l[-q]

Examples:

•Adding new DET devices temporarily with the detctl utility:

root# detctl -a 0 eth0 # creates a det0 device using eth0 as transport device. root# detctl -a 1 eth1 eth2 # creates a det1 device using eth1 and eth2 as aggregated transport devices.

•Removing DET devices with detctl:

root# detctl -d 0 # removes DET device 0 (det0) from the current configuration. root# detctl -d 1 # removes DET device 1 (det1) from the current configuration.

•Listing active DET devices

root# detctl -l # lists all DET devices currently configured.

Please note that aggregating devices usually requires a special switch configuration.

Both devices have the same Ethernet address (MAC), and so there must either be one VLAN for the eth1's and another for the eth2’s, or all the eth1's must be on one Ethernet switch, and all the eth2's on another switch.

Using detctl to add and remove devices is not permanent, as the contents of the /opt/scali/ kernel/scadet.conf configuration file takes presedence. The contents of this file has the following format:

# hca <hca index> <ethernet devices>

Permanent changes must be done by editing opt/scali/kernel/scadet.conf , e.g., to add permanently the example above add the following lines:

hca	0	eth0	eth2
hca	1	eth1

2.2.3.4 Using detstat

To gather transmission statistics (packets transmitted, received and lost) for a DET device use detstat. It can also be used to reset the statistics for DET devices.

detstat has the following syntax :

detstat [-r] [-a] <hca name> Examples:

•root# detstat det0 #listing statistics for the det0 device, if it exists.

•root# detstat -a #listsing statistics for all existing DET devices.

Scali MPI Connect Release 4.4 Users Guide 14