Sun Microsystems 10GBE XFP Tuning for Maximum Performance on a Linux Platform, Net.ipv4.tcpsack =

Page 69

3.To set a particular variable, use the /usr/local/bin/nxge_config if_name set param_name value:

#/usr/local/bin/nxge_config eth4 set class_opt_ipv4_tcp 0xfe0

Tuning for Maximum Performance on a Linux Platform

The following tunings will improve the performance of the Sun x8 Express Dual 10 Gigabit Ethernet device driver on a system running the Linux operating system.

1.Create the conf file (for example, sysctl_nxge.conf) that will be called by the sysctl utility.

### IPV4 specific settings

#turns TCP timestamp support off, default 1, reduces CPU use net.ipv4.tcp_timestamps = 0

#turn SACK support off, default on systems with a VERY fast bus ->

#memory interface this is the big gainer

net.ipv4.tcp_sack = 0

#sets min/default/max TCP read buffer, default 4096 87380 174760 net.ipv4.tcp_rmem = 10000000 10000000 10000000

#sets min/pressure/max TCP write buffer, default 4096 16384 131072 net.ipv4.tcp_wmem = 10000000 10000000 10000000

#sets min/pressure/max TCP buffer space, default 31744 32256 32768 net.ipv4.tcp_mem = 10000000 10000000 10000000

###CORE settings (mostly for socket and UDP effect)

# maximum receive socket buffer size, default 131071 net.core.rmem_max = 524287

# maximum send socket buffer size, default 131071 net.core.wmem_max = 524287

# default receive socket buffer size, default 65535 net.core.rmem_default = 524287

# default send socket buffer size, default 65535 net.core.wmem_default = 524287

# maximum amount of option memory buffers, default 10240 net.core.optmem_max = 524287

# number of unprocessed input packets before kernel starts dropping

# them, default 300

net.core.netdev_max_backlog = 300000

Chapter 5 Configuring the nxge Device Driver Parameters 51

Image 69
Contents Sun Dual 10GbE XFP PCI Express Card User’s Guide Please Recycle Contents Configuring the nxge Device Driver Parameters Network ConfigurationConfiguring the Jumbo Frames Feature Configuring Link AggregationConfiguring VLANs Specifications Diagnostic SoftwareIndex Tables Page Figures Sun Dual 10GbE XFP PCI Express Card User’s Guide January EMC Declaration of ConformityXii Sun Dual 10GbE XFP PCI Express Card User’s Guide January How This Document Is Organized PrefaceUsing Unix Commands Typographic Conventions Shell PromptsRelated Documentation Documentation, Support, and TrainingThird-Party Web Sites Sun Welcomes Your Comments Page Product Overview ComponentsProduct Description Hardware and Software Requirements Features Sun Dual 10GbE XFP PCI Express Card User’s Guide January To Download the Driver on a Solaris Platform Installing and Setting Up the DriverUnpack the tar file Uncompress the gzipped tar fileFor Sparc systems, change to the following directory For x86 systems# /usr/sbin/pkgadd -d Select the packages you want to installTo discover the driver packages run the pkginfo command Verify that the nxge driver is installed on the systemTo remove the driver packages run the pkgrm command To Remove the Driver from a Solaris PlatformDownloading and Installing the Driver on a Linux Platform Login to your system# ifconfig -a grep eth # modprobe nxge To Remove the Driver From a Linux Platform # ethtool -i eth4 driver nxge version# rpm -e nxge-1.0-1 Sun Dual 10GbE XFP PCI Express Card User’s Guide January Installing the Adapter Installing the AdapterTo Install the Adapter Installing the Adapter Sun Dual 10GbE XFP PCI Express Card User’s Guide January Install the optical transceivers Installing an Optical TransceiverTo Install an Optical Transceiver Installing the Adapter To Verify the Hardware Installation Verifying the Hardware InstallationConnect the Ethernet cables Ok show-netsOk .properties Ok cd /pci@7c0/pci@0/pci@8/network@0,1Rebooting the System Ok boot -rOk device-end Configuring the Network Host Files Network Configuration# grep nxge /etc/pathtoinst # ifconfig nxge0 plumb ip-addressup# cat /etc/hosts To Set Up a 10-Gigabit Ethernet Port on a Diskless Client Edit the hosts file in the diskless client’s root directoryExport/root/client-name Network Configuration Find the root directory of the client system # grep client-name /etc/bootparamsOk boot /pci@7c0/pci@0/pci@8/network@0 Proceed with the Solaris Operating System installationShut down and halt the client system # shutdown -i0 -g0 -y# dladm show-dev Refer the boot-device-order image To Boot Over the Network on Solaris x86 and Linux SystemsSystem should reboot after saving the boot configuration Image pxe-mac-addr Press the F12 key to install the OS from the networkSun Dual 10GbE XFP PCI Express Card User’s Guide January Nxge Hardware and Software Overview Configuring the nxge Device Driver ParametersSetting nxge Driver Parameters on a Solaris Platform Setting Parameters Using the ndd UtilityNoninteractive and Interactive Modes To Specify Parameter Values Using the ndd Utility To Specify Device Instances for the ndd UtilityTo modify a parameter value, use the -setoption Setting Parameters Using the nxge.conf File # ndd /dev/nxge1 name to get/set ?Man prtconf Following parameters can be set using To Set Driver Parameters Using an nxge.conf FileLocate the path names and the associated instance numbers Set the parameters for the nxge devicesSun Dual 10GbE XFP PCI Express Card User’s Guide January Configuring the nxge Device Driver Parameters Class-opt-ipv4-tcp = 100 class-opt-ipv4-udp = Reboot the system Tuning for Maximum Performance on a Solaris Platform# reboot -r Save the nxge.conf fileTo Improve Performance on an AMD CPU Based Sun Platform Set ipipsqueuefanout=1 set ipsqueuesoftring=1Set ddimsixalloclimit=1 Configuring the nxge Device Driver Parameters To Set Parameters Using the ethtool Utility Setting Parameters on a Linux PlatformConfiguring the nxge Device Driver Parameters Following are some common parameters that can be changed # ethtool -k eth4 # ethtool -C eth4 rx-usecs # ethtool -c eth4# /usr/local/bin/nxgeconfig eth4 get classoptipv4udp # /usr/local/bin/nxgeconfig eth4 get# /usr/local/bin/nxgeconfig eth4 set classoptipv4tcp 0xfe0 Tuning for Maximum Performance on a Linux PlatformNet.ipv4.tcpsack = Net.core.netdevmaxbacklog =# sysctl -p /etc/sysctlnxge.conf Set up the sysctl utilityConfiguring the Jumbo Frames Feature Checking Jumbo Frames ConfigurationsJumbo Frames Overview # kstat nxge1 grep rdcpackets To Show the Driver Statistics in a Solaris Environment# kstat Nxge38001 Enabling Jumbo Frames in a Solaris EnvironmentBoot -r To Check Layer 2 Configuration# ifconfig nxge mtu Set the maximum MTU for maximum performanceTo Enable Jumbo Frames in a Linux Environment To Check Layer 3 ConfigurationEnsure that the nxge software is installed Enabling Jumbo Frames in a Linux Environment# ifconfig eth4 # ifconfig ethx mtuTo Show the Driver Statistics in a Linux Environment Set the MTU for maximum performanceConfiguring the Jumbo Frames Feature Sun Dual 10GbE XFP PCI Express Card User’s Guide January Configuring Link Aggregation Configuring Link Aggregation in a Solaris EnvironmentOverview of Link Aggregation To Configure Link Aggregation in a Solaris Environment # ifconfig aggr33 plumb Use the dladm show-aggr-scommand to display statistics# ifconfig aggr33 192.168.1.1/24 broadcast + up # dladm show-aggr -sTo Configure Bonding for Multiple nxge Interfaces Configuring Link Aggregation in a Linux EnvironmentLoad the bonding driver # dladm show-aggr -LConfigure bond0 interface Sun Dual 10GbE XFP PCI Express Card User’s Guide January Overview of VLANs Configuring VLANsVlan Configuring VLANs in a Solaris Environment 2Ethernet Tag Header Format To Configure Static VLANsTo Configure VLANs in a Linux Environment Configuring VLANs in a Linux Environment# ifconfig -a Ensure that the nxge module is loaded# /sbin/vconfig add eth2 Configure the nxge Vlan eth2 in this example# ifconfig eth2.5 xxx.xxx.xx.xxx up Insert the Vlan moduleConnectors SpecificationsTable A-1XFP Connector Characteristics Performance SpecificationsPower Requirements Physical CharacteristicsSun Dual 10GbE XFP PCI Express Card User’s Guide January SunVTS Diagnostic Software Diagnostic SoftwareUpdating SunVTS to Recognize the Adapter To Update SunVTS to Recognize the AdapterTo Use the netlbtest Using the SunVTS netlbtestSun Dual 10GbE XFP PCI Express Card User’s Guide January Index SymbolsSun Dual 10GbE XFP PCI Express Card User’s Guide January