Sun Microsystems 10GBE XFP instruction

Sun™ Dual 10GbE XFP PCI Express Card User’s Guide

Sun Microsystems, Inc.

www.sun.com

Part No. 820-0184-13

January 2009, Revision A

Submit comments about this document at: http://www.sun.com/hwdocs/feedback

Image 1

Sun Microsystems 10GBE XFP manual Sun Dual 10GbE XFP PCI Express Card User’s Guide

Contents

Sun Dual 10GbE XFP PCI Express Card User’s Guide Please Recycle Contents Configuring the nxge Device Driver Parameters Network Configuration Configuring Link Aggregation Configuring the Jumbo Frames FeatureConfiguring VLANs Diagnostic Software SpecificationsIndex Tables Page Figures Sun Dual 10GbE XFP PCI Express Card User’s Guide January EMC Declaration of Conformity Xii Sun Dual 10GbE XFP PCI Express Card User’s Guide January How This Document Is Organized Preface Using Unix Commands Typographic Conventions Shell Prompts Documentation, Support, and Training Related DocumentationThird-Party Web Sites Sun Welcomes Your Comments Page Components Product OverviewProduct Description Hardware and Software Requirements Features Sun Dual 10GbE XFP PCI Express Card User’s Guide January To Download the Driver on a Solaris Platform Installing and Setting Up the Driver Unpack the tar file Uncompress the gzipped tar fileFor Sparc systems, change to the following directory For x86 systems # /usr/sbin/pkgadd -d Select the packages you want to install To discover the driver packages run the pkginfo command Verify that the nxge driver is installed on the systemTo remove the driver packages run the pkgrm command To Remove the Driver from a Solaris Platform Login to your system Downloading and Installing the Driver on a Linux Platform# ifconfig -a grep eth # modprobe nxge # ethtool -i eth4 driver nxge version To Remove the Driver From a Linux Platform# rpm -e nxge-1.0-1 Sun Dual 10GbE XFP PCI Express Card User’s Guide January Installing the Adapter Installing the Adapter To Install the Adapter Installing the Adapter Sun Dual 10GbE XFP PCI Express Card User’s Guide January Install the optical transceivers Installing an Optical Transceiver To Install an Optical Transceiver Installing the Adapter To Verify the Hardware Installation Verifying the Hardware InstallationConnect the Ethernet cables Ok show-nets Ok .properties Ok cd /pci@7c0/pci@0/pci@8/network@0,1 Ok boot -r Rebooting the SystemOk device-end Configuring the Network Host Files Network Configuration # grep nxge /etc/pathtoinst # ifconfig nxge0 plumb ip-addressup # cat /etc/hosts Edit the hosts file in the diskless client’s root directory To Set Up a 10-Gigabit Ethernet Port on a Diskless ClientExport/root/client-name Network Configuration Find the root directory of the client system # grep client-name /etc/bootparams Ok boot /pci@7c0/pci@0/pci@8/network@0 Proceed with the Solaris Operating System installationShut down and halt the client system # shutdown -i0 -g0 -y # dladm show-dev Refer the boot-device-order image To Boot Over the Network on Solaris x86 and Linux Systems System should reboot after saving the boot configuration Image pxe-mac-addr Press the F12 key to install the OS from the network Sun Dual 10GbE XFP PCI Express Card User’s Guide January Nxge Hardware and Software Overview Configuring the nxge Device Driver Parameters Setting Parameters Using the ndd Utility Setting nxge Driver Parameters on a Solaris PlatformNoninteractive and Interactive Modes To Specify Device Instances for the ndd Utility To Specify Parameter Values Using the ndd UtilityTo modify a parameter value, use the -setoption # ndd /dev/nxge1 name to get/set ? Setting Parameters Using the nxge.conf FileMan prtconf Following parameters can be set using To Set Driver Parameters Using an nxge.conf FileLocate the path names and the associated instance numbers Set the parameters for the nxge devices Sun Dual 10GbE XFP PCI Express Card User’s Guide January Configuring the nxge Device Driver Parameters Class-opt-ipv4-tcp = 100 class-opt-ipv4-udp = Reboot the system Tuning for Maximum Performance on a Solaris Platform# reboot -r Save the nxge.conf file Set ipipsqueuefanout=1 set ipsqueuesoftring=1 To Improve Performance on an AMD CPU Based Sun PlatformSet ddimsixalloclimit=1 Configuring the nxge Device Driver Parameters To Set Parameters Using the ethtool Utility Setting Parameters on a Linux Platform Configuring the nxge Device Driver Parameters Following are some common parameters that can be changed # ethtool -k eth4 # ethtool -C eth4 rx-usecs # ethtool -c eth4 # /usr/local/bin/nxgeconfig eth4 get classoptipv4udp # /usr/local/bin/nxgeconfig eth4 get # /usr/local/bin/nxgeconfig eth4 set classoptipv4tcp 0xfe0 Tuning for Maximum Performance on a Linux PlatformNet.ipv4.tcpsack = Net.core.netdevmaxbacklog = # sysctl -p /etc/sysctlnxge.conf Set up the sysctl utility Checking Jumbo Frames Configurations Configuring the Jumbo Frames FeatureJumbo Frames Overview # kstat nxge1 grep rdcpackets To Show the Driver Statistics in a Solaris Environment # kstat Nxge38001 Enabling Jumbo Frames in a Solaris Environment Boot -r To Check Layer 2 Configuration# ifconfig nxge mtu Set the maximum MTU for maximum performance To Enable Jumbo Frames in a Linux Environment To Check Layer 3 ConfigurationEnsure that the nxge software is installed Enabling Jumbo Frames in a Linux Environment # ifconfig eth4 # ifconfig ethx mtuTo Show the Driver Statistics in a Linux Environment Set the MTU for maximum performance Configuring the Jumbo Frames Feature Sun Dual 10GbE XFP PCI Express Card User’s Guide January Configuring Link Aggregation in a Solaris Environment Configuring Link AggregationOverview of Link Aggregation To Configure Link Aggregation in a Solaris Environment # ifconfig aggr33 plumb Use the dladm show-aggr-scommand to display statistics# ifconfig aggr33 192.168.1.1/24 broadcast + up # dladm show-aggr -s To Configure Bonding for Multiple nxge Interfaces Configuring Link Aggregation in a Linux EnvironmentLoad the bonding driver # dladm show-aggr -L Configure bond0 interface Sun Dual 10GbE XFP PCI Express Card User’s Guide January Overview of VLANs Configuring VLANs Vlan Configuring VLANs in a Solaris Environment 2Ethernet Tag Header Format To Configure Static VLANs To Configure VLANs in a Linux Environment Configuring VLANs in a Linux Environment# ifconfig -a Ensure that the nxge module is loaded # /sbin/vconfig add eth2 Configure the nxge Vlan eth2 in this example# ifconfig eth2.5 xxx.xxx.xx.xxx up Insert the Vlan module Connectors Specifications Table A-1XFP Connector Characteristics Performance Specifications Power Requirements Physical Characteristics Sun Dual 10GbE XFP PCI Express Card User’s Guide January SunVTS Diagnostic Software Diagnostic Software Updating SunVTS to Recognize the Adapter To Update SunVTS to Recognize the Adapter To Use the netlbtest Using the SunVTS netlbtest Sun Dual 10GbE XFP PCI Express Card User’s Guide January Index Symbols Sun Dual 10GbE XFP PCI Express Card User’s Guide January