HP OneView 197

Part VI Troubleshooting

The chapters in this part include information you can use when troubleshooting issues in your data center, and information about restoring the appliance from a backup file in the event of a catastrophic failure.

Contents

Abstract Page Page Page Page 10 Quick Start: Adding a network to an existing appliance environment 11 Quick Start: Adding an enclosure and connecting its server blades to 12 Quick Start: Configuring an enclosure and server blade for Direct attach to 16 Managing interconnects, logical interconnects, and logical interconnect Page Page Page A Step by step: Configuring an example data center using HP OneView Page Page Page Storage Power and cooling Network Management software One tool and one data set—oneview •Provision the data center (page 16) •Manage and maintain firmware and configuration changes (page 20) •Monitor the data center and respond to issues (page 20) The solution also provides core enterprise management capabilities, including: 1.2.1 Server profiles •Directly assign the server profile to the enclosure bay •Copy the server profile and assign the copy to the enclosure bay 1.2.2 Groups, templates, and sets •Simplify the distribution of configuration changes across your data center Types of groups and sets Define configurations for specific environments For example, to build multiple external web servers: Flexibility in design and deployment 1.2.3 Streamlined process for bringing hardware under management 1.2.4Operating system deployment 1.3.1 Simplified firmware management Set a firmware •Detect when a resource does not comply with the firmware baseline •Identify firmware compatibility issues •Update firmware for an entire enclosure in minutes Isolated management network Automatic configuration for monitoring Agentless and out-of-bandmanagement Management from other platforms using the REST APIs and the SCMB Monitoring the environment and responding to issues 1.4.1 Data center environmental management •Thermal data visualization in 3D •Power delivery infrastructure representation •Physical asset location in 3D 1.4.2 Resource utilization monitoring 1.4.4Hardware and firmware inventory information •Summary and detailed views of firmware bundle contents One encrypted backup file for both the appliance and its database Flexible scheduling and an open interface for backup operations •Schedule a backup process from outside the appliance RBAC Single •Audit logging for all user actions •Use of certificates for authentication over SSL (Secure Sockets Layer) •A firewall that allows traffic on specific ports and blocks all unused ports User interface—efficiencyand simplicity by design For more information about the UI, see REST APIs—automationand integration The REST APIs: Provide an Onboard Administrator HP Integrated Lights-Out HP Insight Control server provisioning •Automating standard workflows and troubleshooting steps •Connecting to service desks Supported networks •Ethernet for data networks Logical interconnects •Uplinks to data center networks as mapped by their uplink sets •Downlinks to the servers Network sets High level overview Server resources Network provisioning resources Network resources Appliance resources The following sections introduce the resources shown in Figure 1 (page 29) •Basic server identification information •Firmware versions •BIOS settings •Boot order •The network or network set to which the server is to be connected •Exactly one server profile resource •Exactly one connection template resource network •Zero or more server profiles •Zero or more servers of the type defined by that server hardware type Zero or one View inventory data Power on or power off the server Page Page •Zero or more interconnects An interconnect has the following types of ports: In the resource model: Page •Zero or more logical interconnects •Zero or more enclosure groups uplink sets A logical interconnect is a single entity for multiple physical interconnects •An uplink set cannot include a network set •A network can be a member of one uplink set only •An uplink set can contain only one Fibre Channel network •An uplink set can contain multiple Ethernet networks •You cannot assign two instances of the same network to the same physical port A network represents a Fibre Channel or Ethernet network in the data center •Zero or more connections •Zero or one uplink set per logical interconnect •For Ethernet networks only, zero or more network sets •A network set can contain zero or more Ethernet networks Page 2.19.1 Data centers In the appliance, you can: View a 3D model of the data center layout that includes a •View temperature history data •More easily locate specific devices for hands-onservicing 2.19.3 Power delivery devices 2.19.4 Unmanaged devices Page Page Page Operating-system-level A setting that enables an authorized support representative to obtain a You can either enable or disable access with this setting ◦Management LAN ◦Production LAN Page 3.5.1 Specifying user accounts and roles The appliance provides the following roles: The Server administrator cannot manage user accounts The Network administrator cannot manage user accounts The Backup administrator cannot restore the appliance from a backup file A breakdown of an audit entry follows: Example 1 Sample audit entries: user login and logout “Algorithms for securing the appliance” (page 54) 3.9.1 Overview The certificate has the following boxes: •Common Name (CN) 3.9.2.1 Verifying a certificate 3.9.2.2 Downloading and importing a self-signedcertificate 3.9.3 Using a certificate authority 1.You generate a CSR (certificate signing request) 2.You copy the CSR and submit it to the CA, as instructed by the CA 3.The CA authenticates the requestor 4.The CA sends the certificate to you, as stipulated by the CA 5.You import the certificate “Managing certificates from a browser” (page 50) Table 1 Required ports Typical legitimate uses for access to the console are: •Troubleshooting network configuration issues •Resetting an appliance administrator password 3.13.1 Enabling or disabling authorized services access 3.13.2 Restricting console access Page Page For general information about browser use, see the following topics: •“Supported browsers” (page 57) •“Required plug-insand settings” (page 57) •“Browser best practices for a secure environment” (page 52) •“Commonly used browser features and settings” (page 57) 4.1.4 Set the browser for US or metric units of measurement Table 3 Set US or metric units of measurement Figure 2 Screen topography icon to expand the menu view selector or the icon Activity control Help control Enclosures Table 4 UI buttons details pane Figure 5 Filters sidebar 1Pin control: Expands or hides the Filters sidebar Help 4.8.1 Status and severity icons 4.8.2 User control icons 4.8.3 Informational icons The Map view was designed to be highly interactive and useful even at scale To open the relationship view for a resource, do one of the following: •Select Map from the view selector •Select the icon 1Z34AB7890 Map ! A, Slot A, Slot 4.Enter a search term in the Search box. Note the following: •Search terms are case insensitive •Cutting and pasting text into the search box is supported •Wildcard characters, such as the asterisk (*), are not supported •Enclosing a search phrase in double quotation marks is not supported “Search help topics” (page 66) Advanced searching and filtering with properties TIP: “Icon descriptions” (page 62) Figure 10 Filter resource instances by their health status Server Hardware All statuses 4.14.1 Reset the health status view Page For general information about REST APIs, see the following topics: •“Resource operations” (page 71) •“Return codes” (page 72) •“URI format” (page 72) •“Resource model format” (page 72) ErrorMessage https: {appl} /rest resource category /rest/version X-API-Version X-API-Version:3 POST /rest/server-profiles 202 Accepted http://www.hp.com/go/oneviewcommunity The online help documents both the UI and the REST APIs, and includes: •Overviews of the appliance and its features •Descriptions of resources and UI screens •Quick-startinstructions for bringing your data center under management •Step-by-stepinstructions for using the UI to perform tasks Page Page Page “Understanding the security features of the appliance” (page 45) 7.2.1 Choosing an Administrator password 7.2.2 Choosing the LAN for the appliance 7.2.3 Choosing whether or not to enable support access “Managing the appliance settings” (page 155) public 7.2.5 Choosing a security certificate policy “Managing SSL certificates” (page 157) “Appliance access over SSL” (page 50) 7.2.6 Determining roles and restrictions for authorized users 7.2.10 Reviewing your firewall access Consider the following information when choosing resource names: To minimize the need for name changes and to make ◦The purpose of the resource. For example: prod for production network resources dev for development network resources testUS_left •The appliance provides default names for many resources. For example: Encl enclosure_name -LI 7.5.2Planning for high availability 7.5.3 Location of the appliance 7.5.4 Separate networks for data and management HP recommends having separate networks for management and data 7.5.5 Time clocks and NTP Page Appliance When you install an appliance update, the appliance is taken offline Enclosures Interconnects and logical interconnects Change the name of the uplink set in the logical interconnect “Networks” (page 85) Server profiles and server hardware “Interconnects and logical interconnects” (page 85) 8.2.1 Adding a network •Networks. Add the network You might need to update multiple logical interconnects “Quick Start: Adding a network to an existing appliance environment” (page 95) 8.2.2 Adding an enclosure •Enclosures. Add the enclosure Page Page Page •Changing the Administrator password “Configure the environment for the first time” (page 91) “Determining your backup policy” (page 80) “Security planning” (page 79) Optional: Local authentication Directory-basedauthentication Ethernet Fibre Channel Page Page Page Page Page Page Page “Managing networks and network resources” (page 115) See for prerequisites and preparation you must complete before you add a server Process “Provisioning eight host servers for VMware vSphere Auto Deploy” (page 238) Page Page Page Page Page This quick start describes the process for adding a rack mount server “Server hardware features supported by the appliance” (page 111) •The server is connected to a live power source Page Page Page •A server profile enables management of your server hardware •Get information about (read) a server profile •Create and apply a server profile •Edit the BIOS settings of a server profile Page Page Page 15.2.1Roles 15.2.2Tasks •Add a Fibre Channel SAN •Delete a Fibre Channel SAN •Edit a Fibre Channel SAN configuration •Providing redundant physical links to the data center networks 15.4.1About Fibre Channel networks You can use Fibre Channel networks to connect to storage devices 15.4.1.1Fibre Channel network types Fabric attach Figure 11 Direct attach and Fabric attach Fibre Channel networks 15.4.1.2 Fabric attach Fibre Channel networks 15.4.1.3 Direct attach Fibre Channel networks 15.4.1.4 Fibre Channel networks and FCoE 15.4.2 About Ethernet networks You use Ethernet networks for data networks Ethernet networks and network sets 15.4.2.1 Ethernet networks and VLAN IDs Network set details •Network sets are supported for use in server profiles only •All networks in a network set must be Ethernet networks •All networks in a network set must be configured in the same appliance •A network can be a member of multiple network sets 16.1.1Roles 16.1.2Tasks 16.1.3About interconnects “About logical interconnects” (page 123) You can update interconnect firmware using an SPP (Service Pack for ProLiant) Connectivity and synchronization with the appliance About unsupported interconnects critical 16.2.1Roles 16.2.2Tasks 16.2.3 About logical interconnects Uplink sets For each logical interconnect: Stacking modes Stacking health Adding a logical interconnect Deleting a logical interconnect 16.2.4 About logical interconnect groups •All interconnects in an enclosure are members of the logical interconnect •Horizontally adjacent interconnects must be the same interconnect type •Existing interconnect types in their existing positions •All physical uplink ports disabled except for stacking links 16.2.5 About SNMP settings Update the logical interconnect configuration from the logical interconnect group Consistent Inconsistent with group 1.From the Logical Interconnects screen, select Actions→Update from group 16.2.7 Configure a port to monitor network traffic 16.2.8Learning more Page Page •Add an enclosure to manage its contents •Remove an enclosure from management •Bring an unmanaged enclosure under management •Claim an enclosure currently being managed by another appliance •Forcibly remove an enclosure if a remove action fails Page Page “Updating the appliance” (page 163) www.hp.com/go/spp Unsupported firmware for firmware bundles Unsupported firmware for enclosures When adding an enclosure, the appliance: •Updates the OA firmware automatically, if below the required minimum •Updates the iLO firmware automatically, if below the required minimum Unsupported firmware for server profiles Page •“Troubleshooting firmware bundles” (page 210) 19.1.1Roles 19.1.2Tasks 19.1.3About power delivery devices 19.1.4 About racks 19.2.1Roles 19.2.2Tasks •Add a data center •Edit a data center •Manipulate the view of a data center visualization 19.2.4Learning more •Minimum required privileges: Infrastructure administrator •Add a user with local authentication •Add a user with directory-basedauthentication •Add a group with directory-basedauthentication •Designate user privileges Table 5 Appliance role types Use Table 6 Action privileges for user roles Table 6 Action privileges for user roles (continued) Page •Enters their name (typically, the Common-Nameattribute, CN) •Enters the password for the group •Specifying the LDAPS (LDAP over SSL) communication port You can: •Allow local logins only, which is the default Page Page http://www.hp.com/go/dataprotector HP provides REST APIs for integration with enterprise backup/restore products •Only one backup operation can run at a time •You do not need to wait for tasks to stop before creating a backup file •Perform a backup operation before and after updating the appliance firmware Non-data •Log files (except the Audit log file) •Appliance network configuration •First time setup configuration files •Firmware bundles Page •“How the appliance handles an unexpected shutdown” (page 153) •“What to do when an appliance restarts” (page 154) For information about how to shut down or restart the appliance, see: •“Shut down the appliance” (page 154) •“Restart the appliance” (page 154) Appliance recovery operations When the appliance restarts, it performs the following operations: Appliance recovery during a firmware update of a managed resource 22.1.2.1What to do when an appliance restarts 22.1.3 Shut down the appliance Restarting the appliance 1.From the Settings screen, select Actions→Restart 2.Select Yes, restart in the dialog box 3.Log in when the login screen reappears 22.2.1Roles 1.Install the host operating system on the server 2.Install the SNMP subsystem on the server 22.2.4Learning more •“Planning the appliance configuration” (page 82) •“Managing appliance availability” (page 153) 22.4.1.1Roles 22.4.1.2Tasks 22.4.2 Managing SSL certificates 22.4.2.1Roles 22.4.2.2Tasks 22.4.3.1Roles 22.4.3.2Tasks 22.4.4Downloading audit logs 22.4.4.1Roles 22.4.4.2Tasks •Specify a license type as part of adding a rack mount server •View licensing status information through license graphs •View a list of server hardware that has been assigned a specific license type 22.5.3About licensing https://www.hp.com/software/licensing 22.5.3.4 View license status Table 7 License graph colors HP OneView w/o iLO HP OneView The license type for which policy type Licensed features 22.5.4.1Server blade licensing at the enclosure level Enclosure licensing policy behavior 22.5.4.2 Rack mount server licensing Rack mount server licensing is managed at the server level Rack mount server licensing behavior When you add a rack mount server to the appliance: • You must choose a license type: HP OneView or HP OneView w/o iLO Table 8 Licensing scenarios 22.6.1Roles 22.6.2Tasks 22.6.3Learning more The only action available for unsupported hardware is Remove unmanaged unsupported disabled Page Page Page 24.1.1 Initial check: the Dashboard 24.1.2 Activities 24.1.3 Utilization graphs “Monitoring power and temperature” (page 179) 24.1.4 Monitor data center temperature To learn more about temperature, see 24.2.1 Best practices for monitoring health with the appliance UI General health monitoring steps Server hardware health monitoring Network health monitoring •For interconnects, a healthy state is Configured For logical interconnects, a healthy state for stacking is 24.2.2 Best practices for monitoring health using REST APIs Page •View activities for a resource •Filter activities by health and status •Filter activities by date •Assign an owner to an alert •Add a note to an alert 24.3.1.1 Activity types: alerts and tasks 24.3.1.1.1 About alerts Resources generate alerts to notify you that some action is required Cleared •Oldest cleared alerts The tasks database is different from the stored alerts 24.3.1.2 Activity states 24.3.1.3 Activity statuses 24.4.1About the Dashboard main menu 24.4.2 Dashboard screen details The following graphs appear on the Dashboard: 24.4.3 How to interpret the Dashboard graphs Table 9 Dashboard graph colors (continued) Status icons Figure 14 Sample Dashboard Page The appliance: •Displays 3D color-codedhardware temperature visualization (UI only) •Collects and reports power metric statistics •Collects and reports temperature metric statistics Displays utilization statistics using customizable utilization graphs (UI only) 25.1.1.1 Manipulating the view of the data center visualization 25.1.2Monitoring power and temperature utilization 25.1.2.1About the Utilization panel 25.1.2.2 About utilization graphs Table 10 Utilization statistics gathered by resource Utilization statistics and licensing “About licensing” (page 159) (horizontal axis) using a line to graph data points 25.1.2.2.1 Power utilization metrics 25.1.2.2.2 Temperature utilization metrics 25.2.1 Update enclosure power capacity settings •Minimum required session ID privileges: Server administrator 25.2.2 Update server hardware power capacity settings To use the SCMB, you must: 1.Use REST APIs to create and download an AMQP certificate from the appliance 2.Connect to the SCMB using one or both of these methods: •Use the “EXTERNAL” authentication mechanism •Connect without sending a user name and password Page task •scmb.# •scmb.tasks Sample queues ChangeType values Example 2 JSON example 1.Download the root CA certificate GET /rest/certificates/ca Save the contents in the response body into a text file named Additional example-specificprerequisites Examples Example 3 .Net C# code example 1 (directly referencing client certificate) Example 4 .Net C# code example 2 (Microsoft Windows certificate store) Trusted Root Certificate Authorities Local Computer •StoreName.Root = Trusted Root Certificate Authorities •StortLocation.LocalMachine = Local Computer The Java code example shows how to connect and subscribe to the SCMB Example 5 Java code example http://pika.readthedocs.org https:// pypi.python.org/pypi/pika Example 6 Python code example (pika) Example 7 Python code example (amqplib) Page Page Page Page Category Activity screen Download an audit log Create a support dump file About syntax errors: About network setup errors: When VM host is down or nonresponsive: •Deletes any existing support dump file •Gathers logs and other information required for debugging •Creates a compressed file The support dump file contains the following: 3.Click Yes, create 27.4 Troubleshooting the appliance 27.4.1 First time setup Appliance network settings are not properly configured External difficulties 27.4.2 Appliance cannot access the network Appliance network not properly configured 27.4.4 Appliance update is unsuccessful 27.4.5 Support dump file creation action fails Insufficient time 27.4.6 Certificate action fails Follow the recommendation if any of these certificate actions fails: 27.4.7 Backup file creation, download, or restore action fails Other related operations are in progress Backup file creation is still in progress The appliance network is down The backup file is incompatible A profile operation was running during the backup Restore operation fails or times out 27.4.8 Restart or shutdown failure Internal server error 27.4.9 VM does not restart when VM host time is manually set 27.4.10 Reinstall the remote console Reinstall the software 27.5.1 Add or remove enclosure is unsuccessful Enclosure is already being managed by some other management software and is claimed by that software Page 27.5.2 Add server blade is unsuccessful Same server blade, different bay error 27.5.3 Certificate Error Reset the OA: 27.6 Troubleshooting firmware bundles 27.6.1 Incorrect credentials 27.6.2 Lost iLO connectivity 27.6.3 HP SUM errors 27.7.1 Interconnect edit is unsuccessful If interconnect edit is unsuccessful: 27.7.2 Interconnect modules are in Maintenance state The license key embedded on the OA has been erased 27.8.2 The license assigned does not match the type specified 27.9 Troubleshooting logical interconnects 27.9.1 I/O bay occupancy errors 27.9.1 I/O bay occupancy errors Change in interconnect Interconnect state errors due to: state 27.9.3 Physical interconnect warnings and errors Interconnect warnings or errors due to: 27.10.1 Network create operation is unsuccessful The prerequisites have not been met The network was configured incorrectly Server is already being managed by some other management software and is claimed by that software 27.11.2Cannot control power on server blade 27.11.3 Lost connectivity to server hardware after appliance restarts resyncPending 27.12.1 Server profile is not created or updated correctly Error Prerequisites and conditions have not been met Server profile has duplicate networks on the same physical port The following conditions have not been met The server hardware or its iLO are powered-offor reset The appliance cannot collect progress information from the iLO Invalid configuration 27.12.2 What to do when you cannot apply the server profile Verify the operational status of the server hardware 27.12.3 Profile operations fail The enclosure is no longer managed by the appliance 27.13.1 Incorrect privileges Your assigned role does not have the correct privileges 27.13.2 Unauthenticated user or group 27.13.3 User public key is not accepted Hidden characters introduced during a copy/paste operation change the key code 27.13.4 Directory service not available Directory service server is down 27.13.5 Cannot add directory service 27.13.6 Cannot add server for a directory service 27.13.7 Cannot add directory user or group User or group not configured in the directory service See also “Post-restorationtasks” (page 224) Firmware baseline HP Service Pack for ProLiant - Base Firmware The appliance is not operational during a restore operation It can take several hours to perform a restore operation Page Before you restore an appliance from a backup file, note the following: •The appliance is not operational during a restore operation •A restore operation cannot be canceled or undone after it has started •The appliance blocks login requests while a restore operation is in progress If you removed, and then Restoring an appliance from a backup file 1.From the Settings screen, select Actions→Restore from backup 2.Read the on-screennotification, then select the check box to confirm 3.From the dialog box that opens, do one of the following: •Click Browse and select the backup file to upload Page Page •Your Service Agreement Identifier (SAID) •Software product name — HP OneView •Hypervisor virtualization platform and version •Messages generated by the appliance •Other HP or third-partysoftware in use 29.4.2 Using your software technical support and update service 29.4.3 Obtaining HP OneView software and firmware updates 29.4.4 Obtaining software and drivers for HP ProLiant products 29.4.5 Warranty Documentation For UI and REST API help For user guides and other manuals Page A.2.1 Sample data center hardware •A pair of SAN switches that connect to data center storage devices •A pair of Ethernet switches that connect to the data center networks All of the hardware is located in the same room Rack mount server HP Virtual Connect FlexFabric 10Gb/24-PortModule port configurations A.2.2 Data center networks A.2.2.1 Fibre Channel networks Fibre Channel networks for the SAN fabrics Figure 17 (page 235) Table 11 SAN A and SAN B Fibre Channel network configurations Fibre Channel networks directly attached to the 3PAR Storage System (Flat SAN) The names of the Direct attach Fibre Channel networks follow: •FlatSAN A •FlatSAN B A.2.2.2 Ethernet Networks Table 15 Development networks Table 16 Test networks Table 17 Ethernet network configuration values A.3.1 Planning for installation of the appliance You follow the recommendations in the A.3.2 Planning for network sets •Production networks •Development networks •Test networks A.3.3 Planning for users and roles A.3.4 Planning resource names Assumptions A.5.1 Workflow 7.“Copying the template server profile to eight servers” (page 255) A.5.2 Downloading the latest firmware bundle and adding it to the appliance 1.Download SPPs from the HP website www.hp.com/go/sppto your local system From the 3.Do one of the following: The value for Uplink Set is “Creating a logical interconnect group and its uplink sets” (page 244) A.5.3.2 Configuring the Ethernet networks 2.Create the ESXi management and vMotion networks: a.For Type, select Ethernet k.Click Create + to create another network 3.Create the production networks a.For Name, enter the name of the network b.For VLAN ID, enter the VLAN ID of the network c.For Purpose, select General A.5.3.3 Configuring the network sets d.Select a network in the network set to receive untagged traffic On the Create network set dialog box, under Networks, locate the first network ii.Select the check box under Untagged e.Click Create + prod networks b.In the search box, enter dev to filter the list of networks c.Select all of the dev networks listed and click Add d.Select Untagged for the first network in the list of networks 5.Create the network set for the test networks: b.In the search box, enter test to filter the list of networks Page 5.Add the uplink set for the SAN B: a.For Name, enter SAN B c.For Network, select SAN B Creating the uplink sets for the Ethernet networks The Add uplink set screen opens Page 4.Click Create to create the logical interconnect group A.5.5 Creating an enclosure group for vSphere (ESXi) hosts 3.For Logical interconnect group, select EsxFlexFabricLIG 4.Click Create A.5.6 Adding the enclosure In this procedure you will add one enclosure: 2.Enter the following information: Viewing server hardware types Editing server hardware types View information about the server hardware A.5.8 Creating a server profile to use as a template Page Add two connections to the esxi mgmt Add one connection to the Page A.5.9 Copying the template server profile to eight servers a.From the main menu, select Server Hardware b.In the master pane, select an instance of server hardware c.Select Actions→Launch console The appliance launches the remote console for the selected server •The appliance is installed A.6.3 Adding the enclosure that is connected to the HP 3PAR Storage System Page Page A.6.4 Creating the server profile Page Page A.6.5 Collecting the WWPNs to use when configuring the HP 3PAR Storage System 3.Record the WWPN information for FlatSAN A and FlatSAN B Because this is an HP ProLiant DL rack mount server: •You cannot use this appliance to provision any networks for the server •You cannot assign a server profile to the server A.7.1 Workflow A.7.2 Adding the server hardware A.7.3 Powering on the server A.7.4 Viewing information about the server Page A.7.5 Adding a license for the server Page Page Page Example 8 “Sample backup.ps1 script” MSVCR100.dll •64 bit: http://www.microsoft.com/download/en/details.aspx?id=14632 Make sure the path environment variable includes the path for cURL Sample script Example 8 Sample backup.ps1 script Page Page Page Page Page Page Page Page Page Example 9 “Sample restore.ps1 script” query-user() 3.Calls uploadTo-appliance() to upload the backup to the appliance Page Example 9 Sample restore.ps1 script