HP-UXIPSec version A.02.00
Administrator’s Guide
Warranty
U.S. Government License
Copyright Notice
Trademark Notice
Contents
Preface: About This Document
1. HP-UXIPSec Overview
2. Installing HP-UXIPSec
3. Configuring HP-UXIPSec
Page
4. Using Certificates with HP-UXIPSec
5. Troubleshooting HP-UXIPSec
Page
6. HP-UXIPSec and IPFilter
7. HP-UXIPSec and HP-UXMobile IPv6
8. HP-UXIPSec and MC/ServiceGuard
Page
9. HP-UXIPSec and Linux
A. Product Specifications
B. Migrating from Previous Versions of HP-UXIPSec
C. HP-UXIPSec Configuration Examples
Page
Page
Tables
Page
Figures
Page
Intended Audience
New and Changed Documentation in This Edition
-entity
-ike
-gateway
-tunnel
-spd_soft
Publishing History
Table
Publishing History Details
Document
Operating Systems
Typographical Conventions
Related Documents
HP Encourages Your Comments
OpenSSL Copyright Notice
NOTE
Page
Page
HP-UXIPSec Overview
Page
Page
Internet Key Exchange (IKE)
Manual Keys
Authentication Header (AH)
Authentication Header (AH)
Symmetric key hash
shared key hash
secret key hash
Figure
Symmetric Key Authentication
HMAC-SHA1
HMAC-MD5
Transport and Tunnel Modes
IPv6
AH in Transport Mode
Tunnel Mode
AH in Tunnel Mode
Encapsulating Security Payload (ESP)
ESP Encryption
Symmetric Key Cryptosystem
symmetric key cryptography
shared key cryptography
Page
ESP Encryption in Transport Mode
ESP in Tunnel Mode
ESP with Authentication and Encryption
Authenticated ESP
Nested ESP in AH
Internet Key Exchange (IKE)
Internet Key Exchange (IKE)
primary authentication
Security Associations (SAs) and IKE Phases
Security Association
Diffie-Hellman
SA Establishment
Generating Shared Keys: Diffie-Hellman
Diffie-HellmanKey Generation
IKE Primary Authentication
IKE Preshared Key Authentication
preshared key
Digital Signatures
security certificates
Public Key Infrastructure (PKI)
Manual Keys
Manual Keys
HP-UXIPSec Topologies
HP-UXIPSec Topologies
Host-to-HostTopology
Figure 1-11IPSec Host-to-HostTopology
Host-to-GatewayTopology
Figure 1-12 Host-to-Gateway(VPN) Topology
Host-to-HostTunnel Topology
Figure 1-13 Host-to-HostTunnel Topology
Gateway-to-GatewayTopology
Figure 1-14IPSec Gateway-to-GatewayTopology
HP-UXIPSec Configuration and Management Features
HP-UXIPSec Configuration and Management Features
Page
Page
Installing HP-UXIPSec
Page
HP-UXIPSec Product Requirements
HP-UXIPSec Product Requirements
Disk Requirements
Security Certificate Configuration Utility Requirements
Step 1: Verifying HP-UXIPSec Installation and Configuration Prerequisites
Step 1: Verifying HP-UXIPSec Installation and Configuration Prerequisites
uname
swlist
Step 2: Loading the HP-UXIPSec Software
Step 2: Loading the HP-UXIPSec Software
root
swinstall
Page
Step 3: Setting the HP-UXIPSec Password
Step 3: Setting the HP-UXIPSec Password
ipsec_admin -newpasswd
Re-establishingthe HP-UXIPSec Password
Step 4: Completing Post-InstallationMigration Requirements
Step 4: Completing Post-Installation
Migration Requirements
Configuring HP-UXIPSec
Page
Maximizing Security
Maximizing Security
Bypass List
Strong End System Model
ndd -set /dev/ipip_strong_es_model
ipsec_config add
ipsec_config batch
ipsec_config delete
ipsec_config show
Using a Profile File with a Batch File
Profile File Structure
Creating a Customized Profile File
IPv6 Networks
nocommit Argument
Configuration Overview
•Host IPSec Policies
•Tunnel IPSec Policies
•IKE Policies
•IKE Authentication Records
•Start-upoptions
gateway IPSec policies
Step
Page
Step 1: Configuring Host IPSec Policies
Step 1: Configuring Host IPSec Policies
Policy Order and Selection
default Host IPSec Policy
ipsec_config add host Syntax
Acceptable Values:
-source and -destination
and
ip_addr[/prefix[/port_number|service_name]]
Default:
Range:
Table
ipsec_config Service Names
Service Names
Service
ipsec_config Service Names (Continued)
FTP-CONTROL
HTTP-TCP
HTTP-UDP
NTP
ICMP
ICMPV6
IGMP
Default: ALL
CAUTION
PASS
DISCARD
transform_name[/lifetime_seconds[/lifetime_kbytes]]
TIP
ipsec_config Transforms
Transforms
Transform Name
Description
ipsec_config Transforms (Continued)
ESP_DES_HMAC_SHA1
ESP_3DES
ESP_3DES_HMAC_MD5
ESP_3DES_HMAC_SHA1
-flags flags
ipsec_config add host Flags
Flags
Flag
EXCLUSIVE
Host IPSec Policy Configuration Examples
Page
ipsec_config add tunnel Syntax
-tsource and -tdestination tunnel_address
Default: None
Page
Page
Page
-action transform_list
transform_name
Tunnel IPSec Policy Configuration Example
Page
Step 3: Configuring IKE Policies
Step 3: Configuring IKE Policies
ipsec_config add ike Syntax
-remote ip_addr[/prefix]
-authentication authentication_type
-authentication
authentication_type
PSK
RSASIG
-hashMD5|SHA1
MD5
SHA1
-encryption
encryption_algorithm
ipsec_config add IKE Command Examples
Step 4: Configuring Preshared Keys Using Authentication Records
Step 4: Configuring Preshared Keys Using Authentication Records
ipsec_config add auth
Remote Multi-homedSystems
Configuring IKE ID Information with Preshared Keys
not
ipsec_config add auth Syntax
ip_addr[/prefix]
Specifying a subnet address filter and a preshared key allows
WARNING
you to configure a single preshared key for an entire subnet
However, HP strongly recommends that you configure an
individual authentication record for each remote system with a
Page
Step 5: Configuring Certificates
Step 5: Configuring Certificates
Step 6: Configuring the Bypass List (Local IPv4 Addresses)
Step 6: Configuring the Bypass List (Local IPv4 Addresses)
Logical Interfaces
Example
ipsec_config add bypass Syntax
add bypass ip_address
Bypass Configuration Example
Step 7: Verify Batch File Syntax
Step 7: Verify Batch File Syntax
ipsec_config batch batch_file_name -nocommit
Step 8: Committing the Batch File Configuration and Verifying Operation
ipsec_config batch batch_file_name
ipsec_config show all
ipsec_admin -status
ipsec_report -cache
ipsec_report -all
-bypass
ipsec_config add startup Syntax
-newpasswd
Step 10: Creating Backup Copies of the Batch File and Configuration Database
Baltimore Configuration Files
VeriSign Configuration Files
Page
Using Certificates with HP-UX
IPSec
Page
Overview
Overview
Security Certificates and Public Key Cryptography
public key cryptography
asymmetric key cryptography
private key
Certificate Revocation List (CRL)
Digital Signatures
IKE Public Key Distribution
Requirements
Using VeriSign Certificates
Using VeriSign Certificates
VeriSign PKI Data Flow
VeriSign Certificate Tasks
Step 1: Verifying Prerequisites
Step 2: Configuring Web Proxy Server Parameters
Using a Remote Display Device
export DISPLAY=display_device:0.0
Step 3: Registering the Administrator
Step 4: Requesting and Receiving Certificates
Page
Page
Using Baltimore Certificates
Using Baltimore Certificates
Baltimore Certificate Tasks
must
Step 2: Requesting the Baltimore Certificate
Step 3: Configuring the Baltimore Certificate
Page
Page
Page
Page
Configuring Authentication Records with IKE IDs
Configuring Authentication Records with IKE IDs
Configuring Authentication Records with Certificate-BasedAuthentication
Determining the IPv4 Address in the SubjectAlternativeName
VeriSign
VeriSign SubjectAlternativeName
Baltimore
Syntax
add auth auth_name -remote ip_addr[/prefix]
add auth
-ltype local_id_type
-ltype
-lvalue local_id
-rtype remote_id_type
-rtype
FQDN
USER-FQDN
X500-DN
CN=commonName,O=organization,C=country[,OU=organizationUnit]
CN
,O
,C=c
[,OU
add auth Zebra1 -remote10.20.20.20 -rtypeIPV4 \ -rid10.20.20.20
add auth Zebra2 -remote192.6.2.21 -rtypeIPV4 \ -rid10.20.20.20
add auth Black -remote10.10.10.10 -ltypeIPV4 \ -lid10.20.20.20
Retrieving the Certificate Revocation List (CRL)
Retrieving the Certificate Revocation List (CRL)
VeriSign
Baltimore
Manually Retrieving a CRL for VeriSign or Baltimore
Page
Troubleshooting HP-UXIPSec
Page
IPSec Operation
IPSec Operation
Establishing Security Associations (SAs)
Security Associations
1.Authenticate Identities
2.Establish ISAKMP/MM SA
3.Establish IPSec/QM SAs
Internal Processing
Outbound Processing
Outbound Data
1.Query the Kernel Policy Engine
2.Query the Policy Manager Daemon
3.Establish an ISAKMP/MM SA
4.Establish IPSec/QM SAs
5.Add IPSec/QM SAs to the Kernel SA Database
Inbound Data
•AH or ESP Packet
Page
•Clear Text Packet
Establishing Tunnel Security Associations
Processing Inbound Tunnel Packets
Page
Troubleshooting Utilities Overview
Troubleshooting Utilities Overview
Getting General Information
Getting General Information
Task
Command
Getting SA Information
Getting SA Information
Getting Policy Information (Continued)
ipsec_report -hostconfigured
ipsec_config show gateway
ipsec_report -gateway
ipsec_report -gateway[active]
Getting Interface Information
Getting Interface Information
ipsec_report -ip
ipsec_report -bypass
Viewing and Configuring Audit Information
Viewing and Configuring Audit Information
ipsec_admin -m[axsize]
max_audit_file_size
ipsec_config add startup
argument_list
Enabling and Disabling Tracing
Troubleshooting Procedures
Troubleshooting Procedures
Checking Status
ipsec_report -all [-file filename]
Page
Isolating HP-UXIPSec Problems from Upper-layer
Problems
ipsec_admin -traceon[ tcp | udp | igmp | all ]
Checking Policy Configuration
Using ipsec_policy
Using
ipsec_policy -sa15.1.1.1 -sp65535 -da15.2.2.2 -dp23 -ptcp -dirout
telnet
Configuring HP-UXIPSec Auditing
Audit Level
Audit Files and Directory
Audit File Size
ipsec_admin -m[axsize] max_audit_file_size
Dynamically Setting Audit Parameters
ipsec_admin [-al audit_level] [-au audit_directory]
Configuring Startup Audit Parameters
ipsec_config add startup [-autobootON|OFF]
[-auditlvl audit_level] [-auditdir audit_directory]
[-auditlvl
[-auditdir
ipsec_report -auditaudit_file
Filtering Audit File Output by Entity
ipsec_report -auditaudit_file -entity entity_name
Reporting Problems
Reporting Problems
netstat
-rn
•Output from ipsec_admin -status
•Output from ipsec_report -all
Page
Troubleshooting Scenarios
Troubleshooting Scenarios
HP-UXIPSec Incorrectly Passes Packets
Problem
Symptoms
Solution
HP-UXIPSec Incorrectly Attempts to
Encrypt/Authenticate Packets
ping
linkloop
HP-UXIPSec Attempts to Encrypt/Authenticate and Fails
ipsec_report -audit file
Additional Information
ipsec_report -audit /var/adm/ipsec/auditdateinfo.log
Main Mode
processing failed, MM negotiation timeout)
Page
ISAKMP Primary Authentication with Preshared Key Fails
ipsec_config show auth
ISAKMP Primary Authentication Fails with
Certificates
ipsec
Negotiation Fails (Quick Mode processing failed, QM negotiation timeout)
Manual Keys Fail
local inbound
remote outbound
local outbound
remote inbound
SADB_ADD for SPI
Page
ipsec_admin
-start
Page
Corrupt or Missing Configuration Database
Using ipsec_migrate
ipsec_admin -stop
ipsec_migrate -s old_config_file -d new_config_file
ipsec_migrate
ipsec_config batch batch_file
Autoboot is Not Working Properly
Administrator Cannot Get a Local VeriSign
Certificate
Page
Kernel
Policy Cache Threshold reached or Kernel Policy Cache Threshold exceeded)
ipsec_config add startup -spd_soft spd_soft_limit
ipsec_config add startup -spd_hard spd_hard_limit
HP-UXIPSec and IPFilter
Page
IPFilter and IPSec Basics
IPFilter and IPSec Basics
IPFilter and IPSec
IPFilter Scenario One
Page
IPSec UDP Negotiation
IPSec UDP Negotiation
IPFilter Scenario Two
Page
When Traffic Appears to be Blocked
When Traffic Appears to be Blocked
Scenario Three
Allowing Protocol 50 and Protocol 51 Traffic
Allowing Protocol 50 and Protocol 51 Traffic
Packet with Encrypted TCP Data
Packet with IPSec-EncryptedTCP Data
Scenario Four
IPSec Gateways
IPSec Gateways
HP-UXIPSec and HP-UXMobile
IPv6
Page
Mobile Node and Home Address
Mobile Node
MN)
home address
home network
Home Agents and Basic Operation
Mobile IPv6 Basic Operation: Correspondent Node to Mobile
Node
Mobile IPv6 Basic Operation: Mobile Node to Correspondent
Route Optimization
Mobile IPv6 Route Optimization
Securing Mobile IPv6 with HP-UXIPSec
Binding Messages
Binding Update
Binding
Acknowledgement messages
Return Routability Messages Between the Home Agent and Mobile Node
Prefix Discovery Packets Between the Home Agent and Mobile Node
Mobile Prefix Advertisement
Payload Packets Routed Through the Home Agent
payload
Page
Understanding Gateway IPSec Policies
Gateway IPSec Policies
Using Manual Keys
Selecting Encryption Keys
Using the HP-UXStrong Random Number Generator
od -Ax -Nnn /dev/random
Troubleshooting Manual Key Problems
Page
Page
-source home_agent_addr
-destination mn_home_addr
-protocolMH
-action transform_name
-in and -out manual_key_sa_specification
auth_key
enc_key
Step 2: (Recommended) Securing Return
Routability Messages Routed Through the
Home Agent
Mobile IPv6 Home Test Init and Home Test Packets
Step 2A:
Step 2B:
Step 2C:
Step 2A: Return Routability Messages: Configuring
the Gateway IPSec Policy for Home Agent
Page
-tunnel rr_tunnel_name -actionFORWARD -flagsMIPV6
-tunnel rr_tunnel_name
ipsec_config add tunnel rr_tunnel_name
Page
Step 3: (Recommended) Securing Prefix
Discovery Messages Between the Home Agent
and Mobile Node
Page
Step 4: (Optional) Securing Payload Packets
Routed Through the Home Agent
Step 4A:
Step 4B:
Step 4C:
-protocolALL [-priority priority_number]
-actionFORWARD -flagsMIPV6
interface_name
-tunnel payload_tunnel_name
-homeclear
Step 4C: Payload Packets: Configuring the Home Agent - Mobile Node Tunnel
ipsec_config add tunnel payload_tunnel_name
Mobile IPv6 Configuration Example
Mobile IPv6 Configuration Example
3ffe::83ff:fef7:1111
3ffe::83ff:fef7:2222
Binding Messages
add host mn_2222_binding
Gateway IPSec Policy for Home Agent - Correspondent Node
Segments
add gateway mn2222_rr_to_cn
3ffe::83ff:fef7:2222
0::0
add host mn2222_prefix
-protoICMPV6 -pri210 -actionESP_AES128_HMAC_SHA1 \ -flagsMIPV6
(Optional) Payload Messages Routed Through the Home Agent
Payload Gateway IPSec Policies
Gateway IPSec Policy for Home Agent - Correspondent Node Segments
Payload Tunnel IPSec Policy
add tunnel mn2222_payload_tunnel
3ffe::83ff:fef7:1111
-protocolALL
Batch File Template
Batch File Template
Page
Page
Page
HP-UXIPSec and
MC/ServiceGuard
Page
cluster
failover packages
fail over
adoptive node
MC/ServiceGuard Cluster
package addresses
clients
Using HP-UXIPSec with MC/ServiceGuard
Package Clients Not Using HP-UXIPSec A.01.07 or Later
Page
MC/ServiceGuard Heartbeat Requirement and
Recommendation
Configuration Steps
Page
Step 1: Configuring a Common HP-UXIPSec Password
Step 1: Configuring a Common HP-UXIPSec Password
Step 2: Configuring HP-UXHost IPSec
Policies for MC/ServiceGuard
Determining MC/ServiceGuard Cluster Information
Configuring Host IPSec Policies for Package Addresses
Configuring PASS Host IPSec Policies for Heartbeat IP Addresses
Source IP
Destination
Source
Address
IP Address
Configuring Host IPSec Policies for MC/ServiceGuard Quorum Server
Cluster Node IPSec Policies for Quorum Server
IP Address
Quorum Server IPSec Policies
Configuring Host IPSec Policies for Remote
Command Execution
Cluster Node IPSec Policies for Remote Command Execution
Page
Remote Command Client Host IPSec Policies
Configuring Host IPSec Policies for ServiceGuard Manager
Cluster Node Host IPSec Policies for ServiceGuard Manager
ServiceGuard Manager Host IPSec Policies
Protoco
Configuring Host IPSec Policies for Cluster Object Manager (COM)
Cluster Node Host IPSec Policies for COM
COM System Host IPSec Policies
Summary: MC/ServiceGuard Port Numbers and
Protocols
MC/ServiceGuard Port Numbers and Protocols
Protocols
MC/ServiceGuard Port Numbers and Protocols (Continued)
Page
Step 3: Configuring HP-UXIPSec IKE policies
Step 3: Configuring HP-UXIPSec IKE policies
Cluster IKE policies
Cluster Client IKE policies
Step 4: Configuring Authentication Records for Preshared Keys
Step 4: Configuring Authentication Records for Preshared Keys
Preshared Key Configuration on Cluster Nodes
Preshared Key Configuration on Client Nodes
Preshared Keys Configuration on Cluster Nodes
Remote IP Address
Key
Preshared Keys Configuration on Client1
Preshared Keys Configuration on Client2
Page
Step 5: Configuring Authentication Records for Certificates
Step 5: Configuring Authentication Records for Certificates
Authentication Records and IKE ID Information
Cluster Node
Cluster Clients
Page
IKE ID Configuration on Cluster Nodes
IKE ID Configuration on Client1 and Client2
Step 6: Verifying and Testing the HP-UXIPSec Configuration
Step 6: Verifying and Testing the HP-UXIPSec Configuration
ipsec_policy -sa15.1.1.1 -da15.2.2.2
ipsec_policy -sa15.1.1.1 -sp65535 -da15.2.2.2 -dp5300 -ptcp
Step 7: Configuring HP-UXIPSec Start-upOptions
Step 7: Configuring HP-UXIPSec Start-up
Options
Step 8: Distributing HP-UXIPSec Configuration Files
Step 8: Distributing HP-UXIPSec
Configuration Files
Page
Step 9: Configuring MC/ServiceGuard
Step 9: Configuring MC/ServiceGuard
Cluster Configuration
Package Configuration
Package Control Script
Monitor Script Polling Interval
Step 10: Starting HP-UXIPSec and MC/ServiceGuard
Step 10: Starting HP-UXIPSec and
MC/ServiceGuard
cmruncl
Adding a Node to a Running Cluster
Page
HP-UXIPSec and Linux
Page
Limitations of HP-UXIPSec Interoperating with Linux FreeSwan
Limitations of HP-UXIPSec Interoperating with Linux FreeSwan
Configuration Example
Configuration Example
Product Specifications
Page
IPSec RFCs
IPSec RFCs
Table A-1
Supported IPSec RFCs
RFC Number
RFC Title
Page
Product Restrictions
Product Restrictions
ISAKMP Limitations
IPv4 ICMP Messages
IPv6 ICMP Messages
HP-UXIPSec Transforms
HP-UXIPSec Transforms
Comparative Key Lengths
Table A-2
AH and ESP Algorithms and Key Lengths
Algorithm
Encryption Algorithms
ESP-DES
Linux FreeSwan
ESP-DES-HMAC-MD5
ESP-DES-HMAC-SHA1
ESP-NULL-HMAC-MD5
ESP-NULL-HMAC-SHA1
Transform Lifetime Negotiation
Migrating from Previous Versions of
HP-UXIPSec
Page
Pre-InstallationMigration Instructions
Pre-InstallationMigration Instructions
MD5 Version Compatibility
ipsec_report -audit audit_file_name [-file output_file_name]
[-file
Migrating from Versions Prior to A.01.03
-stop
Not Re-usingConfiguration Files
Post-InstallationMigration Instructions
Post-InstallationMigration Instructions
Configuration File
/usr/sbin/ipsec_migrate -s config_file -d new_config_file
ipsec_config add startup -autobooton
ipsec_admin -start
Page
C HP-UXIPSec Configuration Examples
Page
Example 1: telnet Between Two Systems
Example 1: telnet Between Two Systems
Apple Configuration
Figure C-1
Example 1: telnet AB
Figure C-2
Example 1: telnet BA
-destination15.2.2.2/32/TELNET
-priority20 -actionESP_AES128_HMAC_SHA1
add host telnetBA
-source15.1.1.1/32/TELNET \ -destination15.2.2.2
-priority30 -actionESP_AES128_HMAC_SHA1
Page
Example 2: Authenticated ESP with Exceptions
Example 2: Authenticated ESP with
Exceptions
Figure C-3
Example 2: Network IPSec Policy with Exceptions
Carrot Configuration
add host potato -destination193.3.3.3 -priority20 \ -actionESP_AES128_HMAC_SHA1
#to modify the default host policy, you must delete
#the existing default policy, then re-addit
Policy Priority
ipsec_config Batch File Entries
add ike potato -remote193.3.3.3 -authenticationpsk
add ike 192.1.1_net -remote192.1.1.0/24 \ -authenticationrsasig
Authentication Record
add auth potato -remote193.3.3.3
Example 3: Host to Gateway
Example 3: Host to Gateway
Figure C-4
Host to Gateway Configuration Example
Blue Configuration
Host IPSec Policy
Tunnel IPSec Policy
-actionESP_DES_HMAC_MD5
add auth torouter -rem16.6.6.6 -pskHello
Example 4: Manual Keys
Example 4: Manual Keys
Dog Configuration
Cat Configuration
add host rlog_dog_to_cat -destination10.2.2.2 \ -source10.4.4.4/32/RLOGIN
Glossary
Diffie-Hellman
Encryption
ESP
Filter
HMAC
Page
Page
Numerics