ZyXEL Communications 50 manual

ZyWALL 50

Internet Security Gateway

User’s Guide

Version 3.50

November 2001

Contents

User’s Guide Copyright Federal Communications Commission (FCC) Interference Statement Information for Canadian Users Standard ZyXEL Limited Warranty Customer Support Table of Contents Chapter 4 General And WAN Setup Chapter 6 Internet Access ADVANCED APPLICATIONS Page Page Chapter 12 Logs Chapter 13 Example Firewall Rules Chapter 14 Content Filtering Page Chapter 18 Firmware and Configuration Maintenance Chapter 19 System Maintenance & Information Chapter 20 Remote Management CALL SCHEDULING AND VPN/IPSEC Chapter 22 Introduction to IPSec Chapter 23 VPN/IPSec Setup Chapter 24 SA Monitor Appendix B PPPoE Appendix C PPTP Appendix D Hardware Specifications Appendix E Important Safety Instructions List of Figures Page Page Page Page Page Page List Of Tables Page Page Page Page Preface Syntax Conventions Bold Times Bold Arial Part I: Getting Started Page Getting to Know Your ZyWALL 1.1The ZyWALL 50 Internet Security Gateway 1.2Features of The ZyWALL Content Filtering Packet Filtering Call Scheduling PPTP Encapsulation Dynamic DNS Support Network Address Translation (NAT) Port Forwarding DHCP (Dynamic Host Configuration Protocol) Full Network Management RoadRunner Support 1.3Applications for the ZyWALL 1.3.1 Secure Broadband Internet Access via Cable or DSL Modem 1.3.2 VPN Application Page Hardware Installation 2.1Front Panel LEDs and Back Panel Ports 2.1.1 Front Panel LEDs 2.2ZyWALL 50 Rear Panel and Connections Figure 2-2ZyWALL 50 Rear Panel and Connections Step 1. Connecting the Console Port Step 2. Connecting the ZyWALL to the Broadband Modem Step 2a Step 2b Step 3. Connecting the ZyWALL to the LAN Step 4. Connecting the Power Adapter to your ZyWALL 2.3Additional Installation Requirements Page Initial Setup 3.1Turning On Your ZyWALL 3.1.1 Initial Screen 3.1.2 Entering the Password 3.2Navigating the SMT Interface 3.2.1 Main Menu 3.2.2 System Management Terminal Interface Summary Page 3.2.3 SMT Menus at a Glance Figure 3-5Advanced Management SMT Menus 3.3Changing the System Password 3.4Resetting the ZyWALL 3.4.1 Methods of Restoring Factory-Defaults 3.4.2 Procedure To Use The Reset Button Page General And WAN Setup 4.1System Name 4.2Dynamic DNS Figure 4-1Configure Dynamic DNS Table 4-1Configure Dynamic DNS Menu Fields 4.3WAN Setup Figure 4-2Menu 2 — WAN Setup Table 4-2WAN Setup Menu Fields Page Page LAN Setup 5.1Introduction 5.2LAN Port Filter Setup 5.3TCP/IP and DHCP for LAN 5.3.1 Factory LAN Defaults 5.3.2 DHCP Configuration 5.3.3 IP Address and Subnet Mask 5.3.4Private IP Addresses 5.3.5 RIP Setup 5.3.6 IP Multicast 5.3.7 IP Alias 5.4TCP/IP and DHCP Ethernet Setup Menu Figure 5-6Menu 3.2 — TCP/IP and DHCP Ethernet Setup Table 5-3DHCP Ethernet Setup Menu Fields 5.4.1 IP Alias Setup Figure 5-7Menu 3.2.1 — IP Alias Setup Table 5-5IP Alias Setup Menu Fields Internet Access 6.1Internet Access Setup 6.1.1 Ethernet Encapsulation 6.1.2 PPTP Encapsulation 6.1.3 Configuring the PPTP Client 6.1.4 PPPoE Encapsulation 6.2Basic Setup Complete Part II: Advanced Applications Remote Node Setup 4.1Remote Node Profile 4.1.1 Ethernet Encapsulation Figure 4-1Menu 11.1 — Remote Node Profile for Ethernet Encapsulation Table 4-1Fields in Menu 4.1.2 PPPoE Encapsulation Figure 4-2Menu 11.1 — Remote Node Profile for PPPoE Encapsulation Outgoing Authentication Protocol Nailed-UpConnection 4.1.3 PPTP Encapsulation Figure 4-3Menu 11.1 — Remote Node Profile for PPTP Encapsulation Table 4-3Fields in Menu 11.1 (PPTP Encapsulation) 4.2Editing TCP/IP Options (with Ethernet Encapsulation) 4.2.1 Editing TCP/IP Options (with PPTP Encapsulation) Figure 4-5Menu 11.3 — Remote Node Network Layer Options Table 4-5Remote Node Network Layer Options Menu Fields 4.2.2 Editing TCP/IP Options (with PPPoE Encapsulation) 4.3Remote Node Filter Figure 4-6Menu 11.5 — Remote Node Filter (Ethernet Encapsulation) Figure 4-7Menu 11.5 — Remote Node Filter (PPPoE or PPTP Encapsulation) Page IP Static Route Setup 5.1IP Static Route Setup Table 5-1IP Static Route Menu Fields Page Network Address Translation (NAT) 6.1Introduction 6.1.1 NAT Definitions 6.1.2 What NAT Does 6.1.3 How NAT Works 6.1.4 NAT Application 6.1.5 NAT Mapping Types Many to Many Overload Many to Many No Overload Server Table 6-2NAT Mapping Types 6.2Using NAT 6.2.1 SUA (Single User Account) Versus NAT 6.2.2Applying NAT [ENTER] to bring up Menu 11.3 - Remote Node Network Layer Options Figure 6-4Menu 11.3 — Applying NAT to the Remote Node Table 6-3Applying NAT in Menus 4 & 6.3NAT Setup 6.3.1 Address Mapping Sets SUA Address Mapping Set Figure 6-7Menu 15.1.255 — SUA Address Mapping Rules Table 6-4SUA Address Mapping Rules User-DefinedAddress Mapping Sets Select Rule Figure 6-8Menu 15.1.1 — First Set Ordering Your Rules Table 6-5Fields in Menu Edit Menu 15.1.1.1 - Address Mapping Rule Global Start/End IPs Figure 6-9Menu 15.1.1.1 — Editing/Configuring an Individual Rule in a Set Table 6-6Menu 15.1.1.1 — Editing/Configuring an Individual Rule in a Set 6.4NAT Server Sets – Port Forwarding 6.4.1 Configuring a Server behind NAT Figure 6-10Menu 15.2 — NAT Server Setup Figure 6-11Multiple Servers Behind NAT Example 6.5General NAT Examples 6.5.1 Internet Access Only 6.5.2 Example 2: Internet Access with an Inside Server 6.5.3 Example 3: Multiple Public IP Addresses With Inside Servers Figure 6-16NAT Example Menu 15.1 - Address Mapping Sets Edit Action Start IP Figure 6-17Example 3: Menu Figure 6-18Example 3: Menu Figure 6-19Example 3: Final Menu Figure 6-20Example 3: Menu 6.5.4 Example 4: NAT Unfriendly Application Programs Figure 6-22Example 4: Menu 15.1.1.1 — Address Mapping Rule Figure 6-23Example 4: Menu 15.1.1 — Address Mapping Rules Part III: Firewall and Content Filters Page Firewalls 7.1What Is a Firewall 7.2Types of Firewalls 7.2.1 Packet Filtering Firewalls 7.2.2 Application-levelFirewalls 7.3Introduction to ZyXEL’s Firewall 7.4Denial of Service 7.4.1 Basics 7.4.2 Types of DoS Attacks Figure 7-2 Three-WayHandshake SYN Attack Figure 7-3SYN Flood LAND Attack brute-force Figure 7-4Smurf Attack Table 7-2ICMP Commands That Trigger Alerts 7.5Stateful Inspection 7.5.1 Stateful Inspection Process 7.5.2 Stateful Inspection and the ZyWALL 7.5.3 TCP Security 7.5.4 UDP/ICMP Security 7.5.5 Upper Layer Protocols 7.6Guidelines For Enhancing Security With Your Firewall 7.6.1 Security In General 7.7Packet Filtering Vs Firewall 7.7.1 Packet Filtering: When To Use Filtering 7.7.2 Firewall When To Use The Firewall Page Introducing the ZyWALL Firewall 8.1Remote Management and the Firewall 8.2Access Methods 8.3Using ZyWALL SMT Menus 8.3.1 Activating the Firewall 8.3.2 Viewing the Firewall Log Page Page Using the ZyWALL Web Configurator 9.1Web Configurator Login and Main Menu Screens WIZARD SETUP ADVANCED Figure 9-1Main Menu 9.2Enabling the Firewall 9.3E-mail 9.3.1 Alerts 9.3.2 Logs Table 9-1 E-mail 9.3.3 SMTP Error Messages 9.3.4 Example E-mailLog 9.4Attack Alert 9.4.1 Threshold Values 9.4.2 Half-OpenSessions TCP Maximum Incomplete and Blocking Time Figure 9-5Attack Alert Table 9-3Attack Alert Page Page Page Creating Custom Rules 10.1 Rules Overview 10.2 Rule Logic Overview 10.2.1 Rule Checklist 10.2.2 Security Ramifications 10.2.3Key Fields For Configuring Rules Action Service Source Address 10.3 Connection Direction 10.3.1 LAN to WAN Rules 10.3.2 WAN to LAN Rules 10.4 Rule Summary Figure 10-3Firewall Rules Summary — First Screen Table 10-1Firewall Rules Summary — First Screen Page 10.5 Predefined Services Page Page 10.5.1 Creating/Editing Firewall Rules 10.5.2 Source and Destination Addresses Figure 10-5Adding/Editing Source and Destination Addresses 10.6 Timeout 10.6.1 Factors Influencing Choices for Timeout Values Figure 10-6Timeout Screen Table 10-5Timeout Menu Page Custom Ports 11.1 Introduction Table 11-1Custom Ports 11.2 Creating/Editing A Custom Port Table 11-2Creating/Editing A Custom Port Logs 12.1 Log Screen Table 12-1Log Screen Example Firewall Rules 13.1 Examples 13.1.1 Example 1: Firewall Rule To Allow Web Service From The Internet Figure 13-1Activate the Firewall Figure 13-2Example 1: E-MailScreen Rule Summary Figure 13-3Example 1: Configuring a Rule Page 13.1.2 Example 2: Small Office With Mail, FTP and Web Servers Figure 13-6Send Alerts When Attacked Figure 13-7Configuring A POP Custom Port Rule Summary Source Address Figure 13-8Example 2: Local Network Rule 1 Configuration Figure 13-9Example 2: Local Network Rule Summary Destination Address Figure 13-10Example: Internet to Local Network Rule Summary 13.1.3Example 3: DHCP Negotiation and Syslog Connection from the Internet Figure 13-12Syslog Rule Configuration Figure 13-13Example 3: Rule Summary Content Filtering 14.1 Categories 14.1.1 Restrict Web Features 14.1.2 Filter List 14.1.3 Days and Times 14.4 Customizing 14.5 Keywords 14.6 Log Records Part IV: Advanced Management Filter Configuration 15.1 About Filtering 15.1.1 The Filter Structure of the ZyWALL Filter Set Execute Filter Rule Figure 15-2Filter Rule Process 15.2 Configuring a Filter Set Edit Comments Menu 21.1.1 - Filter Rules Summary Figure 15-6NetBIOS_WAN Filter Rules Summary Figure 15-7NetBIOS _LAN Filter Rules Summary 15.2.1 Filter Rules Summary Menu 15.2.2 Configuring a Filter Rule 15.2.3 TCP/IP Filter Rule Figure 15-8Menu 21.1.1.1 — TCP/IP Filter Rule Table 15-3TCP/IP Filter Rule Menu Fields Page Page Figure 15-9Executing an IP Filter 15.2.4 Generic Filter Rule Table 15-4Generic Filter Rule Menu Fields 15.3 Example Filter Yes Equal Drop Forward Figure 15-12Example Filter — Menu A = Y Type = IP Pr DP M = N 15.4 Filter Types and NAT 15.5 Firewall 15.6 Applying a Filter and Factory Defaults 15.6.1 LAN traffic 15.6.2 Remote Node Filters Output Filter Sets Call Filter Sets Call Filter Sets when Figure 15-16Filtering Remote Node Traffic Page SNMP Configuration 16.1 About SNMP Figure 16-1SNMP Management Model 16.2 Supported MIBs 16.3 Configuring SNMP 16.4 SNMP Traps Table 16-3SNMP Traps Page System Information & Diagnosis 17.1 System Status System Maintenance - Status Menu 24.1 - System Maintenance - Status Figure 17-2Menu 24.1 — System Maintenance — Status Table 17-1System Maintenance — Status Menu Fields 17.2 System Information and Console Port Speed 17.2.1 System Information 17.2.2 Console Port Speed 17.3 Log and Trace 17.3.1 Viewing Error Log Menu 24.3 - System Maintenance - Log and Trace Figure 17-6Menu 24.3 — System Maintenance — Log and Trace Figure 17-7Examples of Error and Information Messages 17.3.2 UNIX Syslog Page Page 17.3.3 Call-TriggeringPacket 17.4 Diagnostic 17.4.1 WAN DHCP Figure 17-11WAN & LAN DHCP Table 17-4System Maintenance Menu Diagnostic Page Firmware and Configuration Maintenance 18.1 Filename Conventions 18.2 Backup Configuration 18.2.1 Backup Configuration 18.2.2 Using the FTP Command from the Command Line 18.2.3 Example of FTP Commands from the Command Line 18.2.4 GUI-BasedFTP Clients 18.2.5 TFTP and FTP over WAN Will Not Work When 18.2.6 Backup Configuration Using TFTP 18.2.7 TFTP Command Example 18.2.8 GUI-BasedTFTP Clients 18.2.9 Backup Via Console Port 18.3 Restore Configuration 18.3.1 Restore Using FTP or TFTP 18.3.2 Procedure To Restore Using FTP 18.3.3 Restore Using FTP Session Example 18.3.4 Restore Via Console Port 18.4 Uploading Firmware and Configuration Files 18.4.1 Firmware File Upload 18.4.2 Configuration File Upload 18.4.3 FTP File Upload Command from the Command Line Example 18.4.4 FTP Session Example of Firmware File Upload 18.4.5 TFTP File Upload 18.4.6 TFTP Upload Command Example 18.4.7 Uploading Via Console Port 18.4.8 Uploading a Firmware File Via Console Port 18.4.9 Example Xmodem Firmware Upload Using HyperTerminal 18.4.10Uploading a Configuration File Via Console Port 18.4.11Example Xmodem Configuration Upload Using HyperTerminal Figure 18-19Example Xmodem Upload Page System Maintenance & Information 19.1 Command Interpreter Mode 19.2 Call Control Support 19.2.1 Budget Management Figure 19-4Budget Management Table 19-1Budget Management 19.2.2 Call History 19.3 Time and Date Setting Figure 19-7Menu 24.10 System Maintenance — Time and Date Setting Table 19-3Time and Date Setting Fields 19.3.1 Resetting the Time Page Remote Management 20.1 Telnet 20.2 FTP 20.3 Web 20.4 Remote Management 20.4.1 Remote Management Limitations 20.5 Remote Management and NAT 20.6 System Timeout Part V: Call Scheduling and VPN/IPSec Page Call Scheduling 21.1 Introduction To delete a schedule set, enter the set number and press [SPACE BAR] or [DELETE] in the Edit Name field Menu 26.1 - Schedule Set Setup Figure 21-2Schedule Set Setup Duration PPPoE Figure 21-3Applying Schedule Set(s) to a Remote Node (PPPoE) Figure 21-4Applying Schedule Set(s) to a Remote Node (PPTP) Introduction to IPSec 22.1 Introduction 22.1.1 VPN 22.1.2 IPSec 22.1.3 Security Association 22.1.5 VPN Applications 22.2 IPSec Architecture 22.2.1 IPSec Algorithms 22.2.2 Key Management 22.3 Encapsulation 22.3.1 Transport Mode 22.3.2 Tunnel Mode 22.4 IPSec and NAT Table 22-1VPN and NAT VPN/IPSec Setup 23.1 VPN/IPSec Setup 23.2 IPSec Algorithms 23.2.1 AH (Authentication Header) Protocol 23.2.2 ESP (Encapsulating Security Payload) Protocol 23.3 IPSec Summary Secure Gateway IP Address Table 23-2Telecommuter and HQ Configuration Example Figure 23-4Telecommuter’s ZyWALL Configuration Page Table 23-3Menu 27.1 — IPSec Summary 23.3.1 IPSec Setup Figure 23-7Menu 27.1.1 — IPSec Setup Table 23-4Menu 27.1.1 — IPSec Setup 23.4 IKE Setup 23.4.1 IKE Phases 23.4.2 Negotiation Mode 23.4.3 Pre-SharedKey 23.4.4 Diffie-Hellman(DH) Key Groups 23.4.5 Perfect Forward Secrecy (PFS) Figure Page 23.5 Manual Setup 23.5.1 Active Protocol 23.5.2 Security Parameter Index (SPI) Figure 23-10Menu 27.1.1.2 — Manual Setup Table 23-7Menu 27.1.1.2 — Manual Setup Page Page SA Monitor 1.1. Introduction Page IPSec Log Double exclamation marks (!!) denote an error or warning message Table 25-1Sample IKE Key Exchange Logs Table 25-2Sample IPSec Logs During Packet Transmission Table 25-3 RFC-2408ISAKMP Payload Types Part VI: Troubleshooting, Appendices and Index Troubleshooting 26.1 Problems Starting Up the ZyWALL 26.2 Problems with the LAN Interface 26.3 Problems with the WAN interface 26.4 Problems with Internet Access 26.5 Problems with the Password 26.6 Problems with Remote Management Table 26-6Troubleshooting Remote Management Appendix A The Big Picture Page Appendix B PPPoE PPPoE in Action Benefits of PPPoE Traditional Dial-upScenario How PPPoE Works ZyWALL as a PPPoE Client Appendix C PPTP Diagram 5 PPTP Protocol Overview Control & PPP connections Diagram 6 Example Message Exchange between PC and an ANT Page Appendix D Hardware Specifications Appendix E Important Safety Instructions Page Appendix F Boot Commands Diagram 8 Boot Module Commands Appendix G Firewall CLI Commands Page Page Page Page Appendix H Power Adapter Specifications Page Page Index