ZyXEL Communications 10, 50 manual

ZyWALL 10/50

Internet Security Gateway

User’s Guide

Version 3.50

May 2002

Contents

User’s Guide Copyright Federal Communications Commission (FCC) Interference Statement Information for Canadian Users Standard ZyXEL Limited Warranty Customer Support Table of Contents Chapter 5 LAN Setup Chapter 7 Remote Node Setup Chapter 9 Network Address Translation (NAT) FIREWALL AND CONTENT FILTERS Chapter 13 Creating Custom Rules Chapter 15 Logs Page Part I: Getting Started Page Getting to Know Your ZyWALL 1.1The ZyWALL 10/50 Internet Security Gateway 1.2Features Content Filtering Packet Filtering Call Scheduling PPTP Encapsulation Dynamic DNS Support Network Address Translation (NAT) Port Forwarding DHCP (Dynamic Host Configuration Protocol) Full Network Management RoadRunner Support 1.3Applications 1.3.1 Secure Broadband Internet Access via Cable or DSL Modem 1.3.2 VPN Application Page Hardware Installation 2.1Front Panel LEDs and Back Panel Ports 2.1.1 Front Panel LEDs 2.2ZyWALL Rear Panel and Connections Figure 2-2ZyWALL 10 Rear Panel and Connections Figure 2-3ZyWALL 50 Rear Panel and Connections Step 1. Connecting the Console Port Step 2. Connecting the ZyWALL to the Broadband Modem Step 2a Step 2b Step 3. Connecting the ZyWALL to the LAN Step 4. Connecting the Power Adapter to your ZyWALL 2.3Additional Installation Requirements Initial Setup 3.1Turning On Your ZyWALL 3.1.1 Initial Screen 3.1.2 Entering the Password 3.2Navigating the SMT Interface 3.2.1 Main Menu 3.2.2 System Management Terminal Interface Summary Page 3.2.3 SMT Menus at a Glance Figure 3-5Advanced Management SMT Menus 3.3Changing the System Password 3.4Resetting the ZyWALL 3.4.1 Methods of Restoring Factory-Defaults 3.4.2 Procedure To Use The Reset Button Page 4.2 Dynamic DNS 4.1 System Name General and WAN Setup 4.2.1 DYNDNS Wildcard 4.3General Setup 4.3.1 Configuring Dynamic DNS Page 4.4WAN Setup Page LAN Setup 5.1Introduction 5.2LAN Port Filter Setup 5.3TCP/IP and DHCP for LAN 5.3.1 Factory LAN Defaults 5.3.2 DHCP Configuration 5.3.3 IP Address and Subnet Mask 5.3.4Private IP Addresses 5.3.5 RIP Setup 5.3.6 IP Multicast 5.3.7 IP Alias 5.4TCP/IP and DHCP Ethernet Setup Menu Figure 5-6Menu 3.2 — TCP/IP and DHCP Ethernet Setup Table 5-3DHCP Ethernet Setup Menu Fields 5.4.1 IP Alias Setup Figure 5-7Menu 3.2.1 — IP Alias Setup Table 5-5IP Alias Setup Menu Fields Internet Access 6.1Internet Access Setup 6.1.1 Ethernet Encapsulation 6.1.2 PPTP Encapsulation 6.1.3 Configuring the PPTP Client 6.1.4 PPPoE Encapsulation Figure 6-3Internet Access Setup (PPPoE) Table 6-3New Fields in Menu 4 (PPPoE) screen 6.2Basic Setup Complete Page Part II: Advanced Applications Page Remote Node Setup 7.1Remote Node Profile 7.1.1 Ethernet Encapsulation Figure 7-1Menu 11.1 — Remote Node Profile for Ethernet Encapsulation Table 7-1Fields in Menu 7.1.2 PPPoE Encapsulation Figure 7-2Menu 11.1 — Remote Node Profile for PPPoE Encapsulation Outgoing Authentication Protocol Nailed-UpConnection 7.1.3 PPTP Encapsulation Figure 7-3Menu 11.1 — Remote Node Profile for PPTP Encapsulation Table 7-3Fields in Menu 11.1 (PPTP Encapsulation) 7.2Editing TCP/IP Options (with Ethernet Encapsulation) 7.2.1 Editing TCP/IP Options (with PPTP Encapsulation) Figure 7-5Menu 11.3 — Remote Node Network Layer Options Table 7-5Remote Node Network Layer Options Menu Fields 7.2.2 Editing TCP/IP Options (with PPPoE Encapsulation) 7.3Remote Node Filter Figure 7-6Menu 11.5 — Remote Node Filter (Ethernet Encapsulation) Figure 7-7Menu 11.5 — Remote Node Filter (PPPoE or PPTP Encapsulation) Page IP Static Route Setup 8.1IP Static Route Setup Table 8-1IP Static Route Menu Fields Page Network Address Translation (NAT) 9.1Introduction 9.1.1 NAT Definitions 9.1.2 What NAT Does 9.1.3 How NAT Works 9.1.4 NAT Application 9.1.5 NAT Mapping Types Many to Many Overload Many One to One Server Table 9-2NAT Mapping Types 9.2Using NAT 9.2.1 SUA (Single User Account) Versus NAT 9.2.2Applying NAT Yes Menu 11.3 - Remote Node Network Layer Options Figure 9-4Menu 11.3 — Applying NAT to the Remote Node Table 9-3Applying NAT in Menus 4 & 9.3NAT Setup 9.3.1 Address Mapping Sets SUA Address Mapping Set Figure 9-7Menu 15.1.1 — SUA Address Mapping Rules Table 9-4SUA Address Mapping Rules User-DefinedAddress Mapping Sets Figure 9-8Menu 15.1.1 — First Set Ordering Your Rules Table 9-5Fields in Menu Menu 15.1.1.1 - Address Mapping Rule Global Start/End IPs Figure 9-9Menu 15.1.1.1 — Editing/Configuring an Individual Rule in a Set 9.4NAT Server Sets – Port Forwarding 9.4.1 Configuring a Server behind NAT Step 2. Enter 2 to go to Menu 15.2 - NAT Server Setup Start Port No End Port No IP Address Figure 9-10Menu 15.2 — NAT Server Setup Figure 9-11Multiple Servers Behind NAT Example 9.5General NAT Examples 9.5.1 Internet Access Only 9.5.2 Example 2: Internet Access with an Inside Server 9.5.3 Example 3: Multiple Public IP Addresses With Inside Servers Figure 9-16NAT Example Menu 15.1 - Address Mapping Sets Edit Action One-to-One Start IP Figure 9-17Example 3: Menu Figure 9-18Example 3: Menu Figure 9-19Example 3: Final Menu Figure 9-20Example 3: Menu 9.5.4 Example 4: NAT Unfriendly Application Programs Figure 9-22Example 4: Menu 15.1.1.1 — Address Mapping Rule Figure 9-23Example 4: Menu 15.1.1 — Address Mapping Rules Part III: Firewall and Content Filters Page Firewalls 10.1 What Is a Firewall 10.2 Types of Firewalls 10.2.1 Packet Filtering Firewalls 10.2.2 Application-levelFirewalls 10.3 Introduction to ZyXEL’s Firewall 10.4 Denial of Service 10.4.1 Basics 10.4.2 Types of DoS Attacks Figure 10-2 Three-WayHandshake SYN Attack Figure 10-3SYN Flood LAND Attack brute-force Figure 10-4Smurf Attack Table 10-2ICMP Commands That Trigger Alerts 10.5 Stateful Inspection 10.5.1 Stateful Inspection Process 10.5.2 Stateful Inspection and the ZyWALL 10.5.3 TCP Security 10.5.4 UDP/ICMP Security 10.5.5 Upper Layer Protocols 10.6 Guidelines For Enhancing Security With Your Firewall 10.6.1 Security In General 10.7 Packet Filtering Vs Firewall 10.7.1 Packet Filtering: When To Use Filtering 10.7.2 Firewall When To Use The Firewall Page Introducing the ZyWALL Firewall 11.1 Remote Management and the Firewall 11.2 Access Methods 11.3 Using ZyWALL SMT Menus 11.3.1 Activating the Firewall 11.3.2 Viewing the Firewall Log Table 11-1View Firewall Log Page Using the ZyWALL Web Configurator 12.1 Web Configurator Login and Main Menu Screens 12.2 Enabling the Firewall 12.3 E-mail 12.3.1 Alerts 12.3.2 Logs Table 12-1 E-mail 12.3.3 SMTP Error Messages 12.3.4 Example E-mailLog 12.4 Attack Alert 12.4.1 Threshold Values 12.4.2 Half-OpenSessions TCP Maximum Incomplete and Blocking Time Figure 12-4Attack Alert Table 12-3Attack Alert Page Page Creating Custom Rules 13.1 Rules Overview 13.2 Rule Logic Overview 13.2.1 Rule Checklist 13.2.2 Security Ramifications 13.2.3Key Fields For Configuring Rules Action Service Source Address 13.3 Connection Direction 13.3.1 LAN to WAN Rules 13.3.2 WAN to LAN Rules 13.4 Rule Summary Figure 13-3Firewall Rules Summary — First Screen Table 13-1Firewall Rules Summary — First Screen Page 13.5 Predefined Services Page Page 13.5.1 Creating/Editing Firewall Rules 13.5.2 Source and Destination Addresses Figure 13-5Adding/Editing Source and Destination Addresses 13.6 Timeout 13.6.1 Factors Influencing Choices for Timeout Values Figure 13-6Timeout Screen Table 13-5Timeout Menu Page Custom Ports 14.1 Introduction Table 14-1Custom Ports 14.2 Creating/Editing A Custom Port Table 14-2Creating/Editing A Custom Port Logs 15.1 Log Screen Table 15-1Log Screen Example Firewall Rules 16.1 Examples 16.1.1 Example 1: Firewall Rule To Allow Web Service From The Internet Figure 16-1Activate the Firewall Figure 16-2Example 1: E-MailScreen Rule Summary Figure 16-3Example 1: Configuring a Rule Page 16.1.2 Example 2: Small Office With Mail, FTP and Web Servers Figure 16-6Send Alerts When Attacked POP3 is now a predefined service, but you still use the same process for configuring a custom port Figure 16-7Configuring A POP Custom Port Rule Summary Source Address Figure 16-8Example 2: Local Network Rule 1 Configuration Figure 16-9Example 2: Local Network Rule Summary Destination Address Figure 16-10Example: Internet to Local Network Rule Summary 16.1.3Example 3: DHCP Negotiation and Syslog Connection from the Internet Figure 16-12Syslog Rule Configuration Figure 16-13Example 3: Rule Summary Content Filtering 17.1 Categories 17.1.1 Restrict Web Features 17.1.2 Filter List 17.1.3 Time of Day 17.4 Customizing 17.5 Keywords 17.6 Logs Part IV: Advanced Management Page Filter Configuration 18.1 About Filtering 18.1.1 The Filter Structure of the ZyWALL Filter Set Execute Filter Rule Figure 18-2Filter Rule Process 18.2 Configuring a Filter Set Figure 18-5Menu 21.1 — Filter Set Configuration Edit Comments Menu 21.1.1 - Filter Rules Summary Figure 18-6NetBIOS_WAN Filter Rules Summary 18.2.1 Filter Rules Summary Menu 18.2.2 Configuring a Filter Rule 18.2.3 TCP/IP Filter Rule Figure 18-7Menu 21.1.1.1 — TCP/IP Filter Rule Table 18-3TCP/IP Filter Rule Menu Fields Page Page Figure 18-8Executing an IP Filter 18.2.4 Generic Filter Rule Table 18-4Generic Filter Rule Menu Fields 18.3 Example Filter Figure 18-11Example Filter — Menu Figure 18-12Example Filter Rules Summary — Menu 18.4 Filter Types and NAT 18.5 Firewall 18.6 Applying a Filter and Factory Defaults 18.6.1 LAN traffic 18.6.2 Remote Node Filters Figure 18-15Filtering Remote Node Traffic Page SNMP Configuration 19.1 About SNMP Figure 19-1SNMP Management Model 19.2 Supported MIBs 19.3 Configuring SNMP Figure 19-2Menu 22 — SNMP Configuration Table 19-2SNMP Configuration Menu Fields 19.4 SNMP Traps Page System Information & Diagnosis 20.1 System Status System Maintenance - Status Menu 24.1 - System Maintenance - Status Figure 20-2Menu 24.1 — System Maintenance — Status Table 20-1System Maintenance — Status Menu Fields 20.2 System Information and Console Port Speed 20.2.1 System Information 20.2.2 Console Port Speed 20.3 Log and Trace 20.3.1 Viewing Error Log Menu 24.3 - System Maintenance - Log and Trace Figure 20-6Menu 24.3 — System Maintenance — Log and Trace Figure 20-7Examples of Error and Information Messages 20.3.2 UNIX Syslog Page Page 20.3.3 Call-TriggeringPacket 20.4 Diagnostic 20.4.1 WAN DHCP Figure 20-11WAN & LAN DHCP Table 20-4System Maintenance Menu Diagnostic Page Firmware and Configuration Maintenance 21.1 Filename Conventions 21.2 Backup Configuration 21.2.1 Backup Configuration 21.2.2 Using the FTP Command from the Command Line 21.2.3 Example of FTP Commands from the Command Line 21.2.4 GUI-BasedFTP Clients 21.2.5 TFTP and FTP over WAN Will Not Work When 21.2.6 Backup Configuration Using TFTP 21.2.7 TFTP Command Example 21.2.8 GUI-BasedTFTP Clients 21.2.9 Backup Via Console Port 21.3 Restore Configuration 21.3.1 Restore Using FTP or TFTP 21.3.2 Procedure To Restore Using FTP 21.3.3 Restore Using FTP Session Example 21.3.4 Restore Via Console Port 21.4 Uploading Firmware and Configuration Files 21.4.1 Firmware File Upload 21.4.2 Configuration File Upload 21.4.3 FTP File Upload Command from the Command Line Example 21.4.4 FTP Session Example of Firmware File Upload 21.4.5 TFTP File Upload 21.4.6 TFTP Upload Command Example 21.4.7 Uploading Via Console Port 21.4.8 Uploading a Firmware File Via Console Port 21.4.9 Example Xmodem Firmware Upload Using HyperTerminal 21.4.10Uploading a Configuration File Via Console Port 21.4.11Example Xmodem Configuration Upload Using HyperTerminal Figure 21-19Example Xmodem Upload Page System Maintenance & Information 22.1 Command Interpreter Mode 22.2 Call Control Support 22.2.1 Budget Management 22.2.2 Call History 22.3 Time and Date Setting Figure 22-6Menu 24 — System Maintenance Figure 22-7Menu 24.10 System Maintenance — Time and Date Setting 22.3.1 Resetting the Time Page Page Remote Management 23.1 Telnet 23.2 FTP 23.3 Web 23.4 Remote Management Figure 23-2Menu 24.11 – Remote Management Control Table 23-1Menu 24.11 – Remote Management Control 23.4.1 Remote Management Limitations 23.5 Remote Management and NAT 23.6 System Timeout Page Part V: Call Scheduling and VPN/IPSec Page Call Scheduling 24.1 Introduction To delete a schedule set, enter the set number and press [SPACE BAR] or [DELETE] in the Edit Name field Menu 26.1 - Schedule Set Setup Figure 24-2Schedule Set Setup Duration PPPoE Figure 24-3Applying Schedule Set(s) to a Remote Node (PPPoE) Figure 24-4Applying Schedule Set(s) to a Remote Node (PPTP) Introduction to IPSec 25.1 Introduction 25.1.1 VPN 25.1.2 IPSec 25.1.3 Security Association 25.1.5 VPN Applications 25.2 IPSec Architecture 25.2.1 IPSec Algorithms 25.2.2 Key Management 25.3 Encapsulation 25.3.1 Transport Mode 25.3.2 Tunnel Mode 25.4 IPSec and NAT Table 25-1VPN and NAT VPN/IPSec Setup 26.1 VPN/IPSec Setup 26.2 IPSec Algorithms 26.2.1 AH (Authentication Header) Protocol 26.2.2 ESP (Encapsulating Security Payload) Protocol 26.3 IPSec Summary 26.3.1 My IP Address 26.3.2 Secure Gateway Address Page Figure 26-6Menu 27.1 — IPSec Summary Table 26-3Menu 27.1 — IPSec Summary Page Page 26.4 IPSec Setup Page Page 26.5 IKE Setup 26.5.1 IKE Phases 26.5.2 Negotiation Mode 26.5.3 Pre-SharedKey 26.5.4 Diffie-Hellman(DH) Key Groups 26.5.5 Perfect Forward Secrecy (PFS) Figure Page 26.6 Manual Setup 26.6.1 Active Protocol 26.6.2 Security Parameter Index (SPI) Figure 26-10Menu 27.1.1.2 — Manual Setup Table 26-7Menu 27.1.1.2 — Manual Setup Page Page SA Monitor 1.1. Introduction 27.1Using SA Monitor Table 27-1Menu 27.2 — SA Monitor Page Page IPSec Log 28.1 VPN Initiator IPSec Log 28.2 VPN Responder IPSec Log Page Table 28-2Sample IPSec Logs During Packet Transmission Table 28-3 RFC-2408ISAKMP Payload Types Page Page Part VI: Troubleshooting, Appendices and Index Page Troubleshooting 29.1 Problems Starting Up the ZyWALL 29.2 Problems with the LAN Interface 29.3 Problems with the WAN interface 29.4 Problems with Internet Access 29.5 Problems with the Password 29.6 Problems with Remote Management Appendix A The Big Picture Page Appendix B PPPoE PPPoE in Action Benefits of PPPoE Traditional Dial-upScenario How PPPoE Works ZyWALL as a PPPoE Client Appendix C PPTP Diagram 5 PPTP Protocol Overview Control & PPP connections Diagram 6 Example Message Exchange between PC and an ANT Page Page Appendix D Hardware Specifications Page Appendix E Important Safety Instructions Page Appendix F Boot Commands Diagram 8 Boot Module Commands Appendix G Command Interpreter Page Appendix H Firewall Commands Page Page Page Page Page Appendix NetBIOS Filter Commands NetBIOS Filter Configuration Page Page Index