Introduction, Required | HP Host Intrusion Detection System (HIDS)

Conﬁguration

Introduction

Introduction

Once you have installed or updated your HP-UX HIDS software (see the Release Notes cited in “Documentation” on page 2), you need to complete the conﬁguration with the following required and optional steps.

Required

Before you run HP-UX HIDS, you must complete the following conﬁguration steps.

•“Setting Up the HP-UX HIDS Secure Communications” on page 20

You need to create certiﬁcates to ensure secure communication between the Administration system and the Agent systems.

Optional

You may also need to complete one or more of the following steps.

•“Conﬁguring a Multihomed Agent System” on page 25

If you have agent systems with more than one IP address, you may need to specify the correct address to the agent and administration software.

•“Conﬁguring a Multihomed Administration System” on page 27

If you have an administration system with more than one IP address, you may need to specify the correct address to the administration software.

•“Conﬁguring a Loopback System” on page 29

•“Conﬁguring Ports” on page 30

•“Working with NIS” on page 30

•“Working with Firewalls” on page 30

•“Enabling Large Numbers of Agents” on page 31

•“Enabling Over 23 Agents (Thread Limits)” on page 31

If you have many agent systems, you may need to increase the thread limit on the administration system.

•“Enabling Over 20 Inbound Requests” on page 32

•“Accessing Manpages” on page 33

Chapter 2

19

Image 31

HP Host Intrusion Detection System (HIDS) manual Introduction, Required, Optional

Contents

Edition HP-UX Host Intrusion Detection System Administrator’s GuideManufacturing Part Number J5083-90013 December Government License Warranty Iii Trademarks Conventions Contents Schedule Manager Screen System Manager Screen Network Node Screen Host Manager ScreenVii Templates and Alerts Preferences ScreenViii Agent Conﬁguration File Idsagent CommandIdsadmin Command Automated Response Messages TroubleshootingHP Software License Original SSLeay License HP Software License Terms Xii Overview Documentation Summary Loss of Computing Resources Why Do You Need Intrusion Detection?Loss of Financial Assets Loss of Intellectual Property Malicious Code Who Are the Perpetrators?How Are These Threats Realized? Misplaced Trust Firewalls Why Existing Tools Are Only Part of the SolutionExcessive Privilege for Simple Tasks Being Used as a Springboard to Attack the Next Victim Security Auditing Tools Encryption What Is Intrusion Detection? Where Does Intrusion Detection Fit In? What HP-UX Hids Does What HP-UX Hids Does Not Do Graphic Representation HP-UX Hids Components HP-UX Hids Components How the Components Interact to Detect Intrusions Surveillance Schedules HP-UX Hids Secure CommunicationsDetection Templates Surveillance Groups Glossary of HP-UX Hids Terms Node Intrusion Detection DataIntrusion Detection System Kernel Virus System ManagerVulnerability Glossary of HP-UX Hids Terms Chapter Conﬁguration Conﬁguration Required IntroductionOptional Create the X.509 Certiﬁcates Setting Up the HP-UX Hids Secure CommunicationsOverview of Procedures to Set Up Secure Communications Script to Use Where Used End Product $ IDSgenAdminKeys install $ IDSgenAgentCerts Transport the Certiﬁcates TIP $ IDSimportAgentKeys /var/opt/ids/tmp/myhost1.tar.Z myadmin Install the Keys on Each Host Step Conﬁguring a Multihomed Agent System $ nslookup large2 Example To conﬁgure a multihomed administration system Conﬁguring a Multihomed Administration System Edit the agent conﬁguration ﬁle for example To conﬁgure a loopback system Conﬁguring a Loopback System Working with NIS Conﬁguring PortsWorking with Firewalls Select Kernel Conﬁguration Select Conﬁgurable Parameters Enabling Large Numbers of AgentsEnabling Over 23 Agents Thread Limits To change the value of maxthreadproc To view and change the value of tcpconnrequestmax Enabling Over 20 Inbound Requests Files Permissions Accessing ManpagesRestricting Permissions Runtime File Permissions Accessing Manpages Chapter Getting Started Getting Started System Manager Agents Starting HP-UX Hids for the First Time Set up hosts and run schedules See , Host Manager Screen, on Network Node Operations ScreensSchedule Manager Host Manager Sorting Entries Basic Screen ActionsSelecting Entries in Lists Searching Entries Basic Screen Actions Chapter System Manager Screen System Manager Screen System Manager Screen To stop the HP-UX Hids System Manager Starting the HP-UX Hids System ManagerStopping the HP-UX Hids System Manager To start the HP-UX Hids System Manager Status Field Values On the System Manager ScreenStatus Value Description To get the status of agent hosts Getting the Status of Agent HostsOn the System Manager screen To resynchronize agent hosts Resynchronizing Agent Hosts To activate a surveillance schedule on agent hosts Activating a Schedule on Agent HostsChoose the Actions Activate Schedule menu item To stop a surveillance schedule on agent hosts Stopping Schedules on Agent Hosts To start the agent Starting HP-UX Hids Agents To halt agents remotely from the System Manager Halting HP-UX Hids AgentsTo halt the agent locally on the agent host To go to the Schedule Manager screen Accessing Other ScreensGo to Schedule Manager Screen Go to Host Manager Screen Go to Preferences Screen Go to Network Node ScreenReturn to System Manager Screen Accessing Other Screens Chapter Schedule Manager Screen Schedule Manager Screen Schedule Manager To create a surveillance schedule Creating a Surveillance Schedule To close the Schedule Manager screen Displaying the Schedule Manager ScreenClosing the Schedule Manager Screen To display the Schedule Manager screen To create a new surveillance schedule Conﬁguring Surveillance SchedulesCreating a New Surveillance Schedule Copying a Surveillance Schedule To modify a surveillance schedule Modifying a Surveillance Schedule To rename a surveillance schedule Renaming a Surveillance ScheduleChoose File Save Selected Schedule As Undoing and Redoing Changes Deleting a Surveillance ScheduleTo delete a surveillance schedule To save a surveillance schedule Saving a Surveillance ScheduleChoose File Save Selected Schedule To create a new surveillance group Conﬁguring Surveillance GroupsCreating a New Surveillance Group Copying a Surveillance Group To modify a surveillance group Modifying a Surveillance Group Rename Surveillance Group Dialog Renaming a Surveillance GroupTo rename a surveillance group Saving a Surveillance Group Deleting a Surveillance GroupTo delete a surveillance group Modifying a Property Value In a Template Conﬁguring Detection TemplatesTo change the value of a property in a detection template To add a new value Edit List Dialog 11Edit Dialog Edit Suggested Best Practices Some Template Conﬁguration Guidelines Setting Surveillance Schedule Timetables To specify when a schedule will run Specifying When a Schedule Will Run Canceling Changes See Saving a Surveillance Schedule on To view the source of a surveillance schedule Viewing Surveillance Schedule DetailsViewing the Source of a Surveillance Schedule Refreshing the Details Display To clear the display Clearing the Details DisplaySaving the Details Display Save Dialog Predeﬁned Surveillance Schedules Predeﬁned Surveillance Schedules and Groups Predeﬁned Surveillance Schedules Host Manager Screen Host Manager Screen Displaying the Host Manager Screen Managing Hosts Closing the Host Manager Screen To add a new host manually Adding New HostsAdding a New Host Manually Add Host Dialog Address ﬁeld Host NameIP Address To add new hosts from /etc/hosts Adding New Hosts from /etc/hostsName ﬁeld Host Name and IP Address To add new hosts from a ﬁle Adding New Hosts from a FileRules for Host Lists Files Open Dialog To modify a host entry Modifying a Host To delete a host entry Deleting Hosts Enabling and Disabling Hosts To enable or disable an agent host for monitoring Add, modify or delete tags To add a tag Managing Tags To delete a tag To edit a tag Saving the Host List in the Current File Maintaining Host FilesSaving the Host List in a Different File Using Multiple Host Files Using an Alternate Host List File Maintaining Host Files Chapter Network Node Screen 100 To display the Network Node screen for an agent host Network Node ScreenOpening a Network Node Screen Closing a Network Node Screen 102 Alerts Tab HP-UX Hids Alerts What They Mean, What to Do HP-UX Hids Errors What They Mean, What to Do Errors Tab104 Simple Version General OperationsSelecting Entries Selecting with the Mouse Searching for a String Searching for the Next Unseen EntryFind Dialog To search again To delete one or more alerts or errorsDeleting an Entry Marking Entries as Seen or Unseen 108 Unseen Saving a New Log File Set Saving a Log File SetNetwork Node screen from the System Manager screen Saving the Current Log File Set Example Saving the File Set over Another File Set Save Dialog BoxPress Ctrl-A Example Creating a New File Set Log File Rotation Opening a Log File SetOpen Dialog Box 112 Preferences Screen 114 Preferences Screen 116 Option Default DescriptionGeneral Preferences To choosing Actions Status Poll from the System Manager Actions Resync from the System Manager screen 118 Column Name Default DescriptionBrowser Preferences Alert Events Preferences Column Default Description Name Error Events Preferences 120 System Manager Subtab Templates and Alerts Templates AlertsLimitations Property Types Table A-1 Detection Templates Alert SummaryAttack Detected Alert Alert Severity Detection Template 124 Appendix a 125 Examples Unix Regular Expressions126 Appendix a 127 128 Limitations Type I Pathnames to Not Monitor Template Property Types 130 Type II Pathnames/Programs Pairs Type IV UID Pairs Type III UIDs Type VI Time Strings Type V Network Triplets132 Type Viii Scalars Type VII Flags 134 Buffer Overﬂow Template Table A-3 Execute on Stack Alert Properties Name Type Default ValueExecute on Stack Table A-2 Template Properties 136 Table A-4 Unusual Argument Length Alert Properties Unusual Argument Length 138 Argument with Non-printable Character Appendix a 139 140 Table A-6 Template Properties Race Condition Template Table A-7 File Reference Modiﬁcation Alert Properties File Reference Modiﬁcation142 Appendix a 143 Table A-8 Setuid Script Executed Alert Properties Privileged Setuid Script Executed144 Appendix a 145 Table A-9 Template Properties Modiﬁcation of Files/Directories Template146 Properties 148 Table A-10 File Being Modiﬁed Alert Properties File Being Modiﬁed 150 Appendix a 151 Table A-11 Template Properties Changes to Log File Template152 Table A-12 Append-Only File Being Modiﬁed Alert Properties Append-Only File Being Modiﬁed 154 By this template Creation of Setuid File TemplateTable A-13 Template Properties Alerts generated Table A-14 Setuid File Created Alert Properties Setuid File Created156 Appendix a 157 Table A-15 Template Properties Creation of World-Writable File Template158 Table A-16 World-writable File Created Alert Properties World-Writable File Created 160 Appendix a 161 162 Table A-17 Template Properties Modiﬁcation of Another User’s File Template Table A-18 Non-owned File Being Modiﬁed Alert Properties Non-owned File Being Modiﬁed164 Appendix a 165 Limitations 166 Table A-19 Template Properties Login/Logout Template 168 Table A-20 Login/Logout Alert Properties Login/Logout Table A-21 Successful su Detected Alert Properties Successful su Detected170 Appendix a 171 172 Table A-22 Template Properties Repeated Failed Logins TemplateTemplate How this template Table A-23 Failed Login Attempts Alert Properties Failed Login Attempts174 Appendix a 175 Table A-25 Repeated Failed Su Attempts Alert Properties Repeated Failed su Commands TemplateRepeated Failed su Attempts Table A-24 Template Properties Appendix a 177 178 Template Conﬁguration Syntax Appendix a 179 180 Automated Response 182 Response Methods General Guidelines 184 Programming Notes How Automated Response Works in HP-UX HidsAlert Process Security checks 186 Table B-1 Additional Arguments Passed to Response Programs Appendix B 187 Name Value Description Table B-3 Environment Variables Set for Response Programs188 Appendix B 189 190 Programming GuidelinesWriting Perl vs. Shell Response Scripts Writing Privileged Response Programs Code Examples Solution aCode for scriptA.sh 192 Solution BCode for privA program Code for PrivB program Solution C Code for scriptC.sh script #!/usr/bin/sh Code for privC program194 Sample C Language Program Source Code Sample Response ProgramsSample Shell Script Alert Responses 196 Forwarding Information Appendix B 197 198 Halting any further attacks Appendix B 199 200 Preservation of evidence Appendix B 201 202 Restoration of a known good state HP OpenView Operations Smart Plug-In OVO Enablement in HP-UX Hids 204 Idsagent Command 206 Synopsis Options Idsagent Command 208 Example Idsadmin Command 210 Synopsis Startup Options Idsadmin Command 212 Commands Agent Conﬁguration File 214 Forcing Active Agent to Reread Conﬁguration File Agent Conﬁguration File 216 Name Default ValueGlobal Conﬁguration Table E-1 Global Conﬁguration Variables DSP idskernDSP Parameters Data Source Process ConﬁgurationKernel Audit Data DSP Table E-2 218 Table E-3 Remote Communication ConﬁgurationCorrelator Conﬁguration Variables 220 Messages 222 Agent Messages Idsagent failed to start group Idsagent failed to reopen stderr in append modeIdsagent internal error in handling signature groups Idsagent failed to initialize conﬁguration module Idsagent unable to setup Sigsegv signal handler Idsagent unable to setup Sigchld signal handlerIdsagent unable to setup Sighup signal handler Idsagent unable to setup signal handler Idsagent failed to execute correlator corr Idsagent error trying to shutdown a processIdsagent failed to allocate memory Idsagent failed to create schedule path ﬁlename Idsagent failed to initialize schedule in crontab Idsagent internal error no correlator in PMStartProcessesIdsagent internal error occurred in PMStopGroup Idsagent failed to initialize schedule Idsagent out of process table space Idsagent not enough disk space to create scheduleIdsagent not enough disk space to parse schedule Idsagent not enough disk space to save conﬁg ﬁle Internal error unknown state Internal errorUnable to open the response script directory dir System Manager Messages Invalid Property Value value Property Value Error Exception while opening ﬁle filename File Save ErrorIncomplete or Invalid Entry Data Entry Error Invalid Host State Unable to disable host Select Property to be edited Selection Error No more instances of searchstring found Find ErrorOnly one property may be edited at a time Selection Error Searchstring not found Find Error Select Surveillance Schedule to delete Selection Error Select Surveillance Group Name to delete Selection ErrorSelect Surveillance Group to copy Selection Error Select Surveillance Schedule to copy Selection Error 234 Surveillance Schedule not selected Schedule Selection ErrorFollowing hosts are in an invalid state for this command Unable to Overwrite filename File Save Error Unknown IP Address unable to resolve Host Name Unknown Host unable to resolve IP Address IPaddress 236 Troubleshooting 238 Appendix G 239 Agent and System Manager cannot communicate with each other Troubleshooting240 $ /usr/sbin/kmtune -q enableidds Agent does not start on system boot 242 Agent needs further troubleshootingTo clean up the IDS message queues Agent host appears to hang and/or you see message disk full Alerts are not being displayed in the alert browser Agent does not start after installationAgents appear to be stuck in polling status Alert date/time sort seems inconsistent 244 Idsadmin needs installed agent certiﬁcatesBuffer overﬂow triggers false positives Duplicate alerts appear in System Manager IDSgenAdminKeys or idsgui quits early IDScheckInstall fails with a kmtune error 246 Large ﬁles in /var/opt/idsLog ﬁles are ﬁlling up No Agent Available Schedule Manager timetable screen appears to hang SSH does not perform a clean exit after idsgent is started System Manager does not start after idsgui is started System Manager appears to hang248 Unknown program and arguments in certain alert messages Using HP-UX Hids with IPFilter and SecureShellIPFilter rules for HP-UX Hids 250 How to allow the SecureShell daemon to forward X11 trafﬁc Appendix G 251 252 Appendix H 253 HP Software License 254 OpenSSL License Appendix H 255 Original SSLeay License 256 HP Software License Terms 258