Troubleshooting | HP Host Intrusion Detection System (HIDS) guide

G Troubleshooting

Appendix G

237

Image 249

HP Host Intrusion Detection System (HIDS) manual Troubleshooting

Contents

HP-UX Host Intrusion Detection System Administrator’s Guide EditionManufacturing Part Number J5083-90013 December Government License Warranty Iii Trademarks Conventions Contents Schedule Manager Screen System Manager Screen Host Manager Screen Network Node ScreenVii Preferences Screen Templates and AlertsViii Idsadmin Command Idsagent CommandAutomated Response Agent Conﬁguration File Troubleshooting MessagesHP Software License Original SSLeay License HP Software License Terms Xii Overview Documentation Summary Loss of Financial Assets Why Do You Need Intrusion Detection?Loss of Intellectual Property Loss of Computing Resources How Are These Threats Realized? Who Are the Perpetrators?Misplaced Trust Malicious Code Excessive Privilege for Simple Tasks Why Existing Tools Are Only Part of the SolutionBeing Used as a Springboard to Attack the Next Victim Firewalls Security Auditing Tools Encryption What Is Intrusion Detection? Where Does Intrusion Detection Fit In? What HP-UX Hids Does What HP-UX Hids Does Not Do Graphic Representation HP-UX Hids Components HP-UX Hids Components How the Components Interact to Detect Intrusions Detection Templates HP-UX Hids Secure CommunicationsSurveillance Groups Surveillance Schedules Glossary of HP-UX Hids Terms Intrusion Detection System Intrusion Detection DataKernel Node System Manager VirusVulnerability Glossary of HP-UX Hids Terms Chapter Conﬁguration Conﬁguration Introduction RequiredOptional Overview of Procedures to Set Up Secure Communications Setting Up the HP-UX Hids Secure CommunicationsScript to Use Where Used End Product Create the X.509 Certiﬁcates $ IDSgenAdminKeys install $ IDSgenAgentCerts Transport the Certiﬁcates TIP $ IDSimportAgentKeys /var/opt/ids/tmp/myhost1.tar.Z myadmin Install the Keys on Each Host Step Conﬁguring a Multihomed Agent System $ nslookup large2 Example To conﬁgure a multihomed administration system Conﬁguring a Multihomed Administration System Edit the agent conﬁguration ﬁle for example To conﬁgure a loopback system Conﬁguring a Loopback System Conﬁguring Ports Working with NISWorking with Firewalls Enabling Over 23 Agents Thread Limits Enabling Large Numbers of AgentsTo change the value of maxthreadproc Select Kernel Conﬁguration Select Conﬁgurable Parameters To view and change the value of tcpconnrequestmax Enabling Over 20 Inbound Requests Restricting Permissions Accessing ManpagesRuntime File Permissions Files Permissions Accessing Manpages Chapter Getting Started Getting Started System Manager Agents Starting HP-UX Hids for the First Time Set up hosts and run schedules See , Host Manager Screen, on Schedule Manager Operations ScreensHost Manager Network Node Selecting Entries in Lists Basic Screen ActionsSearching Entries Sorting Entries Basic Screen Actions Chapter System Manager Screen System Manager Screen System Manager Screen Stopping the HP-UX Hids System Manager Starting the HP-UX Hids System ManagerTo start the HP-UX Hids System Manager To stop the HP-UX Hids System Manager On the System Manager Screen Status Field ValuesStatus Value Description Getting the Status of Agent Hosts To get the status of agent hostsOn the System Manager screen To resynchronize agent hosts Resynchronizing Agent Hosts Activating a Schedule on Agent Hosts To activate a surveillance schedule on agent hostsChoose the Actions Activate Schedule menu item To stop a surveillance schedule on agent hosts Stopping Schedules on Agent Hosts To start the agent Starting HP-UX Hids Agents Halting HP-UX Hids Agents To halt agents remotely from the System ManagerTo halt the agent locally on the agent host Go to Schedule Manager Screen Accessing Other ScreensGo to Host Manager Screen To go to the Schedule Manager screen Go to Network Node Screen Go to Preferences ScreenReturn to System Manager Screen Accessing Other Screens Chapter Schedule Manager Screen Schedule Manager Screen Schedule Manager To create a surveillance schedule Creating a Surveillance Schedule Closing the Schedule Manager Screen Displaying the Schedule Manager ScreenTo display the Schedule Manager screen To close the Schedule Manager screen Creating a New Surveillance Schedule Conﬁguring Surveillance SchedulesCopying a Surveillance Schedule To create a new surveillance schedule To modify a surveillance schedule Modifying a Surveillance Schedule Renaming a Surveillance Schedule To rename a surveillance scheduleChoose File Save Selected Schedule As Deleting a Surveillance Schedule Undoing and Redoing ChangesTo delete a surveillance schedule Saving a Surveillance Schedule To save a surveillance scheduleChoose File Save Selected Schedule Creating a New Surveillance Group Conﬁguring Surveillance GroupsCopying a Surveillance Group To create a new surveillance group To modify a surveillance group Modifying a Surveillance Group Renaming a Surveillance Group Rename Surveillance Group DialogTo rename a surveillance group Deleting a Surveillance Group Saving a Surveillance GroupTo delete a surveillance group Conﬁguring Detection Templates Modifying a Property Value In a TemplateTo change the value of a property in a detection template To add a new value Edit List Dialog 11Edit Dialog Edit Suggested Best Practices Some Template Conﬁguration Guidelines Setting Surveillance Schedule Timetables To specify when a schedule will run Specifying When a Schedule Will Run Canceling Changes See Saving a Surveillance Schedule on Viewing the Source of a Surveillance Schedule Viewing Surveillance Schedule DetailsRefreshing the Details Display To view the source of a surveillance schedule Saving the Details Display Clearing the Details DisplaySave Dialog To clear the display Predeﬁned Surveillance Schedules Predeﬁned Surveillance Schedules and Groups Predeﬁned Surveillance Schedules Host Manager Screen Host Manager Screen Displaying the Host Manager Screen Managing Hosts Closing the Host Manager Screen Adding a New Host Manually Adding New HostsAdd Host Dialog To add a new host manually Host Name Address ﬁeldIP Address Name ﬁeld Adding New Hosts from /etc/hostsHost Name and IP Address To add new hosts from /etc/hosts Rules for Host Lists Files Adding New Hosts from a FileOpen Dialog To add new hosts from a ﬁle To modify a host entry Modifying a Host To delete a host entry Deleting Hosts Enabling and Disabling Hosts To enable or disable an agent host for monitoring Add, modify or delete tags To add a tag Managing Tags To delete a tag To edit a tag Maintaining Host Files Saving the Host List in the Current FileSaving the Host List in a Different File Using Multiple Host Files Using an Alternate Host List File Maintaining Host Files Chapter Network Node Screen 100 Opening a Network Node Screen Network Node ScreenClosing a Network Node Screen To display the Network Node screen for an agent host 102 Alerts Tab HP-UX Hids Alerts What They Mean, What to Do Errors Tab HP-UX Hids Errors What They Mean, What to Do104 Selecting Entries General OperationsSelecting with the Mouse Simple Version Searching for the Next Unseen Entry Searching for a StringFind Dialog Deleting an Entry To delete one or more alerts or errorsMarking Entries as Seen or Unseen To search again 108 Unseen Network Node screen from the System Manager screen Saving a Log File SetSaving the Current Log File Set Saving a New Log File Set Press Ctrl-A Save Dialog BoxExample Creating a New File Set Example Saving the File Set over Another File Set Opening a Log File Set Log File RotationOpen Dialog Box 112 Preferences Screen 114 Preferences Screen General Preferences Option Default DescriptionTo choosing Actions Status Poll from the System Manager 116 Actions Resync from the System Manager screen Browser Preferences Column Name Default DescriptionAlert Events Preferences 118 Column Default Description Name Error Events Preferences 120 System Manager Subtab Templates and Alerts Limitations AlertsProperty Types Templates Alert Summary Table A-1 Detection TemplatesAttack Detected Alert Alert Severity Detection Template 124 Appendix a 125 Unix Regular Expressions Examples126 Appendix a 127 128 Limitations Type I Pathnames to Not Monitor Template Property Types 130 Type II Pathnames/Programs Pairs Type IV UID Pairs Type III UIDs Type V Network Triplets Type VI Time Strings132 Type Viii Scalars Type VII Flags 134 Buffer Overﬂow Template Execute on Stack Name Type Default ValueTable A-2 Template Properties Table A-3 Execute on Stack Alert Properties 136 Table A-4 Unusual Argument Length Alert Properties Unusual Argument Length 138 Argument with Non-printable Character Appendix a 139 140 Table A-6 Template Properties Race Condition Template File Reference Modiﬁcation Table A-7 File Reference Modiﬁcation Alert Properties142 Appendix a 143 Privileged Setuid Script Executed Table A-8 Setuid Script Executed Alert Properties144 Appendix a 145 Modiﬁcation of Files/Directories Template Table A-9 Template Properties146 Properties 148 Table A-10 File Being Modiﬁed Alert Properties File Being Modiﬁed 150 Appendix a 151 Changes to Log File Template Table A-11 Template Properties152 Table A-12 Append-Only File Being Modiﬁed Alert Properties Append-Only File Being Modiﬁed 154 Table A-13 Template Properties Creation of Setuid File TemplateAlerts generated By this template Setuid File Created Table A-14 Setuid File Created Alert Properties156 Appendix a 157 Creation of World-Writable File Template Table A-15 Template Properties158 Table A-16 World-writable File Created Alert Properties World-Writable File Created 160 Appendix a 161 162 Table A-17 Template Properties Modiﬁcation of Another User’s File Template Non-owned File Being Modiﬁed Table A-18 Non-owned File Being Modiﬁed Alert Properties164 Appendix a 165 Limitations 166 Table A-19 Template Properties Login/Logout Template 168 Table A-20 Login/Logout Alert Properties Login/Logout Successful su Detected Table A-21 Successful su Detected Alert Properties170 Appendix a 171 172 Repeated Failed Logins Template Table A-22 Template PropertiesTemplate How this template Failed Login Attempts Table A-23 Failed Login Attempts Alert Properties174 Appendix a 175 Repeated Failed su Attempts Repeated Failed su Commands TemplateTable A-24 Template Properties Table A-25 Repeated Failed Su Attempts Alert Properties Appendix a 177 178 Template Conﬁguration Syntax Appendix a 179 180 Automated Response 182 Response Methods General Guidelines 184 Alert Process How Automated Response Works in HP-UX HidsSecurity checks Programming Notes 186 Table B-1 Additional Arguments Passed to Response Programs Appendix B 187 Table B-3 Environment Variables Set for Response Programs Name Value Description188 Appendix B 189 Writing Perl vs. Shell Response Scripts Programming GuidelinesWriting Privileged Response Programs 190 Solution a Code ExamplesCode for scriptA.sh Code for privA program Solution BCode for PrivB program 192 Solution C Code for privC program Code for scriptC.sh script #!/usr/bin/sh194 Sample Response Programs Sample C Language Program Source CodeSample Shell Script Alert Responses 196 Forwarding Information Appendix B 197 198 Halting any further attacks Appendix B 199 200 Preservation of evidence Appendix B 201 202 Restoration of a known good state HP OpenView Operations Smart Plug-In OVO Enablement in HP-UX Hids 204 Idsagent Command 206 Synopsis Options Idsagent Command 208 Example Idsadmin Command 210 Synopsis Startup Options Idsadmin Command 212 Commands Agent Conﬁguration File 214 Forcing Active Agent to Reread Conﬁguration File Agent Conﬁguration File Global Conﬁguration Name Default ValueTable E-1 Global Conﬁguration Variables 216 Kernel Audit Data DSP Data Source Process ConﬁgurationTable E-2 DSP idskernDSP Parameters 218 Remote Communication Conﬁguration Table E-3Correlator Conﬁguration Variables 220 Messages 222 Agent Messages Idsagent internal error in handling signature groups Idsagent failed to reopen stderr in append modeIdsagent failed to initialize conﬁguration module Idsagent failed to start group Idsagent unable to setup Sighup signal handler Idsagent unable to setup Sigchld signal handlerIdsagent unable to setup signal handler Idsagent unable to setup Sigsegv signal handler Idsagent failed to allocate memory Idsagent error trying to shutdown a processIdsagent failed to create schedule path ﬁlename Idsagent failed to execute correlator corr Idsagent internal error occurred in PMStopGroup Idsagent internal error no correlator in PMStartProcessesIdsagent failed to initialize schedule Idsagent failed to initialize schedule in crontab Idsagent not enough disk space to parse schedule Idsagent not enough disk space to create scheduleIdsagent not enough disk space to save conﬁg ﬁle Idsagent out of process table space Internal error Internal error unknown stateUnable to open the response script directory dir System Manager Messages Incomplete or Invalid Entry Data Entry Error Exception while opening ﬁle filename File Save ErrorInvalid Host State Unable to disable host Invalid Property Value value Property Value Error Only one property may be edited at a time Selection Error No more instances of searchstring found Find ErrorSearchstring not found Find Error Select Property to be edited Selection Error Select Surveillance Group to copy Selection Error Select Surveillance Group Name to delete Selection ErrorSelect Surveillance Schedule to copy Selection Error Select Surveillance Schedule to delete Selection Error Following hosts are in an invalid state for this command Surveillance Schedule not selected Schedule Selection ErrorUnable to Overwrite filename File Save Error 234 Unknown IP Address unable to resolve Host Name Unknown Host unable to resolve IP Address IPaddress 236 Troubleshooting 238 Appendix G 239 Troubleshooting Agent and System Manager cannot communicate with each other240 $ /usr/sbin/kmtune -q enableidds Agent does not start on system boot To clean up the IDS message queues Agent needs further troubleshootingAgent host appears to hang and/or you see message disk full 242 Agents appear to be stuck in polling status Agent does not start after installationAlert date/time sort seems inconsistent Alerts are not being displayed in the alert browser Buffer overﬂow triggers false positives Idsadmin needs installed agent certiﬁcatesDuplicate alerts appear in System Manager 244 IDSgenAdminKeys or idsgui quits early IDScheckInstall fails with a kmtune error Log ﬁles are ﬁlling up Large ﬁles in /var/opt/idsNo Agent Available 246 Schedule Manager timetable screen appears to hang SSH does not perform a clean exit after idsgent is started System Manager appears to hang System Manager does not start after idsgui is started248 Using HP-UX Hids with IPFilter and SecureShell Unknown program and arguments in certain alert messagesIPFilter rules for HP-UX Hids 250 How to allow the SecureShell daemon to forward X11 trafﬁc Appendix G 251 252 Appendix H 253 HP Software License 254 OpenSSL License Appendix H 255 Original SSLeay License 256 HP Software License Terms 258