OSSIM-Hands-On.pdf

Hands­On OSSIM 2.3

Your Quick & Dirty Guide to  Understanding and Deploying OSSIM  By Houcem Hachicha

Disclaimer All the information in this report is not necessarily correct.

License Not for commercial use.

Abstract This work deals with OSSIM Security Information Management system (version 2.3 as of this writing). The report gives a detailed description of OSSIM's core components: sensor, server, database and framework. Installation process, advanced configuration, and server administration, are all covered. Details about integration of third party devices, including development of custom plugins for unsupported ones, are explained in this writing. Correlation techniques and CALM Risk assessment algorithm are described. OSSEC, OCS Inventory, Nagios, and other open source software are dealt with in their integration within OSSIM. Keywords: SIM, OSSIM, risk, correlation, sensor, server, CALM, OSSEC, open source.

1 / 30

Outline 1. Security Information Management systems.....................................................................................3 2. OSSIM: the Open Source SIM.........................................................................................................4 2.1. General description...................................................................................................................4 2.2. OSSIM's architecture................................................................................................................4 2.2.1. OSSIM sensor...................................................................................................................5 2.2.2. OSSIM server....................................................................................................................6 2.2.3. OSSIM database................................................................................................................7 2.2.4. OSSIM framework............................................................................................................7 3. Installing OSSIM..............................................................................................................................8 3.1. Downloading the .iso image.....................................................................................................8 3.2. Selecting the profile..................................................................................................................9 3.3. Configuring the network.........................................................................................................10 3.4. Setting up the partition table...................................................................................................10 3.5. Installing the base system.......................................................................................................10 3.6. Activating plugins...................................................................................................................11 3.7. Installation settings modification ...........................................................................................12 3.8. Updating the system ...............................................................................................................13 3.9. Accessing the web UI.............................................................................................................13 4. Assets and network discovery........................................................................................................15 5. Data collection ...............................................................................................................................17 5.1. Cisco PIX and Cisco FWSM..................................................................................................17 5.2. Linux server and Apache .......................................................................................................17 5.3. Windows host .........................................................................................................................18 5.4. OCS inventory........................................................................................................................18 6. Risk and correlation setup..............................................................................................................21 6.1. Risk configuration ..................................................................................................................21 6.2. Aggregated Risk and CALM algorithm..................................................................................21 6.3. Correlation directives..............................................................................................................23 7. OSSIM tuning ................................................................................................................................26 7.1. Least privilege account creation ............................................................................................26 7.2. Extending partitions................................................................................................................26 7.3. Filtering noise.........................................................................................................................27 7.4. Tuning panels .........................................................................................................................29 8. Conclusion......................................................................................................................................30

2 / 30

1. Security Information Management systems Infrastructure of information technology companies is getting more and more complicated. Security devices are heterogeneous, ranging from firewalls and IDSs, to AntiViruses and Spam filters. Yet, they all have something in common: they store data on the so called “logs”. A log is a record of events that happens to a particular software. Thus, logs contain all the data a security manager needs. However, logs are too long to be analyzed. Indeed, a study conducted by Gartner, the IT research and advisory firm headquartered in Stamford USA, showed that out of 700 European IT managers questioned, 45% receive over 4,000 security log line every second from their IT systems. Furthermore, according to the same source, one in ten IT departments spends more than three days a week analyzing security log data. A Security Information Management system (SIM), is a tool that fills in that gap by collecting eventlogs into a central repository for trend analysis. Indeed, it: •

centralizes log information;

•

correlates logs to establish cause-effect relationship between events;

•

aggregates security events to a manageable list;

•

prevents possible damage/flaws on the company’s resources;

•

generates a security dashboard for management, assuring compliance with security policies (internal or external);

•

etc.

Hence, deploying a SIM in a company's infrastructure is extremely beneficial.

3 / 30

2. OSSIM: the Open Source SIM Various SIM products exist on the market. Perhaps the most famous closed-source SIM is ArcSight, as it is said to be the leader of the market. An open source popular solution exists: OSSIM.

2.1. General description OSSIM is a fully featured SIM solution that offers all the necessary functionality, ranging from the detection at low-level to high-level reporting. Founded in 2003 by AlienVault, OSSIM is at the time of this writing the de-facto standard in Open Source Security Information Management. Indeed, AlienVault claims to have an installed user base exceeding 10,000 units, which accounts for half of the installed SIM market. Based on GNU/Linux Debian, kernel 2.6, OSSIM takes advantage of the wealth of free software “gems”, by integrating a handy suite of security open source tools, among which: •

OSSEC Host Intrusion Detection System;

•

OpenVAS Vulnerability Assessment Scanner;

•

OCS Inventory;

•

Nagios Availability Monitor;

•

Nmap Port Scanner;

•

etc.

Note that OSSIM and all of these tools are licensed under the strict GPL license. As a vulnerability scanner, earlier versions of OSSIM included Nessus. However, when Tenable company brought Nessus and changed its license to a proprietary one, OSSIM developers integrated OpenVAS, which is a GPL-licensed fork of Nessus. Apart from the advantages of a SIM systems in general, OSSIM has several benefits over proprietary products: •

free: no license fees to afford, just implementation costs;

•

has a strong community support, professional support can be brought later on;

•

runs perfectly on common hardware, no fancy servers needed;

•

has an open architecture;

•

integrates powerful open source software.

2.2. OSSIM's architecture The first thing one must do in order to deploy OSSIM (or any SIM) in its infrastructure, is to decide which architecture to opt for. OSSIM's architecture is based on four main components: •

OSSIM sensor: it collects and normalizes the events generated by the different security equipment. One can deploy as many sensors as needed;

•

OSSIM server: it receives events from sensors, and does Risk Assessment and Correlation tasks; 4 / 30

•

OSSIM database: it is a MySQL database that stores events, configurations, and useful information.

•

OSSIM framework: it is the PHP/Python code that “serves” the information to the web front-end.

The choice of a convenient architecture depends mainly on the number of machines one will allocate to OSSIM. For instance, the recommended architecture using a three machines is the following:

OSSIM simple architecture using three machines In this particular architecture, the management server will play the role of OSSIM framework and OSSIM server. OSSIM server and OSSIM database are deployed on separate machines. This is the recommended architecture because usually OSSIM sensor and database require more hardware resources than other OSSIM components. Let us discover more details about OSSIM by following an example of an event generated by a security equipment that goes through the sensor, server, database, framework, to end in the web front-end.

2.2.1. OSSIM sensor If we suppose that we have a Stonegate Firewall we want to monitor using OSSIM, in this case, we have to set up that Firewall to send its logs to OSSIM sensor. A relevant method to do so is using syslog,   which  is  the  standard  protocol  for   forwarding   log   messages.   Hence, we have Stonegate firewall sending its logs to the sensor via syslog. OSSIM sensor is configured (by default) to receive logs from external devices. Indeed, it has rsyslogd service enabled out of the box. Typically, the received logs will get stored in the /var/log/syslog file. A sample Stonegate log line from that file would be:

5 / 30

Jul 14 16:25:43 "2010-07-14 16:25:41","5928048","x.x.27.253", "Protocol Agent","Notification",,,"x.x.27.3","x.x.186.59","HTTP","TCP","1326","80",,,,,,"407109632",,"HT TP",,,,,,,,,,,,,"FR_Hammemlif node 1", ,,,"2010-07-14 16:25:42" ,"Firewall","HTTP_URLLogged",,,"5365022921374463088",,,,

OSSIM sensor parses incoming events using different plugins. One can choose which plugins to activate in the installation process. Activating Stonegate Firewall plugin will result in OSSIM sensor looking for Stonegate related events in /var/log/syslog. Each plugin has a set of python regular expressions (regexes) that will be matched against log lines in /var/log/syslog. Once the regex matches the log line, useful data will be extracted, like source IP address, destination IP address, source port, destination port, event signature, etc. Out of the previous log line, Stonegate plugin will extract the following information: •

date: Jul 14 16:25:41;

•

source IP address: x.x.27.3;

•

destination IP address: x.x.186.59;

•

source Port: 1326;

•

destination Port: 80;

•

protocol: TCP;

•

event signature: Protocol Agent;

•

etc.

This process is called “normalization”. Normalized data, along with the brute log line are then sent to OSSIM server.

2.2.2. OSSIM server Based on the normalized event, the server will perform risk assessment and correlation tasks. Risk assessment depends on three variables: •

asset value;

•

event's Priority;

•

event's Reliability.

Assets are key for SIM systems. An asset is an equipment that one wants to secure. Hosts, firewalls, IDSs, servers, etc, are all assets. For each asset one defines in OSSIM, an asset-value between 0 and 5 that has to be specified. This value measures the asset's importance. Priority and reliability are both values relative to each event, they range from 0 to 10. Priority measures the relative importance of the event as an attack, while reliability measures the probability of the event itself being real. For instance, IDSs are known for generating many false positives. 6 / 30

Those need to have a weak reliability value. Each of those three variables will be combined in one called Risk. Risk = (Asset * Priority * Reliability) / 25 Therefore, OSSIM server will calculate the risk value for each event. Once that value bypasses 1, the server will generate an alarm to notify the security manager. To illustrate, let us suppose that x.x.27.3 is an asset of ours with a value of 5. Knowing that “Protocol Agent” event has a reliability of 2, and a priority of 5, the risk calculated for source IP is: Risk=(5*5*2)/25=2. An alarm will automatically be generated. However, x.x.186.59 is not one of our assets. It gets the default asset value, which is 2. The risk for destination IP will be: Risk=(2*5*2)/25 SIEM), Cisco related events started to pour in. OSSIM was processing the logs, identifying events, and normalizing them as can be seen in the screen-shot below.

5.2. Linux server and Apache  To monitor the RedHat box using OSSEC, I had two choices: •

forward the logs via syslog to OSSIM; OSSIM can handle those directly using Apache and Linux plugins;

•

install an OSSEC agent on the RedHat box, that will parse the relevant data and communicate it to OSSIM.

OSSIM relies on OSSEC for remote data collection. OSSEC is an independent open source project. It is a Host Intrusion Detection System that is capable of collecting and processing logs from various equipment, including Linux, Windows, HP-UX and IBM AIX systems, Cisco firewalls, Apache etc. OSSEC has a server client architecture and is perfectly integrated in OSSIM. I decided to opt for the second option. I preferred adding another service (ossec-agent service) on the RedHat server, rather than modifying configuration files to forward logs via syslog. And OSSIM already had plenty of syslog data forwarded to OSSIM. OSSEC has an agent server classic architecture. The server is already configured on OSSIM, so we only needed to install the agent on the Linux Box. Behind the scenes, that agent will send data to OSSEC server through UDP 1514. Consequently, OSSEC server will generate logs that OSSEC plugin will parse. Note that OSSEC's agent communicates with its server using a symmetric encryption key (Blowfish algorithm). The agent can be downloaded from the web UI (Tools > Downloads).

17 / 30

OSSIM's available downloads Beside importing the communication key from OSSEC server to the agent, installation is straightforward. I checked in the SIEM section of the web UI that OSSIM was collecting events from the Linux server along with Apache.

5.3. Windows host  OSSIM can harvest data from Windows hosts (and servers) through several different agents, namely: Snare, Osiris, and OSSEC. I tested all the three, and decided to deploy OSSEC. Snare and Osiris are great software, but all they do is log collection. However, OSSEC is more than just a log collection tool. It includes log analysis, file integrity checking, policy monitoring, rootkit detection, real-time alerting, and active response. Installation of OSSEC on a Windows host is straight forward: as usual, I downloaded the agent from the link provided by OSSIM, installed it on the Windows host, imported the symmetric communication key from OSSEC server and finally started the agent.

5.4. OCS inventory OCS inventory is an open source software that allows a network administrator to build an inventory containing details about every machine on the network, like type of the OS, installed software, hardware resources, etc... This tool is perfectly integrated in OSSIM. Like OSSEC, OCS inventory has an agent-server architecture. The server is already installed in OSSIM, while agents can be downloaded from the web UI. 18 / 30

Inventory overview

After deploying a couple of OCS agents on a Linux server and a Windows host, I saw the pie-charts appear under (Dashboards > Inventory) in the web UI. Clicking on the Windows host's name showed a variety of useful information, as the screen-shot below demonstrates.

Inventory details

19 / 30

5.4. Nagios  Nagios is an availability monitor. Like OCS and OSSEC, deploying a Nagios agent on a Linux server allowed me to monitor the availability of its different services.

Nagios availability monitor in OSSIM Once a service is down, Nagios generates events to alert the network administrator.

20 / 30

6. Risk and correlation setup 6.1. Risk configuration  As explained previousely; risk formula in OSSIM is: Risk = (Asset * Priority * Reliability) / 25 It is important to configure each of these three variables. Combined, they will determine the risk of any particular event. Indeed, I already defined the value of my Assets (section 2.2.). Now time to fix the Priority and reliability of plugin-events. Hopefully, those values can be easily modified using the web administration UI.

Availability and Reliability setup

6.2. Aggregated Risk and CALM algorithm CALM (Compromise and Attack Level Monitor) is an assessment algorithm used by OSSIM to aggregate risk. Its input is a high volume of events, and its output is a single indicator of the general state of security of each asset.

21 / 30

Top 10 nets risk As shown in the previous figure, CALM algorithm displays in real time the compromise and attack level of networks. •

Compromise (C), measures the probability that an asset is compromised;

•

Attack (A), measures how frequently an asset is being attacked.

Each asset has its A and C variables. Those are incremented according to three rules: 1. Any possible attack launched from machine 1 on machine 2 will increase the A of machine 2 and the C of machine 1. 2. When there is an attack response (an event that indicates the attack’s success), the value of C will increase for both machines 1 and 2. 3. If the events are internal, the C value will go up only for the originating machine. CALM is intended for real-time monitoring, hence, the algorithm should have a short-term memory that places importance on the most recent events and discards the oldest ones. Indeed, “Threshold” and “Recovery ratio” are two variables intended to periodically lower the C and A levels for each machine by a constant value. Those variables can be set in (Configuration > Main).

22 / 30

Recovery ratio and Threshold setup

6.3. Correlation directives Correlation is perhaps the most interesting feature in a SIM system. Correlation allows the security manager to both detect complex attacks, and avoid false positives. OSSIM has three types of correlation: cross correlation, inventory correlation, and logical correlation. Only logical correlation is part of my project's scope. OSSIM has a set of written logical correlation directives. A correlation directive is a sequence of rules for OSSIM server to follow in order to correlate events. In each rule, one can specify: •

event's signature to detect;

•

reliability of the rule;

•

occurrence of that event along with timeout;

•

source and destination ports/IP addresses;

•

etc.

Note that risk is calculated for each rule using the classic formula. Just like normal events, rules (and directives) can generate alarms.

23 / 30

Correlation directives in OSSIM Creating correlation directives between events related to various devices requires a deep understanding of one's network architecture, security devices, and, most of all, attack scenarios and their impact on each device. Since I don't have that kind of knowledge, I tested the correlation feature in OSSIM by creating simple directives. Once a new directive is created, one should restart OSSIM server in order for that directive to be effective. #/etc/init.d/ossim-server restart Restarting the ossim-server service should prompt error messages if any problem is detected in the structure of the directive.

24 / 30

Correlation directive creation

25 / 30

7. OSSIM tuning  7.1. Least privilege account creation  In Configuration > users, selecting “create a new user” prompts the following:

Web UI Account creation I chose which network related events the new user will be able to see. I also selected which sections of the web UI I wanted the new user to have access to.

7.2. Extending partitions During OSSIM installation, I used LVM to set the disk size of each key repertory. For instance, I assigned 6.5GB to /home Logical Volume. I tried to follow the disc usage in each LV through the days, using #df -h command.

26 / 30

#df ­h Filesystem Size /dev/mapper/OSSIM­root 322M /dev/mapper/OSSIM­home 6.0G /dev/mapper/OSSIM­tmp  368M /dev/mapper/OSSIM­usr 4.6G /dev/mapper/OSSIM­var 2.8G

Used

Avail

Use%

 Mounted on 

161M

145M

 53%  

/

173M  

5.5G  

3%

/home

11M  

339M 

3%

/tmp

1.2G  

3.3G  

27%

/usr

388M

2.3G

15%

/var

The MySQL database stored events under /var. Indeed, I noticed that /dev/mapper/OSSIM-var use percentage grew rapidly through time. Hence I needed to allocate more space to OSSIM-var Logical Volume. #lvextend -L+200G /dev/mapper/OSSIM-var lvextend will allocate more space to the logical volume. But I needed a second command to write changes to the disk effectively. /var is a key partition that cannot be unmounted from the system. Hence, I needed to write changes to OSSIM-var “online”, using resize2fs. #resize2fs /dev/mapper/OSSIM-var Note that in other Unix systems, namely HP-UX and IBM-AIX, LVM tools like resize2fs are not free software.

7.3. Filtering noise Redundant events are likely to trigger false positives. So I needed to filter those. For instance, I had OSSEC plugin running to monitor Apache events, and cisco-pix plugin for the firewall-related events. But OSSEC understands Cisco logs. So, basically, I had two plugins parsing the same log file and generating the same events out of it. Hence, I needed to filter some OSSEC events to solve the issue. To do so, I firstly created a “Plugin Group” named Ossec pix. I inserted in this group all the event signatures I wanted to get rid of.

27 / 30

Ossec­pix plugin group Second, from (Intelligence > Policies), I selected “create a new policy”. I wanted this policy to drop all the events in “Ossec pix” plugin group, so I selected “ANY” as Source, Destination, Time range, and Ossec pix as a plugin group.

Policy creation As a “Policy Consequence”, I opted for “Drop events”. Finally, I got the following policy that filters all OSSEC events related to the Cisco PIX firewall.

28 / 30

Ossec­pix Policy up and running As it can be noticed, creating policies in OSSIM is an easy task, yet, it is very efficient in case one needs to filter some events out, which was the case for me.

7.4. Tuning panels  When browsing the different links of the web UI, I noticed that several tabs where showing unneeded information. Like “Backbone traffic” or “Global throughput” tabs. Those are network traffic related tabs, and I had no network traffic to monitor, so basically, they are useless. Hence, I looked for their PHP code on OSSIM server, and omitted it.

Cisco PIX and Cisco FWSM events

29 / 30

8. Conclusion This document was a brief “hands-on” for OSSIM. Feel free to contact me if you need more details or encounter problems with OSSIM. Reach me at http://about.me/houcem.

30 / 30