ACSE - Administración de correlacionadores OSSIM

Share Embed Donate


Short Description

Guía para la administración y generación de directivas y datasheets de OSSIM....

Description

ACSE AlienVault Certified Security Engineer

1 Thursday, May 3, 12

2 Thursday, May 3, 12

About this document •

ACSE (AlienVault Certified Security Engineer)



Author: AlienVault Training Team ([email protected])



Document Version 1.0



Last revision: 01/2012



Product version used: 3.1

Copyright © Alienvault 2012 All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and publisher. Any trademarks referenced herein are the property of their respective holders

3 Thursday, May 3, 12

Contents • Installation

• Logger

• Updates

• IDM

• CLI

• HIDS

• Event Collection

• Secure Connection

• Data Sources

• Snort

• Policies & Actions

• Dimensioning and Deployment

• Logical Correlation Directives

4 Thursday, May 3, 12

Bubba •

Throughout the document Bubba will give you useful hints and links for further documentation

We come in peace and security!

5 Thursday, May 3, 12

AlienVault Installation Getting to speed

6 Thursday, May 3, 12

Products •

AlienVault Installations Appliances

‣ -

Sensors (X1000, X2000, X3000, X4000)

-

Loggers (L1000, L2000, L3000)

-

SIEM (S1000, S2000, S3000)

Software

‣ -

analog to the Appliances range, installable on custom hardware •

Preferred: AlienVault Appliances

‣ -

7 Thursday, May 3, 12

custom restrictions, special purpose environments, etc.

optimum performance and compatibility

Installation Guide

8 Thursday, May 3, 12



Find the Installation Guide here



http://www.alienvault.com/docs/Installation_Guide.pdf

Hardware recommendations •

For a production system: ‣

At least 4GB Ram



64 Processor



DUAL Core Processor



Depending on the amount of traffic being monitored and the amount of data captured RAM has to be increased, always avoiding SWAP memory usage.



If we don’t have the appropriate hardware: ‣

9 Thursday, May 3, 12

"Divide et vinces"

Network hardware •



10 Thursday, May 3, 12

Requires Intel E1000 cards for capturing ‣

performance



performance



performance

Administration interface can be any card with no known problems

Best practice

11 Thursday, May 3, 12



Always use the latest installation image



If you need performance you can’t use any hardware



disable unused data sources



for the time of installation and customization, try to stick to english for faster support



don’t use Sniffers (ntop, snort, p0f) on interfaces without tap or port/ mirror

Performance is crucial when sniffing is involved. Get rid of unneeded CPU hoggers!

Best practice: VmWare •

12 Thursday, May 3, 12

VMware installations are popular but ‣

minimal memory: 8GB



minimal number of CPUS: 4



take care that all resources are bound singularly to the AlienVault guest system

Best practice: Partitioning •

SIEM only ‣

use 50-100 GB of disk space for /var/lib/mysql



use the maximum available remaining space for /var -



Logger only use at least twice the space required per log interval for /var

‣ -

-

Thursday, May 3, 12

50 GB expected logs/day => /var => 100GB

use the remaining rest for /var/ossim/logs



13

REASON: /var/lib/mysql keeps all the configuration

REASON: if /var runs out of space /var/ossim/logs remains intact

Best practice: Partitioning •

Use a small amount of space for the operating system ‣





14 Thursday, May 3, 12

50-100 GB are absolutely sufficient

be sure to configure enough swap space ‣

golden rule: 2x amount of RAM



additional swap can be configured later with a loopback mount

the storage killer: /var ‣

use at least 80% or more for this partition



depending on the usage you may sub-partition /var further

Best practice: Partitioning •

15 Thursday, May 3, 12

Sensor only ‣

50-100GB for /



2x physical memory (RAM) for swap



remaining space: /var



Example: SIEM - S1000 path

filesystem

size

partition

/boot

ext2

2GB

#1

/

ext3

100GB

#2

swap

swap

16GB

#3

/var

ext3

834GB

#4

/var/lib/mysql

ext3

100GB

#5

Installation profiles •



16 Thursday, May 3, 12

Selecting the server role ‣

Sensor



Server (includes SIEM & Logger)



Database



Framework

can be changed during installation or anytime after installation

Profile: Sensor •

enables Sensor functionality needs access to all the networks being monitored





receives all the network traffic Port Span

‣ -

17 Thursday, May 3, 12

needs to be configured separately



Tap device



Hub

Profile: Sensor •

18 Thursday, May 3, 12

out of the box enabled data sources ‣

Snort (Network Intrusion Detection System)



Ntop (Network and usage Monitor)



OpenVAS (Vulnerability Scanning)



P0f (Passive operative system detection)



Pads (Passive Asset Detection System)



Arpwatch (Ethernet/Ip address parings monitor)



OSSEC (Host Intrusion Detection System)



Nagios (Availability Monitoring)



OCS (Inventory)

Profile: Server •





19 Thursday, May 3, 12

combines SIEM functionality ‣

correlation



risk assessment



etc.

with Logger functionality ‣

forensic long-term storage



digitally signed

The server installation profile also comes with a Sensor with limited functionality to monitor the Server itself

Profile: Database •

enables a mysql server for usage with Server components



is required in most installations SIEM only

‣ -

event and alarm storage

-

metadata

-

framework

Logger



20 Thursday, May 3, 12

-

configuration

-

metadata

-

framework

Profile: Framework

21 Thursday, May 3, 12



enables the Web front-end for the server



is installed on most appliances ‣

low overhead



useful in emergency situations -

power outage on central console

-

connection to central console lost

-

usage as a per department or per location console

Profile: All-In-One •





22 Thursday, May 3, 12

Enables all AlienVault components on a single appliance ‣

Server (SIEM + Logger)



Database (Configuration and event storage + correlation)



Sensor



Framework (Web-Interface)

Useful for: ‣

Testing



Evaluation



Small deployments

activated on every automated install

Installation methods Automated installation

Custom installation

1. Boot the installation system

1. Boot the installation system

2. Configure networking

2. Select the installation language

3. Create and mount the partitions on which AlienVault will

3. Configure keyboard

be installed 4. Watch the automatic download/install/setup/update of the base system. 5. Set up users and passwords 6. Load the newly installed system for the first time

4. Configure location 5. Select the installation AlienVault profiles for this installation 6. Configure networking 7. Create and mount the partitions on which AlienVault will be installed 8. Enter the professional license 9. Watch the automatic download/install/setup/update of the base system. 10.Set up users and passwords

23 Thursday, May 3, 12

Installation checklist

24 Thursday, May 3, 12



Rack Space



Power



Network Configuration ‣

Port mirroring



IP addresses



Professional Key



Internet Access (Required when installing the professional version)



Role of the device to install

INSTALL

25 Thursday, May 3, 12

Installation: Next steps •

Point your web-browser to https://siem_ip/ ‣



Use ssh to login to the appliance ‣

26 Thursday, May 3, 12

username/password = admin

same password as in installation dialog

Be sure to note down the root password you enter in the installation process.

Hands-On: Installation •

Fill in the following tables with the partitioning data for the given profiles ‣

Logger, 4TB of overall disk capacity Logger, 8TB of disk capacity Mountpoint

Capacity

Comment

SIEM, 4 TB of disk capacity Mountpoint

27 Thursday, May 3, 12

Capacity

Comment

Hands-On: Installation •

Given the following SIEM design, which profiles need to be installed on which machines?

SIEM & Console Profiles: ______________________________________ ______________________________________ ______________________________________

Logger: Profiles: ______________________________________ ______________________________________ ______________________________________

28 Thursday, May 3, 12

Sensor: Profiles: ______________________________________ ______________________________________ ______________________________________

AlienVault Updates Keeping your system up to date

29 Thursday, May 3, 12

AlienVault: Update channels AlienVault uses the Advanced Packaging Tool (apt) for software maintenance







reliable



tested

Every AlienVault Appliance is configured to retrieve updates ‣

Base System: Debian repositories



AlienVault Software: AlienVault repositories -

Binary and package updates •

-

/etc/apt/sources.list.d/alienvault-pro.list

AlienVault professional feed •

30 Thursday, May 3, 12

/etc/apt/sources.list.d/alienvault-etpro-pro.list

Get more info on how APT works here: http://en.wikipedia.org/wiki/ Advanced_Packaging_Tool

Updating Process •

Important files (They should never be modified) /etc/apt/sources.list

‣ -

/etc/apt/preferences

‣ -



31 Thursday, May 3, 12

Contains the different software repositories

Contains the priority configuration between the different repositories

The update system in AlienVault: ‣

Updates the AlienVault components as well as the Debian base system



Allows the AlienVault development team preventing software packages from being upgraded (Unstable versions, software causing troubles to other users.

Update AlienVault

32 Thursday, May 3, 12



The system notifies in the Web interface the availability of new versions of the AlienVault components



The system notifies the management console on the availability of new updates to the components of AlienVault



If the update procedure requires any manual change, it will be explained in the updates notification system



To update the whole system, use the following command: ‣

# alienvault-update



debugging: alienvault-update -v

Update AlienVault •

During the update process: ‣

When prompted always select installing the latest configuration files available



Once the system has been updated, if something is not working run: -



33 Thursday, May 3, 12

# alienvault-reconfig

Snort rules, OpenVas scripts, directives and plugins will be updated automatically using software packages

Update AlienVault •

In case new Snort or OpenVas rules are included manually, you will need to run the following scripts to update the information in the database Snort:

‣ -

OpenVas:

‣ -



Thursday, May 3, 12

# perl /usr/share/ossim/scripts/update_nessus_ids.pl

And restart the AlienVault Server ‣

34

# perl /usr/share/ossim/scripts/create_sidmap.pl /etc/snort/rules

# /etc/init.d/ossim-server restart

Package Management •

Following apt’s principle several tools are available to install packages and monitor installed software apt-cache

‣ -

search for packages (apt-cache search )

-

show package requirements, versions etc.

apt-get

‣ -

dpkg

‣ -

35 Thursday, May 3, 12

install individual packages

show installed packages

The Debian APT howtos: http://www.debian.org/doc/manuals/ apt-howto/

Hands-On: Updates

36 Thursday, May 3, 12



Update the system



review the software sources in /etc/apt/sources.list.d



Which versions of the following software is installed: ‣

ossim-server ___________________________



ossim-agent



mysql-server-core ___________________________



ossim-framework ___________________________

___________________________

AlienVault CLI power to the user

37 Thursday, May 3, 12

AlienVault: Base System •



AlienVault Appliances are built around 64 bit Debian 5 Linux ‣

Open



Reliable



Secure



Innovative

AlienVault uses many packages from Debian ‣



38 Thursday, May 3, 12

where possible

AlienVault repositories ‣

for AlienVault software packages



customized Debian packages

AlienVault: Filesystem •

39 Thursday, May 3, 12

Current standard installation is based on ext3 filesystem ‣

recommended filesystem for usage of the AlienVault software



journaled



stable

AlienVault: system access •

Get access to the platform SSH access with superuser “root”

‣ -

HTTPS access with administrative user “admin”



40 Thursday, May 3, 12

use the password supplied by AlienVault or the one from the installation process

-

default password is ‘admin’

-

can be switched to normal HTTP

Configuration Files •



AlienVault ‣

OSSIM Server: /etc/ossim/server/config.xml



OSSIM Agent: /etc/ossim/agent/config.cfg



Frameworkd: /etc/ossim/framework/ossim.conf

Snort ‣



OpenVas ‣



Thursday, May 3, 12

/etc/nagios3/

Database ‣

41

/etc/openvas/openvasd.conf

Nagios ‣



/etc/snort/snort.ethN.conf

/etc/mysql/my.cnf

Configuration Files •

System startup ‣



Logrotate ‣



Thursday, May 3, 12

/etc/rsyslog.conf

Monit ‣

42

/etc/resolv.conf

Rsyslog ‣



/etc/network/interfaces

DNS configuration ‣



/etc/logrotate.d

Network configuration ‣



/etc/rc*

/etc/monit/monitrc

AlienVault: Services •

Stop a service ‣

# /etc/init.d/ stop



# service stop



Start a service ‣

# /etc/init.d/ start



# service start



Restart a service ‣

# /etc/init.d/ restart



# service restart



The parameters that the services will use when starting are usually configured in the following path: ‣

43 Thursday, May 3, 12

# /etc/default/

alienvault-setup /alienvault-reconfig AlienVault Components

Database ossim_setup.conf

ossim-reconfig Integrated Tools

OS Components

44 Thursday, May 3, 12

ossim-setup.conf

45 Thursday, May 3, 12



interface: Network management interface (eth0, eth1...  



language: Language used within AlienVault (en, es, fr...)



profile: Profile or Profiles enabled in the system



version: Version of AlienVault in use



hostname: Name of the system



admin_ip: IP address to manage the system



first_init: Variable to check whether is the first boot or not



email_notify: e-mail to receive notifications

alienvault-setup (Database)

46 Thursday, May 3, 12



acl_db, event_db, ossim_db, osvdb_db, ocs_db: Name for the different databases



db_ip: IP address of the Database



db_port: Listening port of the Database



pass: Password of the Database



type: Type of Database



user: User in the Database



create: If i is set to yes, database will be deleted and created again when running alienvault-reconfig

ossim-setup.conf (Sensor)

47 Thursday, May 3, 12



detectors: Enabled detector plugins (Separated by comma and using the same name that the plugin configuration file has)



interfaces: Listening interfaces (Separated by comma)



ip: IP address that the sensor will use to connect to the AlienVault Server



monitors: Enabled monitor plugins (Separated by comma and using the same name that the plugin configuration file has)



name: Name of the sensor



networks: Local networks that will be monitored from that sensor (In CIDR format and separated by comma)

ossim-setup.conf •



48 Thursday, May 3, 12

FRAMEWORK ‣

framework_ip: IP address of the Web interface



framework_port: Listening port of the frameworkd daemon

SERVER ‣

server_ip: Listening IP address of the AlienVault Server



server_port: Listening port of the AlienVault Server



server_license: Professional license code



server_plugins: Enabled plugins in the Server profile

ossim-setup.conf •



SNMP ‣

snmpd: Enable the Snmp daemon



snmptrap: Enable Snmp traps collection



community: Snmp community

FIREWALL ‣



49 Thursday, May 3, 12

active (firewall): Enable or disable iptables

VPN ‣

vpn_infrastructure: Enable or disable the VPN between the OSSIM components



vpn_net: VPN Network



vpn_port: VPN Port

Agent Configuration •

/etc/ossim/agent/config.cfg



[daemon] ‣

daemon: Daemon mode (True or False)



pid: Path to the PID file (Process identifier)



50 Thursday, May 3, 12

[event-consolidation] ‣

Enable events consolidation at Sensor level



by_plugin: List of plugins that will be consolidated



enable: Enable or disable (True or False)



time: Wait n seconds to consolidate the events before sending them

Agent Configuration •

/etc/ossim/agent/config.cfg



[log] ‣

Configures the verbose level and the path to the different log files



error: File in which the error events will be stored



file: File in which all the agent logs will be stored



stats: File in which the agent stats will be stored (Every 5 minutes)



verbose: Configures the verbose level (Debug, Info, Warning, Error or Critical)



51 Thursday, May 3, 12

[output-plain] ‣

Writes in a log file what is being sent to the AlienVault Server (Useful for debugging and developing purposes)



enable: Enable or disable (True or False)



file: File in which the output-plain will be stored

Agent Configuration •

/etc/ossim/agent/config.cfg



[output-server]





Configures the server to which events are sent



enable: Enable or disable sending events to the server (True or False)



ip: IP address of the AlienVault Server (Logger or SIEM)



port: Listening port of the AlienVault Server (Logger or SIEM)

[plugin-defaults] ‣

52 Thursday, May 3, 12

In this category variables can be defined to be used in the plugins configuration.

Agent Configuration ‣

/etc/ossim/agent/config.cfg



[plugins] -

Defines which Data Source Connectors (detectors and monitors) are enabled

-

name_of_the_plugin=path_to_the_plugin_config_file

-

device= /etc/ossim/agent/plugins/device.cfg

[watchdog]



53 Thursday, May 3, 12

-

Monitor the process associated to each plugin (In case it is running in the same machine)

-

enable: Enable or disable (True or False)

-

interval: Wait X seconds between checks

-

restart_interval: Restart the process every X seconds (This has to be enabled in each plugin)

Agent Configuration •

/etc/ossim/agent/aliases.cfg ‣



Data Source Connectors configuration files ‣

54 Thursday, May 3, 12

Contains predefined regular expressions that can be used when creating new plugins

/etc/ossim/agent/plugins/*.cfg

Data Source Connectors •

Detector plugin configuration (/etc/ossim/agent/plugins/*.cfg)



[DEFAULT]



55 Thursday, May 3, 12



Any var defined inside this category will be sent to the AlienVault Server



plugin_id: Numerical identifier of the plugin within the AllienVault system (Data Source ID)

[config] ‣

type: detector



enable: Enable or Disable plugin (It must be enabled in config.cfg)



source: Source of the events (log, database, wmi)



location: File in which logs can be found



create_file: Create the log file in case it does not exist

Data Source Connectors ‣

Detector plugin configuration (/etc/ossim/agent/plugins/*.cfg)



[config]



56 Thursday, May 3, 12

-

process: Name of the process generating logs (If the process is running in the same system)

-

start: Start the process when the agent starts (yes/no)

-

stop: Stop the process when the agent stops (yes/no)

-

startup: Command that starts the process

-

shutdown: Command that stops the process

The next part of the configuration files includes the regular expressions that collect and normalize the events.

Agent Configuration •

The different configuration variables defined in the config file can be used with the following syntax to help defining new variables:



%()s ‣



When the variable has been defined in the same file:



process=pads



shutdown=killall -9 %(process)s

\_CFG() ‣

When the variable has been defined in the main configuration file(config.cfg)



In /etc/ossim/agent/config.cfg file: restart_interval=3600 ; seconds between plugin process restart



In the Data Source Connector configuration file: restart_interval=\_CFG(watchdog,restart_interval)

57 Thursday, May 3, 12

Server Configuration •

/etc/ossim/server/config.xml



Path to the AlienVault Server log file ‣



Configuration to access the SQL Database ‣







Configuration to access the OSVDB Database ‣

Thursday, May 3, 12



Configuration to access the Snort Database ‣

58





Server Configuration •

/etc/ossim/server/config.xml



Path to the correlation directives ‣



Waiting time between each execution of AlienVault Server scheduled jobs ‣



Thursday, May 3, 12



Listening port, name and listening IP address of the AlienVault Server ‣

59





Web Interface Configuration •

Executive panel configuration ‣



60 Thursday, May 3, 12

/etc/ossim/framework/panel/

/etc/ossim/framework/ossim.conf ‣

Paths to applications and libraries



Configure access to the different databases



Some tools use this file to get the configuration parameters to access the database (ossim-db, create_sidmap.pl...)

Monit •

Monit ‣

Process monitors all the important services in the AlienVault machines and restarts services in case of a process crash



Configuration file /etc/monit/monitrc



Different configuration based on the profile in use



When Stopping any process monit must be stopped first

# Framework check process ossim-framework with pidfile /var/run/ossim-framework.pid group framework start program = "/etc/init.d/ossim-framework start" stop program = "/etc/init.d/ossim-framework stop" if 5 restarts within 5 cycles then timeout

61 Thursday, May 3, 12

When debugging, be sure to turn monit OFF! More Monit infos: http://mmonit.com/monit/

AlienVault: System logging •

All the AlienVault components offer logging



This can and should be used for Debugging

‣ -

errors may not be seen directly in the Web interface but can be spotted in the system logs

-

extra information for the AlienVault Support Team

Diagnosis

‣ -

are all the services up to date

-

did system update generate errors

Verification of Success or Failures



62 Thursday, May 3, 12

-

is my data source connector generating events

-

are idm-events collected

AlienVault: Log files •

OSSIM Server ‣



OSSIM Agent ‣





/var/log/ossim/frameworkd.log

Snort ‣

/var/log/syslog



/var/log/snort (Binary Format)



Other applications ‣

Thursday, May 3, 12

/var/log/ossim/agent.log

OSSIM Frameworkd ‣

63

/var/log/ossim/server.log

Check the variable location in the plugin configuration file

Debug Mode •

OSSIM Server ‣

# ossim-server –D 6 -d



Logfiles in /var/log/ossim/server.log



This does not show information on the terminal, much more info will be logged in the server log file



OSSIM Agent ‣



64 Thursday, May 3, 12

# ossim-agent –vv

OSSIM Frameworkd ‣

# ossim-framework –vv



Never leave an application running in Debug mode in a production

Networks Card Information •



ethtool/mii-tool ‣

Network card stats and link status



set link speed and type for not autonegoiating switch ports

iptraf ‣



65 Thursday, May 3, 12

measure throughput and TCP sessions on a network interface

tcpdump ‣

Check whether the port mirroring is well configured or not



do configured devices really send syslog to the Sensor

Network Configuration •

Rename network interfaces ‣



Edit the file /etc/iftab



Insert a line for each network interface with the following format :





eth0 mac 00:17:31:56:BC:2D



eth1 mac 00:16:3E:2F:0E:9C

Network cards with more than one interface usually have consecutives MAC addresses ‣

66 Thursday, May 3, 12

# apt-get install ifrename

# ifconfig -a | grep HWaddr

Network Configuration •

Additionally rename it with udev ‣



Create the file /etc/udev/rules.d/010_netinterfaces.rules



Reboot & check udev entries ‣

67 Thursday, May 3, 12

# ifconfig -a |grep HWaddr

# udevinfo -a -p /sys/class/net/eth0

Network Configuration •

The network configuration in Debian is stored in the following file: ‣



After modifying the previous file, it is required restarting networking with the following command: ‣

68 Thursday, May 3, 12

/etc/network/interfaces

# /etc/init.d/networking restart

Network Configuration •

Each interface is configured in /etc/network/interfaces with the following template



be sure that the interface is also in the ‘auto’ section to enable automatic startup on system boot auto lo0 eth0 ... eth allow-hotplug eth0 iface eth0 inet static address 192.168.1.133 netmask 255.255.0.0 network 192.168.0.0 broadcast 192.168.255.255 gateway 192.168.1.1 dns-nameservers 192.168.1.100

69 Thursday, May 3, 12

Network Configuration

70 Thursday, May 3, 12



address: IP address given to the interface (In the example eth0).



netmask: Network Mask



network: It is the part of the IP address which is common between all the IP addresses in the network.



broadcast: Broadcast IP of the network.



gateway: IP address of gateway in our network



dns-nameservers: IP addresses of the DNS servers used in our corporation. More than one DNS server can be used (Separated by comma). In there is a local DNS running in your network, t should be placed in first place.

Network Configuration •

Those interfaces in promiscuous mode used to collect the network traffic (Port mirroring) should have an entry in the network configuration file with the following format: ‣

71 Thursday, May 3, 12

up ifconfig eth0 0.0.0.0 promisc -arp

Network Recommendations

72 Thursday, May 3, 12



The collector does a lot of queries to the DNS server to normalize events, so, the local DNS should always be configured in any AlienVault box.



In case there is not a local DNS in your network, the different hostnames and their associated IP addresses should be defined in the the /etc/hosts file in your collectors.



Interfaces in promiscuous mode should only be used to collect network traffic, those interfaces should never have an assigned IP address

Disk Space •

The disk space left in the AlienVault machines should be monitored



The folder /var/ will use the biggest amount of disk space in AlienVault



73 Thursday, May 3, 12



Databases



Log files

The /var/ folder should be separated into another partition with the highest amount of disk available (>80%).

Swap Memory

74 Thursday, May 3, 12



Swap memory usage slows down the system so we need to make sure the system is not using frequently the swap partition



If the system is always using the Swap partition it is required to increase the amount of RAM memory installed in the system.

Munin

75 Thursday, May 3, 12



Munin can monitor the status of various parameters within the operative system such as Interrupts, Load, Memory, network, and processes



Munin is really useful to monitor that our hardware is working properly



Munin can be used distributed since 3.1

AlienVault: system recovery •

76 Thursday, May 3, 12

Backup & Recovery ‣

simple backup script is provided



easy recovery after new installation



Backup script: see next page

AlienVault: system recovery •

Backup script: (Caution!!! - /var/ossim/logs is not included) #!/bin/bash cd /etc/init.d/ process="monit ossim-agent ossim-server ossim-framework apache2 arpwatch exim4 fprobe nagios3 nessus dnfdump nfsen ntop munin-node openvas-scanner openvassd openvas-server osirisd osirismd pads rsyslog postfix samba snort* snmpd tomcat nfdump" for i in $process do /etc/init.d/$i stop > /dev/null; done chmod 000 /etc/cron.hourly/* chmod 000 /etc/cron.daily/* d1r="/var/ossim/backup/db/`date +%F-%H_%M_%S`" dbs=`echo "show databases" | ossim-db | grep -v "Database" | grep -v "information_schema"` p4ss=`grep -i pass= /etc/ossim/ossim_setup.conf |awk -F'=' '{print$2}'` h0st=`grep -i db_ip /etc/ossim/ossim_setup.conf |awk -F'=' '{print$2}'` test -z $h0st && h0st="localhost" for db in $dbs; do test -d $d1r/$db/struct || mkdir -p $d1r/$db/struct mysqldump -d -u root -h $h0st -p$p4ss $db > $d1r/$db/struct/$db-struct-`date +%F-%H_%M_%S`.sql mysqldump -u root -h $h0st -p$p4ss $db > $d1r/$db/$db-`date +%F-%H_%M_%S`.sql done tar --preserve -czvf $d1r/../../complete_backup_`date +%F-%H_%M`.tgz $d1r > /dev/null echo "Generated /var/ossim/backup/complete_backup_`date +%F-%H_%M`.tgz file" tar --preserve -czvf $d1r/../../config_files_`date +%F-%H_%M`.tgz /etc/* > /dev/null echo "Generated /var/ossim/backup/config_files_`date +%F-%H_%M`.tgz file" dpkg -l >> $d1r/../../packages_list_`date +%F-%H_%M` chmod 755 /etc/cron.hourly/* chmod 755 /etc/cron.daily/* alienvault-reconfig -c -v exit 0

77 Thursday, May 3, 12

Hands-On: CLI •



alienvault-setup ‣

change default admin email address



configure monitored networks



enable detector plugins: foo bar and baz



change the time zone, but if ethx to nonpromiscous mode



change the system name

delete and rebuild the complete alienvault-database ‣

78 Thursday, May 3, 12

WARNING: all your data and configuration will be LOST

AlienVault Event Collection Get the data from the network

79 Thursday, May 3, 12

Syslog •

Using Syslog is usually the easiest way to forward events to the AlienVault Sensor



All Linux, BSD and MacOSX include different Syslog implementations by default.



It's simple to configure event forwarding policies using Syslog



A large number of devices and applications allow event logging using the Syslog protocol. If it doesn’t the log files can be logged into Syslog using logger: ‣

80 Thursday, May 3, 12

# tail –f /path/to/file | logger –t application

Snare Agent

81 Thursday, May 3, 12



Snare forwards Windows EventLog events to a remote Syslog server.



When installing Snare, it can also configure a logging policy in the Windows System. Take care if you have already defined a logging policy.



Once Snare has been installed (Configuration -> Collection -> Downloads) download the .reg file to configure it to forward events to the remote Syslog server

WMI

82 Thursday, May 3, 12



WMI (Windows Management Instrumentation) provides an operating system interface through which instrumented components provide information and notification.



WMI allows scripting languages like VBScript or Windows PowerShell to manage Microsoft Windows personal computers and servers, both locally and remotely.



WMI is preinstalled in Windows 2000 and newer OSs. It is available as a download for Windows 95 and Windows 98.



AlienVault includes two data source connectors that allow collecting information using WMI: ‣

Detector: wmi-system-logger.cfg



Monitor: wmi-monitor.cfg

Fw1-Loggraber

83 Thursday, May 3, 12



Fw1-Loggrabber allows collecting events from Checkpoint FW-1 devices using the Checkpoints LEA (Log Export Api) protocol



Fw1-Loggraber has to be installed along with the detector. It will stablish a connection to get the event from the Checkpoint FW-1 device

Cisco SDEE •

The AlienVault Sensor can collect events from Cisco devices using the SDEE protocol.



The detector allows collecting event from the following devices:

• 84 Thursday, May 3, 12



Cisco Network Detection Systems (IPS)



Cisco Switch IDS Cisco IOS routers with Inline Intrusion Prevention System (IPS) functions



Cisco IDS modules for routers



Cisco PIX Firewalls



Cisco Catalyst 6500 Series firewall services modules (FWSMs)



Cisco Management Center for Cisco security agents CiscoWorks Monitoring Center for Security servers

Data Source Connector configuration file: cisco-ips.cfg

Rsyslog •

Rsyslog is the Syslog implementation shipped with AlienVault



Rsyslog is extremely configurable and allows configuring filtering and forwarding in a really easy way



Rsyslog must allow remote connections to collect logs coming from other Syslog servers. This feature has to be enabled in the Rsyslog configuration file (/etc/rsyslog.conf) including the following lines: $ModLoad imudp $UDPServerRun 514 $ModLoad imtcp $InputTCPServerRun 514

85 Thursday, May 3, 12

rsyslog filters •

Filter using Rsyslog ( /etc/rsyslog.d/)



Forward certain events to a local file





if $msg contains 'error' then /var/log/error



if $syslogfacility-text == 'local0' and $msg startswith 'DEVNAME' and ($msg contains 'error1' or $msg contains 'error0') then /var/log/ somelog

Stop processing some events ‣



Regex in Rsyslog ‣

86 Thursday, May 3, 12

if $msg contains 'error' then ~

http://www.rsyslog.com/user-regex.php

rsyslog customization •

Separate incoming logs (syslog) ‣



find a phrase in syslog to classify the logs (hostname, ip-address,...)

Send logs to a different logfile ‣

Create a file with file extension “conf” in /etc/rsyslog.d



e.g. /etc/rsyslog.d/customize.conf



possible commands

# sends logs with “” to cisco.log :source, isequal, "HOSTNAME" /var/log/cisco.log &~ if $msg contains “STRING” then /var/log/xyz.log if $msg contains “STRING” ~ # ip-address examples if $fromhost == 'IP_ADDRESS' then -/var/log/ossim/device.log &~ if $fromhost-ip isequal 'IP_ADDRESS' then -/var/log/cisco-fw.log &~ :fromhost-ip, isequal, "IP_ADDRESS" -/var/log/cisco-fw.log &~

87 Thursday, May 3, 12

The rsyslog webpage provides more useful examples, tricks and howtos: http://www.rsyslog.com/

Log rotation •

When creating new log forwarding rules (Rsyslog), it is important to ensure that the logs will not grow indefinitely



To do this we must create new entries in the logrotate configuration ‣

/etc/logrotate.d/

/var/log/ossim/agent.log /var/log/ossim/agent-plain.log /var/log/ossim/agent_error.log / var/log/ossim/agent_stats.log { daily firstaction test -f /var/log/snort/alert || touch /var/log/snort/alert > /dev/null 2>&1 endscript prerotate /etc/init.d/ossim-agent stop > /dev/null 2>&1 endscript postrotate /etc/init.d/ossim-agent start > /dev/null 2>&1 endscript } 88 Thursday, May 3, 12

Log rotation example •

If you have new separated logfiles just take this example and add your new logfiles to it! ‣

/etc/logrotate.d/alienvault

/var/log/xyz.log /var/log/foo.log ... { rotate 7 # Save the last 7 logs daily # rotate daily missingok # if file doesn’t exist continue notifempty # if log is empty, the log don’t rotate delaycompress # postpone compression of previous log-file to next cycle compress # Compress the log postrotate invoke-rc.d rsyslog reload > /dev/null }

89 Thursday, May 3, 12

see also > man logrotate

Hands-On: CLI •

Define a syslog source and filter to /var/log/foo...



enable logrotation on the source you just configured



alienvault-setup





change default admin email address



configure monitored networks



enable detector plugins: foo bar and baz



change the time zone



change the system name

delete and rebuild the complete alienvault-database ‣

90 Thursday, May 3, 12

WARNING: all your data and configuration will be LOST

Hands-On: CLI •

Logfiles send some example logs to /var/log/syslog

‣ -

use the following script:

while true do cat /var/log/firewall.log | logger -t sleep 10 done



91 Thursday, May 3, 12

filter the log and send it to a separated log file and be sure that this log is not filling up your disk space (rotate the file daily with compression enabled)

AlienVault Data Sources Adapt collection to your organization

92 Thursday, May 3, 12

Types of DS Connectors

93 Thursday, May 3, 12



Two types of Data Source Connectors



Detectors: They offer events (Snort, Firewalls, Antivirus, Web servers, OS events..)



Monitors: They offer indicators (Ntop, Tcptrack, Nmap, Webs, Compromise & Attack...)

Files •

94 Thursday, May 3, 12

Each DS Connector (monitors and detectors) is built on two files: ‣

Plugin.cfg Contains the configuration parameters of the plugins and the rules that an event has to match in order to be collected and normalized.



Plugin.sql Contains the description of every possible event that can be collected using the plugin (Plugin_id, Plugin_sid, Name given to the event, priority and reliability)

Ds Connector: Detector [DEFAULT] plugin_id=4003 # default values for dst_ip and dst_port # they can be overwritten in each rule dst_ip=\_CFG(plugin-defaults,sensor) dst_port=22 [config] type=detector enable=yes source=log location=/var/log/auth.log

Numerical identifier of the plugin

Default fields for every event Type of plugin: Detector Source of the events (log, mssql,mysql or wmi)

create_file=false process=sshd start=no stop=no startup=/etc/init.d/ssh start shutdown=/etc/init.d/ssh stop

Associated process and start/stop options

[ssh - Failed password] # Feb 8 10:09:06 golgotha sshd[24472]: Failed password for dgil from 192.168.6.69 port 33992 ssh2 event_type=event regexp="(\SYSLOG_DATE)\s+(?P[^\s]*).*?ssh.*?Failed password for (? P\S+)\s+from\s+.*?(?P\IPV4).*?port\s+(?P\PORT)" plugin_sid=1 sensor={resolv($sensor)} date={normalize_date($1)} src_ip={$src} dst_ip={resolv($sensor)} src_port={$sport} username={$user}

Type of event Regular expressions

Fields that will be sent to the AlienVault Server

95 Thursday, May 3, 12

Ds Connector: Detector •



96 Thursday, May 3, 12

plugin_id ‣

Data Source ID. User reserved range: 9000-10000



E.g.: plugin_id=3000

source ‣

log: Text file (E.g: SSH, Sudo, Apache...)



mssql: Mssql Database (E.g: panda-se)



mysql: Mysql Database (E.g: moodle)



wmi: Windows Management Instrumentation (wmi-system-logger)

Ds Connector: Detector location

‣ -

Files in which the applications store the events

-

E.g.: location=/var/log/file.log

create_file

‣ -

Create the file in case it does not exist

-

false/true

process / start / stop / startup / shutdown



97 Thursday, May 3, 12

-

Only if the process is running in the same machine that the detector

-

If the process is not running in the machine, is there a process helping us to collect those logs? syslog? fw1-loggrabber?

Ds Connector: Detector •

98 Thursday, May 3, 12

Rules ‣

Rules define the format of each event and how they are normalized



It is composed by a regular expression and the list of fields that the event will include when once it is sent to the AlienVault SIEM or Logger



In some cases only one regular expression will collect every event coming from one application, in some other cases more than one rule will be required

DS Connector: Detector •

99 Thursday, May 3, 12

Rules ‣

Rules are loading in alphabetical order based on the name given to each rule



Once the log matches one the regex of one rule the ossim agent stops processing the event



Generic rules must be the last loaded in memory as they will probably match all the events



The name of the rule is mandatory

DS Connector: Detector

100 Thursday, May 3, 12



The rule must include the event type:



event_type=event



The following fields can be used to normalize the event: plugin_id

plugin_sid

date

sensor

interface

protocol

src_ip

src_port

dst_ip

dst_port

username

password

filename

userdata1

userdata2

userdata3

userdata4

userdata5

userdata6

userdata7

userdata8

userdata9



Values in bold are mandatory



Fields in red include values that always have to be defined in the plugin



Fields in green can will be filled by the AlienVault Agent in case they can not be found in the original log (Don’t include that line when creating the plugin)



Fields in grey are optional

DS Connector: Detector •

Regexp



The regexp field contains the regular expression that defines the format of the events, and extracts the information to normalize the event.

regexp="(\SYSLOG_DATE)\s+(?P[^\s]*).*?ssh.*?Failed password for (?P\S+)\s+.*?(?P\IPV4).*?port\s+(?P\PORT)" regexp=(\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d)\S+ (\S+) (\S+) (\S+) (\d+) (\w+) (\S+) \S+ (\d+)



The regular expressions are written using the Python regular expression syntax: ‣

101 Thursday, May 3, 12

http://docs.python.org/library/re.html

Regular expressions

102 Thursday, May 3, 12

Operator

Meaning

c

A non special character matches with itself

\c

Removes the special meaning of the character c; The RE \$ matches with $

^

Indicates located at the beginning of the line

$

Indicates located at the end of the line

.

Any individual character

[…]

One or any of the characters …; accepts intervals of the type a-z, 0-9, A-Z

[^…]

A char different from … ; Accepts intervals of the type a-z, 0-9, A-Z

Useful tools for testing regular expressions: Online: http://www.regexpal.com Windows: RegEx Tester Linux: http://kodos.sf.net/ Regexhibit (OSX): http://homepage.mac.com/roger_jolly/software/

Regular expressions

103 Thursday, May 3, 12

Regular expression

Matches with

a.b

axb aab abb aSb a#b ...

a..b

axxb aaab abbb a4$b ...

[abc]

a b c (one character srtings)

[aA]

a A (one character srtings)

[aA][bB]

ab Ab aB AB (two character srtings)

[0123456789]

0123456789

[0-9]

0123456789

[A-Za-z]

A B C ... Z a b c ... Z

[0-9][0-9][0-9]

000 001 .. 009 010 .. 019 100 .. 999

[0-9]*

empty_chain 0 1 9 00 99 123 456 999 9999 ...

[0-9][0-9]*

0 1 9 00 99 123 456 999 9999 99999 99999999 ...

^.*$

A full line

Regular expressions

104 Thursday, May 3, 12

Operator

Meaning

r*

0 or more occurrences of the RE r

r+

1 or more occurrences of the RE r

r?

0 or an occurrences of the RE r, and no more

r{n}

No occurrences of the RE r

r{,m}

0 or at most m occurrences of the RE r

r{n,m}

N or more occurrences of the RE r, but at most m

r1|r2

The RE r1 or the RE r2

Regular expression

Matches with

[0-9]+

0 1 9 00 99 123 456 999 9999 99999 99999999 ..

[0-9]?

empty_string 0 1 2 .. 9

(ab)*

empty_string ab ababab abababababab

([0-9]+ab)*

empty_string 1234ab 9ab9ab9ab 9876543210ab 99ab99ab ...

Regular expressions

105 Thursday, May 3, 12

Regular expression

Matches with

Equals

\d

Any decimal character

[0-9]

\D

Any non decimal character

[^0-9]

\s

Any space character

[ \t\n\r\f\v]

\S

Any non space character

[^ \t\n\r\f\v]

\w

Any alphanumeric character and “_”

[a-zA-Z0-9_]

\W

Any non alphanumeric character

[^a-zA-Z0-9_]

\Z

End of line

Regular expressions Pattern b,c,X,8

Description Ordinary characters just match themselves exactly. The meta-characters which do not match themselves because they have special meanings are: . ^ $ * + ? { [ ] \ | ( )

. \w

Matches any single character except newline (\n). Lowercase w matches a "word" character: a letter or digit or under-bar [a-zA-Z0-9_]. It only matches a single word char, not a whole word.

\W

Uppercase w matches any non-word character.

\s

Lowercase s matches a single whitespace character -- space, newline, return, tab, form [ \n\r\t\f].

\S

Upper case s matches any non-whitespace character.

\d

Lowercase d matches a single Decimal digit [0-9]

\D

Uppercase d matches any non decimal character

\t

Matches a tab character

\n

Matches a newline character

\r

Matches a return character

\Z

Matches only at the end of the string.

\

Escapes special characters. If you are unsure if a character has special meaning, such as '@', you can put a slash in front of it, \@, to make sure it is treated just as a character.

106 Thursday, May 3, 12

Regex aliases •

/etc/ossim/agent/aliases.cfg



/etc/ossim/agent/aliases.local (For user custom aliases)



This file contains predefined regular expressions that can be used to simplify the process of writing new plugins



Usage Example: ‣

107 Thursday, May 3, 12

\SYSLOG_DATE\s+\IPV4\s+\IPV4

Regular Expressions •

The information extracted by the regular expression from the log can be accessed by: Position: (\d\d):(\d\d):(\d\d)

‣ -

hour={$1}

-

minutes ={$2}

-

seconds={$3}

Tags: (?P\d\d):(?P\d\d)(?P\d\d)



108 Thursday, May 3, 12

-

hour={$hour}

-

minutes ={$minutes}

-

seconds={$seconds}

Functions •

The AlienVault SIEM and Logger must receive normalized events, as an example the addresses have to use IPV4 format and the date has to use the following format YYYY-MM-DD HH:MM:SS (2010-12-31 22:57:00)



To simplify the process of normalizing events some functions can be used



resolv() Translate hostnames into IPV4 addresess (DNS queries)





normalize_date() The normalize_date function translate many format dates into the format accepted by the SIEM or Logger

‣ -



109 Thursday, May 3, 12

YYYY-MM-DD hh:mm:ss

More functions can be found and defined in ParserUtils.py A very useful tool for testing plugin is located in /usr/share/ossim/scripts/ regexp.py

Translation Tables

110 Thursday, May 3, 12



Translations can be configured to be done once the event has been collected



E.g.: When the event id is not numeric, but plugin_sid has to be numeric



Translations have to be defined inside a category called [translation]



Translate using the function translate().

Even more info can be found here: http://www.alienvault.com/docs/ AlienVault%20Building%20Collector %20Plugins.pdf

Testing your plugin •

Useful tool: regexp.py ‣

will analyze a set of logs to a set of regular expressions or a plugin



will give you statistics on matched lines and rules

# /usr/share/ossim/scripts/regexp.py /var/log/netscreen.log netscreen-firewall.cfg q Multiple regexp mode used, parsing netscreen-firewall.cfg ----------------------------------------------------------------------------Rule: netscreen-firewall-BA-traffic Matched 5586 times Rule: netscreen-firewall-BB-traffic Matched 0 times Rule: netscreen-firewall-EF-system Matched 0 times Rule: netscreen-firewall-ZZZ-generic Matched 50 times Counted 5636 lines. Matched 5636 lines.

111 Thursday, May 3, 12

Hands-On: plugins •

firewall logs are sent to /var/log/firewall.log while true do cat /var/log/firewall.log | logger -t sleep 10 done

112 Thursday, May 3, 12

Hands-On: plugins •



113 Thursday, May 3, 12

write a firewall plugin ‣

copy existing similar plugin:



plugin_id number start on custom range



write your regex rule to match the loglines

write a firewall sql file ‣

copy existing similar sql file:



change the fields to your custom plugin rules



activate your plugin on the CLI (alienvault-setup)



write your sql file to the database

AlienVault Policies & Actions Controlling flow of information between AlienVault Components

114 Thursday, May 3, 12

Event Filtering Levels

115 Thursday, May 3, 12



Policy rules



AlienVault Sensor Configuration



Data Source Connector Configuration



Collection method used



Data Source Configuration

SIEM & Logger

SIEM

Logger

116 Thursday, May 3, 12

•  Database storage (MySQL) •  The number of events that can be stored will depend on the hardware in use •  As we increase the number of stored events, the analysis gets slower •  Max number of stored events: 20 millions •  Certain events don't offer useful information for correlation and analysis •  Need to filter useless events

•  Disk storage •  No more storage limits apart from the available disk space •  Everything should be stored in the Logger without exception

Filter examples

117 Thursday, May 3, 12



Filter all events generated by a tool



Filter certain events from a tool



Filter all events from one host or network



Filter events generated between two different hosts



Filter some events during the night



Filter some events every weekend



Filter all events from one service

Filter in Policy 1. Create a DS group with the types of events that have to be filtered 2. Select source and destination Assets (Hosts, Networks, ANY...) 3. Select the ports 4. Select the DS group 5. Select time period in which the policy applies 6. Select policy consequences 1.

Process in the SIEM?

2.

Process in the Logger?

3.

Store the event?

4. Correlate only?

118 Thursday, May 3, 12

Agent Configuration •

If we don’t want to collect any event from one of the plugins remove the plugin from the file: ‣



in the variable named: ‣



Thursday, May 3, 12

detectors

And run the following command: ‣

119

/etc/ossim/ossim_setup.conf

# alienvault-reconfig

Filtering in DS Connectors •

Some events can be filtered during the collection process editing the configuration file for each plugin:



Using the option exclude_sids (E.g.: Apache DS Connector) ‣



120 Thursday, May 3, 12

exclude_sids=404,200,403

Modifying the regular expressions to avoid matching certain events

Hands-on: Policies •

121 Thursday, May 3, 12

Write a policy for a Nagios event / service down ‣

use external command and write to /tmp/alert



disable ssh on the device in question



does your external command write to /tmp/alert



Send email after a prioritized event



Ticket creation (e.g. if only one user is allowed/responsible to subscribe new tickets)

Logical Correlation Directives Put intelligence into your SIEM

122 Thursday, May 3, 12

Logical Correlation Directives

123 Thursday, May 3, 12



Logical Correlation uses Correlation directives to detect attacks and problems in the monitored networks



By default AlienVault includes a set of Correlation directives



Users can create custom correlation rules in both Unified and Open Source edition



AlienVault Professional Feed provides more than 500 extra correlation rules

Logical Correlation Directives •

Logical Correlation is implemented in the AlienVault SIEM



Correlation Directives are stored in the following path ‣



Web-based Correlation Directives Editor ‣

124 Thursday, May 3, 12

/etc/ossim/server/

Intelligence -> Correlation Directives

Rules •

Directives are composed by rules



Each rules defines the conditions that event must match to be correlated by the directive



Whenever the conditions defined by a rule are matched, a new event is generated



Some of the values of the rule will be used to generate the new event (In the event of a successful correlation of that rule)



It is possible to have multiple rules in each correlation level (Except in the first correlation level)



125 Thursday, May 3, 12

Example: Brute Force Brute-Force Attack against SSH Server Login Attempt Login Attempt Login Attempt Login Attempt

Root Login Refused Invalid Password Invalid User User not allowed User in DenyUsers ALARM: Brute-Force Attack Against 192.168.1.1 Total events captured 12.392 First Event 10-02-2011 23:30:23 Last event 10-02-2011 23:56:19

Illegal User

Normalized Events SIEM

126 Thursday, May 3, 12

SENSOR

Example: Brute-Force •

Think about all possible events that may be generated User: root Password: root



Root Login not allowed (SSH Server Configuration) ‣



Root Login allowed but incorrect password ‣



Thursday, May 3, 12

Event generated: Invalid Password

Root account locked ‣

127

Event generated: Root Login Refused

Event generated: User not allowed because account is locked

Example: Brute-Force •

Global properties of the correlation directive Name of the directive

‣ -

This is the name that will take all events generated within the correlation of this directive

ID of the directive

‣ -

All events generated within this directive will use 1505 as plugin_id (Data Source ID) and the ID of the directive as plugin_sid (Event type) •

Priority of the directive (Impact of this Attack or problem in your network)

‣ -

128 Thursday, May 3, 12

500000-100000 is the reserver range for user-created directives

All events generated within this directive will have as their priority the global priority value of the correlation directive

Example: Brute-Force •

Global properties of the directive:



Same plugin_id in all events during Logical Correlation: Plugin_id 1505

The Plugin_sid of the new event is the ID of the directive that generated the event

The name (Signature) of the new event is given by the name of the directive

The priority of the new event is given by the global priority of the directive

plugin_id=”1505” plugin_sid=”500000” name=”SSH Brute Force Attack Against 192.168.2.2” priority=”4”

129 Thursday, May 3, 12

SQL Storage Correlation Risk Assessment Policy Collection

EVENTS

Event Generated during Logical Correlation

SIEM

Example: Brute Force •

Events will first try to get in those directive whose correlation has started and then they will try to match with the first correlation level of the correlation rules that are enabled while the SIEM runs



The same event can be correlated in multiple correlation directives

SIEM Logical Correlation Directives in the correlation engine

130 Thursday, May 3, 12

Directives waiting to be correlated

Example: Brute-Force

131 Thursday, May 3, 12



First correlation level



Special Conditions ‣

Only detector rules can be used in the first correlation level



Only one rule



This rule doesn’t have time out



Only one occurrence (Correlation of the directive will start with the first event matching the conditions of the rule

Example: Brute-Force •

First correlation level How many events?

‣ -

What type of events should match the rule?

‣ -

-

Hosts, Host Groups or Networks

-

Source and Destination Ports

Any other special condition?

‣ -

Thursday, May 3, 12

Plugin_id and Plugin_sid of the events

What sources and destinations should match the rule?



132

Always a single event in the first correlation level

E.g.: Avoid events with a certain username from matching the rule

Example: Brute-Force •

First correlation level What type of events should match the rule?

‣ -

Check at Configuration -> Collection -> Data Sources

-

Only one Data Source in each rule (One plugin_id with one or multiple plugin_sid)

What sources and destinations should match the rule?



133 Thursday, May 3, 12

-

The first rule is usually generic (We do not know yet where the attack is coming from)

-

ANY / HOME_NET (Assets in the AlienVault inventory)

Example: Brute-Force •

First correlation level



type: detector (First rule is always detector) and we will collect the events using a detector DS Connector



reliability: When calculating the risk, it will also be 0 -

134 Thursday, May 3, 12

risk= (Asset Value * Priority * Reliability)/25



from/to: Both are set to any because we don't know yet who will be the attacker and his target



port_from/port_to: Set to ANY, not important in this type of directive



plugin_id/plugin_id: List of the events in the SSHD DS Connector that refer to fail authentications

Example: Brute-Force •

This directive would be useless with only one level



This directive with a single level will never generate an event becoming alarm (Reliability is set to 0)





Directives with one level can be used to rename events

WIth a reliability of 0 this event will never become alarm, but it will be stored in the SIEM SQL (SIEM -> Analysis) List of the IP Addresses of the Web Servers

135

Thursday, May 3, 12

Example: Brute-Force Second correlation level

‣ -

Authentication successful •

Same Source and Same destination that matched the first correlation level



Time out before this possibility is discarded?



Would this indicate a big problem? (Authentication successful after 1 failed)

-

136 Thursday, May 3, 12

10 more authentication failed •

Same Source and Same destination that matched the first correlation level



Time out before this possibility is discarded?



Would this indicate a big problem? (A total of 11 Authentication failed in X seconds)

Logical Correlation

137 Thursday, May 3, 12

Logical rule example

138 Thursday, May 3, 12

Logical rule example •

139 Thursday, May 3, 12

In the directive with id 27 every time the condition given by a rule is met an event with priority 2 is generated. The reliability will be taken from the rule that has matched: ‣

Maximum risk will be in the fifth rule



Priority 2, Reliability 10



With this directive the event will get as much a risk of 4 (When one of the assets involved has a value of 5)



(2*10*5) / 25 =4

Example: Worm SIEM

140 Thursday, May 3, 12

Alarm

Example: Worm SIEM

140 Thursday, May 3, 12

Alarm Alarm

Example: Worm SIEM

140 Thursday, May 3, 12

Alarm Alarm Alarm

Example: Worm SIEM

140 Thursday, May 3, 12

Example: Intrusion 1.1. 2.

“Hi!”

“At your command!”

F

3. Persistence

4.

141 Thursday, May 3, 12

Behaviour

Writing Correlation Rules •

All directives have a unique numeric identifier, a name and a global priority that will be used in every event generated during the correlation of the directive.



The events generated when correlation the directive will always have 1505 as plugin_id and the id of the directive as plugin_sid.





143 Thursday, May 3, 12

In the following rules in many cases we will have to make sure that some fields have the same value than the event that started the correlation of the directive

Writing Correlation Rules

144 Thursday, May 3, 12



To open a second correlation level the following tags have to be used: and



More than one rule can be included in each correlation level (Not in the first one)



When one of the multiple rules of each correlation level is matched, the rest of the rules of that level are discarded

Writing Correlation Rules •

145 Thursday, May 3, 12

Two types of rules can be used when writing directives: ‣

Detector rules: Detector rules for incoming events from the different detector plugins (SSH, Snort, Firewalls...)



Monitor rules: Monitor rules request for information to different applications and devices through the collector (Session information, permissions, availability...)

Writing Correlation Rules •

146 Thursday, May 3, 12

Common fields (Monitor and detector rules): ‣

plugin_id: Numerical identifier of the tool that provides the information (Events in detector rules and indicators in monitor rules)



plugin_sid: Numerical identifier of the type of even within the tool defined by plugin_id or request or query that has to be executed (In Monitor rules)



reliability: Reliability value of every event generated within the directive. It can be an absolute value 0-10 or incremental +2, +6



time_out: Waiting time before the rule expires and the directive process defined in that rule is discarded. The first rule doesn’t have a time_out value.

Writing Correlation Rules •

Detector rule: sensor:



ANY ‣



Address IP (x.x.x.x) ‣





IP address list separated by commas (E.g: 192.168.1.2,192.168.2.3) Sensor name

‣ •

Sensor name (As configured in Configuration -> SIEM Components) Relative values

‣ ‣ •

It is possible to relate variable values to previous correlation levels. 1:SENSOR refers to the sensor IP address in the first correlation level. Denied values

‣ ‣

Thursday, May 3, 12

An IP address (E.g: 192.168.1.9) Several IP addresses



147

Any IP address

It is possible to include denied elements as value of the target field: "!192.168.2.203,INTERNAL_NETWORK”

Writing Correlation Rules •

Detector rule: from/to:



ANY Any IP address





Address IP (x.x.x.x) An IP address (E.g: 192.168.1.9)





Several IP addresses IP address list separated by commas (E.g: 192.168.1.2,192.168.2.3)





Network name Network name (Defined in the paragraph Policy in the Web interface)





Relative values It is possible to relate variable values to previous correlation levels:







-

2:DST_IP refers to the target IP address in the second correlation level.

It is possible to include denied elements as a value of the origin field:

All networks in the inventory ‣

Thursday, May 3, 12

1:SRC_IP refers to the origin IP address of the event in the first correlation level.

Denied values ‣

148

-

HOME_NET

Writing Correlation Rules •

Detector rule: port_to/port_from:



This field can adopt an only port or a list of ports separated by commas as value. The keyword ANY is a list of all the ports.



It is possible to use relative values of ports referring to other correlation values: ‣

1:DST_PORT refers to the values of the target port of the first correlation value.



3:DST_PORT refers to the value of the target port of the third correlation value.



To deny ports we put the symbol “!” before the port we want to deny: ‣

149 Thursday, May 3, 12

port="!22,25,110,!21"

Writing Correlation Rules •

Detector rule: protocol:



The field Protocol refers to the protocol in which the communication was established where the event took place.



Protocol can adopt the following values:



150 Thursday, May 3, 12



TCP



UDP



ICMP



Host_ARP_Event



Host_OS_Event



Host_Service_Event



Host_IDS_Event



Information_Event

It is also possible to specify the protocol with the protocol number.

Writing Correlation Rules •



151 Thursday, May 3, 12

Sticky ‣

When the events arrive to the correlation engine they will try to be correlated inside directives whose correlation has been started



Using sticky we avoid those events to start the correlation of the same directive again, as they may also meet the conditions given by the same directive.

Sticky Different ‣

This variable can be associated to any field in rules with more than one occurrence, to make all the occurrences have a different value in one of the fields



E.g.: sticky_different=“DST_PORT”



All the events matching the rule must have a different destination port (Port scanning detection)

Writing Correlation Rules

152 Thursday, May 3, 12



Monitor rules define a condition that has to be checked using monitor plugins



The following variables to, from, port_to, port_from, protocol, sensor, username, filename, password and userdata1-9 will have the values that will be sent to the collector to be used in the monitor plugin. Those values don not have to be met, they will only be used to build the monitor request.



Monitor rules specific fields: ‣

value



condition



interval



absolute

Writing Correlation Rules

153 Thursday, May 3, 12



Monitor rule: condition



Establishes a logical relation between the value field and the value returned in the monitor plugin request. eq

equal

ne

non equal

lt

less than

gt

greater than

le

less or equal than

ge

greater or equal than

Writing Correlation Rules •

Monitor rule: value ‣

This field sets the value that has to be compared with the value returned by the collector after doing the monitor request.



Value must be an integer.



Monitor rule: interval ‣



154 Thursday, May 3, 12

This value of this field sets the waiting time between each monitor request before the rule is discarded because the time defined by time_out is over.

Monitor rule: absolute ‣

This value sets if the value that has to be compared is relative or absolute.



Absolute true: If the host has more than 1000 bytes sent during the next 60 seconds. There will be an answer if in 60 seconds this value is reached.



Absolute false: If the host shows an increase of more than 1000 bytes sent. There will be an answer if the host shows this increase in 60 seconds.

Recommendations

155 Thursday, May 3, 12



The objective should not always be generating alarms



Not all directives should generate alarms with a high risk value (Rate the importance of the attack or problem detected by the directive)



In some cases the last rule level should be used to keep collecting events to avoid having the same directive in the correlation engine so often



A lot of directives can be created using the same directive template (Brute force, Worms, Policy Violation...)



Use sticky to avoid directives from being generated exponentally

AlienVault Logger Forensic Storage

156 Thursday, May 3, 12

Hands-On: Logger •

create the indexed Logger install logger package, called:

‣ -

157 Thursday, May 3, 12

alienvault-logger



check if the indexing process is running



after this you should see the indexed search possibility in the UI

Future versions of AlienVault will only have one Query button combining Raw and Indexed queries

Hands-On: Logger •

158 Thursday, May 3, 12

create a remote logger ‣

ssh-keygen



ssh-copy-id -i [pub_identity_file] user@remotelogger_IP



example:



try to connect without password



create the remote logger under “AlienVault Components”



test the remote logger

AlienVault IDM Identity Monitoring & User awareness

159 Thursday, May 3, 12

AlienVault IDM

160 Thursday, May 3, 12



IDM - Identity Monitoring



Introduced in AlienVault 3.1



every source or destination IP can be enriched with user data



Available user information ‣

username



domain



hostname



IP address



MAC address

IDM: information flow ip=”192.168.2.1” username=”alien” hostname=”alienshome” domain=”alienvault.com”

SIEM Send IDM events

Send IDM

Send

IDM

Sensors

events

even ts

Sensors

Sen d ID

Me ven ts

Sensors

Sensors ip=”192.168.24.2” username=”Administrator” hostname=”dc01” domain=”alienvault.com”

161 Thursday, May 3, 12

IDM: how does it work •



162 Thursday, May 3, 12

User information on the network ‣

logins



ActiveDirectory



LDAP servers



VPN access

Information can be retrieved by plugins ‣

event=idm-event



Rule will set username, hostname, domain etc.

IDM: how does it work

163 Thursday, May 3, 12



This information will then be sent to SIEM and associated with IPs



every Event can have IDM information attached

Installing IDM •

Install alienvault-idm on all the sensors and servers for IDM apt-get install alienvault-idm





On the sensor, tell the agent to send IDM events to the SIEM sensor: /etc/ossim/agent/config.cfg

‣ -

section [output-idm] for sending to a single server

-

section [idm-server-list] for sending to several SIEM/IDM servers

-

[output-idm] enable=True ip=The_IP_of_the_SIEM_server port=40002 [idm-server-list] FQDN1=The_IP_of_the_Second_SIEM_server;40002 FQDN2=The_IP_of_the_Third_SIEM_server;40002

164 Thursday, May 3, 12

Installing IDM •

on SIEM server add IDM in /etc/ossim/server/config.xml

‣ -



-

configure IDM for the GUI •

165 Thursday, May 3, 12

echo “update config set value=1 where conf=‘idm_enable’” | ossim-db

Installing IDM •

Check if the agent already sends IDM events works by default in some plugins

‣ -



166 Thursday, May 3, 12

OSSEC

start integrating your own IDM data

Writing IDM plugins

167 Thursday, May 3, 12



Start with a normal plugin skeleton



Insert rules for log messages generated with IDM relevant data



event-type = idm-event Event attribute

associated IDM value

username

IDM username

domain

IDM domain-name

hostname

IDM hostname

mac

IDM mac address

ip

IDM associated IP

IDM: example plugin

168 Thursday, May 3, 12



Example: Ossec-IDM data source



event-type = idm-event



extracted fields from regex: username, hostname, domain, ip

Hands-On: IDM

169 Thursday, May 3, 12



install the required packages on the infrastructure to support IDM



do all the necessary configuration to enable the flow of IDM information across the infrastructure



from these two loglines, develop a IDM data source that reports username, hostname, domainname and IP to the SIEM ‣

User [email protected] logged in via 192.168.2.1. Assigned IP address: 10.10.1.15



Login from trusted source 10.10.2.123, user alice, domain aliceslair.com, hostname aliceslaptop -

do you receive IDM events

-

do you see IDM events in the console

AlienVault HIDS Extending client security

170 Thursday, May 3, 12

Hands-On: OSSEC •

Configuration on the server side (AlienVault machine), two possibilities:



1) creating the agent in the UI



171 Thursday, May 3, 12

Hands-On: OSSEC •

2) creating the agent on the terminal ‣

172 Thursday, May 3, 12

important scripts in: /var/ossec/bin: •

manage_agents (add agent / extract key / list agents / remove agents)



list_agents (all / connected / not connected)



agent_control (more options than list_agents)

Hands-On: OSSEC •

173 Thursday, May 3, 12

Install ossec-agent on a windows machine ‣

download ossec-agent from the AlienVault UI



install it



configure it: -

OSSEC Server IP:

-

Authentication key:

Hands-On: OSSEC •



After finishing the OSSEC connection between server and agent: ‣

enable OSSEC plugin (ossec.cfg) in alienvault-setup



check if you get the events in the UI or agent.log



check if the OSSEC events arrive in the SIEM

If not check the troubleshooting page ‣

174 Thursday, May 3, 12

next page

Hands-On: OSSEC •

Troubleshoot To check if events are arriving on the server

‣ -

If not, check if the agent is properly connected



175 Thursday, May 3, 12

/var/ossec/logs/alerts/alerts.log

-

view ossec-agent log file on the windows machine

-

check ossec-agent configuration file on the windows machine

-

check if the OSSEC udp port 1514 is open

AlienVault Secure Connection Aliens love encrypted transports

176 Thursday, May 3, 12

Hands-on: OpenVPN •

Connecting a sensor to a SIEM with OpenVPN ‣

SIEM (OpenVPN server) side



#alienvault-reconfig --add_vpnnode=



cd /etc/openvpn/nodes



restart/start OpenVPN process: /etc/init.d/openvpn restart

alienvault-vm:/etc/openvpn/nodes# ifconfig tun0 + ifconfig tun0 tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 inet addr:10.67.68.1 P-t-P:10.67.68.2 Mask:255.255.255.255 UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:65 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:100 RX bytes:0 (0.0 B) TX bytes:4340 (4.2 KiB)

177 Thursday, May 3, 12

Hands-on: OpenVPN •

Connecting a sensor to a SIEM with OpenVPN ‣

send .tar.gz that includes all data we need on the sensor via scp to user@:/etc/openvpn



uncompress on the sensor side: tar -xvzf .tar.gz

alienvault-sensor:/etc/openvpn/# tar -xvzf .tar.gz + tar -xvzf .tar.gz / /keys/ /keys/.csr /keys/.key /keys/ca.crt /keys/.crt .conf openssl.cnf



178 Thursday, May 3, 12

restart OpenVPN: /etc/init.d/openvpn restart

Hands-on: OpenVPN ‣

restart OpenVPN: /etc/init.d/openvpn restart



tunnel should be established



troubleshoot: -

tail -f /var/log/syslog |grep ovpn --color=auto

Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1871]: TUN/TAP device tun0 opened Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1871]: TUN/TAP TX queue length set to 100 ... Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1879]: TCPv4_SERVER link local (bound): [undef]:33800 Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1879]: TCPv4_SERVER link remote: [undef] Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1879]: MULTI: multi_init called, r=256 v=256 Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1879]: IFCONFIG POOL: base=10.67.68.2 size=62 Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1879]: MULTI: TCP INIT maxclients=1024 maxevents=1028 Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1879]: Initialization Sequence Completed

179 Thursday, May 3, 12

Hands-on: OpenVPN •



180 Thursday, May 3, 12

Adapt sensor configuration to tunnel IP addresses ‣

edit ossim-setup.conf on the sensor



change all to mostly 10.67.68.1



run alienvault-reconfig

Check events arriving over the tunnel ‣

Security Events (SIEM) / Logger



or tcpdump -i tunX

Snort Network IDS

181 Thursday, May 3, 12

What is Snort? •

182 Thursday, May 3, 12

Snort is an NIDS (Network Intrusion Detection System) ‣

It is Open Source (GPL v2)



Snort combines signature, protocol and anomaly-based inspection,



It has an active development and new rules are added daily



Snort is the most widely deployed IDS technology worldwide

Brief History

183 Thursday, May 3, 12



Snort was released as an Open Source product in 1998. Since then its source code has been distributed using the GPL license



Snort is developed by Sourcefire, company founded by the original creator of Snort (Martin Roesch).



New Snort versions are released often, including new functionalities and fixing bugs and errors.

IDS

184 Thursday, May 3, 12



Intrusion Detection System, security tool in charge of monitorizing events on a system in search of traces of an intrusion.



An intrusion attempt is an attempt to compromise the confidentiality, integrity or availability of a computer system or to circumvent it’s security measures.

NIDS •

Network level IDS



Monitor network traffic



Advantages



185 Thursday, May 3, 12



Monitor the entire network traffic



No impact on the network

Disadvantages ‣

Unable to analyze encrypted information



Requires continuous rule update in order to catch new attacks



Requires specific network configuration (Port mirroring or Network Tap)

NIDS: Network Traffic

186 Thursday, May 3, 12



If the network traffic is not forwarded to the NIDS machine, only the traffic generated that has as source or destination that machine will be analyzed.



Network traffic can be forwarded using the following methods: ‣

Network taps



Hubs [+ listen only network cables]



Switches with a mirroring or span port

Why a NIDS? •

187 Thursday, May 3, 12

Some attacks and problems can not be detected using a firewall ‣

Attacks tunneling the network traffic



Attacks using vulnerabilities in the applications



Attacks from our internal network against the internal or external network.

Snort modes •



188 Thursday, May 3, 12

Snort can be used in the following modes: ‣

Sniffer: Monitor the activity of the network



Packet logger: All the network activity is stored in capture files



IDS : The network traffic is compared with predefined patterns in search of anomalies, attacks or policy violations

In AlienVault only the IDS mode of Snort is used

Snort and AlienVault

189 Thursday, May 3, 12



Snort is a very important tool within AlienVault and it has been used by AlienVault since the first OSSIM release for several reasons:



Detailled attack information from passive listening ‣

Security problems (Trojans, Virus, Worms...)



Policy Violation (Pornography, p2p, messenger…)



Bad configurations



Snort is the main source of information for correlation directives



Near to Zero configuration



Snort events are used in logical, inventory and cross correlation

Snort and AlienVault

190 Thursday, May 3, 12



Using snort, we can collect information that could also be collected from many applications such as Web servers and Proxies logs.



The main advantage of Snort is that all the events will be generated in a passive mode (URL visited, program versions, Web server response...)

Snort and AlienVault

191 Thursday, May 3, 12



Snort is integrated as a detector plugin in AlienVault



The AlienVault Sensor collects logs from the Snort Data Source agent and send them normalized to the SIEM or Logger



Snort should only be used in those interfaces collecting the network traffic



Snort loads all signatures in memory to analyze the network traffic in real time. If all rules are enabled and a lot of traffic has to be analyzed high performance hardware will be required.



AlienVault Appliances (Sensors) are built for this task



All the events generated by the snort rules will have the plugin_id 1001. The events generated by the Snort preprocessors will have a different plugin_id for each preprocessor within the range 1002-1500.

Snort Architecture •

192 Thursday, May 3, 12

The Snort engine can be divided into the following components: ‣

Decoders



Preprocessors



Detection engine



Output plugins

Architecture: Decoders •

193 Thursday, May 3, 12

Decoders: The decoders collect the network packets and prepare them to be analyzed by the preprocessors and the detection engine.

Architecture: Preprocessors •

194 Thursday, May 3, 12

Snort's preprocessors are divided in two categories. ‣

A number of attacks cannot be detected by signature matching via the detection engine. so some preprocessors are used to examine packets for suspicious activity.



The other preprocessors are responsible for normalizing traffic so that the detection engine can accurately match signatures.

Architecture: Preprocessors

195 Thursday, May 3, 12



frag3: Its main objective is avoiding detection evasion techniques. It reassembles network packets to be analyzed by the detection engine.



stream5: Reassembles TCP traffic and monitors TCP and UDP sessions



http_inspect: Normalizes and searches anomalies in http traffic



rpc_decode: Normalizes and searches anomalies in rpc traffic



bo: Detects back orifice traffic in the network



ftp_telnet: Normalizes and searches anomalies in ftp and telnet traffic



smtp: Normalizes and searches anomalies in smtp traffic



sfportscan: Detects network scans in the network

Architecture: Preprocessors

196 Thursday, May 3, 12



arpspoof: Decodes ARP packets and detects ARP attacks, anomalies and inconsistence's



ssh: Detects the use of different exploits against the ssh protocol



dcerp: Detects and decode SMB and DCE/RCP traffic



dns: Detects the use of different exploits against the DNS service



ssl: Detects the SSL/TSL traffic and determines whether it has to be analyzed or not

Architecture: Detection Engine

197 Thursday, May 3, 12



Detection engine: The Detection engine analyzes every network packet to determine if there is any malicious or forbidden activity on it.



The Detection engine uses rules that are loaded when Snort starts.



Rules are compared with every network packet. If one rule is matched, an event is generated.

Architecture: Detection Engine •

alert tcp $HOME_NET any -> 10.1.1.0/24 80 (flags: SF; msg: “SYN-FIN Scan”;)



Header:





alert: Action that has to be done if the rule is matched



tcp: protocol



$HOME_NET any: Source



10.1.1.0/24 80: Destination

Options: ‣

198 Thursday, May 3, 12

flags: SF; msg: “SYN-FIN Scan” : Conditions that have to be met and name of the event if the rule is matched.

Architecture: Detection Engine •

Rules to detect access miscelaneous web activities: ‣

alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"ET POLICY Megaupload file download service access"; flow:to_server,established; content:"GET "; depth: 4; uricontent:"/?d="; content:"|0d 0a|Host\: "; content:"megaupload.com"; within:25; nocase; classtype:policy-violation; reference:url,doc.emergingthreats.net/2009301; reference:url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/POLICY/POLICY_Download_Services; sid:2009301; rev:2;)



alert tcp $HOME_NET any -> $EXTERNAL_NET 5050 (msg:"ET POLICY Yahoo Chat Signin Inside Webmail"; flow:established,to_server; content:"content-length\:"; nocase; depth:15; content:"
View more...

Comments

Copyright ©2017 KUPDF Inc.
SUPPORT KUPDF