OSSEC Log Mangement With Elasticsearch

 

OSSEC Log Management with Elasticsearch Vic Hargrave | [email protected] | @vichargrave

1

 

$ whoami •

•

Software Architect for Tren Micro !ata Anal"tics #ro% &logger for Tren Micro Secrit" 'ntelligence an Sim%l" Secrit"

•

Email( vichargrav [email protected] [email protected]

•

)e*site( vichargrave.com

•

Twitter( @vichargrave

•

Lin+e'n( www.lin+ein.com,in,vichargrave

2

 

OSSEC oes S'EMs Syslog

syslog

commercial or open source

SIEM

Syslog

Syslog

3

 

Commercial S'EMs are great- *t

+

commercial

=

SIEM

4

 

/ow there0s a whole new 1o%en2sorce3 *allgame

Logstash

Kibana

5

 

OSSEC Log Management with Elasticsearch



 

Elasticsearch •

•

•

O%en sorce- istri*te- fll te4t search engine &ase on A%ache Lcene Stores ata as strctre 5SO/ ocments

•

•

•

•

S%%orts single s"stem or mlti2noe clsters Eas" to set % an scale 6 7st a more noes 8rovies a 9ESTfl A8' 'nstalls with 98M or !E& %ac+ages an is controlle with a service scri%t.

!

 

Elasticseach Elements •

•

•

'ne4 6 contains ocments- ≅   table  !ocment 6 contains fiels- ≅   row  :iel 6 contains string- integer- 5SO/ o*7ect- etc.

•

Shar smaller ivisions of ata that can *e store across6noes •

9e%lica 6 co%" of the %rimar" shar

"

 

Elasticsearch Mlti2noe Configration # default configuration file - /etc/elasticsearch/elasticsearch.yml ######################### ############## ########### Cluster ############## ######################### ########### # Cluster name identifies your cluster for auto-discovery # cluster.name:  cluster.name:  ossec-mgmt-cluster

########################## ############## ############ Node ############## ########################### ############# # Node names are generated dynamically on startup, so you're relieved # from configuring them manually. You can tie this node to a specific name: # node.name:  node.name:  "es-node-" # e.g. !lasticsearch nodes numered   N ########################## ############## ############ $aths ############### ########################## ########### # $ath to directory %here to store inde& data allocated for this node. # path.data:  path.data:  /data/, /data/

#

 

Logstash •

•

•

•

•

 

Log aggregator an %arser  S%%orts transferring %arse ata irectl" to Elasticsearch Controlle *" a configration file that s%ecifies in%tfiltering 1%arsing3 an ot%t ;e" to aa%ting Elasticsearch to other log formats 9n logstash in logstash home irector" as follows( in/logstash conf (logstash config file)

1$

 

OSSEC 6 logstash.conf  input * # stdin*+   udp *   port )    type ) "syslog"   + +   filter *   if type0  "syslog" *                

gro1 * # 2!! N!34 2567! + mutate * remove8field )  "syslog8hostname", "syslog8message", "syslog8pid", "message", "9version", "type", "host" 0 + +

+   output * # stdout * # codec ) ruydeug # +   elasticsearch8http *   host ) "..."   + + 11

 

OSSEC Alert Alert 8arsing •

OSSEC s"slog alert

an ; :  :< lert 5ev 5evel: = ? @ u le: A N 7  /in/su

•

gro+ < =

match ) * "message" ) "G*2Y25H46F!24>F$:syslog8timestamp+ G*2Y25HI24:syslog8host+   G*7>4>:syslog8program+: G*7>4>:syslog8program+ : >lert 5evel: G*NNN!H6N4:>lert85evel+ G*NNN!H6N4:>lert85evel+? ? @ule: G*NNN!H6N4:@ule+ G*NNN!H6N4:@ule+  - G*7>4>:7escription+ G*7>4>:7escription+? ? 5ocation: G*7>4>:5ocation+ G*7>4>:5ocation+? ? Jsrcip: G*6$:2rc86$+?G*2$>C!+KL        

Jdstip: G*6$:7st86$+?G*2$>C!+KL Jsrc8port: G*NNN!H6N4:2rc8$ort+?G*2$>C!+KL Jdst8port: G*NNN!H6N4:7st8$ort+?G*2$>C!+KL Juser: G*E2!@:Eser+ G*E2!@:Eser+?G*2$>C!+KLG ?G*2$>C!+KLG*H@!!7Y7>4>:7etails+ *H@!!7Y7>4>:7etails+" "

+ add8field )  "ossec8server", "G*host+" 0

12

12  

;i*ana •

•

•

#eneral %r%ose >er" ?' 5avascri%t im%lementation er" Elasticsearch withot coing

•

•

'ncles man" wigets 9n ;i*ana in *rowser as follows(  

http://(%e server ip):(port)/(1iana path)

13  

;i*ana 6 config.7s  /MM 9scratch /configuration/config.s/A /configuration/config.s/A   M  elasticsearch   M   M 4he E@5 to your elasticsearch elasticsearch server. You You almost certainly don't   M %ant Ohttp://localh Ohttp://localhost:BO ost:BO here. !ven if Piana and !lasticsearch M are on the same host. Qy default this %ill attempt to reach !2 at the M same host you have h ave 1iana installed on. You proaly %ant to set it i t to M the RS7N of your elasticsearch host   M/  elasticsearch: http://O"(elas http://O"(elasticsearch ticsearch node 6$)"O":B" 6$)"O":B", ,

14  

15  

1  

Elasticsearch Clster Management •

•

•

ElasticH Elasticsearch %lg2in 'nstall from Elastic Elasticsearch search home irector"(   in /plug in -inst sta allroyruss sso o/elast sti icse sea arch-I S

•

8rovies clster clster an noe management metrics an controls

1!  

1"  

1#  

And now for something completely dierent dierent..    The OSSEC virtual virtual appliance

2$  

&ac+ to 9ealit"

Free

21  

Elasticsearch Secrit" Caveats •

•

•

!esigne to wor+ in a trste environment /o *ilt in secrit" Eas" to erase all the ata curl 37!5!4! http://(server): http://(server):B/8all B/8all

•

?se with a %ro4" that %rovies athentication an re>est filtering sch as /gin4  –

htt%(,,wi+i.ngin4.org,Main

22  

:rther 'nformation •

Elasticsearch  –

•

Logstash  –

•

htt%(,,www.elasticsearch.org,ove htt%(,,www .elasticsearch.org,overview,+i*ana, rview,+i*ana,

ElasticH  –

•

htt%(,,logstash.net

;i*ana  –

•

htt%(,,www.elasticsearch.org

htt%(,,elastich>.org

Elasticsearch for Logging  –

 –

htt%(,,vichargrave.com,ossec2log2management2with2elasticsearch, htt%(,,egeofsanit".ne htt%(,,egeofsanit" .net,article,B,,D,elasticsearch2for2log t,article,B,,D,elasticsearch2for2logging.html ging.html

23  

Than+s for attening

 An" >estionsF