Exploring ³Big data´ Security Analytics: Use Cases and More
Dave Shackleford, SANS and Voodoo Security Mark Seward, Sr. Director, Security and Compliance, Splunk
7KH6$16,QVWLWXWH- www.sans.org
Recap: Webinar #1 Security and Big Data: What's all the Hype About? ± Defined ³Big data´ ± Core use cases
Incident Response Root cause analysis Security intelligence KPI analytics
7KH6$16,QVWLWXWH- www.sans.org
2
The Need for Security Intelligence More and more, we need bigger data sets to analyze IT operational data can provide incredibly useful context and correlation points ± DB information All Security Security Relevant Relevant Data Data ± App data ± OS data SIEM gives us a lot of info, but not SIEM enough 7KH6$16,QVWLWXWH- www.sans.org
3 3
³Ok, Ok, I¶m convinced. Big data and the big data phenomenon is real. So what do I do with it?´
7KH6$16,QVWLWXWH- www.sans.org
4
The Big Data Powered Business Less µGut Feeling¶ ± More µEvidence¶ based decisions
Seen as a way to increase top line revenues and reduce expenses New dependence on understanding the data Hurting my business means messing with, stealing, or interrupting the flow of my data and µchanging¶ my decisions
³Epsilon leverages big data and analytics. Revenues increased by 20%.´ IBM ³60% potential increase in operating margins possible with big data. McKinsey and Co. June 2011
The Need to µThink Differently¶
Creativity consists of Convergent and Divergent Thinking 6
Big-data and Creative Security Thinking Divergent Thinking: ʹ ʹ ʹ
The Aha moment / Spontaneous epiphany Remote associative processes Pattern-based thinking
Convergent Thinking: ʹ ʹ
ʹ
ʹ
About analysis and attention The act of µun-concealing¶ ± chiseling away at a problem Write a symphony / poem / solve an algebraic equation Stick with a problem till it µcries uncle¶ 7
Security Intelligence Requires: µThinking Like a Criminal¶ 9
9 9 9
µNormal¶ IT Services Data
What¶s the modus operandi of the attacker? What are the most critical data sets owned by the business? What physical or virtual assets have the data? What patterns of weak-signals in µnormal¶ IT activities would represent µabnormal¶ human or machine behaviors?
Application Physical Security DHCP Data
GPS
AD/LDAP
Netflow
DNS
VPN
Where are my big data experts? Meet your µnew¶ virtual security team
Traditional security folk SOXV«GRPDLQNQRZOHGJH
Where will µbig data experts¶ be hired
Constituents will be partners and partners constituents
µHub and Spoke¶ design
Fosters data-driven decision making
Finance Team Legal Department
Finance Team
Business Service Providers
Traditional Security Team
Development IT Operations
Business Line Owners
Finance Team
³Lets define a new thinking process´
7KH6$16,QVWLWXWH- www.sans.org
10
Big Data, Big Thinking, New Process
What will cause the business to stop functioning?
What¶s normal?
Data SMEs from the business and security teams figure out ± µwhat¶s normal¶ and what would not be normal
Analysis options categorized with combinations of R/T and historic searches
Support for agile interpretation and iteration
Adapted from͞The "Human Element" of the Big Data Equation͟ by Steve Durbin ISF , CRM magazine, November 2012
Copyright © 2012, Splunk Inc.
11
Using the Process - Example The Steps
The Response
Business Issue
Service degradation causes monetary damage and customer satisfaction issues.
Construct one of more hypothesis (team creativity required)
Unwanted bots can degrade service and steal content.
Gather data sources and expertise
What combinations of data would be considered definitive evidence? What might be the first signs of trouble? List all data in which this might be reflected.
Determine the analysis to be performed
Determine the types of data searches appropriate
Interpret the results
Do the results represent false positives of false positives or false negatives? Are there good bots and bad bots? 12
Copyright © 2012, Splunk Inc.
Detecting Account Take-over
Statistical analytics and thresholds
Behavior of logins and password changes and resets Analysis of same IP ± multiple password resets Multiple IPs -- resetting the same account
How many times people change their bank information
How many times they change their credit card information
Does the IP address (location) match the browser language or time zone
Unknown Threat Attack Pattern -- Example
14
Attack Pattern Modeling ± Questions to Ask Is this the first time this
person has received email from the recipient?
Is the website in the email on
a known list of bad websites?
Are their changes to host
Host based Analytics
config files closely tied to a website visit?
If so ± import PCAP and
Flowdata
Are there DNS requests to
known bad sites or are the IP addresses of the DNS URL request and responses the same or different?
Monitor port and protocol
usage unusual amounts or types 15
Network based Analytics
Is Big Data Changing Security? Oh yeah. Zions Bancorporation presented at RSA 2012 on how analytics would change their security model forever The goal? Actionable, real-time security intelligence over petabytes of data.
7KH6$16,QVWLWXWH- www.sans.org
16
Zion Case Study: Components Looked to drive deeper forensics and build complex stats models ± Needed years of data Logs are still centralized ± Using Hadoop and unstructured data file stores Storing: ± DB logs ± FW logs/events ± Antivirus logs ± IDS logs ± Wire ACS transfers ± Credit data
7KH6$16,QVWLWXWH- www.sans.org
17
Even More Use Cases Fraud Detection ± Patterns of user behavior vs. ³other users´ Intellectual Property Theft ± Data access patterns over long time periods, with many sources Security Monitoring Optimization ± Where are best locations for sensors and event monitoring? ± What are best/optimal data sources? 7KH6$16,QVWLWXWH- www.sans.org
18
So What is Splunk?
7KH6$16,QVWLWXWH- www.sans.org
19
Splunk Collects and Indexes Any Machine Data Customer Facing Data
Outside the Datacenter
Shopping cart data Online transaction data
Logfiles
Windows
Click-‐stream data
Registry Event logs File system sysinternals
Copyright © 2011, Splunk Inc.
Configs
Linux/Unix
Configurations syslog File system ps, iostat, top
Messages
Traps Alerts
Virtualization & Cloud
Hypervisor Guest OS, Apps Cloud
Metrics
Scripts
Changes
Applications
Web logs Log4J, JMS, JMX .NET events Code and scripts
DĂŶƵĨĂĐƚƵƌŝŶŐ͕ůŽŐŝƐƚŝĐƐ͙ CDRs & IPDRs Power consumption RFID data GPS data
Tickets
Databases
Networking
Configurations Audit/query logs Tables Schemas
Listen to your data.
Configurations syslog SNMP netflow
So What is Splunk?
+ Text Based Search
Statistical Analysis
Time Index Ingestion Text Base Search Nested Search Cross Data-‐type Search cApend Abstract Cluster Bucket Multikv Scrub Join Rare 7KH6$16,QVWLWXWH- www.sans.org
Cluster Associate Stats AVG Transaction Addtotals Delta Eval Stddev Rare Outlier Streamstats Timechart 21
Splunk: Big Data Security Intelligence Platform Security Intelligence for Business
Machine Data
Security Visualizations for Executives Statistical Analysis s
Proactive Monitoring Search and Investigation 22
7KH6$16,QVWLWXWH- www.sans.org
22
Enabling IT Risk Scenarios
Security Relevant Data Confidentiality / Integrity / Availability
CSO / CIO / CEO Views
App Mgmt
IT Ops
Compliance
Web Analytics
Applying IT Risk Scenarios µFinding Abnormal Behaviors¶
Business Analytics
7KH6$16,QVWLWXWH- www.sans.org
23
Open Discussion What are the operational challenges with security big data analytics? Political issues?
7KH6$16,QVWLWXWH- www.sans.org
24
Questions?
7KH6$16,QVWLWXWH- www.sans.org
25
Contact Follow-up:
[email protected] Dave Shackleford
[email protected] Splunk Mark Seward
[email protected]
7KH6$16,QVWLWXWH- www.sans.org
26
