Splunk

Searcching an nd Repo orting witth Splun nk 4.2 cllass labss Lab typ pographical conven ntions {student t number} ind dicates you sho ould replace this with your stu udent number. {server-name} indicate es you should substitute the server s name asssigned to this class. There are three t sourcetyp pes used in the e labs. The lab b instructions re efer to these so ourcetypes by the t types of da ata they repressent. The data a types are as follows: f Store data – access_* or o access_com mbined Firewall da ata – cisco_ws sa* Email data – cisco_esa a

Lab 1 – Fields Overview O Descripttion This is a sh hort lab to familiarize you with h the data used d in this course e.

Steps Task: Log into Splunk on classroom serrver. 1. 2. 3.

Direct your web brow wser to the classs lab system (for example, http://{serv ver-name}.sp plunk.com:8 8000) Login with the creden ntials your instrructor assigned d. amine the data sources on the e Summary pa age. Take a minute to exa

orm basic searrches on the sto ore data. Task: Perfo 4.

To the e right of the se earch box, set the t time range to Last 24 hou urs.

5.

Search h for all events s with the acce ess_combined d sourcetype (sstore data).

6. 7.

Take a few moments s to examine th he fields that we ere automatica ally extracted. Create e a table that in ncludes the clientip, and status s fields.

Resultts Example:

8. 9.

clientip

sttatus

1

0 192.1.2.40

20 00

2

192.1.2.40 0

20 00

3

67.230.133

40 04

4

…

…

Modifyy the search to only include evvents where ac ction=”purc chase”. Pipe to o the rename command c to re ename the cli ientip field to o customer.

Resultts Example: customerr

sttatus

1

192.1.2.40 0

20 00

2

192.1.2.40 0

20 00

3

67.230.133

40 04

4

…

…

21-Sep-11

1

Task: Perfo orm basic searrches on the firewall data 10. Search h for all events s in the last 24 hours for the cisco_wsa* c s sourcetype (fire ewall data). 11. Take a few moments s to examine th he fields that we ere automatica ally extracted. 12. Create e a table that displays d the cs_username an nd usage fields.

Resultts Example: cs_userna ame

us sage

1

grumpy@d demo.com

Bu usiness

2

grumpy@d demo.com

Pe ersonal

3

grumpy@d demo.com

Bu usiness

4

…

…

**CHALLE ENGE LAB 13. 14. 15. 16. 17.

Search h for all events s in the Last 24 4 hours for the cisco_esa sourcetype s (em mail data). Take a few moments s to examine th he fields that we ere automatica ally extracted. Search h for the term OUTBREAK_*. O Add th he rex comm mand to extract a new field called threat for the t threat inform mation. Add th he top command to display th he top values of o the threat field.

Resultts Example: threat

count

pe ercent

1

AK_0002499 hass threat level 3 OUTBREA

91

2..199662

2

OUTBREA AK_0002476 hass threat level 3

91

2..199662

3

OUTBREA AK_0002445 hass threat level 3

90

2..175489

4

…

…

…

Lab 2 – Basic Sttatistics Descripttion This lab reinforces the co ommands you learned for bassic statistics.

Steps Task: Rep port on top and rare values. 1. 2. 3.

Search h the sourcet type=access_ _combined fo or all events in the t last 24 hou urs where the referer_dom r main is not *myflowersh hop*. Use th he top command to display th he top 3 referre er domains. Add th he fields com mmand to modiify the report to o remove the percent field from the resultss.

Resultss Example: referer_do omain

count

1

http://www w.google.com

2842

2

http://www w.yahoo.com

154

3

http://www w.bing.com

147

21-Sep-11

2

4. 5.

Using the same data a, find the top status s codes fo or each web host. u the fields status s and ho ost. hint: use Add th he sort command to sort by the count field d in descending order.

Resultss Example: host

6. 7. 8.

status

count

percent

1

www2

200

907

77.987962

2

www1

200

900

78.809107

3

www3

400

774

8.168530

4

…

…

…

…

h sourcetype e=cisco_wsa* for all eventss in the last 24 hours. Search Use th he top command to display th he top usage tyypes, grouped by user. hint: use u the field cs s_username Add th he sort command to sort by the count field d in descending order.

Resultss Example: cs_userna ame

9.

usa age

coun nt

percent

1

grumpy@d demo.com

Personal

5189

57.19166 68

2

happy@de emo.com

Personal

4590

66.91937 76

3

doc@dem mo.com

Unknown

3926

58.18882 25

4

…

…

…

…

Using the same data a, find the mostt rare mime tyypes. u the field cs s_mime_type.. hint: use

Resultss Example: cs_mime_ _type

count

percent

1

application n/x-elc

1

0.003685

2

audio/mpe eg

1

0.003685

3

audio/x-ms s-wma

1

0.003685

4

…

…

…

e the stats command and asso ociated functions. Task: Use 10. Search h sourcetype e=access_combined for pu urchase events in the last 24 hours. hint: action=”purc a chase” 11. Use th he stats comm mand to count t the events byy productId. 12. Add th he sort command to sort by the count field d in descending order.

Resultss Example: productId d

count

1

AV-CB-01

533

2

AV-SB-02

230

3

FI-FW-02

119

4

…

…

21-Sep-11

3

13. Search h sourcetype e=access_combined to view w all the activitty for the online e flowershop in n the last 24 ho ours. 14. Use th he stats comm mand to get a distinct d countt of JSESSIONI IDs for each host. Resultss Example: host

dc(JSESSIO ONID)

1

www1

464

2

www2

557

3

www3

488

g a distinct count c of clien ntip for each host. 15. Modifyy the report to get

Resultss Example: host

dc(clientip))

1

www1

20

2

www2

21

3

www3

21

16. Use th he stats comm mand to create e a new report that t gets a sum m of bytes being served for each e file. Resultss Example: file

sum(by ytes)

1

cart.do

951390 0

2

category.s screen

976233 3

3

product.sc creen

827834 4

4

…

…

17. Modifyy the report to get g an average e instead of a sum. s Resultss Example: file

avg(by ytes)

1

cart.do

2111.48 88069

2

category.s screen

2160.55 52463

3

product.sc creen

2097.27 79805

4

…

…

18. Create e a new search h for events in sourcetype= s =cisco_wsa* that include the e term BLOCK_ _* in the last 24 hours s. 19. Use th he stats comm mand to list all the values off the x_webroo ot_threat_n name field with hin the results. Resultss Example: values(x_ _webroot_threatt_name) 1

21-Sep-11

"AntivirusX XPPro Fakealert"" "Paypopup p Cookie" "Trojan-Ba ackdoor-Zbot" "Trojan-Do ownloader-Suurcch" "Trojan-Do ownloader.Gen" "Unknown" "Virus-Otw wycal" "zhongsou u zztoolbar" -

4

Task: Use the eventstats command. 20. Search h sourcetype e=cisco_wsa* for all eventss in the last 24 hours. 21. Use th he stats comm mand to get a count c of all evvents grouped by usage.

Resultss Example: usage

count

1

e Borderline

2962

2

Business

5995

3

Personal

23505

4

…

…

he eventstats command to o add a sum of the count fielld to each even nt in a field called total. 22. Add th

Resultss Example: usage

count

to otal

1

e Borderline

2962

44 4588

2

Business

5995

44 4588

3

Personal

23505

44 4588

4

…

…

…

Lab 3 – Calculatting and Formatting F g Descripttion This lab reinforces the ev val and where e commands.

Steps Task: Use the eval comm mand to convertt field values. 1. 2.

Search h sourcetype e=cisco_wsa* for all eventss in the last 24 hours. Use th he stats comm mand to get a sum s of bytes grouped g by use er name as a fie eld called totallBytes. hint: use the sc_by ytes and cs_u username field ds.

Resultss Example: cs_userna ame

3.

tottalBytes

1

grumpy@d demo.com

227 72853

2

bashful@d demo.com

175 5084

3

doc@dem mo.com

185 5035786

4

…

…

Add th he eval command to set a ne ew field called MB. M Divide the totalBytes field by 104857 76 to populate the MB field. hint: the format is …| eval = (/10 ( 048576)

21-Sep-11

5

Resultss Example: cs_userna ame

4.

totalBy ytes

M MB

1

grumpy

227285 53

2.1765342

2

bahsful

175084 40

1.669744

3

doc

185035 5786

17 76.463877

4

…

…

…

Save the t search and d name it {stud dent number} Bandwidth B Us sage by User.

Task: Rou und field values s 5.

Using the search you u just created, modify the eva al command to o round the field value for the MB field to 2 decimal points. Resultss Example: cs_username

6.

totalBytes

MB

1

-

0

0

2

bashful@de emo.com

1750 0840

1.75

3

doc@demo o.com

1850 035786

176.46

4

…

…

…

Save the t search and d name it {stud dent number} MB M Per User

mpare field valu ues. Task: Com 7. 8.

Search h sourcetype e=access_combined for ac ction=”purch hase” produ uctId=”*”. Use th he eventstats command to o add the avera age value of the e price field to o each event in n a field called avera agePrice e from the pri 9. Add th he eval command to set a ne ew field called difference. d Subtract the averagePric a ice to popula ate the differ rence field. 10. Create e a table of th he results that includes i the pr roduct_name, averagePri ice, price, an nd differenc ce fields.

Resultss Example: product_n name

av veragePrice

price

difference

1

Sweet Splendor Bouquet

15 53.771429

49

29 -104.77142

2

Sweet Dre eams Bouquet

15 53.771429

89

-64.771429 9

3

Birthday Bouquet B

15 53.771429

299

145.228571 1

4

…

…

…

11. Save the t search and d name it {stud dent number} Product P Price Scale Task: Form mat field values s. 12. Modifyy the report you u just created to round the av veragePrice e and differe ence fields to 2 decimal points.

21-Sep-11

6

Resultss Example: product_n name

av veragePrice

price

difference

1

Sweet Splendor Bouquet

15 53.77

49

-104.77

2

Sweet Dre eams Bouquet

15 53.77

89

-64.77

3

Birthday Bouquet B

15 53.77

299

145.23

4

…

…

…

f the valu ues of the price field to prepen nd with a dollarr sign ($) and append a with a 13. Modifyy the report to format decimal and trailing zeroes z (.00) hint: Add an additio onal eval comm mand before crreating the tab ble, and use th he tostring function.

Resultss Example: product_n name

av veragePrice

price

difference

1

Sweet Splendor Bouquet

15 53.77

$49.00

-104.77

2

Sweet Dre eams Bouquet

15 53.77

$89.00

-64.77

3

Birthday Bouquet B

15 53.77

$299.00

145.23

4

…

…

…

Task: Use conditional sta atements. 14. Search h sourcetype e=access_combined for all events in the last l 24 hours. 15. Use th he eval command to set a ne ew field called reqPerforma r ance. Use the if function to group all even nts with st tatus=”200” ” into a value called “ok”, an nd all other eve ents into a value e called “fail led”. hint: you must include the quotes around "ok" an nd "failed" he stats comm mand to get a count c by reqP Performance e. 16. Add th

Resultss Example: reqPerforrmance

co ount

1

ok

71 12

2

failed

25 566

Task: Filterr results with th he where comm mand. he saved searc ch you created {student num mber} MB Per User U 17. Run th 18. Add th he where comm mand to only display results iff the value of th he MB field is greater g than 1.

Resultss Example: cs_userna ame

totalBy ytes

M MB

1

doc

185035 5786

17 76.46

2

sleepy

608961 1848

58 80.75

3

happy

413877 7926

39 94.70

4

…

…

…

21-Sep-11

7

Lab 4 – Charting g Descripttion Use the Ad dvanced Charting view to crea ate charts and timecharts.

Steps Task: Crea ate a basic colu umn chart. 1. 2.

3.

Naviga ate to the Adva anced Charting g view. Select Views V > Advanced Charting g. Create e a report for so ourcetype=a access_combi ined that displays how manyy of each produ uct was purcha ased in the las st 24 hours. Search for acti ion=”purchas se”, and use the t chart com mmand to displa ay a count of eventss by product_ _name. Set the e Chart type to o column.

Chart Example: E

4.

Save the t search and d name it {stud dent number} Daily D Product Sales

Task: Crea ate a multi-seriies chart and work w with forma atting options. 5.

e a report for so ourcetype=c cisco_wsa* th hat displays ea ach user’s Interrnet usage type es in the last 24 Create hours s. Use the char rt command to o display a cou unt of events with w cs_usern name as the X--axis, split by usage.

Chart Example: E

6. 7. 8. 9.

Chang ge the Stack Mode M to Stacke ed. Underr Format, click the x-axis linkk to display optiions for the X-a axis. Enter a tittle for the X-axxis. Underr Format, return to General options. o Chang ge the Chart ty ype to bar.

21-Sep-11

8

10. Underr Legend Place ement, select Bottom. B Chart Example: E

11. Save the t search and d name it {stud dent number} Internet I Usage e by User Task: Crea ate a basic time echart. e a timechart t for sourcetype=cisco_w wsa* that displays a count of o Internet usag ge types over time t 12. Create for the e last 24 hours s. 13. Set the e Chart type to o line and the Multi-series mode m to combiined. Chart Example: E

e a timechart t with a line ch hart type for so ourcetype=ac ccess_combi ined action= =purchase tha at 14. Create displayys a sum of the e price field by b product_na ame for the las st 24 hours. 15. Renam me the X-axis to t revenue. 16. Toggle e the Multi-serries mode betw ween split and combined and d note the disp play difference. Remember to o click apply when changing the multi-series mode. Task: Crea ate a report tha at buckets value es. n to the Search h view. 17. Return 18. Search h sourcetype e=access_combined for pu urchase eventts in the last 24 4 hours. 19. Use th he bucket com mmand to sort the t results by the t _time field d in 1 hour span ns. hint: bucket

21-Sep-11

9

20. Use th he stats comm mand to get a sum s of the pri ice field and po opulate a new field called hou urlySales. Group G the ressults by the _time field. hint: stats sum(< ) as () by Resultss Example: _time

hourlySale es

1

11/7/10 9:00:00.000 AM

712

2

11/7/10 10 0:00:00.000 AM

12356

3

11/7/10 11 1:00:00.000 AM

22633

…

…

Lab 5 – Correlatting Eventts Descripttion Reinforce creating, c searc ching, and repo orting on transa actions.

Steps Task: Crea ate a transactio on using common fields. 1. 2. 3. 4.

n to Search. Select S Last 4 hours for the tim me range. Return Search h for all events s in the email da ata. (sourcetype=”cisco_ _esa”) Note th he number of events. e Add th he transaction command to t the search, and a use the mi id, dcid, and icid i fields to create c the transa actions. Add th he search com mmand to searcch within the trransactions for REJECT.

ate a transactio on using common fields and maxspan, m maxp pause. Task: Crea 5. 6. 7.

h for all store data d in the last 24 hours. Search Create e a transacti ion based on the t clientip p field with a ma ax span of 10 minutes m and max pause of 2 minute es. Add th he stats comm mand to count by useragent t

Lab 6 – Creating g and Usin ng Lookup ps Descripttion Create and d use a new loo okup that will id dentify a browsser, version, and os based on the useragen nt field in the store s data.

Steps Task: Add d a lookup table e file. 1. 2. 3. 4. 5. 6. 7.

Save the t file browse er_lookup.cs sv to your com mputer. (Provide ed by your insttructor) Go to Manager >> Lookups L >> Lo ookup table filles. N to display the Add New page. Click New Verify the Destinatio on app is Searrch. B to loca ate and upload browser_loo okup.csv Click Browse In the Destination filename field, type t browser_ _lookup.csv v Click Save. S

21-Sep-11

10

Task: Crea ate a lookup de efinition. 8. 9. 10. 11. 12. 13. 14. 15.

Naviga ate back to the e main Lookups page. Click Lookup L definittions. Click New N to display the Add New page. Verify the Destinatio on app is Searrch. pe browser_l lookup. In the Name field, typ Verify the Type is File-based. t Lookup file e menu, select browser_look kup.csv. From the Click Save. S

Task: Use e the lookup in a report. 16. Return n to Search. 17. Search h for all events s in sourcetyp pe=access_c combined for th he last 24 hou urs. 18. Add th he lookup comm mand to call br rowser_looku up and referen nce the userag gent field as th he input field. OUTPU UT the browse er, version, and a os fields. Note the new fields are a now availab ble in the field picker. p 19. Add th he top command to display th he top browserrs.

Resultss Example: browser

co ount

percent

1

MSIE

97 70

30.152341

2

Safari

88 82

27.416874

3

Googlebott

48 82

14.389651

4

…

…

Task: Conffigure the looku up to run autom matically 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34.

Naviga ate to Manager >> Lookups >> Automatic c lookups. Click New N to display the Add New page. Verify the Destinatio on app is Searrch. pe browser_L LOOKUP In the Name field, typ From the t Lookup table menu, sele ect browser_lo ookup. Verify that sourcetyp pe is selected in the Apply to o menu. In the Named field, type access_combined. seragent in the e left field. In the Lookup inputt fields, type us ut fields, type browser in the e left field. In the Lookup outpu A another fiield. Click Add Type version v in left field. f Click Add A another fiield. type os o in the left field. Click the t Overwrite field f values ch heckbox. Click Save. S

Task: Use the automatic lookup 35. Return n to Search. 36. Search h sourcetype e=access_combined for all events in the last l 24 hours. 37. Examiine the fields list and notice that t browser, os, and version fields are now automaticcally extracted.

21-Sep-11

11

38. Use th he stats com mmand to create e a report that displays a count for each bro owser / os com mbination. Resultss Example: browser

os s

count

1

Firefox

W Windows

505

2

Googlebott

N//A

557

3

MSIE

W Windows

593

4

…

…

…

Lab 7 – Summary Indexin ng Descripttion Search and d create a repo ort from a summ mary index. NOTE: Fo or this lab a sum mmary index an nd summary se earch have alre eady been crea ated. You will be searching th he summary in ndex using a search named purchasedPro p oducts.

Steps Task: Sea arch a summary y index. 1. 2. 3.

Search h the summary y index for the last l 7 days using the purcha asedProducts search. hint: syntax is inde ex= search_ _name= Use th he stats comm mand to count t by product_ _name. Chang ge the time fram me to last 30 days. d

Task: Unde erstand the pop pulating summary search The search h used to populate the summa ary index is: sourcety ype="access_ _*" action=" "purchase" | sistats count c by pr roduct_name 4.

Would d the following search s generatte a report? Wh hy or why not? ?

5.

x="summary" search_nam me="purchase edProducts" | stats co ount by prod duct_name index | eva al revenue = "$" + pri ice + ".00" Create e a summary se earch that capttures: • prroduct name an nd productId • to otal revenue forr each product

6.

Save the t search as {student { numb ber} Summary y Sales. Set pe ermissions so everyone can Read. R Compa are search hes as a class. NOTE E: The purpose e of steps 5 and d 6 are to allow w you to practicce forming usefful summary se earches. You will w not sch hedule or confi figure the searcch to populate a summary ind dex.

21-Sep-11

12

Lab 8 – Creating g and Usin ng Macross Descripttion Create and d use macros.

Steps Task: Crea ate a basic mac cro 1. 2. 3. 4. 5. 6.

Naviga ate to Manager >> Advanced d search. Selectt Add new nex xt to the Search h macros item. Verify the Destinatio on app is set to o Search. Name the macro webusage. d, type the follo owing search string: s In the Definition field cetype="cisc co_wsa*" | transaction n s_hostnam me, cs_usern name sourc Save the t macro.

Task: Use a basic macro 7. 8. 9. 10.

Return n to the Search h app. Set the e time range to o Last 24 hourrs. In the search bar, typ pe `webusage e` and hit Ente er. Examine the e transactions. Add th he where comm mand. Filter th he results to on nly return transa actions where usage=”Busi u iness” and durat tion > 0. hint: enclose each argument a for th he where comm mand in parenthesis, and sep parate with AND D. hint: You must use quotes when in ndicating the fie eld/value usage="Business" 11. Add th he table comm mand to create e a report that displays d durat tion, usage, and a cs_usern name.

Resultss Example: duration

usage

c cs_username

1

3.02

Business

s sleepy

2

3

Business

h happy

3

6.21

Business

d doc

4

…

…

…

ate a macro witth arguments. Task: Crea 12. Naviga ate to Manager >> Advanced d search >> Search S macros s >> Add new 13. Name the macro acttivityByHost(2 2) g that searchess sourcetype=access_com mbined for varriable action and a host values. 14. Enter a search string hint: Format is fiel ldname=$arg gument$ 15. Add th he stats comm mand to get a count c by prod duct_name. 16. In the Arguments fie eld, enter the arguments, a sep parated by a co omma. ument (no $’s) hint: argument, argu t macro. 17. Save the Task: Use the macro with h arguments in a search 18. Return n to the Search h app. 19. Use th he macro, and pass the argum ments action= =purchase an nd host=www2 2 hint: `macroname(v ` value, valu ue)`

21-Sep-11

13

20. Run th he search again n with the follow wing argumentts remove and d www1 Resultss Example: product_n name

count

1

Birthday Bouquet B

25

2

Day Spa Certificate C

12

3

Tulip Bouq quet

18

4

…

…

21-Sep-11

14