Automated Analysis and Deobfuscation of Android Apps & Malware.pdf

Automated Analysis and Deobfuscation of Android Apps & Malware Jurriaan Bremer @skier t Freelance Security Researcher

June 10, 2013

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

1 / 44

Introduction

I

June 10, 2013

Who am I?

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

2 / 44

Introduction

I

Who am I? I I I

June 10, 2013

Student (University of Amsterdam) Freelance Security Researcher Cuckoo Sandbox Developer (Malware Analysis System)

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

2 / 44

Introduction

Android?

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

3 / 44

Introduction

Android? I

Smartphones

I

Runs custom Linux

I

Millions of Devices

I

Hundreds of thousands of applications

I

etc..

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

3 / 44

Android Applications Android Applications?

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

4 / 44

Android Applications Android Applications? I

Application Package File (APK) I I I I

June 10, 2013

Download from Google Play Zip file Some Metadata (Manifest, Images, ..) classes.dex

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

4 / 44

Android Applications Android Applications? I

Application Package File (APK) I I I I

I

All your code are belong to classes.dex I

June 10, 2013

Download from Google Play Zip file Some Metadata (Manifest, Images, ..) classes.dex More on this later.

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

4 / 44

Android Applications Android Applications? I

Application Package File (APK) I I I I

I

All your code are belong to classes.dex I

I

More on this later.

Resources I I I

June 10, 2013

Download from Google Play Zip file Some Metadata (Manifest, Images, ..) classes.dex

Images Data files Native libraries

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

4 / 44

Running Code on Android

There are two ways. I

Running native libraries I I

June 10, 2013

Extremely awesome This talk does not focus on native

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

5 / 44

Running Code on Android

There are two ways. I

Running native libraries I I

June 10, 2013

Extremely awesome This talk does not focus on native

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

5 / 44

Running Code on Android

There are two ways. I

Running native libraries I I

I

Running Dalvik Bytecode I I I I

June 10, 2013

Extremely awesome This talk does not focus on native Dalvik is Compiled Java Dalvik != Java classes.dex (More on this later)

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

6 / 44

Dex File Format (I)

I I

Dalvik Executable Format classes.dex I

June 10, 2013

Container format to store Dalvik Bytecode with Metadata

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

7 / 44

Dex File Format (I)

I I

Dalvik Executable Format classes.dex I I

Container format to store Dalvik Bytecode with Metadata Various Data Pools I I I I

June 10, 2013

Strings ”Hello World” Classes Ljava/lang/String; Fields Ljava/lang/String;->value Prototypes (I)Ljava/lang/String;

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

7 / 44

Dex File Format (I)

I I

Dalvik Executable Format classes.dex I I

Container format to store Dalvik Bytecode with Metadata Various Data Pools I I I I

I

Lots of headers I I I I

June 10, 2013

Strings ”Hello World” Classes Ljava/lang/String; Fields Ljava/lang/String;->value Prototypes (I)Ljava/lang/String; Complex Cross-references between fields and headers The Classname is a String A Prototype has a String as return value A method links to a Prototype, etc..

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

7 / 44

Dex File Format (II)

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

8 / 44

Dalvik Bytecode Example public static void hello() { System.out.println(”Hello AthCon”); } ->

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

9 / 44

Dalvik Bytecode Example public static void hello() { System.out.println(”Hello AthCon”); } -> sget-object v0, Ljava/lang/System;->out:Ljava/io/PrintStream;

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

9 / 44

Dalvik Bytecode Example public static void hello() { System.out.println(”Hello AthCon”); } -> sget-object v0, Ljava/lang/System;->out:Ljava/io/PrintStream; const-string v1, ”Hello AthCon”

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

9 / 44

Dalvik Bytecode Example public static void hello() { System.out.println(”Hello AthCon”); } -> sget-object v0, Ljava/lang/System;->out:Ljava/io/PrintStream; const-string v1, ”Hello AthCon” invoke-virtual v0, v1, Ljava/io/PrintStream;->println(Ljava/lang/String;)V

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

9 / 44

Dalvik Bytecode Example public static void hello() { System.out.println(”Hello AthCon”); } -> sget-object v0, Ljava/lang/System;->out:Ljava/io/PrintStream; const-string v1, ”Hello AthCon” invoke-virtual v0, v1, Ljava/io/PrintStream;->println(Ljava/lang/String;)V return-void

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

9 / 44

What’s your point?

I

Decompiling is mostly trivial

I

JEB - http://android-decompiler.com/

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

10 / 44

What’s your point?

I

Decompiling is mostly trivial

I

JEB - http://android-decompiler.com/

I

Smali/Baksmali allows you to quickly modify code

I

Based on .smali files, a wrapper around Dalvik bytecode

I

Free and Open Source https://code.google.com/p/smali/

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

10 / 44

Let’s welcome Obfuscators

I

Commercial solutions

I

Make Reverse Engineering harder

I

Make automated analysis harder (what to look at?)

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

11 / 44

Let’s welcome Obfuscators

I

Commercial solutions

I

Make Reverse Engineering harder

I

Make automated analysis harder (what to look at?)

I

What can we do..?

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

11 / 44

Let’s welcome Obfuscators

I

Commercial solutions

I

Make Reverse Engineering harder

I

Make automated analysis harder (what to look at?)

I

What can we do..?

I

Deobfuscate the obfuscated code!

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

11 / 44

Let’s welcome Obfuscators

I

Commercial solutions

I

Make Reverse Engineering harder

I

Make automated analysis harder (what to look at?)

I

What can we do..?

I

Deobfuscate the obfuscated code!

But first..

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

11 / 44

Introduction to Our Tools readdex(1) I

Custom utility to read .dex files

I

Not very strict

I

Works in cases where traditional tools fail

I

E.g., dexdump, dex2jar, sometimes even JEB

I

(Will report JEB bugs later)

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

12 / 44

Introduction to Our Tools readdex(1) I

Custom utility to read .dex files

I

Not very strict

I

Works in cases where traditional tools fail

I

E.g., dexdump, dex2jar, sometimes even JEB

I

(Will report JEB bugs later) Handles the following cases correctly

I

I I I I I

June 10, 2013

Invalid checksum hashes (fails dexdump) Unused opcodes (fails dex2jar/dexdump) Invalid Data Pool Indices (dexdump/dex2jar) Unicode function names (IDA Pro?!) Etc..

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

12 / 44

Introduction to Our Libraries

I

June 10, 2013

Dalvik Disassembler

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

13 / 44

Introduction to Our Libraries

I I

Dalvik Disassembler Basic Dalvik Emulator I I

June 10, 2013

Supports most Dalvik Instructions Supports simple Java Classes (Strings, etc.)

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

13 / 44

Introduction to Our Libraries

I I

Dalvik Disassembler Basic Dalvik Emulator I I

I

Dex File Parser I

June 10, 2013

Supports most Dalvik Instructions Supports simple Java Classes (Strings, etc.) Dex File Creator is Work in Progress

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

13 / 44

Introduction to Our Libraries

I I

Dalvik Disassembler Basic Dalvik Emulator I I

I

Dex File Parser I

I

June 10, 2013

Supports most Dalvik Instructions Supports simple Java Classes (Strings, etc.) Dex File Creator is Work in Progress

Totalling more than 5kloc C (including readdex)

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

13 / 44

Introduction to Our Libraries

I I

Dalvik Disassembler Basic Dalvik Emulator I I

I

Supports most Dalvik Instructions Supports simple Java Classes (Strings, etc.)

Dex File Parser I

Dex File Creator is Work in Progress

I

Totalling more than 5kloc C (including readdex)

I

Not to mention basic Python wrappers

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

13 / 44

Introduction to Our Libraries

I I

Dalvik Disassembler Basic Dalvik Emulator I I

I

Supports most Dalvik Instructions Supports simple Java Classes (Strings, etc.)

Dex File Parser I

Dex File Creator is Work in Progress

I

Totalling more than 5kloc C (including readdex)

I

Not to mention basic Python wrappers

I

All of it will be Open Source soon (TM)

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

13 / 44

What’s next? This stuff is actually useful?

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

14 / 44

What’s next? This stuff is actually useful?

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

14 / 44

Class & Function Name Obfuscation Used by for example Dexguard & Freedom.apk..

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

15 / 44

Class & Function Name Obfuscation Used by for example Dexguard & Freedom.apk.. Welcome to China..

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

15 / 44

Class & Function Name Obfuscation Used by for example Dexguard & Freedom.apk.. Welcome to China..

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

15 / 44

Class & Function Name Obfuscation

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

16 / 44

China?

I

Unreadable identifiers

I

Problematic when Modifying Dalvik Code (.smali)

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

17 / 44

China?

I

Unreadable identifiers

I

Problematic when Modifying Dalvik Code (.smali)

I

unchina.py to the rescue!

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

17 / 44

unchina.py

I

Walks the Dex file

I

Enumerates all classes and methods

I

Renames Chinese names with something readable

I

”zmagic ” + number

I

(For now, can be changed of course..)

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

18 / 44

unchina.py

I

Walks the Dex file

I

Enumerates all classes and methods

I

Renames Chinese names with something readable

I

”zmagic ” + number

I

(For now, can be changed of course..)

I

Simple Python script using some hacky functionality

I

Rewrites parts of the Dex file as needed

I

Writes a new Dex file (still kind of experimental)

I

Sounds easier than it is!

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

18 / 44

unchina.py Demo

Demo of Unchina.py..

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

19 / 44

Obfuscated Strings (I)

Used by for example Dexguard, Whatsapp.apk, Freedom.apk

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

20 / 44

Obfuscated Strings (I)

Used by for example Dexguard, Whatsapp.apk, Freedom.apk

I

Instead of using Hardcoded Strings

I

Build strings up at runtime

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

20 / 44

Obfuscated Strings (I)

Used by for example Dexguard, Whatsapp.apk, Freedom.apk

I

Instead of using Hardcoded Strings

I

Build strings up at runtime Makes it harder to analyze

I

I I

June 10, 2013

Strings usually have meaningful information (Function names, Debug information, URLs, etc.)

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

20 / 44

Obfuscated Strings (I)

Used by for example Dexguard, Whatsapp.apk, Freedom.apk

I

Instead of using Hardcoded Strings

I

Build strings up at runtime Makes it harder to analyze

I

I I

I

More code in the binary I I

June 10, 2013

Strings usually have meaningful information (Function names, Debug information, URLs, etc.) Normally one string Now entire functions for decoding, function calls, etc..

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

20 / 44

Obfuscated Strings (II)

We want to reconstruct the obfuscated strings

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

21 / 44

Obfuscated Strings (II)

We want to reconstruct the obfuscated strings I

Use our Simple Dalvik Emulator

I

Combined with some heuristics (in the future)

I

For now a bit hardcoded..

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

21 / 44

Three different String Obfuscation examples

I

Whatsapp.apk

I

Freedom.apk

I

A Dexguarded binary

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

22 / 44

Whatsapp (I)

#1 - Whatsapp.apk I

Defines for lots of classes I I

June 10, 2013

Class Initialization function Called when the class is being loaded

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

23 / 44

Whatsapp (II)

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

24 / 44

Whatsapp (III)

I I

We emulate the method Intercept the sput-object instruction I

I

June 10, 2013

sput-object v0, mb->z:Ljava/lang/String;

”Assign Static Class Variable”

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

25 / 44

Whatsapp (III)

I I

We emulate the method Intercept the sput-object instruction I

sput-object v0, mb->z:Ljava/lang/String;

I

”Assign Static Class Variable”

I

We now have the deobfuscated string

I

(or multiple strings, in some cases)

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

25 / 44

Whatsapp (III)

I I

We emulate the method Intercept the sput-object instruction I

sput-object v0, mb->z:Ljava/lang/String;

I

”Assign Static Class Variable”

I

We now have the deobfuscated string

I

(or multiple strings, in some cases)

I

Roughly 5000 strings deobfuscated!

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

25 / 44

Freedom (I)

#2 - Freedom.apk I

Has xor decryption methods

I

Calls functions with magic decoding value

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

26 / 44

Freedom (II)

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

27 / 44

Freedom (III)

I

The xor decryption methods have a specific signature

I

Their prototype is always (B)Ljava/lang/String;

I

(Accepts an 8bit integer, returns a String.)

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

28 / 44

Freedom (III)

I

The xor decryption methods have a specific signature

I

Their prototype is always (B)Ljava/lang/String;

I

(Accepts an 8bit integer, returns a String.)

I

We scan every method in the Dex file

I

Function Call to Decryption Method ->Decrypt the String

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

28 / 44

Freedom (III)

I

The xor decryption methods have a specific signature

I

Their prototype is always (B)Ljava/lang/String;

I

(Accepts an 8bit integer, returns a String.)

I

We scan every method in the Dex file

I

Function Call to Decryption Method ->Decrypt the String

I

Roughly 600 strings deobfuscated!

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

28 / 44

Dexguard (I)

#3 - Dexguard is a Commercial Obfuscator As example we use an obfuscated Cyanide.apk

I

Root exploit for some Motorala device

I

(Thanks to Justin Case for the sample)

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

29 / 44

Dexguard (II)

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

30 / 44

Dexguard (III)

I

Dexguard initializes a lookup table on

I

Decrypts strings using this lookup table

I

One dedicated decryption method

I

Signature (III)Ljava/lang/String;

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

31 / 44

Dexguard (IV)

I

Dexguard is a combination of Whatsapp and Freedom

I

(With regards to techniques)

I

First emulate

I

To obtain the lookup table

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

32 / 44

Dexguard (IV)

I

Dexguard is a combination of Whatsapp and Freedom

I

(With regards to techniques)

I

First emulate

I

To obtain the lookup table

I

Then scan every method in the Dex file

I

Find function calls to the decryption method

I

Decrypt strings!

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

32 / 44

Dexguard (IV)

Original Dexguarded Cyanide.apk

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

33 / 44

Rewriting the Dex file (I)

Rewriting Whatsapp, Freedom and Dexguarded Cyanide.apk I

We have the decrypted strings

I

Obfuscated code always takes more instructions than deobfuscated code

I

Patching time..!

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

34 / 44

Rewriting the Dex file (II)

Some problems.. I

We have to introduce new strings I I

June 10, 2013

Extend the String Data Pool Shuffle around half the Dex..

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

35 / 44

Rewriting the Dex file (II)

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

36 / 44

Rewriting the Dex file (III)

Some problems.. I

We have to introduce new strings I I

June 10, 2013

Extend the String Data Pool Shuffle around half the Dex..

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

37 / 44

Rewriting the Dex file (III)

Some problems.. I

We have to introduce new strings I I

I I

Patch Dalvik instructions (straightforward) Remove obsolete functions I I I

June 10, 2013

Extend the String Data Pool Shuffle around half the Dex..

String Decryption Methods are now unused Quite painful.. Dex file-wise *Work in Progress*

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

37 / 44

Rewriting the Dex file (IV)

I

We move all strings to EOF

I

We fixup other data structures

I

Demo time

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

38 / 44

Rewriting the Dex file (V)

Demo of reconstructing Dexguarded Cyanide.apk

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

39 / 44

How do we go from here?

I

Generic Deobfuscation I

June 10, 2013

Based on Heuristics with Prototypes etc

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

40 / 44

How do we go from here?

I

Generic Deobfuscation I

I

Classification based on stripped down binaries I I I

June 10, 2013

Based on Heuristics with Prototypes etc One binary can have many obfuscated representations Deobfuscate to something like the original binary Allows more accurate classification

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

40 / 44

How do we go from here?

I

Generic Deobfuscation I

I

Classification based on stripped down binaries I I I

I

June 10, 2013

Based on Heuristics with Prototypes etc One binary can have many obfuscated representations Deobfuscate to something like the original binary Allows more accurate classification

Did I mention plaintext strings?

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

40 / 44

How do we go from here?

I

Generic Deobfuscation I

I

Based on Heuristics with Prototypes etc

Classification based on stripped down binaries I I I

One binary can have many obfuscated representations Deobfuscate to something like the original binary Allows more accurate classification

I

Did I mention plaintext strings?

I

Plaintext Strings!

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

40 / 44

Automated Malware Analysis!

Yesterday a new malware was found in the wild..

http://www.securelist.com/en/blog/8106/The_most_ sophisticated_Android_Trojan

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

41 / 44

High Expectations Asian Dad strikes again!

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

42 / 44

Backdoor.AndroidOS.Obad.a

I I

Seems like a pretty advanced android malware Multiple obfuscation layers (for strings) I I

I

Some Plaintext Strings.. I I I I

June 10, 2013

Got a start, but far from complete.. *Quick Demo* Tries to enable Bluetooth getSimSerialNumber .. (I need some more time)

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

43 / 44

Questions?

Any questions? Cheers to.. p1ra, nex‘, rep, blasty, thuxnder, diff-, jcase, George, jduck, .. Interested in Android Security? Join #droidsec on irc.freenode.org (thanks jduck!)

June 10, 2013

Jurriaan Bremer

@skier t

Analysis and Deobfuscation of Android Apps

44 / 44