Introduction to debugging linux applications

Spenser Reinhardt
mplsCTFgames.org DC612
SpenserReinhardt@gmail.com

General Overview
Introduction to ELF (Executable and Linkable Format)
Layout
Assembly Primer
Number Bases
Registers and Memory Addressing
Basic Instructions
Debugging Tools and Ideas
Principal of Confirmation
GDB, GDBtui, DDD, Insight
Working with GDB
Example Errors

Executable & Linkable Format

ELF is a format for storing programs or fragments of programs on
disk, created as a result of compiling and linking. An ELF file is divided into
sections. For an executable program, these are the text section for the
code, the data section for global variables and the rodata section that
usually contains constant strings. The ELF file contains headers that describe
how these sections should be stored in memory. ELF is a format for storing
programs or fragments of programs on disk, created as a result of compiling
and linking. An ELF file is divided into sections. For an executable
program, these are the text section for the code, the data section for global
variables and the rodata section that usually contains constant strings. The
ELF file contains headers that describe how these sections should be stored
in memory.
Types of Files: .O, regular executables, shared libraries and core dumps

ELF Structure
Elf Header: Start of the file and a description of
it’s organization.
Program Header Table: Instructs the system
how to execute the file. (optional)
.text: Contains all program instructions
.bss: Holds all uninitialized data
.data: Holds all initialized data
.rodata: Holds read only data
.debug: Contains debug symbols (optional)
Section Header Table: Assists in locating each
internal section (optional)

What is Assembly Language?

The language known as Assembly or ASM, is really a collection of CPU
independent instructions that vary depending on the platform. Each
different CPU and sometimes each revision within a family of processors, will
have it's own version or interpretation of asm instructions. This variation in
what we know as an individual language, is due to the close relationship of
the hardware how machine instructions are almost directly derived from
asm instructions.

Assembly language is a translator language that allows total control over
every individual machine instruction generated by the translator program or
assembler.

Computers = Numbers

Computers only speak in numbers, however they do not count
with numbers as we think of them. They work in a mixed fashion
of both binary and hexadecimal.

Binary = Base 2 = Digits 0 1
Decimal = Base 10 = Digits 0 – 9
Hexadecimal = Base 16 = Digits 0 – 9, A - F

Decimal Calculations

1 15 100 870434
+2 +03 +86 + 37201
3 18 186 907635

9 53 234 829548
-5 - 36 - 75 - 829321
4 17 159 227

Binary Calculations

1B 1111B 1100100B 11010100100000100010B
+10B +0011B +1010110B + 1001000101010001B
11B 10010B 10111010B 11011101100101110011B

1001B 110101B 11101010B 11001010100001101100B
- 101B -100100B - 1001011B -11001010011110001001B
100B 10001B 10011111B 11100011B

Hexadecimal Calculations

1H 15H 64H D4822H
+2H +03H +56H + 9151H
3H 12H BAH DD973H

9H 35H EAH CA86CH
- 5H - 24H - 4BH - CA789H
4H 11H 9FH E3H

CPU Registers

Within a CPU there are special small storage compartments for very fast access, these are called
registers. Much like the rest of asm these registers are very processor specific, however many
generalizations can be made.

8-bit 16-bit 32-bit 64-bit Description

AL AX EAX RAX General purpose register
BL BX EBX RBX General purpose register
CL CX ECX RCX General purpose register
DL DX EDX RDX General purpose register

IP EIP RIP Points to current instruction location (Instruction Pointer)
BP EBP RBP Points to bottom of current stack frame (Base Pointer)
SP ESP RSP Points to top of current stack frame (Stack Pointer)
SI ESI RSI Used for special operations (Source Index)
DI EDI RDI Used for special operations (Destination Index)
CS, DS, SS, ES, FS, GS Segment Registers (16-bit)

Endianness
The order of importance and direction to read byte values. The systems CPU determines
endianness.

Little Endian: Read from right to left, with the most significant byte stored on the right.
(x86, x86-64)

Big Endian: Read from left to right, with the most significant byte stored on the left and not
flipped when read. (PowerPC, IBM Mainframes)

Bi Endian: Can potentially interpret either values either way. (MIPS, IA32, IA64)

The Stack
• Stores data temporarily as an application may need it.
• ESP = Top of the Stack EBP = Bottom of the Stack, or top of previous
• Addressed by offsets of espebp or direct memory locations
• Last in, First out (LIFO) or First in, Last out (FILO)
• Push [value] – Adds to top of the Stack, then decreases ESP accordingly
• Pop [value] – Removes from top of the Stack, then increases ESP
• Dynamically allocated, 32 bits wide
• Grows from higher memory down

Memory Layout
• Almost identical to on-disk ELF layout
• Definitions of sections in ELF, directly applies
• Also has Stack and Heap sections
• Heap space is dynamically allocated as programs
request or deallocate it.
• Heap is allocated in otherwise free space and
does not need to be in any order or specific
location
• Application sees 4GB of virtual memory
• Some or most space may be paged out

ASM Instructions - mnemonics
• Usually one command per line
• First or only operand is usually the destination operand, unless
specifically noted in the instruction details.
• R/8,16,32,64 Register size
• M/8,16,32,64 Memory size
• I/8,16,32,64 Immidate Data
• D/8,16,32,64 Displacement
• SR Segment Register

mov eax, ‘WXYZ’ Save WXYZ into eax
Move ZYXZ into eax, and
zero any remaining space in the register

ASM Instructions - Arithmetic

Instruction Description
add r/m32, r/m32 Combines operands though addition and stores in first
sub r/m32, r/m32 Subtracts operands and stores in first
mul r/m32, eax Multiplies operands* and stores in ax and dx when operands are
greater than 8 bits
div r/m32, eax Divides operands* and

* When mul and div are used the “A” register is used implicitly as the second
operand. “A” register could be AL, AX, EAX, or RAX.

ASM Instructions – Unary Operators
and r/m32, r/m32 Compares operands and sets to one if both are equal or zero if not.
or r/m32, r/m32 Compares operands and sets to one if at least one, is not zero.
xor r/m32, r/m32 Compares operands and sets to one if not equal and zero if equal.
not r/m32 Sets one to zero, and zero to one.
neg r/m32 Sets value equivalent negative value
inc r/m32 Increments operand by 1.
1.
dec r/m32 Decrements operand by 1.
1.

ASM Instructions – Bit Manipulation
shl r/m32, count Shifts bits left [count] times, stores overflow in CF, inserts zero
shr r/m32, count Shifts bits right [count] times, stores overflow in CF, inserts zero
rol r/m32, count Rotates bits from left and inserts on right, no CF use
ror r/m32, count Rotates bits from right and inserts on left, no CF use
rcl r/m32, count Rotates left to right, storing the first value rotated off,
and stored in CF, previous CF is set as right most value
rcr r/m32, count Rotates left to right, storing the first value rotated off,
and stored in CF, previous CF is set as right most value

ASM Instructions – Push Pop Mov
push r/m32 Pushes data onto the stack and lowers ESP
pusha Pushes all 16-bit general purpose registers at once
pushad Pushes all 32-bit general purpose registers at once
pushf Pushes Flags register onto the stack
pop r/m32 Pull data from the stack, store at location provided and raise ESP
popa Pull top 16 bytes from the stack and sets into each register !SP
popad Pull top 32 bytes from stack and and sets into each register !ESP
popf Pull top 2 bytes and store into Flags
mov r/m32, r/m32 Moves data from one location of memory to another

Debugging?

Debugging is the process within software development where applications
and code are tested to be accurate to the developers expectations. This can
include programmatic errors, unexpected data values, infinite loops, and
potentially security risks. Debugging is generally a recursive process
performed until all known bugs are located and corrected, and preformed
again when new issues are found.

Principle of Confirmation

The principle of confirmation, is a process of validating that assumptions you
as a programmer make, actually are true within execution. If something is
not as expected you have likely found a bug, or part of it.

GDB

• TextCLI based by default
• Semi GUI or uses other frontends
• -tui or ctrl-X-A to access console
analogue interface
• Extremely fast
• Low visual input

Insight
• Red HatCent OSFedora based • Frontend to GDB
• Removed from Debian repositories
• Full GUI, including console
• Fast and stable

DDD

• Works in almost all distributions

• Fast but not as stable (IMO)

• Full GUI and supporting console

• Virtually identical to Kdbg

GBD Commands
-tui Used while starting for semi-gui
Break [line] Stops execution at set line and allows for inspection
Tbreak [line] Stops execution at set line the first time hit only
Watch [condition] Performs commands for condition arguments set
Print [variable] Displays a variables value while execution is stopped
Frame [number] Diplays trace of set stack frame
Backtrace Displays entire stack layout

GDB Instructions

Run [arguments] Starts program execution with supplied arguments
Continue Continues normal execution after being paused
Step Executes line
Stepi Executes next ASMmachine instruction
Next Executes next line then pauses, skips over called
functions
Nexti Executes next ASMmachine instruction and pauses

Credits
The Art of Debugging With GDB, DDD, and Eclipse
Norman Mattloff and Peter Jay Salzman – No Starch Press 2008

Assembly Language Step by Step Programming With Linux
Jeff Duntemann – Wiley 2009

C++ Programming Today
Barbara Johnston – Pearson Prentice Hall 2008

Hacking The Art of Exploitation
Jon Erickson – No Starch Press 2008

Introduction to debugging linux applications

More Related Content

What's hot

Similar to Introduction to debugging linux applications

Recently uploaded

Introduction to debugging linux applications