CCY3102

Malware Analysis and Reverse Engineering

Dr. Mohamed Elhamahmy
Week One
Sun 16/Feb/2025
Tuesday 18/Feb/2025
 CCY3102 - Malware Analysis and Reverse Engineering
Course Overview
This course provides computer science students with foundational
knowledge in malware analysis and reverse engineering. The course
emphasizes defensive security practices, ethical considerations, and
responsible disclosure.

Prerequisites
- Strong programming background (C/C++, x86 Assembly)
- Operating Systems fundamentals
- Basic networking concepts
- Computer architecture understanding
Learning Objectives
By the end of this course, students will be able to:
1. Set up and maintain secure malware analysis environments
2. Perform static and dynamic analysis of suspicious programs
3. Use industry-standard reverse engineering tools
4. Understand common malware behaviors and detection techniques
5. Apply ethical principles in security research
Supplemental Resources
Required Tools
- IDA Free/Ghidra
- x64dbg/WinDbg
- Process Monitor/Process Explorer
- Wireshark
- Volatility
- Python scripting environment
- Virtualization software
Recommended Reading
1. "Practical Malware Analysis" by Michael Sikorski and Andrew Honig
2. "The Art of Memory Forensics" by Michael Hale Ligh et al.
3. "Reversing: Secrets of Reverse Engineering" by Eldad Eilam
4. "The IDA Pro Book" by Chris Eagle

Online Resources
- SANS Reading Room
- FireEye Threat Research Blog
- Microsoft Security Blog
- VirusTotal Blog
- Malware Traffic Analysis
Safety Guidelines
1. Never analyze malware on production systems
2. Always use isolated environments
3. Handle samples with proper precautions
4. Follow responsible disclosure practices
5. Adhere to ethical guidelines

Assessment Structure
- Final Test(40%)
- Midterm Test (30%)
- Final Project (20%)
- Course Work (10%)
Final Project Requirements
Students will perform a complete analysis of an assigned sample, including:
1. Environment setup documentation
2. Static analysis findings
3. Dynamic analysis results
4. Network behavior analysis
5. Complete technical report
6. Presentation of findings

Ethics Statement
This course is designed for educational purposes only. Knowledge gained
should be used responsibly and ethically for defensive security purposes.
Students must sign an ethics agreement before participating in labs.
 Lecture 1
Agenda:
1. Understanding Malware
2. Definition and classification of malware types
• Viruses, worms, trojans, ransomware, rootkits
• Current malware landscape and trends
3. Malware infection vectors and propagation methods
4. Common malware behaviors and objectives
 What is the malware?
Definition: Malware is a Malicious Software - malicious executable or
binary.
Purpose: Attackers use malware to:
• Spy on targets (e.g., remote access tools, keyloggers).
• Steal or destroy data.
• Encrypt data for ransom (ransomware).

Malware is a short for “Malicious Software" refers to any software

designed to harm, exploit, or otherwise compromise computers,
networks, or users. There are many types of malware, each with distinct
characteristics and purposes.
 What is Malware Analysis?
Definition: The process of analyzing a malware sample or binary to
extract information.
Goals:
• Understand the malware's functionality and scope.
• Determine how the system was infected.
• Enable prevention of future attacks.
 Objectives of Malware Analysis
Why Analyze Malware?
Key Objectives:
• Understand the type of malware and its capabilities.
• Determine how the system was infected (e.g., targeted or
phishing attack).
• Analyze how the malware communicates with the attacker.
• Extract indicators for generating signatures for future detection.
 Methods of Malware Analysis
Static Analysis:
• Analyzing malware without executing it.
• Extracting metadata like strings and PE headers.
Dynamic Analysis:
• Executing the malware and analyzing its behavior.
• Typically monitored in a debugger.
 Malware Analysis Fundamentals
Types of Malware
Setup Safe Environment
Basic Static Analysis
Basic Dynamic Analysis
 Malware Reverse Engineering
Demonstrate Windows OS Internals including (x86)
Basic understanding of assembly language
Using disassembler in advanced static analysis
Using debugger in advanced dynamic analysis
 Malware Analysis in-depth
Using new methods to analyze specific formats of malware
Scripting and interpreted languages (eg: Autoit, Python)
Office documents (eg: Word, PPT, Excel, …etc.)
PDF and JavaScript
Android applications
Viruses
•Description: Viruses attach themselves to clean files and infect other clean
files. They can spread uncontrollably, damaging system functionality and
deleting or corrupting files. It need a human intervention to start spreading.
•Example: ILOVEYOU Virus (2000) - Spread via email, it overwrote files and
sent itself to everyone in the victim's address book.
Worms
•Description: Worms replicate themselves to spread to other computers,
without need for permission or human intervention, often over a network.
They do not need to attach to a program and can cause harm by consuming
bandwidth or overloading systems.
•Example: Conficker Worm (2008) - Exploited Windows vulnerabilities to
create a botnet and steal sensitive information.
Trojans (Trojan Horses)
•Description: Trojans disguise themselves as legitimate software but perform
malicious actions once executed. They often create backdoors for attackers to
gain unauthorized access.
•Example: Zeus Trojan - A banking Trojan that stole financial data by logging
keystrokes and form submissions.
Ransomware
•Description: Ransomware encrypts a victim's files and demands payment
(usually in cryptocurrency) for the decryption key. It can cripple organizations
and individuals.
•Example: WannaCry (2017) - Exploited a Windows vulnerability to encrypt
files and demanded Bitcoin payments.
Spyware
•Description: Spyware secretly monitors user activity, collecting sensitive
information such as passwords, credit card numbers, and browsing habits.
•Example: FinFisher - A government-grade spyware used for surveillance,
capable of monitoring communications and extracting data.
A rootkit
is a type of malware designed to gain unauthorized access to a system
while hiding its presence. It operates at a deep level, often in the
kernel or firmware, making it difficult to detect and remove. Rootkits
can enable attackers to maintain persistent access, steal data, or
execute malicious activities without the user's knowledge
The Sony BMG Rootkit (2005) was a notorious example where Sony
embedded a rootkit in music CDs to enforce DRM (Digital Rights
Management). Once installed, the rootkit hid itself and any files or
processes starting with "sys," creating a backdoor for potential malware.
This not only violated user privacy but also exposed systems to additional
security risks. The incident led to widespread criticism and legal
repercussions for Sony.
Adware
•Description: Adware displays unwanted advertisements, often redirecting
users to malicious sites. While not always harmful, it can degrade system
performance and compromise privacy.
•Example: Fireball - A adware that hijacked browsers to redirect users to ads
and track their activity.
 Malware Behaviors
Executes malicious code to exploit system vulnerabilities.
Spreads via phishing, infected files, or network
vulnerabilities.
Evades detection using encryption, obfuscation, or
polymorphism.
Establishes persistence via registry edits, scheduled
tasks, or boot sectors.
Communicates with command-and-control (C2) servers
for instructions.
 Malware Objectives
Steal sensitive data (credentials, financial info,
intellectual property).
Disrupt operations via ransomware, DDoS, or system
corruption.
Gain unauthorized access for espionage or further
attacks.
Monetize through cryptojacking, ad fraud, or data sales.
Create botnets for large-scale attacks or spam campaigns.