pk.org: Computer Security/Lecture Notes

Malware and Defenses

Study Guide

Paul Krzyzanowski – 2026-11-05

Malware is defined by what the people responsible for a system authorize, not by what a program does. A malware infection passes through a sequence of activities, from how the code first executes to how it spreads. Each activity has its own mechanisms and its own defenses.

What Malware Is

Malware is software deliberately deployed to violate the security interests or authorized policies of the people responsible for a system. Three parts of that definition carry the weight:

A potentially unwanted program (PUP) is software whose behavior or installation practices conflict with the user’s expectations or interests: intrusive advertising, unwanted browser changes, or bundled installations with inadequate disclosure. Classifying one depends on what the installer disclosed, what the user authorized, and whether the user can reverse the changes.

Malware Labels

Labels describe independent properties, so a program can have several. Propagation labels describe how a program spreads, payload labels describe what it does, and other labels describe deception or concealment:

Propagation and persistence are independent. A worm can live in memory and vanish at reboot, while a backdoor that never replicates can stay for years.

Backdoors in Legitimate Code

A backdoor need not be a separate program. An altered login routine, a hidden account, or an added key is a backdoor too. So is a debugging feature left enabled in a production release, whether or not harm was intended.

A backdoor can be planted in a library that other programs load. A compromised compiler can insert a backdoor into the programs it builds, including the next compiler, while both sources stay clean. Source code reviews show what people wrote, not what the tools produced, so they can’t rule out a backdoor inserted during the build.

The Malware Lifecycle

The classic model of a virus has three functions: an infection mechanism that finds and infects the next host, a trigger that decides when an action occurs, and a payload that performs the action.

Modern malware is described with a longer list of activities. The list is a teaching order rather than a sequence every program follows, since malware can omit, repeat, or combine them:

  1. Infection: how the malware arrives and first executes.

  2. Delivery: a dropper or loader deploys the actual payload, often in stages.

  3. Persistence: surviving logouts and reboots.

  4. Trigger: the condition under which the payload activates.

  5. Communication: contact with an operator for instructions and results.

  6. Payload: what the malware does.

  7. Propagation: spreading to other systems.

Concealment runs alongside the activities rather than being one of them. Detection is the defender’s view of all of them.

Infection

Malware enters a computer by exploiting a vulnerability in running software, convincing a user to run something, or through physical access, such as a removable drive. Stolen credentials and compromised update channels are further routes that need neither a new exploit nor a user mistake.

Four terms describe the boundaries an attacker crosses:

Exploits

An exploit uses a vulnerability to make a system perform an unintended operation. The attacker starts with only the privileges of the compromised process, so escaping a sandbox or controlling the kernel requires further steps. Fixing the vulnerability closes the entry route but doesn’t remove a payload that has already installed itself.

A zero-day vulnerability is a flaw for which no protective patch is available; the developers have had zero days of warning to prepare a fix. The term describes the patch status, not the attack technique or whether it requires user interaction.

An N-day vulnerability has been public for N days, usually because a patch was released. Attackers compare the patched code with the previous version to find the flaw and build an exploit. Machines that have not yet installed the patch are exposed until they do, and most exploitation uses such flaws.

An exposed network service can be both the entry point and the source of attacks on other machines. Each infected host becomes another scanner, so infections grow exponentially at first, faster than people can respond.

Social Engineering

Social engineering manipulates people into revealing information or taking actions that weaken security. It targets the decision that grants authority rather than the mechanism that enforces it. It needs no vulnerability, so it is cheap, and anyone with access can be an entry point.

Attackers rely on a small set of persuasion principles: authority, urgency and scarcity, trust and familiarity, fear, curiosity, reward, helpfulness and reciprocity, and consistency and commitment. The strongest lures combine several. Two related habits help the attacker:

Three forms of attack appear repeatedly:

Attackers can supply every trust signal a reader relies on: a logo, a deadline, personal details, and even a real conversation history from a compromised account. HTTPS shows only that the connection to the domain in the URL is encrypted, not that the domain belongs to the organization shown on the page.

The mismatches that deserve scrutiny concern the requested action:

Independent verification tests the claim through a channel the message didn’t provide.

Lookalike Domains

A phishing link needs an address that looks right:

The defenses act on the address rather than on judgment:

Phone and Text Attacks

Vishing uses phone calls, giving the attacker a live conversation to answer objections and apply pressure. Caller ID can be spoofed. Smishing uses text messages that imitate familiar notifications, with shortened links that conceal the destination.

A one-time code doesn’t stop a real-time phishing attack. The attacker’s page collects the password and the code and uses them on the real site within the code’s validity period.

A code from an authenticator app is better than a text message because there is no message to intercept, but it doesn’t stop relay attacks because the code doesn’t depend on which site is asking for it. A SIM swap moves the victim’s phone number to the attacker’s phone.

Phishing-resistant authentication is bound to the legitimate site. A passkey signs a challenge that includes the site’s domain, so a lookalike page receives a signature the real site rejects.

A user’s decision to run a program does not have to grant it every permission the user holds. Application control, macro policy, sandboxes, and the Principle of Least Privilege limit what a deceived user’s action can do.

Removable Media and Devices

An air gap separates a system from direct network connections but not from removable storage, maintenance laptops, or imported files, so those transfers become the entry points.

A drive does not execute code merely by storing it. Execution needs a mechanism:

Defense depends on the mechanism: application control and macro policy for files the user runs, disabling AutoRun, a firmware setting to prevent booting from external media, and patching parser flaws.

A USB drop attack leaves a drive where a target will find it and relies on the finder to connect it and open its contents. People who intend to return a drive still open what it carries, so the controls that limit what an unknown drive may do are stronger than training.

BadUSB alters a device’s firmware so it reports an interface it shouldn’t have, such as a keyboard. Injected keystrokes run with the privileges of the logged-in session, so file scanning, reformatting, and blocking removable storage do not stop it. Device-control policy that restricts which USB interfaces are accepted, and locking unattended workstations, do.

Documents

A macro is a program stored in a document and run by the application that opens it, with all the permissions of the user. A malicious macro needs macro execution enabled, not a vulnerability, which is why Office now blocks macros in files from the Internet by default.

A parsing flaw is the other document attack: malformed content corrupts memory in the application that reads it. The two need different defenses: blocking macros doesn’t fix a parser bug, and fixing parser bugs doesn’t stop an authorized macro. A document can also include a link or QR code to a phishing page and execute nothing.

Drive-by and Zero-click Attacks

A drive-by download delivers software through a visited web page, usually by exploiting the browser, so visiting the page supplies the input that runs the code. A zero-click attack does not need any interaction, because a messaging application parses the attacker’s content when it arrives or generates a preview. Zero-click describes the user interaction required and says nothing about whether the flaw is new.

Trusted Channels

In a supply chain compromise, malicious code arrives through a channel the victim deliberately trusts: a compromised update or a malicious dependency inside a library a legitimate program loads. Code signing protects signed content from later changes, but a compromised build or signing process produces correctly signed malicious code.

A counterfeit utility downloaded from an attacker’s site is a different case. Nothing in the publisher’s channel was compromised; the failed control was the user’s judgment of the source. A stolen administrator account can likewise install software through a legitimate management service that works exactly as designed.

Delivery

Multi-stage delivery divides the work among components. The code that first executes may be small, such as the few bytes an exploit can inject, while the payload needs much more. A small initial component launches a later one, chosen for the target’s platform and replaceable without changing the lure.

Each required download is also a dependency that defenders can interrupt and an event they can observe. A delivery stage is not a stage of an intrusion: a whole chain of downloads may accomplish only execution.

The components have standard names:

Fileless malware is execution that does not depend on a payload executable stored on disk. Memory is volatile, so persistence still needs something stored that reloads the code. Living off the land uses programs already present on the system, such as an administration scripting environment. A valid signature on an interpreter authenticates the interpreter, not the script it is given.

The attacker-supplied scripts and loader in a chain are malware even when they do not steal anything themselves, because each exists to run code the user did not authorize. The legitimate utilities they run are not.

Persistence

Persistence allows malicious code to resume execution after logout, process termination, or reboot. Every mechanism is a stored program or setting plus an event that triggers it. Malware can register itself at three layers, which run at different points in startup:

  1. Registration points the operating system and applications already provide: services, login items, scheduled tasks, shell profile files, document templates, and library loading.

  2. The boot code that runs before the operating system.

  3. The firmware that runs before the boot code.

A registered program’s privileges come from the account configured for it. Who can install persistence depends on who may write the registration point: a service’s configuration needs administrative access to change, while a login item or profile file can be written by its user.

In library hijacking, a program loads a library by name, and the attacker places a malicious library with that name where the program looks first.

Cleaning a machine requires removing both the running program and the settings that would start it again.

Boot Code and Firmware

At power-on, a PC runs firmware, which initializes the hardware and starts a boot loader, which loads the kernel and transfers control to it. Code that runs earlier controls what runs later. Anything before the kernel runs before the kernel’s protections and before security software.

Three kinds of malware use that position:

Early execution lets a bootkit modify later code in memory even when that code has a valid signature on disk. Installing it still requires authority to modify the startup components or a vulnerability that permits it.

Secure Boot verifies each boot component against an authorization policy before letting it run. The firmware holds trusted public keys and a revocation list, and each stage checks the next. A signature proves who signed the code, not that the code is safe: an old signed boot loader with a known flaw still passes unless it has been revoked, so fixing a boot-code vulnerability also requires revocation.

Rootkits

A rootkit conceals unauthorized activity or access by altering system programs, libraries, or the kernel:

A rootkit is found by looking somewhere it hasn’t changed. For a kernel rootkit there are two approaches:

Driver signing restricts what can load into the kernel but does not prove a driver is harmless, since code-signing private keys can be compromised and signed drivers can have vulnerabilities.

Triggers

A trigger is the condition under which the payload activates: a date, a count, a user action, or the presence of a particular target. A logic bomb is malicious code that stays dormant until a specified condition occurs, usually planted by someone who already has access. Not every payload waits: an infostealer runs at startup, and a worm’s replication is itself the harm.

Communication

Command and control (C2) is the exchange through which an operator directs compromised systems and receives results. Exfiltration is the unauthorized transfer of collected information out of the environment. Fully autonomous malware does not require command and control.

A botnet is a collection of compromised machines, called bots or zombies, that one operator directs at once; it can be rented out for different jobs. In a centralized botnet, bots contact control servers. In a peer-to-peer botnet, bots relay commands to one another, so there is no single server for defenders to seize.

Hiding the traffic relies on what the network already permits:

Malware also needs to find its operator again after a server is blocked:

Encryption hides the content of a connection but not its existence. Network monitoring still records timing, volume, and destination, and monitoring on the machine can identify the originating process. Beaconing is the repeated check-in that announces availability, and its regularity can expose it.

Sinkholing redirects traffic for malicious infrastructure to a defender’s server. It interrupts contact and identifies victims but doesn’t remove malware or invalidate stolen passwords.

Payloads

An infostealer collects browser passwords, session tokens, wallet secrets, and documents. A session token lets a site recognize a user who has already logged in, so a copied token reuses the session without the password or second factor. Multifactor authentication protects login, not the session afterward, and removing the infostealer doesn’t recall what was copied.

Ransomware encrypts files with a symmetric key, encrypts that key with the attacker’s public key, and discards the unencrypted copy. Recovery requires the attacker’s private key, which was never on the victim’s machine, and the malware needs no connection to the attacker while it runs. Double extortion copies documents before encrypting them and threatens publication, which a backup cannot undo. A wiper discards what recovery would require, whatever its screen says.

Other payloads provide remote control through a backdoor or RAT, consume resources through cryptocurrency mining or spam, flood a target in a distributed denial-of-service (DDoS) attack from a botnet, or sabotage industrial equipment while showing its operators normal readings.

Propagation

A worm searches for reachable systems and makes them execute a copy of itself; each copy then repeats the process independently. Replication alone can be the damage. Blocking propagation protects uninfected machines but leaves the payload active on infected ones.

Detection

A malware signature is a pattern used to recognize known malicious code, usually a sequence of bytes that an analyst chose because it occurs in the malware and is unlikely to occur in legitimate programs. A pattern from a part that variants share matches the whole family.

A hash of the whole file is the simplest signature but matches only one exact copy, and changing one byte defeats it. A malware signature is unrelated to a digital signature.

Static analysis examines a program without running it. A heuristic flags a program by suspicious features without a match to a known sample, which is how previously unseen malware and new variants are caught:

Attackers evade static detection by changing the bytes:

Behavioral detection evaluates actions during execution: file access, process creation, memory changes, configuration changes, and network connections. A sequence of events is stronger evidence than any one action. An endpoint detection and response (EDR) system records and correlates those events so the chain from download to script to connection can be reconstructed later.

Behavioral rules make mistakes in both directions. A false positive flags legitimate activity, and a false negative misses malicious activity, so the policy needs context about which actions are authorized.

An analysis sandbox runs a suspicious program in an isolated environment and records what it does. It is a tool for vendors and security teams rather than users. A clean run proves little: the malware may detect the sandbox, wait for user activity, or delay past the observation window.

Code that exists only in memory does not have a file to scan, but it runs in a process, something loaded it, and it usually communicates, so defenders inspect memory, the loader, and network activity. An interpreter can hand its decoded script to security software before running it. A kernel rootkit can alter reports or disable the monitor, but an independent network record still shows that a connection happened.

Reputation-based detection uses prior observations of a file, publisher, address, or domain, and telemetry from many machines exposes rare relationships invisible from one. A rare file isn’t malicious by itself, and a familiar publisher doesn’t authorize every use of its programs.

The detector belongs to the trusted computing base: it runs with broad privileges and parses hostile input, and a compromised monitor can corrupt the evidence administrators rely on.

What You Don’t Need to Study

These exclusions apply to the malware material. Network attacks and defenses in general, web security, steganography and covert channels, and AI security are not part of this material; we will cover those topics later. The malware-related network concepts covered here remain in scope: command and control, DNS tunneling, botnets, and DDoS as a payload.

The following are not required for the exam:

Terms you should know