Malware that has arrived, delivered its payload, and arranged to run again still has to decide when to act, and most of it has to reach its operator. A network connection lets the program retrieve instructions or transfer stolen data, and the operator needs that connection to survive server or domain blocking.

We finish the malware lifecycle with triggers, the conditions that activate a payload; communication with an operator, including how malware hides its traffic and finds its controller again; the payloads themselves; and propagation. We end with detection, the defender’s view of every stage, and a conclusion that looks back over all four parts.

Triggers

A trigger is a condition that activates a malware payload: a date, a count of executions, a user action, or the presence of a particular target. Delaying activation can keep the malware concealed until the attacker is ready or the intended target is reached.

A logic bomb is malicious code that stays dormant until its trigger, a specified condition such as a date or an event, occurs. It can be hidden inside an otherwise legitimate program.

On March 4, 2002, a logic bomb planted by a former systems administrator at UBS PaineWebber, a brokerage, began deleting files on about 1,000 computers. The trigger was the scheduled date and time. Assessing and repairing the damage cost more than $3 million, and the administrator was sentenced to 97 months in prison in December 2006.

A trigger can also depend on the equipment the malware finds. Stuxnet checked for a specific industrial controller configuration before carrying out its sabotage. An infected computer that was not connected to the target equipment never activated the destructive payload.

Not every payload waits for a condition. An infostealer that runs immediately may not have any trigger beyond startup, and a worm’s replication can consume enough resources to cause harm.

Communicating with an Operator

Malware may need to retrieve instructions, download a later payload, or transfer stolen information. These activities can share a communication channel, but they need not use the same protocol or destination. The attacker wants these exchanges to look like ordinary network activity so that defenders do not block them.

Command and control (C2, also written C&C) is communication through which an operator directs compromised systems and receives status or results. Exfiltration is the unauthorized transfer of collected information out of the environment.

Malware can hide its messages in encrypted web connections, uploads to cloud services, or domain name lookups. Continued contact also requires a way to reconnect when a server or domain is blocked.

A botnet is a collection of compromised machines, called bots or zombies, under coordinated remote control. An operator can direct thousands or millions of bots to send spam, flood a target with traffic, install another program, or collect and upload data.

In a centralized botnet, bots contact one or more control servers to obtain instructions and report results. In a peer-to-peer botnet, bots relay commands to one another: the operator introduces commands into the network, and bots pass them on, so there is no single server for defenders to seize.

An operator can use a botnet for different jobs or sell access to other groups. The Emotet and QakBot botnets delivered other groups’ malware for a fee.

Fully autonomous malware can operate without C2. A worm can replicate based on embedded instructions, and ransomware can encrypt files with an embedded public key.

Concealing Traffic in Web and Cloud Services

An infected machine can open an outbound HTTPS connection to a control server. A firewall that blocks unsolicited inbound connections does not prevent this exchange if it allows outbound web traffic.

HTTPS uses Transport Layer Security (TLS) to encrypt requests and responses. Malware can receive commands in a web response and upload stolen data as a file. Someone monitoring the network can see the connection but can’t read its contents.

Malware can also use an account on a legitimate cloud-storage service to retrieve instructions or upload stolen files. Blocking the entire service would also disrupt employees who use it for ordinary work.

Tunneling Data through DNS

Applications depend on the Domain Name System (DNS) to translate names such as www.rutgers.edu into IP addresses. Organizations usually allow these lookups even when they restrict other traffic. Every machine makes many lookups a day, so extra requests don’t look unusual. Malware can exploit that allowance by putting its own messages inside DNS requests and replies.

DNS tunneling embeds commands or other data in DNS traffic. For example, suppose an attacker controls control.example and its DNS server:

1. Malware requests a TXT record, a DNS record containing text, for ready.17.control.example. In this example, 17 identifies the infected machine, and ready announces that it is waiting for work.

2. The organization’s DNS lookup service obtains the record from the attacker’s DNS server. The queried name tells the attacker that machine 17 is ready.

3. The server returns a TXT record containing wait=60. The malware interprets that text as an instruction to contact the server again after 60 seconds.

For data theft, malware can encode fragments of stolen information into queried names and send them through repeated lookups. Encoding makes the data fit into a domain name, but encryption is needed to hide its contents.

Finding Replacement Control Servers

If defenders block a control server or investigators seize it, malware needs another destination. It can store several addresses and attempt another when one fails. As long as a server remains reachable, the operator can send an updated list.

A dead-drop resolver is a public web page or account that publishes the current control-server address. The attacker updates the page when the address changes, and malware reads it to find the new server. The installed malware does not need to be replaced.

A domain generation algorithm (DGA) calculates candidate domain names from shared inputs, such as the current date. The operator runs the same algorithm, registers some of those names, and points them at control servers. Infected machines attempt to contact the generated names until they reach a server. Defenders who recover the algorithm can also predict and block future names.

Detecting and Disrupting Hidden Traffic

Encryption hides message contents, but network monitoring can still record connection times, data volumes, and destination IP addresses. Monitoring software on the infected machine can identify which process opened a connection. For example, a document editor contacting an unfamiliar server immediately after opening a downloaded file deserves investigation.

Beaconing is repeated communication that reports availability or retrieves instructions. Fixed intervals, such as one connection every 60 seconds, are easy to spot in logs, but legitimate update and monitoring tools also communicate periodically. Malware can vary the intervals to make the timing less regular.

DNS tunnels can stand out through unusually long names or large numbers of queries to the same domain. These signs also occur in legitimate traffic, so defenders examine the program making the requests and its other activity. An organization can require its machines to use its own DNS lookup service and record the queries for investigation.

Investigators can seize control servers, suspend accounts used to publish server addresses, or redirect domains. Sinkholing redirects traffic intended for a malicious server to a server controlled by defenders. This can interrupt commands and help identify infected machines, but redirecting traffic does not remove malware or invalidate stolen passwords.

Investigators who gain control of the servers can also turn the botnet’s own command channel against it. In January 2021, they used Emotet’s update mechanism to deliver a module that cut the bots off from their operators, and in August 2023 they redirected QakBot traffic to their own servers and sent an uninstall command. Removing QakBot left any other malware already installed, so those machines still required investigation and cleanup.

Payloads

Once malware is running, its payload carries out the attacker’s objective using the available permissions. A single program can combine theft, remote control, and file encryption.

Stealing Credentials and Documents

An infostealer collects data that attackers can sell or use to access other systems. Targets include saved browser passwords, session tokens, private keys for cryptocurrency wallets, and private documents.

A session token lets a website recognize a user who has already logged in. The browser stores the token, often in a cookie, and includes it with later requests. Stealing a valid token can let an attacker reuse the session without repeating the password or second-factor step.

Infostealers target browser files and memory to obtain these secrets. Removing the malware does not invalidate passwords or tokens already stolen. Recovery includes ending affected sessions and replacing exposed credentials.

Extorting Victims and Destroying Data

Ransomware blocks access to files or systems and demands payment to restore it. A common design encrypts files with a symmetric key, then encrypts that key with the attacker’s public key and discards the unencrypted copy. Recovering the file-encryption key requires the corresponding private key, which the attacker keeps.

An attacker can also copy documents before encryption and demand payment to avoid publication, a practice called double extortion. Restoring a backup does not stop the attacker from publishing the stolen copies.

A wiper is malware designed to destroy data. It can overwrite files, corrupt the information needed to locate them, or encrypt data and discard the key.

Providing Remote Control

A backdoor provides access that bypasses the system’s normal authentication or access controls. A remote access trojan (RAT) gives an attacker remote control, including executing commands, transferring files, viewing the screen, and installing software.

Legitimate support tools can perform the same operations. Investigators must verify who installed the tool, who controls it, and whether its use was approved.

Consuming Processor and Network Resources

Other payloads use the victim’s computing power or network connection. Common uses include:

• Cryptocurrency mining. Malware uses the victim’s computing power to earn cryptocurrency for the attacker. This consumes electricity and slows other programs.

• Spam distribution. Malware sends bulk email from infected machines or stolen email accounts. Using many machines spreads the sending activity across different network addresses.

• Distributed denial-of-service (DDoS). A botnet uses many devices to flood a target with traffic, overwhelming its servers or network connection.

Sabotaging Software and Physical Equipment

A payload can alter a program’s output or misuse software that controls physical equipment. Industrial controllers change motor speeds, open valves, and adjust temperatures. Malicious code can change these commands and falsify monitoring data to hide the changes from operators.

Propagation

A worm searches for reachable systems and attempts to make them execute a copy of itself. Each new copy repeats the process independently. Disconnecting the first infected machine does not stop the others from spreading.

Replication can cause damage on its own. In January 2003, SQL Slammer congested networks with rapid attempts to infect other machines, without a separate destructive payload.

Blocking propagation protects uninfected machines. Machines already infected still need malware removal and recovery from any damage.

Defending Systems through Malware Detection

File scanners use stored rules to recognize malicious files before execution. Other attacks use a legitimate program, such as PowerShell, to run malicious instructions. Detecting those attacks requires inspecting the instructions or observing the program’s actions. Security products combine inspection before execution with monitoring during execution.

Detecting Known Code and Suspicious Structure

When analysts examine a new piece of malware, they look for a byte sequence that appears in it but is unlikely to appear in legitimate programs. A scanner can then search every file for that sequence.

A malware signature is a pattern used to recognize malicious code. Most signatures are byte patterns of this kind, chosen by an analyst and distributed to scanners in regular updates. A pattern taken from a part of the program that variants share recognizes the whole family. This differs from a digital signature, which authenticates a publisher.

The simplest signature is a cryptographic hash of the entire file, which identifies one exact copy. Hashes are useful for blocklisting known files, but changing one byte changes the hash, so a hash match fails against any modified copy. A byte-pattern signature based on an unchanged part of the program still matches.

Static analysis examines a program without executing it, using information such as its instructions, text strings, and file structure. A heuristic flags a program based on suspicious features without requiring a match to a known malware sample. Its purpose is to catch unseen malware, including new variants of known families.

Heuristics come in two kinds, depending on what they examine. A static heuristic examines the file without running it: unusual structure, code that decrypts itself, or instructions that appear in malware and rarely in ordinary programs. A dynamic heuristic, described below, examines the program’s actions while it runs. Most anti-malware products combine signature matching with static heuristics, since both can run before a file executes.

Evading Static Detection with Packing and Mutation

Packing compresses or encrypts an executable and includes startup code that restores it during execution. A file scanner sees the transformed code rather than the original instructions. Detection methods include recognizing unpacking routines, restoring code for inspection, and scanning memory after unpacking.

Malware can also vary its code between copies to make fixed byte patterns harder to match:

• Polymorphic malware varies the encryption key and decryption routine between copies. After decryption, each copy exposes the same underlying program, which a memory scanner can match.

• Metamorphic malware rewrites the program’s instructions while preserving its behavior. For example, it can replace an instruction with an equivalent sequence or reorder independent instructions. The executable code itself varies, making detection harder even after loading.

Detecting Malicious Behavior

A document editor that starts PowerShell deserves closer inspection if PowerShell then reads browser passwords and uploads data. The sequence provides more evidence than any single action.

Behavioral detection identifies suspicious activity from actions performed during execution, such as file access, process creation, and network connections. Dynamic analysis examines a program while it runs, either during a controlled test or through monitoring on a working machine. A dynamic heuristic judges a program by what it does rather than how it is built, and it also targets previously unseen malware. Behavior tells more than structure because a mutated program still has to perform the same actions, but behavioral detection tends to produce more false positives because legitimate programs often perform the same actions.

An endpoint detection and response (EDR) system records activity on individual computers and helps defenders investigate and stop attacks. Defenders use its records to trace a downloaded script, the process that ran it, and the server it contacted. EDR also provides controls to stop a process or isolate a machine from the network.

A false positive labels legitimate activity as malicious. A false negative misses malicious activity. For example, blocking an authorized backup as a data-theft attempt is a false positive. Investigators consider which program ran, which account ran it, and whether the activity was approved.

Defending with Analysis Sandboxes

Security teams run suspicious programs in an analysis sandbox, an isolated environment that records their actions. For example, an email security service tests an attachment in a sandbox before delivering the message.

Evading Sandbox Analysis

Malware evades sandbox analysis by checking for virtual machines, waiting for user activity, or delaying execution. For example, a payload that activates only after a mouse click stays inactive in a test that supplies no mouse input.

Observing no malicious activity during a test does not prove that the program is safe.

Detecting Memory-resident Malware and Rootkits

A payload can execute from memory without being saved as a separate executable file. Defenders inspect memory, the script that loaded the payload, and its network activity.

The Antimalware Scan Interface (AMSI) lets Windows applications submit content to security software for scanning. PowerShell uses AMSI to submit script content before execution, including code decoded from a download.

Kernel-mode rootkits undermine local detection by falsifying system reports or disabling monitoring. Investigators compare local reports with independent network records or inspect the disk after booting from trusted recovery media.

Detecting Malware with Reputation and Telemetry

A new program contacting a server already associated with malware gives defenders a reason to investigate, even without a matching file signature.

Reputation-based detection uses prior observations about files, publishers, and network destinations. Security vendors pool telemetry, records such as process starts and network connections, from many machines to identify shared signs of an attack.

New legitimate programs also have little history, so rarity alone does not prove that a file is malicious.

Protecting the Detector

Security software often runs with broad privileges and reads files attackers create. It belongs to the trusted computing base, the components whose correct operation is needed to enforce security. If a malicious file exploits the detector’s parser to execute code, that code runs with the detector’s privileges.

Conclusion

Malware is deliberately deployed to violate the security interests or policies of the people responsible for a system. Legitimate software also reads documents, encrypts files, and accepts remote commands. Intent and authorization distinguish malicious use of those operations.

Defenses block different parts of an attack. Patching closes exploitable flaws, application control blocks unapproved code, and sandboxing and the Principle of Least Privilege limit what running code can access. Deception alone does not give code unrestricted access.

Each defense changed the attacker’s route rather than ending attacks. BlackLotus exploited a vulnerable boot loader whose valid signature Secure Boot still trusted, and revocation then blocked reuse of that loader.

Detection depends on trustworthy evidence. Kernel rootkits falsify local system reports, so investigators compare those reports with independent network records or inspect the disk from trusted recovery media.

We analyze an unfamiliar attack by tracing:

1. How its code starts executing.

2. Which permissions it obtains and what its payload does.

3. Whether it persists after a restart or spreads to other machines.

4. Which communications it uses.

5. Which controls block its actions.

6. Which records and observations reveal its activity.

Lecture 9: Part 1 | Part 2 | Part 3 | Part 4
Lecture 9 Study Guide | List of terms