Focus on what each mechanism and model does, its goals, what it protects against, and where it falls short. The lecture notes contain dates, names of people, some examples, and command syntax that provide context, but you are not expected to remember these items.

Protection and the Operating System

Access control decides which subjects may perform which operations on which objects and enforces those decisions. A security policy defines what access should be allowed, and protection is the set of mechanisms that enforce it.

An operating system provides processes with controlled access to shared resources: the processor, memory, files and devices, and the network. Programs execute instructions and access mapped memory directly, within limits enforced by the hardware. A system call transfers control to the kernel to request an operating system service, and the kernel checks whether the request is allowed before performing it.

Three things have to be true for protection to work at the operating system level. The operating system must:

1. Protect itself from the programs it runs.

2. Protect programs from each other.

3. Stay in control, so that a process cannot hold the processor forever.

The kernel and the hardware that enforce these properties form the core of the trusted computing base, the set of components whose failure means the security of the whole system fails.

The hardware provides three mechanisms:

• Preemption: A hardware timer that applications cannot disable interrupts the processor periodically and gives the kernel’s scheduler a chance to run. This is preemptive multitasking. Under cooperative multitasking (used in older operating systems), one program that failed to yield could freeze the machine.

• Memory isolation: the memory management unit (MMU) hardware translates every address a program uses, and the kernel sets up each process so that it can reach only the memory it has been assigned.

• Privilege levels: in user mode, applications cannot execute privileged instructions or touch kernel memory. The kernel runs in kernel mode, where every instruction is available.

Execution transfers from user mode to kernel mode through controlled entry points defined by the kernel at boot time. The transfer occurs in three ways:

1. A system call, which the program makes deliberately to request a service.

2. A fault, when an instruction cannot complete without kernel intervention. A fault can indicate an invalid operation or a valid memory access that requires the kernel to supply a page.

3. A hardware interrupt, when a device or the timer needs attention.

The Reference Monitor

The reference monitor is the part of a system that checks every attempt by a subject to reach an object against the policy. It has three requirements:

1. It is invoked on every access.

2. It is tamperproof: the programs it controls cannot change it.

3. It is small enough to be analyzed and tested.

In an operating system, the kernel’s access checks and the hardware mechanisms it configures implement the reference monitor. The MMU enforces memory permissions, while system calls mediate requests for kernel services. User-mode code cannot modify the kernel, but these protections do not prevent bugs in the kernel itself. The size of a modern kernel makes the third requirement difficult to satisfy. A database server or a web application is its own reference monitor for the data it manages. Its enforcement is only as good as its code.

Subjects, Objects, and the Access Matrix

A subject is an active entity that requests access, almost always a process. A principal is the identity on whose behalf a subject acts, usually a user account. An object is a passive resource: a file, a device, a socket, a region of memory, or another process. A right is an operation a subject may perform on an object.

Authorization is the access decision in the following sequence:

1. Identification is a claim of identity.

2. Authentication verifies the claim.

3. Authorization decides what the authenticated principal may do.

The access matrix describes every permission in a system: one row per subject, one column per object, and in each cell the set of rights the subject has over the object, such as read, write, execute, append, and delete.

A row defines a protection domain, the complete set of objects and rights available to a process. Several processes can share the same protection domain. A process operates within a protection domain and exercises that domain’s rights. Deleting an object removes its column from the matrix, while revoking a right removes only that right from a cell.

Other rights govern how permissions are managed or which domain a process uses:

• Owner: the subject that owns an object may grant or revoke rights in that object’s column.

• Copy: a subject may pass a right it holds to another subject if that right permits copying.

• Domain transfer: a subject may switch to another row and acquire that row’s rights. Logging in is a domain transfer, and so is running a setuid program.

The matrix is a model, not a data structure. A real one would be enormous and almost entirely empty, so systems store it in one of two compressed forms:

• An access control list (ACL) stores a column with its object: a list of principals or groups and their rights. Inspecting an object’s entries and changing or removing an access right are easy. Finding everything a principal can access requires searching every object’s list.

  • “Who has permission to read this file?” is easy to answer by inspecting the file’s ACL. “Which files can Alice access?” requires searching the ACLs of many files.

• A capability list stores a domain’s row as a list of capabilities held by a process. Each capability in the list is an unforgeable token that names an object and a set of rights the process has over it. Checking and delegation are easy. Revocation requires a way to invalidate capabilities that have already been distributed.

  • “Which files can Alice’s process access?” is easy to answer by inspecting its capabilities. “Which processes have read access for this file?” requires finding the capabilities held by other processes.

An open file descriptor is a capability: the permission check happens once, when the file is opened, and the descriptor is then presented on every read and write without rechecking the file’s permissions. A descriptor passed to another process carries the access with it.

A confused deputy is a program that holds authority of its own and can be tricked into using that authority for a caller who lacks it. The deputy’s rights are checked and the caller’s are not, so any program that runs with more privilege than its caller must check what the caller is entitled to. Capabilities can address the problem by having the caller supply a capability for each object. The program uses that capability rather than its own authority, and the system enforces the permissions the capability grants.

Discretionary and Mandatory Control

Access control is divided into two families, based on who controls access permissions:

• In discretionary access control (DAC), the owner of an object decides who may access it. Unix and Windows file permissions are examples.

• In mandatory access control (MAC), access rules are defined by the organization rather than by the object’s owner, and the system does not allow users, including owners, to override them.

Nearly every system implements some of both.

Unix File Permissions

Unix file permissions are a foundation for understanding access control because the same basic model is used by Linux, Android, BSD, QNX, and macOS. These systems span servers, phones, personal computers, and embedded devices. In this section, “Unix” refers broadly to systems that use this permission model.

Every file has an owner, a group, and nine permission bits, stored in the file’s inode with the rest of its metadata. One set of three bits applies to the owner, one to members of the file’s group, and one to everyone else (which Unix calls “other”), and within each set one bit grants read, one write, and one execute permissions. This is a compact ACL with three fixed entries that fit in the inode, so reading the file’s metadata also retrieves its permissions without an additional disk read.

The permission check examines only one set of bits: the owner’s if the process’s effective user ID matches the owner, otherwise the group’s if the process belongs to the file’s group, otherwise the other bits.

A binary file can be executable without being readable, because the kernel loads it; a script cannot, because the interpreter has to read it.

For a directory, read access allows listing the names in the directory, write access allows creating, deleting, or renaming entries, and execute access allows the directory to be searched to reach something inside it. Deleting a file requires write and execute permission on the directory, not write permission on the file, because deletion removes a name from the directory.

Default Permissions and the umask

Most programs (particularly scripts and cross-platform environments like Java and Python) request read and write permission for everyone when creating files and leave the restriction to the environment. The umask is an inherited per-process value listing the permission bits to remove at file creation. The usual default removes write permission for the group and everyone else, which leaves new files readable by every user on the machine.

The Access Check Happens at Open

The permission check runs once, when a file is opened, and not on each read or write. A process that holds an open descriptor keeps its access even if the file’s permissions change afterward. A program that creates a file with permissive bits and then restricts them leaves a window of time in which another process can open the file and keep it open. To avoid this race condition, the file should be created with the right permissions in the first place.

Users, Groups, and Root

A principal on a Unix system is represented by a numeric user ID. A process also contains an effective group ID and a list of supplementary groups. Membership in any of them counts as membership in a file’s group.

User ID 0 is root, the superuser. Root can bypass most discretionary permission checks. However, mandatory controls such as SELinux (Security-Enhanced Linux, described below) apply to root too.

A process inherits its user ID from its parent. The login program runs as root and takes on the user’s identity before starting the user’s shell, which is a domain transfer. In this case, the change is permanent: the process gives up its ability to restore root privileges.

Running with Someone Else’s Authority

A process has more than one user ID. Its real user ID identifies the account that started it, and its effective user ID is the one the kernel uses for permission checks. These are usually the same.

Some legitimate actions, such as changing a password, require a privilege the user should not hold in general. If a program’s setuid (“set user ID”) bit is set, a process that runs it gets the file owner’s user ID as its effective user ID rather than the ID of whoever ran it. The real user ID is unchanged, so the kernel still knows who started the process. A setgid bit does the same for the group ID.

Running a setuid program is a domain transfer, and its code must enforce which privileged operations the caller is entitled to request. The program is a deputy, and exploiting a bug in a setuid-root program can give an attacker root privileges.

Several defensive rules follow from that:

• Install as few setuid programs as possible.

• Give up the privilege temporarily when it is not in use, by lowering the effective user ID to the real one, and raise it again only for the moments that need it. A saved user ID keeps a copy of the privileged ID so that this is possible.

• Give up privileges permanently once they are no longer needed, so the process cannot restore them later. Verify that the privileges were successfully removed.

• Concentrate privilege in one heavily reviewed program, which is what the sudo command does.

• Split root’s power by granting only the privileges a program needs. Linux supports this through capabilities, which divide root’s authority into separate privileges. We will cover how they work later. These Linux privileges are distinct from the capability lists described earlier.

A setuid program that checks whether its caller may access a file by name and then opens the file by name has a time-of-check to time-of-use race condition: the filename can be made to refer to a different file between the check and the use, allowing an attacker to access a different file.

Least Privilege and Privilege Separation

The principle of least privilege states that every program and every user should operate with the smallest set of privileges needed to complete the job. It limits damage rather than preventing mistakes. On Unix the principle is applied with identities: each service runs as its own user, a server that must start as root gives up root before it reads anything from the network, and administrators use sudo for individual privileged commands.

Privilege separation is the structure for programs that cannot give up root entirely. The program is divided into a small privileged part that does only the operations needing privilege and a larger unprivileged part that does everything else, connected by a narrow channel. The unprivileged part handles everything an attacker can send, so a bug there yields a process with little authority, and the attack surface of the privileged code shrinks to the requests the channel accepts.

Full Access Control Lists

The nine Unix permission bits express only three entries: owner, group, and others. Linux and macOS also support access control lists (ACLs) with entries for additional users and groups, while Windows NT and its successors have used ACLs from the start. A full ACL takes a variable amount of space, and the filesystem determines how it is stored.

Some ACL systems support explicit deny entries as well as allow entries. Inheritance lets newly created files and directories receive permissions from their parent directory.

Full ACLs can express any column of the access matrix. The cost is that nobody can see the matrix: with inherited entries, named users, and deny entries, determining a user’s effective permissions requires evaluating all applicable rules.

Why Discretionary Control Is Not Enough

Discretionary control has three limits that no setting of the permission bits can fix:

• A program runs with the authority of the user who runs it, so a malicious program can use the user’s permissions to read or disclose files without the user’s intent.

• Rights propagate. A user who can read a file can copy its contents into a file they own and give others access to the copy.

• Owners control permissions. Discretionary control alone cannot enforce an organization-wide rule that owners are forbidden to override.

Mandatory access control addresses these limits by enforcing rules that users and programs cannot override.

Multilevel Security and Bell-LaPadula

The Bell-LaPadula model describes multilevel security: a system that holds information at several levels of sensitivity and prevents it from flowing downward. Objects are given a classification from an ordered set of levels (Unclassified, Confidential, Secret, Top Secret), and a user’s clearance sets the highest level at which their processes may operate. A process can run at that level or a lower one. Two rules protect confidentiality:

1. The simple security property, or no read up: a subject may read an object only if its operating level is at least as high as the object’s classification. For example, a subject operating at Secret may read a Confidential file but cannot read a Top Secret file.

2. The star property, or no write down: a subject may write an object only if the object’s classification is at least as high as the subject’s operating level. For example, a subject operating at Secret may write to a Top Secret file but cannot write to a Confidential file.

Here, writing means adding information without reading the object. An operation that both reads and writes must satisfy both rules, so the subject and object must be at the same level.

A third rule, the discretionary security property, says that ordinary permissions still apply within those limits: an owner can restrict access further but cannot extend it past what the labels allow.

The star property prevents programs from disclosing information to lower classification levels. A program operating at Secret could read a Secret file and attempt to write its contents to an Unclassified one. The star property forbids that write, whether the disclosure is deliberate or accidental. Under this rule, information may flow to the same or a higher classification level, but never to a lower one.

The model has three limits:

1. Information cannot flow down, so declassifying and sanitizing require trusted subjects that are exempt from the star property, and the security of the system then rests on those programs.

2. It says nothing about integrity: a low-clearance subject may write (or overwrite) a Top Secret file.

3. Covert channels can bypass the modeled access controls. For example, a Secret process could signal bits to an Unclassified process by varying its CPU use, causing delays the other process can measure.

Compartments and Multilateral Security

Levels separate information vertically. Need to know separates it horizontally, within a level: a clearance establishes that a person may be trusted at a level, and a separate decision establishes which information at that level the person needs. A compartment is a category of information, such as information about a specific project. Clearance at Secret does not grant access to every Secret compartment. The full Bell-LaPadula label combines a level with a set of compartments. Separating access by compartment as well as by level is called multilateral security.

A subject may read an object only if two tests both pass: its operating level is at least as high as the object’s classification, and its compartment set includes every compartment in the object’s label. When both pass, the subject’s label dominates the object’s. Dominance replaces the level comparison in both rules: a subject may read only what its label dominates and write only to what dominates its label. A subject missing even one of the object’s compartments cannot read it.

Two labels can be incomparable, meaning neither dominates the other. The set of labels ordered by dominance forms a lattice, which can be represented as a directed graph. Each node is a label, and arrows point toward labels that dominate it. Information may flow along these arrows or remain at the same label.

The cost is administrative: every distinct combination of compartments is in effect a new label, and combining information from two compartments produces a label that readers of either compartment alone cannot see.

Outside government, Bell-LaPadula has seen little use, because organizations’ access policies rarely reduce to an ordering of levels.

The Biba Integrity Model

Bell-LaPadula does not protect trustworthy data from being corrupted by sources that should not write it. The Biba model has the same structure and the opposite goal. Subjects and objects carry integrity levels, where a high level means the data is trusted and the subject is trusted to produce it. The two rules are Bell-LaPadula’s reversed:

1. No read down: a subject may read an object only if the object’s integrity level is at least as high as the subject’s. A trusted process may not take input from an untrusted source.

2. No write up: a subject may write an object only if the subject’s integrity level is at least as high as the object’s. An untrusted process may not modify trusted data.

The threat is contamination. The strict rules are a poor fit for real systems because programs that process untrusted input are the normal case.

Windows Mandatory Integrity Control implements a restricted form of Biba: processes and securable objects have integrity levels, and the default rule prevents a process from modifying an object at a higher level regardless of the object’s discretionary permissions. It does not enforce Biba’s no-read-down rule. Its first main use was running a browser at low integrity, so that an exploited browser could not modify the user’s or the system’s files.

The Chinese Wall Model

The Chinese Wall model is for firms that serve clients who compete with each other. Its access decision depends on what a subject has already done. Objects are organized in a two-level hierarchy: each object belongs to one company dataset, which is all the data the firm holds about one client, and each dataset belongs to one conflict of interest class, which groups the datasets of companies that compete.

The read rule is: a subject may read an object in a company dataset only if everything the subject has previously read is either in the same dataset or in a different conflict of interest class.

Before the first read, every dataset is available. After an employee reads one airline’s data, the other airlines in that class are denied, and companies in other classes are still available. The system must record every access, and the record persists across login sessions.

For example, an advertising agency places Rutgers and NYU in the same conflict of interest class. Once an employee reads data from the Rutgers dataset, they cannot read data from the NYU dataset.

The write rule prevents information from passing between competing clients through another dataset. A subject may write to a company dataset only if it may read that dataset and has no permission to read data from any other company dataset, except sanitized data. Sanitized data has been altered so that it cannot be traced to its source company. Reading sanitized data does not create a conflict, and it may flow between datasets. Creating sanitized data requires a trusted procedure outside the ordinary write rule.

For example, the advertising agency also serves Delta Air Lines, whose dataset belongs to a different conflict of interest class from the universities. Alice may read Rutgers and Delta data, while Bob may read NYU and Delta data. Without the write rule, Alice could copy Rutgers data into Delta’s dataset, where Bob could read it. The write rule prevents this by denying Alice permission to write to Delta’s dataset while she has permission to read Rutgers data.

The model is a dynamic form of separation of duties, the principle that a task capable of causing serious harm is divided so that one person cannot complete it alone.

Type Enforcement

Bell-LaPadula and Biba compare ordered labels. The threat on a server is a compromised program, and the policy a server needs is a list of which programs may perform which operations on which files, with nothing implied beyond the list. With ordered labels, permissions follow from positions in a hierarchy. Allowing a program to read one file may also allow it to read files at lower levels, even if the policy should forbid those reads. Type enforcement allows each permission to be specified independently.

Type enforcement is an implementation of the access matrix in which the rows and columns are classes rather than individuals. Every subject is assigned a domain, which is a class of subjects, and every object a type, a class of objects. A policy is a list of rules, each allowing a domain to perform certain operations on a type, and anything the policy does not allow is denied. There is neither a hierarchy nor a notion of up or down. Grouping processes and files into classes reduces the number of rules needed to describe their permissions, hence keeping the size of the matrix manageable.

SELinux is the implementation of type enforcement in the Linux kernel. Its check is a second reference monitor that runs after the ordinary permission bits, and either check can deny access. The benefit appears when a program is compromised: the process is still running in its domain, so even if running as root, it can touch only what the policy allows that domain. The cost is a potentially large policy set, and a missing rule can make a program fail with a permission error.

Role-Based Access Control

In an organization, rights are associated with jobs. Role-based access control (RBAC) models that with four kinds of entities:

• Users are the people or programs that need access.

• Roles are named sets of permissions corresponding to job functions.

• Permissions are rights to perform operations on objects.

• Sessions are the periods during which a user has some of their roles active.

Permissions are assigned to roles and users are assigned to roles. A user never holds a permission directly, and the roles active in a user’s session define what the user can do.

The benefit is administrative: permissions associated with a job can remain stable while users change as people are hired, leave, or change positions.

Roles resemble the Unix groups: both let an administrator manage permissions for many users together. A group is a collection of users, while a role is a collection of permissions associated with a responsibility, such as processing payroll. In RBAC, users receive permissions through their assigned roles. Group-based systems may also grant permissions directly to individual users.

A session records which of a user’s assigned roles are currently active, allowing the user to activate only the permissions needed for a specific task. Switching active groups can serve a similar purpose. RBAC makes the distinction between assigned roles and active roles an explicit part of the access control model.

Roles usually correspond to job functions, with permissions such as approving a payment or issuing a refund, but they can also group file access or narrower technical permissions. RBAC is used within applications, databases, directory services, and cloud platforms. Its defining feature is how permissions are assigned and activated, rather than the kinds of operations they control.

RBAC fails when roles multiply: a role for every combination of permissions anyone has ever needed produces role explosion, with the structure harder to audit than the permissions were. Attribute-based access control (ABAC) generalizes roles to rules that test attributes of the subject, the object, and the environment, such as the user’s department, the document’s classification, and the time of day.

The Models Compared

Each model decides the same question, whether a subject may perform an operation on an object, from a different kind of label and a different rule:

A real system runs more than one model, with one layered above another. Bell-LaPadula assumes discretionary permissions beneath it, and a Linux server with SELinux runs the Unix bits with type enforcement above them. An Android phone runs three layers:

1. Unix permissions, with each app as its own user, keep apps out of each other’s files.

2. SELinux confines each app’s process to what the policy grants.

3. The Android framework’s permission model decides whether an app may use the camera or read contacts.

Each layer catches what the one below it cannot express.

What You Don’t Need to Study

Here’s what you don’t need to study, along with disclaimers.

• Dates, and the names of the people and organizations behind each model or mechanism. Know the names of the mechanisms themselves, such as Bell-LaPadula, Biba, the Chinese Wall model, type enforcement, SELinux, and RBAC.

• Stories, like the Capital One breach or the CrowdStrike outage, PwnKit, and the OpenSSH and Chrome designs. Know the concepts they illustrate: a confused deputy and a role with far more rights than it needed, kernel code failing with kernel consequences, a setuid program as a path to root, and privilege separation.

• How the memory management unit, page tables, and address translation work, protection rings, and hypervisors. Know that the hardware confines each process to the memory the kernel assigned it, that privileged instructions are unavailable in user mode, and that system calls, faults, and hardware interrupts transfer execution to controlled kernel entry points.

• KeyKOS, seL4, and Google’s Fuchsia. Know that an open file descriptor is a capability.

• Octal and binary permission values such as 0644 and 0666, and the syntax of chmod, chown, and chgrp. Know what the nine bits mean, how the check picks one set, and what read, write, and execute mean for a directory.

• The usage of the setuid, seteuid, setgid, initgroups, chown, chmod, and access system calls, the exact sequence of calls in the login process, and the difference between temporary and permanent privilege drops. Know the concepts of user IDs, real and effective user IDs, group IDs, and the setuid bit.

• The sticky bit.

• The syntax of Linux, macOS, and Windows ACLs, the POSIX ACL mask, and how the three systems order and evaluate their entries. Know that full ACLs hold named entries beyond the three Unix permission sets and why full ACLs are hard to audit.

• The implementation and usage of Windows access control: security identifiers, access tokens, DACLs and SACLs, canonical ordering, and NTFS storage. Know that Windows NT and its successors have used full ACLs with deny entries and inheritance from the start.

• The implementation and usage of Windows Mandatory Integrity Control: the names of its levels, where the label is stored, and AppContainer. Know that Windows enforces a Biba-style no-write-up rule by default, allows reading down, and first used this mechanism prominently to confine a browser.

• Sensitive Compartmented Information, detailed lattice mathematics, and the specific label examples. Know what dominance is, that two labels can be incomparable, and why compartments multiply labels.

• Role hierarchies, RBAC constraints, and static versus dynamic separation of duties. Know the general principle of separation of duties.

• The SELinux policy language and its syntax, the size of the policy, the audit log tools, Android’s SELinux history, and AppArmor. Know what type enforcement is, what a domain and a type are, that anything a rule does not allow is denied, and why that confines a compromised program.

Terms we covered