Modern Operating Systems: Global Edition

CHAPTER 1 "INTRODUCTION"

1.1 WHAT IS AN OPERATING SYSTEM? 3
1.1.1 The Operating System as an Extended Machine 4
1.1.2 The Operating System as a Resource Manager 5

1.2 HISTORY OF OPERATING SYSTEMS 6
1.2.1 The First Generation (1945-55): Vacuum Tubes 7
1.2.2 The Second Generation (1955-65): Transistors and Batch Systems 8
1.2.3 The Third Generation (1965-1980): ICs and Multiprogramming 9
1.2.4 The Fourth Generation (1980-Present): Personal Computers 15
1.2.5 The Fifth Generation (1990-Present): Mobile Computers 19

1.3 COMPUTER HARDWARE REVIEW 20
1.3.1 Processors 21
1.3.2 Memory 24
1.3.3 Disks 27
1.3.4 I/O Devices 28
1.3.5 Buses 32
1.3.6 Booting the Computer 34

1.4 THE OPERATING SYSTEM ZOO 35
1.4.1 Mainframe Operating Systems 35
1.4.2 Server Operating Systems 35
1.4.3 Multiprocessor Operating Systems 36
1.4.4 Personal Computer Operating Systems 36
1.4.5 Handheld Computer Operating Systems 36
1.4.6 Embedded Operating Systems. 37
1.4.7 Sensor-Node Operating Systems 37
1.4.8 Real-Time Operating Systems 37
1.4.9 Smart Card Operating Systems 38

1.5 OPERATING SYSTEM CONCEPTS 38
1.5.1 Processes 39
1.5.2 Address Spaces 41
1.5.3 Files 41
1.5.4 Input/Output 45
1.5.5 Protection 45
1.5.6 The Shell 45
1.5.7 Ontogeny Recapitulates Phylogeny 47

1.6 SYSTEM CALLS 50
1.6.1 System Calls for Process Management 53
1.6.2 System Calls for File Management 56
1.6.3 System Calls for Directory Management 57
1.6.4 Miscellaneous System Calls 59
1.6.5 The Windows Win32 API 60

1.7 OPERATING SYSTEM STRUCTURE 62
1.7.1 Monolithic Systems 63
1.7.2 Layered Systems 64
1.7.3 Microkernels 65
1.7.4 Client-Server Model 68
1.7.5 Virtual Machines 69
1.7.6 Exokernels 73

1.8 THE WORLD ACCORDING TO C 73
1.8.1 The C Language 73
1.8.2 Header Files 74
1.8.3 Large Programming Projects 75
1.8.4 The Model of Run Time 76

1.9 RESEARCH ON OPERATING SYSTEMS 77

1.10 OUTLINE OF THE REST OF THIS BOOK 78

1.11 METRIC UNITS 79

1.12 SUMMARY 80

CHAPTER 2 "PROCESSES AND THREADS"

2.1 PROCESSES 85
2.1.1 The Process Model 86
2.1.2 Process Creation 88
2.1.3 Process Termination 90
2.1.4 Process Hierarchies 91
2.1.5 Process States 92
2.1.6 Implementation of Processes 94
2.1.7 Modeling Multiprogramming 95

2.2 THREADS 97
2.2.1 Thread Usage 97
2.2.2 The Classical Thread Model 102
2.2.3 POSIX Threads 106
2.2.4 Implementing Threads in User Space 108
2.2.5 Implementing Threads in the Kernel 111
2.2.6 Hybrid Implementations 112
2.2.7 Scheduler Activations 113
2.2.8 Pop-Up Threads 114
2.2.9 Making Single-Threaded Code Multithreaded 116

2.3 INTERPROCESS COMMUNICATION 119
2.3.1 Race Conditions 119
2.3.2 Critical Regions 121
2.3.3 Mutual Exclusion with Busy Waiting 122
2.3.4 Sleep and Wakeup 127
2.3.5 Semaphores 130
2.3.6 Mutexes 132
2.3.7 Monitors 137
2.3.8 Message Passing 144
2.3.9 Barriers 146
2.3.10 Avoiding Locks: Read-Copy-Update 148

2.4 SCHEDULING 149
2.4.1 Introduction to Scheduling 150
2.4.2 Scheduling in Batch Systems 156
2.4.3 Scheduling in Interactive Systems 158
2.4.4 Scheduling in Real-Time Systems 164
2.4.5 Policy Versus Mechanism 165
2.4.6 Thread Scheduling 166

2.5 CLASSICAL IPC PROBLEMS 167
2.5.1 The Dining Philosophers Problem 167
2.5.2 The Readers and Writers Problem 171

2.6 RESEARCH ON PROCESSES AND THREADS 172

2.7 SUMMARY 173

CHAPTER 3 "MEMORY MANAGEMENT"

3.1 NO MEMORY ABSTRACTION 182

3.2 A MEMORY ABSTRACTION: ADDRESS SPACES 185
3.2.1 The Notion of an Address Space 186
3.2.2 Swapping 187
3.2.3 Managing Free Memory 190

3.3 VIRTUAL MEMORY 194
3.3.1 Paging 195
3.3.2 Page Tables 198
3.3.3 Speeding Up Paging 201
3.3.4 Page Tables for Large Memories 205

3.4 PAGE REPLACEMENT ALGORITHMS 209
3.4.1 The Optimal Page Replacement Algorithm 209
3.4.2 The Not Recently Used Page Replacement Algorithm 210
3.4.3 The First-In, First-Out (FIFO) Page Replacement Algorithm 211
3.4.4 The Second-Chance Page Replacement Algorithm 212
3.4.5 The Clock Page Replacement Algorithm 212
3.4.6 The Least Recently Used (LRU) Page Replacement Algorithm 213
3.4.7 Simulating LRU in Software 214
3.4.8 The Working Set Page Replacement Algorithm 215
3.4.9 The WSClock Page Replacement Algorithm 219
3.4.10 Summary of Page Replacement Algorithms 221

3.5 DESIGN ISSUES FOR PAGING SYSTEMS 222
3.5.1 Local versus Global Allocation Policies 222
3.5.2 Load Control 225
3.5.3 Page Size 225
3.5.4 Separate Instruction and Data Spaces 227
3.5.5 Shared Pages 228
3.5.6 Shared Libraries 229
3.5.7 Mapped Files 231
3.5.8 Cleaning Policy 232
3.5.9 Virtual Memory Interface 232

3.6 IMPLEMENTATION ISSUES 233
3.6.1 Operating System Involvement with Paging 233
3.6.2 Page Fault Handling 234
3.6.3 Instruction Backup 235
3.6.4 Locking Pages in Memory 237
3.6.5 Backing Store 237
3.6.6 Separation of Policy and Mechanism 239

3.7 SEGMENTATION 240
3.7.1 Implementation of Pure Segmentation 243
3.7.2 Segmentation with Paging: MULTICS 243
3.7.3 Segmentation with Paging: The Intel x86 247

3.8 RESEARCH ON MEMORY MANAGEMENT 252

3.9 SUMMARY 253

CHAPTER 4 "FILE SYSTEMS"

4.1 FILES
4.1.1 File Naming
4.1.2 File Structure
4.1.3 File Types
4.1.4 File Access
4.1.5 File Attributes
4.1.6 File Operations
4.1.7 An Example Program Using File-System Calls

4.2 DIRECTORIES
4.2.1 Single-Level Directory Systems
4.2.2 Hierarchical Directory Systems
4.2.3 Path Names
4.2.4 Directory Operations

4.3 FILE SYSTEM IMPLEMENTATION
4.3.1 File-System Layout
4.3.2 Implementing Files
4.3.3 Implementing Directories
4.3.4 Shared Files
4.3.5 Log-Structured File Systems
4.3.6 Journaling File Systems
4.3.7 Virtual File Systems

4.4 FILE-SYSTEM MANAGEMENT AND OPTIMIZATION
4.4.1 Disk-Space Management
4.4.2 File-System Backups
4.4.3 File-System Consistency
4.4.4 File-System Performance
4.4.5 Defragmenting Disks

4.5 EXAMPLE FILE SYSTEMS
4.5.1 The MS-DOS File System
4.5.2 The UNIX V7 File System
4.5.3 CD-ROM File Systems

4.6 RESEARCH ON FILE SYSTEMS

4.7 SUMMARY

CHAPTER 5 "INPUT/OUTPUT"

5.1 PRINCIPLES OF I/O HARDWARE
5.1.1 I/O Devices
5.1.2 Device Controllers
5.1.3 Memory-Mapped I/O
5.1.4 Direct Memory Access
5.1.5 Interrupts Revisited

5.2 PRINCIPLES OF I/O SOFTWARE
5.2.1 Goals of the I/O Software
5.2.2 Programmed I/O
5.2.3 Interrupt-Driven I/O
5.2.4 I/O Using DMA

5.3 I/O SOFTWARE LAYERS
5.3.1 Interrupt Handlers
5.3.2 Device Drivers
5.3.3 Device-Independent I/O Software
5.3.4 User-Space I/O Software

5.4 DISKS
5.4.1 Disk Hardware
5.4.2 Disk Formatting
5.4.3 Disk Arm Scheduling Algorithms
5.4.4 Error Handling
5.4.5 Stable Storage

5.5 CLOCKS
5.5.1 Clock Hardware
5.5.2 Clock Software
5.5.3 Soft Timers

5.6 USER INTERFACES: KEYBOARD, MOUSE, MONITOR
5.6.1 Input Software
5.6.2 Output Software

5.7 THIN CLIENTS

5.8 POWER MANAGEMENT
5.8.1 Hardware Issues
5.8.2 Operating System Issues
5.8.3 Application Program Issues

5.9 RESEARCH ON INPUT/OUTPUT

5.10 SUMMARY

CHAPTER 6 "DEADLOCKS"

6.1 RESOURCES
6.1.1 Preemptable and Nonpreemptable Resources
6.1.2 Resource Acquisition

6.2 INTRODUCTION TO DEADLOCKS
6.2.1 Conditions for Resource Deadlocks
6.2.2 Deadlock Modeling

6.3 THE OSTRICH ALGORITHM

6.4 DEADLOCK DETECTION AND RECOVERY
6.4.1 Deadlock Detection with One Resource of Each Type
6.4.2 Deadlock Detection with Multiple Resources of Each Type
6.4.3 Recovery from Deadlock

6.5 DEADLOCK AVOIDANCE
6.5.1 Resource Trajectories
6.5.2 Safe and Unsafe States
6.5.3 The Banker's Algorithm for a Single Resource
6.5.4 The Banker's Algorithm for Multiple Resources

6.6 DEADLOCK PREVENTION
6.6.1 Attacking the Mutual Exclusion Condition
6.6.2 Attacking the Hold and Wait Condition
6.6.3 Attacking the No Preemption Condition
6.6.4 Attacking the Circular Wait Condition

6.7 OTHER ISSUES
6.7.1 Two-Phase Locking
6.7.2 Communication Deadlocks
6.7.3 Livelock
6.7.4 Starvation

6.8 RESEARCH ON DEADLOCKS

6.9 SUMMARY

CHAPTER 7 "VIRTUALIZATION AND THE CLOUD"

7.1 HISTORY

7.2 REQUIREMENTS FOR VIRTUALIZATION

7.3 TYPE 1 AND TYPE 2 HYPERVISORS

7.4 TECHNIQUES FOR EFFICIENT VIRTUALIZATION
7.4.1 Virtualizing the Unvirtualizable
7.4.2 The Cost of Virtualization

7.5 ARE HYPERVISORS MICROKERNELS DONE RIGHT?

7.6 MEMORY VIRTUALIZATION

7.7 I/O VIRTUALIZATION

7.8 VIRTUAL APPLIANCES

7.9 VIRTUAL MACHINES ON MULTICORE CPUS

7.10 LICENSING ISSUES

7.11 CLOUDS
7.11.1 Clouds as a Service
7.11.2 Virtual Machine Migration
7.11.3 Checkpointing

7.12 CASE STUDY: VMWARE
7.12.1 The early history of VMware
7.12.2 VMware Workstation
7.12.3 Challenges in Bringing Virtualization to the x86
7.12.4 VMware Workstation: Solution Overview
7.12.5 The Evolution of VMware Workstation
7.12.6 ESX Server: VMware's type-1 hypervisor

7.13 RESEARCH ON VIRTUALIZATION AND THE CLOUD

CHAPTER 8 "MULTIPLE PROCESSOR SYSTEMS"

8.1 MULTIPROCESSORS
8.1.1 Multiprocessor Hardware
8.1.2 Multiprocessor Operating System Types
8.1.3 Multiprocessor Synchronization
8.1.4 Multiprocessor Scheduling

8.2 MULTICOMPUTERS
8.2.1 Multicomputer Hardware
8.2.2 Low-Level Communication Software
8.2.3 User-Level Communication Software
8.2.4 Remote Procedure Call
8.2.5 Distributed Shared Memory
8.2.6 Multicomputer Scheduling
8.2.7 Load Balancing

8.3 DISTRIBUTED SYSTEMS
8.3.1 Network Hardware
8.3.2 Network Services and Protocols
8.3.3 Document-Based Middleware
8.3.4 File-System-Based Middleware
8.3.5 Object-Based Middleware
8.3.6 Coordination-Based Middleware

8.4 RESEARCH ON MULTIPLE PROCESSOR SYSTEMS

8.5 SUMMARY

CHAPTER 9 "SECURITY"

9.1 THE SECURITY ENVIRONMENT
9.1.1 Threats
9.1.2 Attackers

9.2 OPERATING SYSTEMS SECURITY
9.2.1 Can We Build Secure Systems?
9.2.2 Trusted Computing Base

9.3 CONTROLLING ACCESS TO RESOURCES
9.3.1 Protection Domains
9.3.2 Access Control Lists
9.3.3 Capabilities

9.4 FORMAL MODELS OF SECURE SYSTEMS
9.4.1 Multilevel Security
9.4.2 Covert Channels

9.5 BASICS OF CRYPTOGRAPHY
9.5.1 Secret-Key Cryptography
9.5.2 Public-Key Cryptography
9.5.3 One-Way Functions
9.5.4 Digital Signatures
9.5.5 Trusted Platform Module

9.6 AUTHENTICATION
9.6.1 Authentication Using a Physical Object
9.6.2 Authentication Using Biometrics

9.7 EXPLOITING SOFTWARE
9.7.1 Buffer Overflow Attacks
9.7.2 Format String Attacks
9.7.3 Dangling Pointers
9.7.4 Null Pointer Dereference Attacks
9.7.5 Integer Overflow Attacks
9.7.6 Command Injection Attacks
9.7.7 Time of Check to Time of Use (TOCTOU) Attacks

9.8 INSIDER ATTACKS
9.8.1 Logic Bombs
9.8.2 Back Doors
9.8.3 Login Spoofing

9.9 MALWARE
9.9.1 Trojan Horses
9.9.2 Viruses
9.9.3 Worms
9.9.4 Spyware
9.9.5 Rootkits

9.10 DEFENSES
9.10.1 Firewalls
9.10.2 Antivirus and Anti-Antivirus Techniques
9.10.3 Code Signing
9.10.4 Jailing
9.10.5 Model-Based Intrusion Detection
9.10.6 Encapsulating Mobile Code
9.10.7 Java Security

9.11 RESEARCH ON SECURITY

9.12 SUMMARY

CHAPTER 10 "CASE STUDY 1: UNIX, LINUX, AND ANDROID"

10.1 HISTORY OF UNIX AND LINUX
10.1.1 UNICS
10.1.2 PDP-11 UNIX
10.1.3 Portable UNIX
10.1.4 Berkeley UNIX
10.1.5 Standard UNIX
10.1.6 MINIX
10.1.7 Linux

10.2 OVERVIEW OF LINUX
10.2.1 Linux Goals
10.2.2 Interfaces to Linux
10.2.3 The Shell
10.2.4 Linux Utility Programs
10.2.5 Kernel Structure

10.3 PROCESSES IN LINUX
10.3.1 Fundamental Concepts
10.3.2 Process Management System Calls in Linux
10.3.3 Implementation of Processes and Threads in Linux
10.3.4 Scheduling in Linux
10.3.5 Booting Linux

10.4 MEMORY MANAGEMENT IN LINUX
10.4.1 Fundamental Concepts
10.4.2 Memory Management System Calls in Linux
10.4.3 Implementation of Memory Management in Linux
10.4.4 Paging in Linux

10.5 INPUT/OUTPUT IN LINUX
10.5.1 Fundamental Concepts
10.5.2 Networking
10.5.3 Input/Output System Calls in Linux
10.5.4 Implementation of Input/Output in Linux
10.5.5 Modules in Linux

10.6 THE LINUX FILE SYSTEM
10.6.1 Fundamental Concepts
10.6.2 File System Calls in Linux
10.6.3 Implementation of the Linux File System
10.6.4 NFS: The Network File System

10.7 SECURITY IN LINUX
10.7.1 Fundamental Concepts
10.7.2 Security System Calls in Linux
10.7.3 Implementation of Security in Linux

10.8 ANDROID

10.9 SUMMARY

CHAPTER 11 "CASE STUDY 2: WINDOWS 8"

11.1 HISTORY OF WINDOWS THROUGH WINDOWS 8.1
11.1.1 1980s: MS-DOS
11.1.2 1990s: MS-DOS-based Windows
11.1.3 2000s: NT-based Windows
11.1.4 Windows Vista
11.1.5 2010s: Modern Windows

11.2 PROGRAMMING WINDOWS
11.2.1 The Native NT Application Programming Interface
11.2.2 The Win32 Application Programming Interface
11.2.3 The Windows Registry

11.3 SYSTEM STRUCTURE
11.3.1 Operating System Structure
11.3.2 Booting Windows
11.3.3 Implementation of the Object Manager
11.3.4 Subsystems, DLLs, and User-Mode Services

11.4 PROCESSES AND THREADS IN WINDOWS
11.4.1 Fundamental Concepts
11.4.2 Job, Process, Thread, and Fiber Management API Calls
11.4.3 Implementation of Processes and Threads

11.5 MEMORY MANAGEMENT
11.5.1 Fundamental Concepts
11.5.2 Memory Management System Calls
11.5.3 Implementation of Memory Management

11.6 CACHING IN WINDOWS

11.7 INPUT/OUTPUT IN WINDOWS
11.7.1 Fundamental Concepts
11.7.2 Input/Output API Calls
11.7.3 Implementation of I/O

11.8 THE WINDOWS NT FILE SYSTEM
11.8.1 Fundamental Concepts
11.8.2 Implementation of the NT File System

11.9 WINDOWS POWER MANAGEMENT

11.10 SECURITY IN WINDOWS 8
11.10.1 Fundamental Concepts
11.10.2 Security API Calls
11.10.3 Implementation of Security
11.10.4 Security Mitigations

11.11 SUMMARY

CHAPTER 13 "OPERATING SYSTEM DESIGN"

13.1 THE NATURE OF THE DESIGN PROBLEM
13.1.1 Goals
13.1.2 Why Is It Hard to Design an Operating System?

13.2 INTERFACE DESIGN
13.2.1 Guiding Principles
13.2.2 Paradigms
13.2.3 The System Call Interface

13.3 IMPLEMENTATION
13.3.1 System Structure
13.3.2 Mechanism versus Policy
13.3.3 Orthogonality
13.3.4 Naming
13.3.5 Binding Time
13.3.6 Static versus Dynamic Structures
13.3.7 Top-Down versus Bottom-Up Implementation
13.3.8 Useful Techniques

13.4 PERFORMANCE
13.4.1 Why Are Operating Systems Slow?
13.4.2 What Should Be Optimized?
13.4.3 Space-Time Trade-offs
13.4.4 Caching
13.4.5 Hints
13.4.6 Exploiting Locality
13.4.7 Optimize the Common Case

13.5 PROJECT MANAGEMENT
13.5.1 The Mythical Man Month
13.5.2 Team Structure
13.5.3 The Role of Experience
13.5.4 No Silver Bullet

13.6 TRENDS IN OPERATING SYSTEM DESIGN
13.6.1 Virtualization
13.6.2 Multicore Chips
13.6.3 Large Address Space Operating Systems
13.6.4 Networking
13.6.5 Parallel and Distributed Systems
13.6.6 Multimedia
13.6.7 Battery-Powered Computers
13.6.8 Embedded Systems
13.6.9 Sensor Nodes

13.7 SUMMARY

CHAPTER 14 "READING LIST AND BIBLIOGRAPHY"

14.1 SUGGESTIONS FOR FURTHER READING
14.1.1 Introduction and General Works
14.1.2 Processes and Threads
14.1.3 Memory Management
14.1.4 Input/Output
14.1.5 File Systems
14.1.6 Deadlocks
14.1.7 Virtualization and the CLoud
14.1.8 Multiple Processor Systems
14.1.9 Security
14.1.10 UNIX, Linux, and Android
14.1.11 Windows 8
14.1.12 Design Principles

14.2 ALPHABETICAL BIBLIOGRAPHY
show more