spcl.inf.ethz.

ch
@spcl_eth

ADRIAN PERRIG & TORSTEN HOEFLER

Networks and Operating Systems (252-0062-00)

Chapter 1:
Introduction to Operating Systems

spcl.inf.ethz.ch
@spcl_eth

Administrivia
Two parts:
Networks Adrian Perrig
Operating Systems Torsten Hoefler

Lecture:
Thu 8-10am, CAB G61
Fri 10am-noon, CAB G61

Practice sessions (subset of the ones in eDoz!):

Thu 3-6pm, ML F 40
Fri 1-4pm, CHN G 22
Fri 1-4pm, CAB G 57

Go to one of these sessions!

And participate!
Well, and participate in the lecture as well

spcl.inf.ethz.ch
@spcl_eth

More Administrivia
Course webpage (the authoritative information source)
http://spcl.inf.ethz.ch/Teaching/2014-osnet/
All slides will be there before the lecture (so you can take notes)

Exercises are:
Theoretical: Analysis of performance properties
Practical: Trying out stuff + Programming exercises

We assume you know both C and Java.

Exercises start today!

There is a mailing list for questions to the Tas

You are not subscribed but can sign up at (if you want)
https://spcl.inf.ethz.ch/cgi-bin/mailman/listinfo/2014-osnet-ta

Please register during the break

put your name into lists at front desk of lecture hall

spcl.inf.ethz.ch
@spcl_eth

Exam
(No mid-term.)
Final exam: tbd (Session)
Material:
Covered in the lectures, and/or
Learned during the lab exercises

We will not follow the books closely.

All pieces will be in books though

Optional extra readings may appear on the web

spcl.inf.ethz.ch
@spcl_eth

Course Outline
20.02.: OS Introduction
21.02.: Processes
27.02.: Scheduling
28.02.: Synchronization
13.03.: Memory Management
14.03.: Demand Paging

10.04.: File System Abstractions

11.04.: File System Implementations
08./09.05.: I/O Subsystem
15.05.: Virtual Machine Monitors
16.06.: Reliable Storage, Specials

06.03.: Network Intro / OSI Model

07.03.: Physical Layer
20.03.: skip
21.03.: Data Link Layer
27./28.03.: Network Layer
03./04.04.: Reliable Data Transfer
17.04.: Naming, Name Resolution
02.05.: Network Programming

22.05.: Applications
23.05.: Security
30.05.: New Internet Architectures
5

spcl.inf.ethz.ch
@spcl_eth

Birds-eye perspective
Networks
bridge space

Databases
bridge time

Networks, Operating Systems, Databases

they all manage resources
OS, DB: all resources (storage, computation, communication)
Networks: focus on communication

spcl.inf.ethz.ch
@spcl_eth

Today: We start on Operating Systems!

Introduction: Why?
Roles of the OS
Referee
Illusionist
Glue

Structure of an OS

spcl.inf.ethz.ch
@spcl_eth

Goals

Demystify operating systems themselves

What is an OS? What does it do?

What is its structure?
How do the OS and applications relate to each other?
What services does the OS provide?

Quintessential systems problem

Non-idealizable / non-reducible
Scaling, emergent properties
Concurrency and asynchrony

spcl.inf.ethz.ch
@spcl_eth

The Book
On the web:
http://ospp.cs.washington.edu/

spcl.inf.ethz.ch
@spcl_eth

Also worth a look

Jerome H. Saltzer and
M. Frans Kaashoek:
Principles of
Computer System
Design
Focus on principles,
with illustrative
examples

10

spcl.inf.ethz.ch
@spcl_eth

Also worth a look

Andrew S.
Tanebaum:
Modern Operating
Systems
Must be at least 3rd
Edition!
Very broad lots of
references to
recent (2006)
research.
11

spcl.inf.ethz.ch
@spcl_eth

Introduction to
Operating Systems

12

spcl.inf.ethz.ch
@spcl_eth

Why learn about Operating Systems?

One of the most complex topics in Computer Science!
Very few simplifying assumptions
Dealing with the real world
Intersection of many areas

Mainstream OSes are large:

Windows 7 ~ 50 million lines of code
Linux rapidly catching up in complexity

Most other software systems are a subset

Games, browsers, databases, servers, cloud, etc.

13

spcl.inf.ethz.ch
@spcl_eth

Why learn about Operating Systems?

One of the most complex topics in Computer Science!
Very few simplifying assumptions
Dealing with the real world
Intersection of many areas

Mainstream OSes are large:

Windows 7 ~ 50 million lines of code
Linux rapidly catching up in complexity (~15 million LOC)

Most other software systems are a subset

Games, browsers, databases, servers, cloud, etc.

14

spcl.inf.ethz.ch
@spcl_eth

Why learn about Operating Systems?

One of the most complex topics in Computer Science!
Very few simplifying assumptions
Dealing with the real world
Intersection of many areas

Mainstream OSes are large:

Windows 7 ~ 50 million lines of code
Linux rapidly catching up in complexity (~15 million LOC)

Most other software systems are a subset

Games, browsers, databases, servers, cloud, etc.

15

spcl.inf.ethz.ch
@spcl_eth

There are lots of operating systems concepts

Systems calls
Concurrency and asynchrony
Processes and threads
Security, authorization, protection
Memory, virtual memory, and paging
Files and file systems, data management
I/O: Devices, Interrupts, DMA
Network interfaces and protocol stacks

16

There are lots of

operating systems

spcl.inf.ethz.ch
@spcl_eth

17

Goals:
what makes a good OS?

spcl.inf.ethz.ch
@spcl_eth

18

Goals:
what makes a good OS?

spcl.inf.ethz.ch
@spcl_eth

Reliability: does it keep working?

And availability

19

Goals:
what makes a good OS?

spcl.inf.ethz.ch
@spcl_eth

Reliability: does it keep working?

And availability

Security: can it be compromised?

And isolation: is it fair?

20

Goals:
what makes a good OS?

spcl.inf.ethz.ch
@spcl_eth

Reliability: does it keep working?

And availability

Security: can it be compromised?

And isolation: is it fair?

Portability: how easily can it be retargeted?

21

Goals:
what makes a good OS?

spcl.inf.ethz.ch
@spcl_eth

Reliability: does it keep working?

And availability

Security: can it be compromised?

And isolation: is it fair?

Portability: how easily can it be retargeted?

Performance: how fast/cheap/hungry is it?

22

Goals:
what makes a good OS?

spcl.inf.ethz.ch
@spcl_eth

Reliability: does it keep working?

And availability

Security: can it be compromised?

And isolation: is it fair?

Portability: how easily can it be retargeted?

Performance: how fast/cheap/hungry is it?
Adoption: will people use it?

23

spcl.inf.ethz.ch
@spcl_eth

Operating Systems

Applications

Operating
System

Hardware

24

spcl.inf.ethz.ch
@spcl_eth

Operating Systems

Applications

Operating
System

Operating Systems
(rest of this course!)

Hardware

25

spcl.inf.ethz.ch
@spcl_eth

Operating Systems

Applications

Operating
System

Parallel Programming
Operating Systems
(rest of this course!)

Hardware

26

spcl.inf.ethz.ch
@spcl_eth

Operating Systems

Applications

Operating
System

Hardware

Parallel Programming
Operating Systems
(rest of this course!)
Computer Architecture and
Systems Programming

27

spcl.inf.ethz.ch
@spcl_eth

Operating System Roles

28

spcl.inf.ethz.ch
@spcl_eth

OS roles

Referee

Illusionist

Glue

29

spcl.inf.ethz.ch
@spcl_eth

The Referee:
Resource Manager

Application

Application

Application

System calls
Operating
System
Hardware

30

spcl.inf.ethz.ch
@spcl_eth

The OS as Referee
Sharing:
Multiplex hardware among applications
CPU, memory, devices
Applications shouldnt need to be aware of each other

Protection:
Ensure one application cant r/w anothers data
In memory, on disk, over network
Ensure one application cant use anothers resources
CPU, storage space, bandwidth,

Communication:
Protected applications must still communicate

31

spcl.inf.ethz.ch
@spcl_eth

The OS as Referee
Sharing:
Multiplex hardware among applications
CPU, memory, devices
Applications shouldnt need to be aware of each other

Protection:
Ensure one application cant r/w anothers data
In memory, on disk, over network
Ensure one application cant use anothers resources
CPU, storage space, bandwidth,

Communication:
Protected applications must still communicate

32

spcl.inf.ethz.ch
@spcl_eth

The OS as Referee
Sharing:
Multiplex hardware among applications
CPU, memory, devices
Applications shouldnt need to be aware of each other

Protection:
Ensure one application cant r/w anothers data
In memory, on disk, over network
Ensure one application cant use anothers resources
CPU, storage space, bandwidth,

Communication:
Protected applications must still communicate

33

spcl.inf.ethz.ch
@spcl_eth

Resource management goals

Fairness:
No starvation, every application makes progress

Efficiency:
Best use of complete machine resources
Minimize e.g. power consumption

Predictability:
Guarantee real-time performance

34

spcl.inf.ethz.ch
@spcl_eth

Resource management goals

Fairness:
No starvation, every application makes progress

Efficiency:
Best use of complete machine resources
Minimize e.g. power consumption

All in mutual
Guarantee real-time performance
contradiction

Predictability:

35

spcl.inf.ethz.ch
@spcl_eth

Example: Threads
Threads are virtual CPUs
Physical resource: CPUs
Virtual resource: Threads
Mechanism: pre-emption, timeslicing, context switching, scheduling

More on this later in the course

36

spcl.inf.ethz.ch
@spcl_eth

The Illusionist
Virtualization:
OS creates illusion of a real resource
Processor, storage, network, links,

Virtual resource looks a bit like a physical resource

However, is frequently quite different
Simpler, larger, better,

37

spcl.inf.ethz.ch
@spcl_eth

How?
1.

Multiplexing

2.

Emulation

3.

Divide resources up among clients

Create the illusion of a resource using software

Aggregation

Join multiple resources together to create a new one

38

spcl.inf.ethz.ch
@spcl_eth

Why?
1.

Sharing

2.

Sandboxing

3.

Prevent a client from accessing other resources

Decoupling

4.

Enable multiple clients of a single resource

Avoid tying a client to a particular instance of a resource

Abstraction

Make a resource easier to use

39

spcl.inf.ethz.ch
@spcl_eth

Example: Virtual Memory

Easier memory to manage

Physical resource: RAM

Virtual resource: Virtual Memory
Method: Multiplexing
Mechanism: virtual address translation

40

spcl.inf.ethz.ch
@spcl_eth

Example: Paged virtual memory

More memory than you really have

Physical resource: RAM and disk

Virtual resource: paged virtual memory
Method: multiplexing and emulation
Mechanism: virtual memory + paging to/from disk

Much more on this later in the course

41

spcl.inf.ethz.ch
@spcl_eth

Example: virtual machines

Quite popular topic commercially right now:

Xen, VMware, HyperV, kvm, etc.

Many uses:

Run one OS on another

Consolidate servers
Migrate running machines around datacenter
Run hundreds of honeypot machines
Deterministic replay of whole machines
Etc.

42

spcl.inf.ethz.ch
@spcl_eth

Example: Files (or database!)

Virtual resource: persistant memory

Physical resource: disk
Method: multiplexing, emulation
Mechanism: block allocation, metadata

Again, more later

43

Example: Windows

spcl.inf.ethz.ch
@spcl_eth

(not the Microsoft OS)

Physical resource: Frame buffer and/or GPU

Method: Multiplexing and emulation
Mechanism: Windows as separate
bitmaps/textures

44

spcl.inf.ethz.ch
@spcl_eth

Example: virtual circuits

Physical resource: network link
Virtualization method: multiplexing
Mechanism: VC identifiers, VC switching

Virtual
circuits

Real circuits

VCI=346

VCI=1044

VCI=1044

VCI=56

45

spcl.inf.ethz.ch
@spcl_eth

Example: VLANs
Methods: multiplexing
Mechanisms: port assignment, tags

C D

I
J
K

M
S1

S2

S1

S2

N
N

G H

46

spcl.inf.ethz.ch
@spcl_eth

Glue: the OS as
Abstract Machine

Applications
Virtual machine interface
Operating
System
Physical machine
interface
Hardware

47

spcl.inf.ethz.ch
@spcl_eth

The OS as Glue
Provides high-level abstractions
Easier to program to
Shared functionality for all applications
Ties together disparate functions and services

Extends hardware with added functionality

Direct programming of hardware unnecessary

Hides details of hardware

Applications decoupled from particular devices

48

spcl.inf.ethz.ch
@spcl_eth

Services provided by an OS

Program execution
Load program, execute on 1 or more processors

Access to I/O devices

Disk, network, keyboard, screen,

Protection and access control

For files, connections, etc.

Error detection and reporting

Trap handling, etc.

Accounting and auditing

Statistics, billing, forensics, etc.

49

spcl.inf.ethz.ch
@spcl_eth

Operating System Structure

50

spcl.inf.ethz.ch
@spcl_eth

General OS structure

Application

Application

Server process
(daemon)

System Library

System Library

System Library

System calls

User mode

Privileged mode
Kernel

CPU

CPU

Device

Device
51

spcl.inf.ethz.ch
@spcl_eth

Privileged Mode and User Mode

As we saw in Computer Architecture,
most CPUs have a privileged mode:
ia32 protection rings
VAX kernel mode
Etc.

Most Operating Systems use this for protection

In particular, protecting the OS from applications!

52

spcl.inf.ethz.ch
@spcl_eth

General OS structure

Application

Application

Server process
(daemon)

System Library

System Library

System Library

System calls

User mode

Privileged mode
Kernel

CPU

CPU

Device

Device
53

spcl.inf.ethz.ch
@spcl_eth

Kernel
That part of the OS which runs in privileged mode
Large part of Unix and Windows (except libraries)
Small part of L4, Barrelfish, etc. (microkernels)
Does not exist in some embedded systems

Also known as:

Nucleus, nub, supervisor,

54

spcl.inf.ethz.ch
@spcl_eth

The kernel is a program!

Kernel is just a (special) computer program.
Typically an event-driven server.
Responds to multiple entry points:
System calls
Hardware interrupts
Program traps

May also include internal threads.

55

spcl.inf.ethz.ch
@spcl_eth

General OS structure

Application

Application

Server process
(daemon)

System Library

System Library

System Library

System calls

User mode

Privileged mode
Kernel

CPU

CPU

Device

Device
56

spcl.inf.ethz.ch
@spcl_eth

System Libraries
Convenience functions
strcmp(), etc.
Common functionality

System call wrappers

Create and execute system calls from high-level languages
See man syscalls on Linux

57

spcl.inf.ethz.ch
@spcl_eth

General OS structure

Application

Application

Server process
(daemon)

System Library

System Library

System Library

System calls

User mode

Privileged mode
Kernel

CPU

CPU

Device

Device
58

spcl.inf.ethz.ch
@spcl_eth

Daemons
Processes which are part of the OS
Microkernels: most of the OS
Linux: increasingly large quantity

Advantages:
Modularity, fault tolerance
Easier to schedule

59

spcl.inf.ethz.ch
@spcl_eth

Entering and exiting the kernel

60

spcl.inf.ethz.ch
@spcl_eth

When is the kernel entered?

Startup
Exception: caused by user program
Interrupt: caused by something else
System calls

61

spcl.inf.ethz.ch
@spcl_eth

Recall: System Calls

RPC to the kernel
Kernel is a series of syscall event handlers
Mechanism is hardware-dependent

User mode

Privileged mode

System calls

62

spcl.inf.ethz.ch
@spcl_eth

Recall: System Calls

RPC to the kernel
Kernel is a series of syscall event handlers
Mechanism is hardware-dependent

User process
runs

User mode

Privileged mode

System calls

63

spcl.inf.ethz.ch
@spcl_eth

Recall: System Calls

RPC to the kernel
Kernel is a series of syscall event handlers
Mechanism is hardware-dependent

User process
runs

Execute
syscall

User mode

Privileged mode

System calls

64

spcl.inf.ethz.ch
@spcl_eth

Recall: System Calls

RPC to the kernel
Kernel is a series of syscall event handlers
Mechanism is hardware-dependent

User process
runs

Execute
syscall

User mode

Privileged mode

System calls

65

spcl.inf.ethz.ch
@spcl_eth

Recall: System Calls

RPC to the kernel
Kernel is a series of syscall event handlers
Mechanism is hardware-dependent

User process
runs

Execute
syscall

User mode

Privileged mode
Execute kernel
code

System calls

66

spcl.inf.ethz.ch
@spcl_eth

Recall: System Calls

RPC to the kernel
Kernel is a series of syscall event handlers
Mechanism is hardware-dependent

User process
runs

Execute
syscall

Process resumes

User mode

Privileged mode
Execute kernel
code

System calls

67

spcl.inf.ethz.ch
@spcl_eth

System call arguments

Syscalls are the way a program requests
services from the OS kernel.
Implementation varies:
Passed in processor registers
Stored in memory (address in register)
Pushed on the stack
System library (libc) wraps as a C function
Kernel code wraps handler as C call

68

spcl.inf.ethz.ch
@spcl_eth

When is the kernel exited?

Creating a new process
Including startup

Resuming a process after a trap

Exception, interrupt or system call

User-level upcall
Much like an interrupt, but to user-level

Switching to another process

69