DV1492 / DV1556:

Realtime- (and) Operating-Systems

08:15-10:00 Tuesday August 29th, 2017

An introduction to operating systems.

Chapter 1 - 1.3 (pg 1-35)

Table of Contents
What is an OS?
Motivating example programs.
Course overview.

1. Introduction

  • Hello! Welcome...
    • Carina - Examinator - cnl@bth.se
    • Andrew - Lectures - awm@bth.se
    • Cuong & Erik - Labs
  • Natural questions to ask:
    • Am I in the right place?
    • What is an Operating System?
    • What is the course about?
    • Why do I want to learn about this?
    • What will I gain from the course?
    • What must I put into the course?
    • The book is large and scary...

2. Motivation: The missing code

  • Let us recall a basic C program.
  • Visible code is simple.
  • Implicit interface to the environment is complex.
  • Talks to several physical devices.
  • Works on any model of keyboard.
  • Text output is probably rendered into a window.
  • (implicit) Memory Management.
  • I/O to shell (args and exit code).
int main(int argc, char **argv) { char name[80]; printf("What is your name?\n"); gets(name); printf("Hello %s!", name); return 0; } /* Cue pychadelic flashback to Intro to C */

3. Motivation: What lies beneath

  • So where is all the missing code?
    • We see library definitions.
    • "Stubs" in the language runtime.
      • Small function wrappers to call something else.
    • Somehow these calls need to turn into requests to physical hardware.
    • Other programs are probably using the same hardware.
    • Negotiate access to resources shared with other programs.
    • Low-level control of the device (model / brand).

4. What is an OS?

  • No standard ("official") definition.
  • Several definitions, depending on context / perspective.
Simplification / Abstraction
Provide a high-level machine to work in, hide machine details.
Common Code
A collection of codes useful in many programs.
Resource Manager
Arbitration: negotiate shared access between programs.
Consistent Interface
Provide same user-interface for different programs.
Controls access to the machine.

5. What is an OS: Simplification / Abstraction

Simplifying a task by hiding irrelevant information
  • Why do we need to simplify the architecture of the computer?
    • Hardware is complex and messy.
    • S-ATA: 450 pages of documention to access the disk.
    • The ofstream class: a few slides to explain how to use it.
  • Task: writing a file to the disk.
    • The abstraction is designed to support this operation.
    • We only need to know how to use the interface (ofstream).
    • We don't care about the low-level details: irrelevant.
  • The OS is a program running on the real machine.
    • It provides a simpler "extended machine" for other programs.

6. What is an OS: Simplification / Abstraction

  • Abstraction is good strategy for handling complexity.
    • Reuse it: abstractions of abstractions.
    • Conventionally these are drawn as layer diagrams.
    • Each layer knows the details of one layer below.
    • Each layer acts as a simple interface for the layers above.
  • When the abstraction fits a task well:
    • Program only needs high-level interfaces.
    • e.g. browser uses sockets / UI.
  • When the abstraction breaks down:
    • Details from the layers below leak.
    • e.g. thread-unsafe libraries, security.

7. What is an OS: Abstraction and Safety

  • Programs that can directly access hardware run on "bare metal".
    • Writing one complex program on bare metal is difficult.
    • Can do anything; must provide everything.
    • Yet we expect to execute more than one program at a time.
  • The OS is the only program that runs on bare metal.
    • It manages the execution of all other programs: they run "on" the OS.
    • If a program can access hardware it can cause other programs to break.
    • Finding the source of faults becomes impossible.
  • Isolating hardware access inside the OS also provides safety.
    • Programs will only break because of their own bugs.

8. What is an OS: Safety

  • Users expect that program behaviour...
    • ...should depend on data and interactions.
    • ...should not be affected by other programs.
  • Programs (and their bugs) should not interfere with one another.
  • Each program should be isolated in the system.
    • Programs rely on memory to hold their data.
    • Unexpected changes to data cause faults.
    • Each program needs its own private memory.
  • The processor should do what the program instructs; each program needs a private processor.

9. What is an OS: Managing Resources

  • Many hardware devices should be shared among multiple programs.
    • Normally the device will not have a multi-user interface.
    • Managing different sessions, switching operation is costly.
    • Assume the OS can perform these functions.
  • Multiplexing: dividing a resource (device) into pieces.
    • Time-multiplexing: programs take turns to use the resource (e.g. a printer).
    • Space-multiplexing: programs use different pieces of the resource (e.g. disk-space).
  • Hide the details from programs (avoid repeating the multiplexing code).
  • Hide the details from hardware (avoid extra circuitry).
OS tracks allocation of resources/sessions.
Merges interactions into a single stream of requests / responses.

10. What is an OS: Consistency

  • Different kinds of device behave differently.
    • Interface for a mouse is vastly different to a hard-disk.
  • Similar kinds of devices also behave differently.
    • Different manufacturers use different protocols.
    • Different sets of features are programmed differently.
    • Different models require different "work-arounds".
Simplicity is prerequisite for reliability.
- Edsger Dijkstra

11. What is an OS: Common code

  • Each of the perspectives presented overlaps somewhat.
    • The boundaries of what is the OS tend to shift over time.
    • Kernel (obviously), Shell, Libraries, System utilities...
    • Old definition: its what is shipped on the CD.
    • Package managers / App Stores / Automatic updates have blurred this boundary.
    • The boundary does not shrink - things are added over time.
  • Here is a rough working definition:
    • The probability that a piece of code will become part of the OS depends on:
      • How many programs use that code.
      • How difficult it is for a program to deal with its absence.
      • How old the code is.

12. Some motivation: overall approach

  • There are two ways to teach an (undergrad) OS course:
    • Take an "architectural" approach - avoid any specifics (Silberschatz)
    • Take an "implementation" approach - show you how to implement pieces.
  • I don't think teaching architecture first has any value.
    • It is more useful to understand the details first.
    • Then generalise them to look for patterns.
  • The Tenenbaum text explains writing an OS as a programming problem.
    • Here are problems, algorithms / data-structures / code as solutions.
  • Processes, threads, event-despatch, IPC, blocking I/O...
    • Features to solve problems in programs that you have not written yet.
  • Generic types of programs: batch, bare-metal, desktop, server, game.
    • Simple sketches to provide some context for where the book starts.

13. Motivating programs: batch

int main(int argc, char **argv) { /* Process argc/argv to decide what to do */ /* May open additional files to process... */ while( read(stdin,...) != EOF) { /* Do stuff */ } return 0; }
  • Non interactive: e.g. command-line utilities, compilers etc
  • Blocking I/O: procedure-call performs an operation, returns when complete.
  • The order of control-flow in the program shows the sequence of logical operations on the stream.
  • Reads and writes are discrete operations: easy to reason about the program.

14. Motivating programs: bare metal I

PSUEDO_DECL { /* Cmds to hardware by setting bits */ unsigned char *control = 0x10ffee; *control = 0x80; /* Dummy operation - timing loop */ for(i=0; i<10000; i++) x = x*2; ... (continued) ...
  • Firmware / Driver / Embedded / (older) Demo-scene
  • Tied to the I/O requirements of hardware.
  • Control-flow matches the sequence of physical requests / responses.
  • "Bit banging" is a style of commanding hardware in realtime.
  • Physical device needs time to respond: delay loops.

Break (15mins)


15. Motivating programs: bare metal II

... (continued) ... unsigned char status, *port = 0x10ffef; /* Wait for response */ while((status = *port) & 0x80) ; /* Do some processing using status */ }
  • Difficult to do other processing.
  • Have to hide it inside the timing loops.
  • Program cut into small incremental pieces; interrupted and resumed.
  • We are not free to define control flow.
  • It is largely determined by the access to hardware.
  • Access to hardware = program can do anything (no security).

16. Motivating programs: desktop

int WINAPI WinMain( ... ) { Init(); /* Allocate resources, create window etc */ while (GetMessage(&msg, NULL, 0, 0)) { TranslateMessage(&msg); DispatchMessage(&msg); /*big switch lives here*/ } }
  • Desktop application: interaction via message queue.
  • Control-flow is broken up - no large scale serial processing.
    • Incremental processing in event handlers (asynchronous style).
  • The GetMessage call is blocking (synchronous) to make this easy.
  • Alternative implementation is to register a callback for processing.
  • Time is easy to handle this this model: SetTimer(), message arrives "later".

17. Motivating programs: server

int main(int argc, char **argv) { int server = socket(...); /* Create */ bind(...); /* Attach to port */ while(1) { int newCon = accept(server, ...); int pid = fork(); if(pid) { processClient(newCon); /* In new process */ exit(0); } else close(newCon); /* In old process */ } }
  • Despatching connection in a server (e.g. a web-server).

18. Motivating programs: server II

int pid = fork(); if(pid) { processClient(newCon); /* In new process */ exit(0); } else close(newCon); /* In old process */
  • When a program forks:
    • Splits into two copies that both continue running concurrently.
    • Use the same code for both.
  • One program calls processClient and handles the interaction (then quits).
  • The other program does not need the file handle for this socket.
    • Closes it (still open in the other program).
    • Continues the loop (accepts more connections, does more forks).

19. Motivating programs: server III

int newCon = accept(server, ...); int pid = fork(); if(pid) ... else ...
  • The OS is supplying blocking I/O on the accept.
  • The OS allows a program to split itself into multiple running pieces.
    • This is ideal encapsulation (separate program for each connection).
    • A bug (attack) in one connection is isolated.
    • Because the sub-programs do less, we could drop privileges.
  • On a machine with multiple CPUs we expect parallelisation.
    • Programs forked off for connections can run across multiple cores / CPUs.
  • Very nice high-level code because of the OS primitives.

20. Motivating programs: game

float frameTime = 1 / 60.0; /* Avoid peasantry */ float lastTime = getTime(); while (1) { sampleInput(); float realTime = getTime(); while( (realTime-lastTime) < frameTime * 3.0/4.0; { doSimulationStep(); realTime = getTime(); } renderWorld(); }
  • Maximises the number of world updates between renders.
    • Tries to keep constant frame-rate.
    • The 3:1 simulation:rendering ratio would be an "empirical fact".
  • Serial, synchronous style (realtime): burn 100% CPU to meet deadlines.

21. What is the course about?

  • The major subsystems in the OS:
    • Processes and Scheduling.
    • Storage (Memory, File-Systems).
    • I/O.
  • Cross-cutting concerns:
    • Security
  • In each case:
    • Design Issues - what to support / motivating problems.
    • Important algorithms - normally a range of solutions.
    • Implementation details.
In theory there is no difference between theory and practice. In practice...

22. What is the course about? (again)

  • Same question: answer from a different perspective.
  • Bare-metal programming is ugly.
    • It's hard, insecure and tied to the low-level details of a machine.
  • Each of the other styles:
    • More productive.
    • Safer.
    • More portable.
Central Question
How can we write one large complex bare-metal program ... that executes all of the other kinds ... concurrently
  • It takes about eight weeks to answer this question properly...

23. Why do I want to learn about this?

  • A good programmer understands what their program does:
    • Understanding the language.
    • Understanding the platform.
  • Part of the platform is the definition of the libraries.
    • But part of how the platform works is only implied.
    • To understand it you must understand the layer underneath - the OS.
  • Three reasons:
    • More complete as a developer (systems programming, embedded, devops).
    • Write better code (using the system more effectively).
    • To become a better engineer (understanding the contraints you work within).

24. What will I gain from the course?

  • Direct programming experience:
    • Project: implement a file-system.
    • Good step in expanding beyond the intro courses.
    • Medium size project, clear spec, put some of the material into practice.
  • Indirect programming experience:
    • We can't get you to write a full OS in eight weeks.
    • Studying how one works is the next best thing.
    • Expand knowledge of data-structures and algorithms.
    • See how these are applied.
  • Lab with an intro to linux.

25. What must I put into the course?

  • Study the textbook.
  • Lectures.
    • Will highlight key points - not a replacement for studying textbook.
    • Idea is to understand the central points of the chapters we cover.
    • Ask questions.
    • Expand upon it in self-study.
  • Labs.
  • Coursework.
  • Exam.

26. Wrapping up

  • Next time we review common hardware.
  • Look at the history of Operating Systems.
  • Start looking at central concepts.
    • How system calls work.
  • Questions?