OS 14 File System

Download as pdf or txt
Download as pdf or txt
You are on page 1of 102

File system

Slides borrowed from Galvin, with many of our modifications


Operating System Concepts – 9th Edition Silberschatz, Galvin and Gagne ©2013
How should the OS efficiently manage a persistent
device (your hdd / ssd)?

Operating System Concepts – 9th Edition 11.2 Silberschatz, Galvin and Gagne ©2013
How should the OS efficiently manage a persistent
device (your hdd / ssd)?
● What are data read/write abstractions?
● What are the APIs?
● How to implement file system related functions?

Operating System Concepts – 9th Edition 11.3 Silberschatz, Galvin and Gagne ©2013
At the very beginning: You just
have a disk

Operating System Concepts – 9th Edition 11.4 Silberschatz, Galvin and Gagne ©2013
Disk Structure
■ What is a disk: OS sees a set of blocks that can be accessed
using memory addresses – logical view

■ Disk can be subdivided into partitions

■ Disk or partition can be used raw – without a file system, or


formatted with a file system

■ Partitions also known as minidisks, slices

■ Entity containing file system known as a volume

Operating System Concepts – 9th Edition 11.5 Silberschatz, Galvin and Gagne ©2013
A Typical File-system Organization

Operating System Concepts – 9th Edition 11.6 Silberschatz, Galvin and Gagne ©2013
Disk Structure (contd.)
■ What is a disk: OS sees a set of blocks that can be accessed
using memory addresses – logical view

■ Disk can be subdivided into partitions

■ Disk or partition can be used raw – without a file system, or


formatted with a file system

■ Partitions also known as minidisks, slices

■ Entity containing file system known as a volume

■ Each volume containing file system also tracks that file


system s info in device directory or volume table of contents

Operating System Concepts – 9th Edition 11.7 Silberschatz, Galvin and Gagne ©2013
So what is a file system?

File system = Set of Files + Directory


Analogy: Book = Chapters + Index

Operating System Concepts – 9th Edition 11.8 Silberschatz, Galvin and Gagne ©2013
What is a file?

Operating System Concepts – 9th Edition 11.9 Silberschatz, Galvin and Gagne ©2013
File: R/W abstraction for storage
■ Named collection of related information – smallest allotment of logical
secondary usage

■ Types:
● Data: Numeric, character, binary
● Program files which can be executed

■ The meaning of the file content defined by file’s creator


● Many types
4 Consider text file, source file, object file, executable file
4A file often has a structure, e.g., sequence of characters

Operating System Concepts – 9th Edition 11.10 Silberschatz, Galvin and Gagne ©2013
File Structure
■ None - sequence of words, bytes

■ Simple record structure


● Lines, Fixed length, Variable length

■ Complex Structures
● Special formatted document
● Can simulate by inserting appropriate control characters

■ Who decides the file structure:


● Operating system
● Sometimes programs define their special file structure

Operating System Concepts – 9th Edition 11.11 Silberschatz, Galvin and Gagne ©2013
File Types – Name, Extension

Operating System Concepts – 9th Edition 11.12 Silberschatz, Galvin and Gagne ©2013
Attributes stored with files
■ Name – only information kept in human-readable form
■ Identifier – unique tag (number) identifies file within file system
■ Type – needed for systems that support different types
■ Location – pointer to file location on device
■ Size – current file size
■ Protection – controls who can do reading, writing, executing
■ Time, date, and user identification – data for protection, security,
and usage monitoring
■ Sometimes extra information, like checksum to know file corruption.

Information about files are kept in the directory


structure, which is maintained on the disk (persistent)

Operating System Concepts – 9th Edition 11.13 Silberschatz, Galvin and Gagne ©2013
File info on Linux

■ stat notes.txt
■ file notes.txt
■ ls –l notes.txt

Operating System Concepts – 9th Edition 11.14 Silberschatz, Galvin and Gagne ©2013
Recap

File system = Set of Files + Directory


Analogy: Book = Chapters + Index

Operating System Concepts – 9th Edition 11.15 Silberschatz, Galvin and Gagne ©2013
What is a directory structure for
Files?

Operating System Concepts – 9th Edition 11.16 Silberschatz, Galvin and Gagne ©2013
Directory Structure

■ A collection of nodes containing information about all files

Directory

Files
F1 F2 F4
F3
Fn

Both the directory structure and the files reside on disk

Operating System Concepts – 9th Edition 11.17 Silberschatz, Galvin and Gagne ©2013
Directory design: System model

■ System model
● There are multiple users in a machine
● Each user might need a bunch of files (e.g., in their
home directory)
● Users act independently

Operating System Concepts – 9th Edition 11.18 Silberschatz, Galvin and Gagne ©2013
Desirable properties of directories

■ Efficiency – locating/searching for a file quickly

■ Naming – convenient to users


● Two users can have same name for different files

■ Grouping – logical grouping of files by properties, (e.g.,


all Java programs, all games, all pdf files, …)

■ Advanced: Sharing files between users / programs


● The same file can have several different names

Operating System Concepts – 9th Edition 11.19 Silberschatz, Galvin and Gagne ©2013
1. Single-Level Directory
■ A single directory for all users

■ Problem: Searching might be linear


■ Problem: Two different files can not have same name
■ Problem: No grouping capability

Operating System Concepts – 9th Edition 11.20 Silberschatz, Galvin and Gagne ©2013
2. Two-Level Directory
■ Separate directory for each user

■ Good:
■ Efficient searching
■ Can have the same file name for different user (using path name)
■ Problem:
■ No grouping capability

Operating System Concepts – 9th Edition 11.21 Silberschatz, Galvin and Gagne ©2013
3. Tree-Structured Directories

root/spell/mail/prog/obj/filename

Operating System Concepts – 9th Edition 11.22 Silberschatz, Galvin and Gagne ©2013
3. Tree-Structured Directories (Cont.)
■ Absolute or relative path name
● Creating a new file is done in current directory [command: mkdir]

■ Good:
● Efficient searching (tree traversal is better than linear search
● Grouping Capability (using subdirectories)

■ Current directory (working directory) [command: pwd]


● cd /spell/mail/prog [change directory]

Operating System Concepts – 9th Edition 11.23 Silberschatz, Galvin and Gagne ©2013
3. Tree-Structured Directories (Deletion)
■ Delete a file
rm <file-name>
■ Creating a new subdirectory is done in current directory
mkdir <dir-name>
Example: if in current directory /mail
mkdir count

● Deleting mail Þ deleting the entire subtree rooted by mail


4 MS-DOS: If a directory contains file / subdir, cannot delete it
4 Unix: give a “recursive” option, delete all (rm -r)

Operating System Concepts – 9th Edition 11.24 Silberschatz, Galvin and Gagne ©2013
4. Acyclic-Graph Directories: Enable file
sharing
■ Have shared subdirectories and files

■ New directory entry type


● Link – another name (pointer) to an existing file
● Resolve the link – follow pointer to locate the file

Operating System Concepts – 9th Edition 11.25 Silberschatz, Galvin and Gagne ©2013
Hard and soft linking
■ In acyclic graph directories
● Directories can share files using links between files
● These links can be hard links or soft links
■ Hard links
● link to an actual data blocks in the disk
● Command: ln <filename> <inkname>
● cannot link directories
● Hard links always refer to source files, even after its moved
■ Soft links (symbolic links)
● Symbolic link to the path where the file is stored
● Command: ln -s <filename> <inkname>
● Soft links can link directories
● Destroyed when the file moved

Operating System Concepts – 9th Edition 11.26 Silberschatz, Galvin and Gagne ©2013
Problem: How to ensure “acyclic”
■ In if not acyclic what is the problem?
● You have no idea how many nodes are in the tree using
directory tree traversal
● Say you have directory a and then create a link within a called
c. c points to a
● you go /a/c , /a/c/a, /a/c/a/c …

■ Solution
● Allow only links to file not subdirectories
● Every time a new link is added use a cycle detection algorithm
to determine whether it is OK

Operating System Concepts – 9th Edition 11.27 Silberschatz, Galvin and Gagne ©2013
Problem: How to ensure “acyclic”
■ In if not acyclic what is the problem?
● You have no idea how many nodes are in the tree using
directory tree traversal
● Say you have directory a and then create a link within a called
c. c points to a
● you go /a/c , /a/c/a, /a/c/a/c …

■ Solution
● Allow only links to file not subdirectories
● Every time a new link is added use a cycle detection algorithm
to determine whether it is OK

■ What is actually done?


● Bypass links during directory traversal

Operating System Concepts – 9th Edition 11.28 Silberschatz, Galvin and Gagne ©2013
File system = Set of Files + Directory

You can have a file system on a thumb drive :


How can you tell that to an OS about the file
system after plugging it in?

Operating System Concepts – 9th Edition 11.29 Silberschatz, Galvin and Gagne ©2013
File System Mounting
■ A file system must be mounted before it can be accessed
■ An unmounted file system (i.e., Fig. below) is mounted at a
mount point
/

users

bill fred sue jane

help doc
prog

(a) (b)

Operating System Concepts – 9th Edition 11.30 Silberschatz, Galvin and Gagne ©2013
Mount Point

users

bill
fred
sue
ja

help

prog
fred sue jane
(a)
(b)
help doc
prog

(b)

Operating System Concepts – 9th Edition 11.31 Silberschatz, Galvin and Gagne ©2013
How should the OS efficiently manage a persistent
device (your hdd / ssd)? Files, directory structures,
mounting
● What are data read/write abstractions?
● What are the APIs?
● How to implement file system related functions?

Operating System Concepts – 9th Edition 11.32 Silberschatz, Galvin and Gagne ©2013
How should the OS efficiently manage a persistent
device (your hdd / ssd)? Files, directory structures,
mounting
● What are data read/write abstractions?
● What are the APIs?
● How to implement file system related functions?

Operating System Concepts – 9th Edition 11.33 Silberschatz, Galvin and Gagne ©2013
Basic set of “FILE” Operations

■ File is an abstract data type


■ Six basic operation
● Create
● Write – at write pointer location
● Read – at read pointer location
● Reposition within file - seek
● Delete
● Truncate ( … > notes.txt or truncate –s <size>
filename ) why is this operation needed?
■ Copying a file?

Operating System Concepts – 9th Edition 11.34 Silberschatz, Galvin and Gagne ©2013
#include<stdio.h>
void main() {
FILE *fp;
fp = fopen(”OS.txt", "w");


fclose(fp);
}

Operating System Concepts – 9th Edition 11.35 Silberschatz, Galvin and Gagne ©2013
Two ingredients for performing these
operations: open, close

Operating System Concepts – 9th Edition 11.36 Silberschatz, Galvin and Gagne ©2013
Open and Close Files
■ Open(Fi)
● search the directory structure on disk for entry Fi (file name)
● Checks privilege of the process to access file
● Creates an entry in array of open files in the OS memory with
reference to the file
● The entry also contains the current r/w position in the file
● Returns index of this file in the array – your file descriptor

■ Close (Fi)
● That particular space for entry in your in-memory open file table
can be reused

Operating System Concepts – 9th Edition 11.37 Silberschatz, Galvin and Gagne ©2013
Open Files: Data structures needed
■ Open(Fi)
● search the directory structure on disk for entry Fi (file name)

● Checks privilege of the process to access file

● Creates an entry in array of open files in the OS memory with


reference to the file

● The entry also contains the current r/w position in the file

● Returns index of this file in the array – your file descriptor

Operating System Concepts – 9th Edition 11.38 Silberschatz, Galvin and Gagne ©2013
Open Files: Data structures needed
■ Open(Fi)
● search the directory structure on disk for entry Fi (file name)

● Checks privilege of the process to access file


Access rights: per-process
access mode information

● Creates an entry in array of open files in the OS memory with


reference to the file
Open-file table: tracks open files
(per process or system?)

● The entry also contains the current r/w position in the file
File pointer: pointer to last read/write location,
per process that has the file open
● Returns index of this file in the array – your file descriptor

Operating System Concepts – 9th Edition 11.39 Silberschatz, Galvin and Gagne ©2013
Close Files: Data structures needed
■ Close (Fi)
● That particular space for entry in your in-memory open file table
can be reused
● Requirement: OS should not close a file (remove the entry
from main memory) even if at least one process has it open

Operating System Concepts – 9th Edition 11.40 Silberschatz, Galvin and Gagne ©2013
Close Files: Data structures needed
■ Close (Fi)
● That particular space for entry in your in-memory open file table
can be reused
● Requirement: OS should not close a file (remove the entry
from main memory) even if at least one process has it open

File-open count: counter of number of times a


file is open – to allow removal of data from
open-file table when last processes closes it

So open file table entry will not be removed till


this count is 0

Operating System Concepts – 9th Edition 11.41 Silberschatz, Galvin and Gagne ©2013
Accessing Files after opening
■ File pointer moves in the file to last r/w location. How?
■ Sequential Access
read next
write next
reset

■ Direct Access – file is fixed length logical records (like an


array)
read n
write n
rewrite n
n = relative block number (like an array index)

■ Relative block numbers allow OS to control which disk


block the user can start r/w from
Operating System Concepts – 9th Edition 11.42 Silberschatz, Galvin and Gagne ©2013
Simulation of Sequential Access on
Direct-access File

Operating System Concepts – 9th Edition 11.43 Silberschatz, Galvin and Gagne ©2013
Basic set of “DIRECTORY” Operations
■ Six basic operations
● Search for a file (command: find)
● Create a file (command: mkdir, touch)
● Delete a file (command: rm –rf )
● List a directory (command: ls)
● Rename a file (command: rename)
● Traverse the file system (command: cp -r )

Operating System Concepts – 9th Edition 11.44 Silberschatz, Galvin and Gagne ©2013
How should the OS efficiently manage a persistent
device (your hdd / ssd)? Files, directory structures,
mounting
● What are data read/write abstractions?
● What are the APIs? Create, open, close, read
write etc.
● How to implement file system related functions?

Operating System Concepts – 9th Edition 11.45 Silberschatz, Galvin and Gagne ©2013
How should the OS efficiently manage a persistent
device (your hdd / ssd)? Files, directory structures,
mounting
● What are data read/write abstractions?
● What are the APIs? Create, open, close, read
write etc.
● How to implement file system related functions?

Operating System Concepts – 9th Edition 11.46 Silberschatz, Galvin and Gagne ©2013
How should the OS efficiently manage a persistent
device (your hdd / ssd)? Files, directory structures,
mounting
● What are data read/write abstractions?
● What are the APIs? Create, open, close, read
write etc.
● How to implement file system related functions?
4Implement open(), close() etc. API and
directory
4Allocate storage portion to files
4 Manage free space for efficient operation

Operating System Concepts – 9th Edition 11.47 Silberschatz, Galvin and Gagne ©2013
Background: Disk addressing
■ We have hard drive/ssd/thumb drives in our computer
and we want to use them via File systems
● Disk exports its data as a logical array of blocks
[0…N]

● These blocks are the smallest unit of transfer

● This is called Logical Block addressing (LBA)

● We will call these blocks on secondary storage as


“physical blocks”

Operating System Concepts – 9th Edition 11.48 Silberschatz, Galvin and Gagne ©2013
Background: Logical structure
of file system
■ File structure
● Logical storage unit
● Collection of related information
■ File system resides on secondary storage (disks)
● Provided user interface to storage, mapping logical to physical
● Provides efficient and convenient access to disk by allowing data to
be stored, located retrieved easily
■ Disk provides in-place rewrite and random access
● I/O transfers performed in blocks of sectors (usually 512 bytes)
■ File system organized into layers

Operating System Concepts – 9th Edition 11.49 Silberschatz, Galvin and Gagne ©2013
Background: Layers of a File System

All book keeping metadata


Process A open file X etc.

Map logical blocks to


Physical blocks

Retrieve and store raw


blocks of data

Device drivers, r/w data


to/from controller cards

■ File control block – storage structure consisting of information about a file


■ Device driver controls the physical device

Operating System Concepts – 9th Edition 11.50 Silberschatz, Galvin and Gagne ©2013
Implement API:
Data structures

Operating System Concepts – 9th Edition 11.51 Silberschatz, Galvin and Gagne ©2013
On-Disk data structures
■ Boot control block
● One per partition
● Information (address) of the boot loader (assembly code)
● Boot loader loads OS code/data in the volume
● Usually first block of the volume, may be empty
■ Volume control block (superblock, master file table)
● One per partition
● Total # of blocks, block size, # of free blocks, block size, free block
pointers, free File control block (FCB) or inode count, FCB/inode
pointer array
■ Directory structure
● One per partition
● Unix: File names and associated inode numbers,
● NTFS: same thing, stored as master file table

Operating System Concepts – 9th Edition 11.52 Silberschatz, Galvin and Gagne ©2013
On-Disk data structures (contd.)
■ File control block
● One per file
● Unique identifier (Unix inode) number
● permissions, size, dates
● NFTS stores as part of in master file table using relational DB
structures (one row, one file)

Operating System Concepts – 9th Edition 11.53 Silberschatz, Galvin and Gagne ©2013
In-memory data structures
■ Mount Table
● Contains information about each mounted volume
■ Directory-Structure Cache
● Holds directory information about recently accessed directories
■ System-Wide Open-File Table
● Contains a copy of the FCB, i.e., inode, of each open file
■ Per-Process Open-File Table
● Contains a pointer to the appropriate entry in the System-Wide
Open-File Table
■ I/O Memory Buffers
● Holds file-system blocks while they are being read from or written to
disk

Operating System Concepts – 9th Edition 11.54 Silberschatz, Galvin and Gagne ©2013
Implement API:
create, open, read, close

Operating System Concepts – 9th Edition 11.55 Silberschatz, Galvin and Gagne ©2013
create() : Implementation

■ Mechanism for create():


● Application calls logical file system
● Logical file system allocates a new FCB/inode
4Allocated from free memory
4selected from a pre-allocated set of free-FCB, i.e.,
free-inodes
● OS reads corresponding (to the create call ) directory in-
memory
● updates directory with the new file name and FCB
● writes it back directory to the disk

Operating System Concepts – 9th Edition 11.56 Silberschatz, Galvin and Gagne ©2013
open() : Implementation
■ Mechanism for open():
● open() call passes a file name to the logical file system
● Searches system-wide open-file table to check if the file is already in use by
another process
4 Yes: a per-process open-file table entry created pointing to existing
system-wide open-file table
§ No: directory structure is searched for the given file name (use directory
caches). Once file is found, FCB is copied into a system-wide open-file
table (with #process that called open()). Then same as the "Yes" case.
● return a pointer to entry in per-process open-file table (the “file descriptor”)

Operating System Concepts – 9th Edition 11.57 Silberschatz, Galvin and Gagne ©2013
read() : implementation
■ Mechanism for read():
● read() call passes the file descriptor (index in per-process open-file table)
● The per-process open-file table entry at index points to the system-wide
open-file table entry
● The system-wide open-file table entry contain the FCB
● Pass this information to the file-organization module and the layers below

Operating System Concepts – 9th Edition 11.58 Silberschatz, Galvin and Gagne ©2013
close(): implementation
■ Mechanism for close():
● per-process open-file table entry removed
● system-wide open-file table-entry’s open count is decreased by 1
● If the system-wide open-file table-entry’s open count is 0
4 updated metadata is copied back to secondary storage directory
4 the system-wide open-file table entry is removed

Operating System Concepts – 9th Edition 11.59 Silberschatz, Galvin and Gagne ©2013
Implementation note
■ open(), close(), read(), write() etc. are nice understandable system calls
● OS like Unix/Linux provide similar system calls also for other things like
networking or basically any device your connect
● system-wide open-file table-entries are not only FCB (i.e., inode) but also
similar information for network connections and devices
● Thus the adage: “Everything is a file” in Unix/Linux
● More: https://en.wikipedia.org/wiki/Everything_is_a_file

■ Why cache
● Most OS keep all information about an open file in memory, except for the
actual data blocks
● in-memory buffer caches for both read() and write()
● Average FS cache hit rate of Unix is 85%!

Operating System Concepts – 9th Edition 11.60 Silberschatz, Galvin and Gagne ©2013
Implement Directory

Operating System Concepts – 9th Edition 11.61 Silberschatz, Galvin and Gagne ©2013
Directory Implementation
■ Challenge: Map the file name (i.e., full path to the file with directory names)
to the physical data blocks on secondary storage

■ Linear list of file names with pointer to the data blocks


● Simple to program
● Time-consuming to execute
4 Linear search time
4 Could keep ordered alphabetically via linked list or use B+ tree

■ Hash Table – linear list + Extra hash data structure


● Decreases directory search time
● Collisions – situations where two file names hash to the same location
● Only good if entries are fixed size, or use chained-overflow method

Operating System Concepts – 9th Edition 11.62 Silberschatz, Galvin and Gagne ©2013
How should the OS efficiently manage a persistent
device (your hdd / ssd)? Files, directory structures,
mounting
● What are data read/write abstractions?
● What are the APIs? Create, open, close, read
write etc.
● How to implement file system related functions?
4Implement open(), close() etc. API and
directory
4Allocate storage portion to files
4 Manage free space for efficient operation

Operating System Concepts – 9th Edition 11.63 Silberschatz, Galvin and Gagne ©2013
File allocation methods

■ An allocation method refers to how disk blocks are allocated for


files:
● Contiguous allocation
4 Extent based allocation

● Linked allocation
4 FAT tables

● Indexed allocation
4 Multi-level indexed allocation

Operating System Concepts – 9th Edition 11.64 Silberschatz, Galvin and Gagne ©2013
How to know good or bad?

■ Fragmentation
● External: lots of holes
● Internal: Whatever is allocated is not used

■ Can you grow the file over time?

■ Complexity of reading the files across blocks

■ Easy implementation

■ Storage overhead

Operating System Concepts – 9th Edition 11.65 Silberschatz, Galvin and Gagne ©2013
1. Contiguous allocation

■ Allocate files like continuous memory


● Use base and limit values
● User specifies length, file system allocates space all at once
in continuous blocks
● Metadata: start of the file and size

Operating System Concepts – 9th Edition 11.66 Silberschatz, Galvin and Gagne ©2013
Contiguous Allocation: Schematic

Assign each file a set of contiguous blocks on the disk

Operating System Concepts – 9th Edition 11.67 Silberschatz, Galvin and Gagne ©2013
Contiguous Allocation: Good and Bad
■ Good
● Easy to implement
● Low storage overhead
4 only storing starting location (block #) and length (number of
blocks) are required
● Fast sequential access: continuous blocks
● Fast random access: accessing just an array index

■ Bad
● Wastes a lot of space
4 External fragmentation
4 Compaction (and related downtime)
● Files cannot grow
4 Subsequent blocks may be occupied by other files

Operating System Concepts – 9th Edition 11.68 Silberschatz, Galvin and Gagne ©2013
Contiguous Allocation: Mapping
■ Mapping from logical address (LA) to physical
● Logical address: relative to file, e.g., access LA’th byte in the file
● Physical address: the actual physical block number on storage
● Each physical block = 512 byte

Q = LA // 512
R = LA mod 512

Physical Block to be accessed = Q + starting address


Displacement into block = R

Operating System Concepts – 9th Edition 11.69 Silberschatz, Galvin and Gagne ©2013
1.1. Extent-based Allocation
■ Extent-based file systems allocate disk blocks in
extents
• An extent is a contiguous fixed-size disk-block
• Extents are allocated for file allocation
• A file consists of one or more extents

■ File = set of extents


• Metadata: Small array of entries pointing to extent
start address

Operating System Concepts – 9th Edition 11.70 Silberschatz, Galvin and Gagne ©2013
Extent Allocation: Good and Bad
■ Good
● Easy to implement
● Low storage overhead (a few entries to specify extents)
● File can grow (unless run out of extents)
● Fast sequential access
● random access: Simple to calculate extent number and offset

■ Bad
● Internal fragmentation
● External fragmentation can still happen when files grow and shrink
dynamically

Operating System Concepts – 9th Edition 11.71 Silberschatz, Galvin and Gagne ©2013
File allocation methods

■ An allocation method refers to how disk blocks are allocated for


files:
● Contiguous allocation
4 Extent based allocation

● Linked allocation
4 FAT tables

● Indexed allocation
4 Multi-level indexed allocation

Operating System Concepts – 9th Edition 11.72 Silberschatz, Galvin and Gagne ©2013
2. Linked Allocation
o Linked allocation – each file a linked list of blocks
o Directory entry = address of first block
o File ends at nil pointer

Operating System Concepts – 9th Edition 11.73 Silberschatz, Galvin and Gagne ©2013
Linked Allocation: Good and Bad
■ Good
● No compaction, external fragmentation
● Files can easily grow
● Easy to implement but need to space space per block!

■ Bad
● Storage overhead for pointers
● Linear search -- Locating a block can take many I/Os and disk
seeks, slow sequential
● Difficult to compute block number for random access
● Reliability : Pointers can be broken

■ Improve efficiency by clustering blocks into groups but increases


internal fragmentation (wasted allocated space)

Operating System Concepts – 9th Edition 11.74 Silberschatz, Galvin and Gagne ©2013
Linked Allocation : Mapping
■ Each file is a linked list of disk blocks: blocks may be scattered
anywhere on the disk
block = pointer

■ Pointer size = 4 bytes


● Mapping from logical to physical
Q = LA // 508
R = LA mod 508
Block to be accessed is the Qth block in the linked chain of blocks
representing the file.

Displacement into block = R + 4

Operating System Concepts – 9th Edition 11.75 Silberschatz, Galvin and Gagne ©2013
2.2. Special Linked allocation: FAT
■ FAT (File Allocation Table)
● Idea: Store linked-list pointers outside block in File
Allocation Table
● Beginning of volume has table, indexed by block number
● Much like a linked list, but faster on disk and cacheable
● New block allocation simple

■ FAT (File Allocation Table) working


● Directory contains first block number for a file
● FAT entry indexed by the block number == next block
● Go to next block and again consult FAT for next block
● Last block == special EOF value in FAT entry
Operating System Concepts – 9th Edition 11.76 Silberschatz, Galvin and Gagne ©2013
File-Allocation Table

Operating System Concepts – 9th Edition 11.77 Silberschatz, Galvin and Gagne ©2013
File-Allocation Table

Question: you have a 512 GB hard disk and each block size
is 4 KB. If your File system contains FAT, what is the smallest
amount of memory used for FAT?

Operating System Concepts – 9th Edition 11.78 Silberschatz, Galvin and Gagne ©2013
File-Allocation Table

Question: you have a 512 GB hard disk and each block size
is 4 KB. If your File system contains FAT, what is the smallest
amount of memory used for FAT? = 27 bits * 2^27 blocks

Operating System Concepts – 9th Edition 11.79 Silberschatz, Galvin and Gagne ©2013
FAT: Good and Bad
■ Good
● Fast random access, only search cached FAT in memory

■ Bad
● Storage overhead for FAT
● Slow sequential access, repeated FAT lookup

Operating System Concepts – 9th Edition 11.80 Silberschatz, Galvin and Gagne ©2013
File allocation methods

■ An allocation method refers to how disk blocks are allocated for


files:
● Contiguous allocation
4 Extent based allocation

● Linked allocation
4 FAT tables

● Indexed allocation
4 Multi-level indexed allocation

Operating System Concepts – 9th Edition 11.81 Silberschatz, Galvin and Gagne ©2013
3. Indexed Allocation

■ Indexed allocation
● Each file has its own index block(s) of pointers to its data blocks
● Directory entry points to index block
● entry i in the index block points to the i-th physical block for the file

■ Schematic view

Index block Actual file blocks

index table

Operating System Concepts – 9th Edition 11.82 Silberschatz, Galvin and Gagne ©2013
Example of Indexed Allocation

Operating System Concepts – 9th Edition 11.83 Silberschatz, Galvin and Gagne ©2013
Indexed Allocation (Cont.)

■ Need index table

■ Allow random access without external fragmentation


● Dynamic increase is easy, but have overhead of index block

■ Mapping from logical to physical in a file of maximum size of 64K


bytes, block size of 512 bytes and entry size 32 bits
● We need only 1 block for index table. Why?

Q = LA // 512
R = LA mod 512

Q = displacement into index table


R = displacement into block

Operating System Concepts – 9th Edition 11.84 Silberschatz, Galvin and Gagne ©2013
Indexed allocation: Good and Bad
■ Good
● Easy to implement
● No external fragmentation
● Files can be easily grown (but has a maximum limit)
● Fast random access, use index block

■ Bad
● Storage overhead for index
● Slow sequential access, repeated index table lookup

Operating System Concepts – 9th Edition 11.85 Silberschatz, Galvin and Gagne ©2013
A slight improvement : Linked scheme
■ Increase maximum size of file
■ Create an index block as a disk block
■ Then if necessary add a pointer to the end to another new index block
● Linked scheme – Link blocks of index table (no limit on size)

Operating System Concepts – 9th Edition 11.86 Silberschatz, Galvin and Gagne ©2013
3.2.Even better : multilevel index

Operating System Concepts – 9th Edition 11.87 Silberschatz, Galvin and Gagne ©2013
Multilevel index
■ First-level index block is a disk block which holds address for second-
level index block
■ If level = 2, then second-level index points to data blocks

■ What is the maximum size of files for two level index with 4kB sized
blocks and 4 byte pointers?
● Each 4KB index block, number of entries = (4 x 2^10 bytes) / 4
bytes = 2^10 pointers
● First index block = 2^10 pointers to 2^10 index blocks
● So, total #entries in all second level tables = 2^10 x 2^10
● Each entry in each second level table points to a 4 KB block
● So maximum file size = 2^10 x 2^10 x 4 KB = 4 GB

Operating System Concepts – 9th Edition 11.88 Silberschatz, Galvin and Gagne ©2013
Multilevel index
■ First-level index block is a disk block which holds address for second-
level index block
■ If level = 2, then second-level index points to data blocks
■ What is the maximum size of files for two level index with 4kB sized
blocks and 4 byte pointers?
● Each 4KB index block, number of entries = (4 x 2^10 bytes) / 4
bytes = 2^10 pointers
● First index block = 2^10 pointers to 2^10 index blocks
● So, total #entries in all second level tables = 2^10 x 2^10
● Each entry in each second level table points to a 4 KB block
● So maximum file size = 2^10 x 2^10 x 4 KB = 4 GB
■ Exercise: What would be the max. file size for n level index with 4m
byte block sizes and 4 byte pointer sizes? [Ans: 4 x m^(n+1)]
■ Exercise: Find the logical file specific byte to physical block
mapping in this two–level scheme. Assume a block size of 512
bytes and pointer size of 4 bytes. [ Hint: First find the block number
in 2nd index table ]

Operating System Concepts – 9th Edition 11.89 Silberschatz, Galvin and Gagne ©2013
Indexed allocation: Combined Scheme:
Used in UNIX
4K bytes per block, 32-bit addresses

More index blocks than can be addressed with 32-bit file pointer

Operating System Concepts – 9th Edition 11.90 Silberschatz, Galvin and Gagne ©2013
So which one to choose?
■ Best method depends on file access type
● Contiguous great for sequential and random
● Linked good for sequential, not random
● Declare access type at creation -> select either contiguous or
linked
■ Indexed more complex
● Single block access could require 2 index block reads then data
block read
● A hybrid is more preferable: contiguous for small files and indexed
for large files

Operating System Concepts – 9th Edition 11.91 Silberschatz, Galvin and Gagne ©2013
How should the OS efficiently manage a persistent
device (your hdd / ssd)? Files, directory structures,
mounting
● What are data read/write abstractions?
● What are the APIs? Create, open, close, read
write etc.
● How to implement file system related functions?
4Implement open(), close() etc. API and
directory
4Allocate storage portion to files
4 Manage free space for efficient operation

Operating System Concepts – 9th Edition 11.92 Silberschatz, Galvin and Gagne ©2013
Free-Space Management: Bit map
approach
■ File system maintains free-space list to track available blocks/clusters
● (Using term block for simplicity)
■ Bit vector or bit map (n blocks)

0 1 2 n-1

!"#
1 Þ block[i] free
bit[i] =
0 Þ block[i] occupied

Block number calculation

(number of bits per word) *


(number of 0-value words) +
offset of first 1 bit
CPUs have instructions to return offset within word with first 1 bit

Operating System Concepts – 9th Edition 11.93 Silberschatz, Galvin and Gagne ©2013
Free-Space Management (Cont.)
■ Bit map requires extra space
● Example:
block size = 4KB = 212 bytes
disk size = 240 bytes (1 terabyte)
n = 240/212 = 228 bits (or 32MB)
if clusters of 4 blocks -> 8MB of memory

■ Easy to get contiguous files

Operating System Concepts – 9th Edition 11.94 Silberschatz, Galvin and Gagne ©2013
Free-Space Management: Linked List
approach
■ Linked list (free list)
● No waste of space
● Expensive to traverse the list
● Hard for contiguous
assignment
● But OS often needs one free
block for a file
●The first one is returned

Operating System Concepts – 9th Edition 11.95 Silberschatz, Galvin and Gagne ©2013
Summary

§ What are data read/write abstractions?


Files, directory structures, mounting

§ What are the APIs?


Create, open, close, read write etc.

§ How to implement file system related functions?


§ open(), close() etc. API and directory
In memory and on disk data structures, FCB/inode,
open file tables, directory = Linear list + Hash table
§ Allocate storage portion to files
Contiguous (extent), linked (FAT),
indexed (multi-level index)

§ Manage free space for efficient operation


bitmap, linked list

Operating System Concepts – 9th Edition 11.96 Silberschatz, Galvin and Gagne ©2013
Extra concept: R/W optimization

■ Performance improvement
● Buffer cache – separate section of main memory for frequently
used blocks
● Synchronous writes sometimes requested by apps or needed
by OS
4 No buffering / caching – writes must hit disk before
acknowledgement
4 Asynchronous writes more common, buffer-able, faster
● Free-behind and read-ahead – techniques to optimize
sequential access
4 Remove a used block and read requested block +
subsequent blocks
● Reads frequently slower than writes.
4 Why? Read ahead

Operating System Concepts – 9th Edition 11.97 Silberschatz, Galvin and Gagne ©2013
Extra Concept: Recovery of File
systems
■ Why do you need recovery?
● Imagine your code has 4 system calls, first two creates a FCB and
add it to directory, second two writes file data to disk blocks
● Now what if system crash after first two?
● The FCB is there, but no file data à inconsistent state

■ How to deal with inconsistency?


● scan all data on directory and compares with data in each block to
determine consistency
● fsck in Linux, chkdsk in Windows
● Super slow
● May require human intervention to resolve conflicts

Operating System Concepts – 9th Edition 11.98 Silberschatz, Galvin and Gagne ©2013
Extra concept of File system security:
Protection
■ File owner/creator should be able to control:
● what can be done
● by whom
■ Types of access
● Read
● Write
● Execute
● Append
● Delete
● List

Operating System Concepts – 9th Edition 11.99 Silberschatz, Galvin and Gagne ©2013
Access Lists and Groups
■ Mode of access: read, write, execute
■ Three classes of users on Unix / Linux
RWX
a) owner access 7 Þ 111
RWX
b) group access 6 Þ 110
RWX
c) public access 1 Þ 001

■ For a particular file (say game) or subdirectory, define an


appropriate access.

Attach a group to a file


chgrp G game

Operating System Concepts – 9th Edition 11.100 Silberschatz, Galvin and Gagne ©2013
A Sample UNIX Directory Listing

■ Question: What is the meaning of chmod 777, chmod 644, chmod 700
■ Check : http://www.filepermissions.com/articles/what-are-file-permissions-
in-linux-and-mac

Operating System Concepts – 9th Edition 11.101 Silberschatz, Galvin and Gagne ©2013
Operating System Concepts – 9th Edition 11.102 Silberschatz, Galvin and Gagne ©2013

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy