DSA, Database System & Operating System β Memory Management, File Systems & Administration, NEC licence examination syllabus (Nepal Engineering Council).
File System Implementation
The on-disk structures β boot block, superblock, inode table, data blocks β and the three allocation methods.
π Where this lives: when a filesystem is corrupted, recovery follows this layout exactly: fsck reads the superblock (or a backup copy, which is why filesystems keep several), validates the inode table against the block bitmap, then repairs directory entries. Understanding the layout makes "superblock corrupt, trying backup superblock" a meaningful message rather than a frightening one. It is also why formatting is fast and secure erasure is slow β mkfs writes only metadata, leaving your data physically intact. Search "ext4 superblock backup fsck recovery".
On-disk layout
A typical UNIX filesystem, in block order:
ββββββββββββββ¬βββββββββββββ¬ββββββββββ¬ββββββββββββ¬ββββββββββ
β BOOT BLOCK β SUPERBLOCK β BLOCK β INODE β DATA β
β β β BITMAP β TABLE β BLOCKS β
ββββββββββββββ΄βββββββββββββ΄ββββββββββ΄ββββββββββββ΄ββββββββββ
BOOT BLOCK block 0. Code to load the OS. Present even on
non-bootable filesystems, for uniformity.
SUPERBLOCK the filesystem's own metadata:
total blocks, block size, free-block count
inode count and free-inode count
pointers to the bitmaps and inode table
mount state, last-check time, magic number
β CRITICAL. Corrupt it and the filesystem is
unreadable, which is why BACKUP copies are
written at known offsets.
BLOCK BITMAP one bit per data block: free or allocated.
COMPUTED for a 1 TB disk with 4096-byte blocks:
1 TB / 4 KB = 268,435,456 blocks
bitmap = 268,435,456 bits = 32 MB
β 32 MB to track a terabyte. Cheap.
INODE TABLE a fixed array of inodes, sized at format time.
β THE CONSEQUENCE: a filesystem can run out of
INODES while having free space. Millions of
tiny files exhaust the table and writes fail
with ENOSPC even though `df` shows space free;
`df -i` reveals it. Modern filesystems (APFS,
btrfs, XFS) allocate inodes dynamically.
DATA BLOCKS the file contents.
IN-MEMORY STRUCTURES the OS also maintains:
mount table which filesystems are mounted where
system-wide open-file table one entry per open FILE
per-process open-file table one entry per file DESCRIPTOR
buffer cache / page cache cached blocks
THE TWO-LEVEL OPEN-FILE TABLE matters: two processes opening
the same file get separate descriptors with separate FILE
POINTERS but share one inode entry. That is why two processes
can read the same file at different positions β and why a
descriptor inherited across fork() SHARES the position, since
both point at the same system-wide entry.
MEASURED ON THIS MACHINE:
filesystem APFS
block size 4096 bytes
allocation 4096 bytes
The three allocation methods
1. CONTIGUOUS ALLOCATION
each file occupies a consecutive run of blocks. The directory
stores START BLOCK and LENGTH.
β FASTEST sequential access β no seeks between blocks
β direct access is trivial: block = start + n
β EXTERNAL FRAGMENTATION, exactly as with contiguous memory
β the file size must be known at creation
β growing a file may require moving it entirely
Used by CD-ROM/DVD (write-once, size known), and in a relaxed
form by extent-based filesystems.
2. LINKED ALLOCATION
each block contains a pointer to the next. The directory
stores only the FIRST block.
β NO external fragmentation β any free block will do
β files grow freely
β DIRECT ACCESS IS O(n): to reach block 50 you must read
blocks 0..49 to follow the chain
β a pointer consumes space in every block, so the usable size
is not a power of two β awkward for aligned I/O
β ONE CORRUPTED POINTER loses the rest of the file
FAT VARIANT: move all pointers into one File Allocation Table
at the start of the disk.
β the FAT can be cached, so traversal happens in memory
β direct access becomes feasible
β a single point of failure β hence FAT keeps two copies
3. INDEXED ALLOCATION
each file has an INDEX BLOCK listing all its block numbers.
β NO external fragmentation
β DIRECT ACCESS in O(1): look up entry n in the index
β the index block is overhead even for a tiny file
β a single index block LIMITS the file size
MAXIMUM FILE SIZE with one index block:
block 4 KB, pointer 4 bytes β 4096/4 = 1024 pointers
max file = 1024 Γ 4 KB = 4 MB
4 MB is far too small, which forces the multilevel scheme
below.
THE UNIX INODE β indexed allocation, made practical:
12 DIRECT pointers β 12 blocks
1 SINGLE INDIRECT pointer β a block full of pointers
1 DOUBLE INDIRECT pointer β a block of blocks of pointers
1 TRIPLE INDIRECT pointer β three levels
MAXIMUM FILE SIZE, 4 KB blocks, 4-byte pointers
(1024 pointers per block) β all values COMPUTED:
direct 12 Γ 4 KB = 48 KB
single indirect 1024 Γ 4 KB = 4 MB
double indirect 1024Β² Γ 4 KB = 4 GB
triple indirect 1024Β³ Γ 4 KB = 4 TB
βββββββββββββββββ
TOTAL β 4 TB + 4 GB + 4 MB + 48 KB
THE ELEGANCE: a small file uses only the 12 direct pointers,
so it costs ONE inode read and ONE data read β two I/Os. A
huge file remains addressable, paying extra indirection only
for its distant blocks.
ACCESS COST BY FILE POSITION:
first 48 KB 1 extra read (the inode) β fastest
up to 4 MB + 1 read (single indirect)
up to 4 GB + 2 reads (double indirect)
beyond + 3 reads (triple indirect)
A deliberate optimisation for the common case: most files are
small.
The inode's asymmetry is the design lesson: 12 direct pointers handle the overwhelming majority of files at minimum cost, while three levels of indirection make the rare huge file merely slower rather than impossible. Optimising the common case and degrading gracefully for the rest is the same principle as the TLB, the multilevel page table, and the B-tree.
Free-space management and consistency
FREE-SPACE TRACKING:
BIT VECTOR one bit per block. A hardware bit-scan finds the
first free block quickly.
32 MB tracks a 1 TB disk at 4 KB blocks.
LINKED LIST each free block points to the next free block.
β no extra space at all
β finding n contiguous blocks needs n disk reads
GROUPING the first free block holds the addresses of n
free blocks, the last pointing to the next
group. Fewer reads.
COUNTING store (first block, count) pairs, exploiting the
fact that free blocks are usually contiguous.
CONSISTENCY β the problem journaling solves:
Creating a file touches several structures:
1. allocate an inode (inode bitmap)
2. write the inode (inode table)
3. allocate data blocks (block bitmap)
4. write the data
5. add the directory entry
A crash between any two leaves the filesystem INCONSISTENT β
an allocated inode nothing points to (a leak), or a directory
entry pointing at a free inode (corruption).
fsck SCANS THE WHOLE FILESYSTEM to find and repair these.
β time proportional to filesystem SIZE, not to the damage.
Hours for a large volume.
JOURNALING writes the intended changes to a LOG first:
1. write "I am about to do steps 1β5" to the journal
2. commit the journal entry
3. perform the real updates
4. mark the journal entry complete
After a crash, replay any committed-but-incomplete entries.
β recovery time depends on the JOURNAL size, not the disk
size β seconds instead of hours
β this is EXACTLY the write-ahead logging of ACtE0704. Same
problem, same solution, different layer.
JOURNALING MODES (ext4):
journal metadata AND data journalled β safest, slowest
ordered metadata journalled, data written first
(the default)
writeback metadata journalled, data unordered β fastest,
can expose stale data after a crash
COPY-ON-WRITE FILESYSTEMS (ZFS, btrfs, APFS) take another
route: never overwrite in place. Write new blocks, then
atomically update the pointer. There is no inconsistent
intermediate state to repair, and snapshots become nearly free β
which is how APFS on this machine makes a snapshot instantly.
π Go further: the "out of inodes with free space" failure is worth seeing once. Fill a filesystem with empty files and writes fail with ENOSPC while df reports space available β only df -i shows inode exhaustion. It used to hit mail servers and build caches routinely. Modern filesystems fixed it by allocating inodes dynamically, one of the clearest examples of a static design decision becoming a production limit. Search "no space left on device inodes df -i".
π‘ Exam angle: draw the on-disk layout (boot block, superblock, bitmaps, inode table, data blocks) and say what the superblock holds. Compare the three allocation methods on external fragmentation, direct-access cost and growth. The maximum file size calculation for a UNIX inode is a guaranteed numerical β given block and pointer sizes, compute the direct, single, double and triple indirect contributions. Know why a filesystem can run out of inodes with free space, and explain journaling as write-ahead logging that makes recovery proportional to the journal rather than the disk.
Syllabus points
Allocation: contiguous, linked, indexed
Create a free account to tick topics off, take notes as you read, watch the video lessons and get a day-by-day study plan built around your exam date.
Related topics in Memory Management, File Systems & Administration