Chapter 11: Persistence Beyond RAM (File I/O)

Learning Objectives

• Understand the difference between volatile RAM and persistent storage.
• Learn how programs negotiate with the Operating System to access hardware.
• Master reading and writing text files.
• Understand File Streams and Buffers.

Prerequisites

Chapter 10: Error Handling (crucial for dealing with missing files).

Why Does This Exist?

Every variable, array, and object we have created so far lives in RAM (Random Access Memory).

RAM is volatile. The moment your program ends, or the computer loses power, the electricity dissipates and everything is permanently erased.

If you build a word processor, the user needs their essay to survive a reboot.

We needed a way to write data onto physical, persistent storage media (Hard Drives or SSDs). We needed File I/O (Input/Output).

History

In the 1950s, persistence meant punching physical holes in paper cards or spinning magnetic tape reels.

As operating systems evolved, they had to create a unified abstraction. A programmer shouldn't need to write different code depending on whether the user has a Toshiba hard drive or a Samsung SSD.

The OS created the "File" abstraction. To your code, a file is just a continuous stream of bytes. The OS handles the incredibly complex physics of actually writing magnetic fields or trapping electrons on the hardware.

Mental Model

Think of your program (in RAM) as a person writing on a whiteboard. It's incredibly fast, but at the end of the day, the janitor wipes it clean.

Think of the Hard Drive as a massive, heavy filing cabinet in the basement.

To save your work, you cannot just throw your whiteboard into the basement. You must pick up a phone, call the basement clerk (the Operating System), and say: "Please open a new folder called data.txt, and write these exact words into it."

When you are done, you must explicitly tell the clerk: "I am done, close the folder."

Internal Working

When you ask to open a file, your program makes a "System Call" to the OS kernel.

1. The OS checks permissions (Does this user have the right to read this file?).

2. The OS locates the file on the physical disk hardware.

3. The OS hands your program a File Descriptor—a temporary ID badge that proves you have an open connection to that file.

When you write data, it doesn't instantly go to the disk (disks are slow). It goes into a Buffer in RAM. Only when you close() the file does the OS "flush" the buffer, physically writing the electrons to the SSD.

Syntax

In Python, the with open() syntax is the gold standard.

1# Writing to a file
2with open("scores.txt", "w") as file:
3    file.write("High Score: 999\n")
4
5# Reading from a file
6with open("scores.txt", "r") as file:
7    content = file.read()
8    print(content)

Token breakdown:

• with: A magical keyword called a Context Manager. It guarantees that when the indented block ends, it will automatically close() the file for you, even if the program crashes in the middle!
• "w": Write mode. (WARNING: This instantly deletes the old file and starts fresh).
• "r": Read mode.
• \n: The newline character. Files don't have visual lines; you must explicitly insert the "Enter key" character.

Visual Explanation

Tiny Example

Appending data instead of overwriting it.

1with open("log.txt", "a") as f:
2    f.write("User logged in.\n")

Walkthrough

• Line 1: The "a" stands for Append mode. Instead of destroying the file, the OS finds the very end of the existing file and places the cursor there.
• Line 2: Writes the new log entry.
• End of block: The with statement silently calls f.close(), flushing the buffer to the SSD.

Common Mistakes

1f = open("data.txt", "w")
2f.write("Hello")
3# Forgot f.close() !

Debugging

File I/O is the most error-prone part of programming. Files get deleted, permissions change, hard drives get full.

Professional code ALWAYS wraps file reads in a try/except block.

try:
    with open("config.json", "r") as f:
        pass
except FileNotFoundError:
    print("Using default settings.")

Mini Project

Time: 20 minutes.

Goal: The Guest Book.

Write a script that asks the user for their name using input(). Open a file called guests.txt in append mode ("a") and write their name to it, followed by a newline. Then, open the file in read mode ("r") and print the entire list of all past guests.

# Log file writer — demonstrating file I/O
import os
from datetime import datetime

LOG_FILE = "app.log"

def write_log(level, message):
    timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
    entry = f"[{timestamp}] [{level:7}] {message}
"
    
    # 'a' = append mode — each write adds to the end, not overwrites
    with open(LOG_FILE, "a") as f:
        f.write(entry)
    print(f"  Logged: {entry.strip()}")

def read_logs(level_filter=None):
    if not os.path.exists(LOG_FILE):
        print("  No log file found.")
        return
    
    with open(LOG_FILE, "r") as f:
        lines = f.readlines()
    
    print(f"
=== Log File ({len(lines)} entries) ===")
    for line in lines:
        if level_filter is None or f"[{level_filter}]" in line:
            print(f"  {line.strip()}")

# Simulate application events
write_log("INFO",    "Application started.")
write_log("INFO",    "User 'tarun' logged in.")
write_log("WARNING", "Memory usage at 80%.")
write_log("ERROR",   "Failed to connect to database.")
write_log("INFO",    "Retrying database connection...")
write_log("INFO",    "Database connection restored.")

read_logs()
read_logs(level_filter="ERROR")

# Clean up
os.remove(LOG_FILE)

Bigger Project

Time: 1 hour.

Goal: CSV Data Processor.

Create a text file containing rows of data: Name,Age,Score\nAlice,20,95\nBob,22,80. Write a script that reads this file line by line using a for line in file: loop. Split each line by the comma into an array. Calculate the average score of all students and print it.

# CSV-based contact book — file persistence with structured data
import csv
import os

CONTACTS_FILE = "contacts.csv"
FIELDNAMES = ["name", "email", "phone", "company"]

def load_contacts():
    if not os.path.exists(CONTACTS_FILE):
        return []
    with open(CONTACTS_FILE, "r", newline="") as f:
        reader = csv.DictReader(f)
        return list(reader)

def save_contacts(contacts):
    with open(CONTACTS_FILE, "w", newline="") as f:
        writer = csv.DictWriter(f, fieldnames=FIELDNAMES)
        writer.writeheader()
        writer.writerows(contacts)

def add_contact(name, email, phone, company=""):
    contacts = load_contacts()
    # Check for duplicate email
    if any(c["email"] == email for c in contacts):
        print(f"  ⚠️  Contact with email {email} already exists.")
        return
    contacts.append({"name": name, "email": email, "phone": phone, "company": company})
    save_contacts(contacts)
    print(f"  ✅ Added: {name}")

def search_contact(query):
    contacts = load_contacts()
    results = [c for c in contacts if query.lower() in c["name"].lower() or query in c["email"]]
    return results

def list_contacts():
    contacts = load_contacts()
    print(f"
=== Contact Book ({len(contacts)} contacts) ===")
    for c in sorted(contacts, key=lambda x: x["name"]):
        print(f"  {c['name']:20} {c['email']:30} {c['phone']}")

# Demo
add_contact("Tarun Sharma",    "tarun@example.com",   "+91-9876543210", "Tech Corp")
add_contact("Alice Johnson",   "alice@startup.io",     "+1-415-555-0100", "Startup Inc")
add_contact("Bob Williams",    "bob@enterprise.com",   "+44-20-7946-0001", "Enterprise Ltd")
add_contact("Charlie Chen",    "charlie@freelance.dev","+1-650-555-0199", "Freelance")
add_contact("Alice Johnson",   "alice@startup.io",     "duplicate")     # Should warn

list_contacts()

print("
=== Search: 'al' ===")
for c in search_contact("al"):
    print(f"  Found: {c['name']} ({c['email']})")

# Clean up
os.remove(CONTACTS_FILE)

Production Usage

At an enterprise level, raw text files are rarely used for complex data. Instead, engineers serialize objects into structured formats like JSON (JavaScript Object Notation).

Databases like PostgreSQL are ultimately just massive, highly optimized File I/O engines built on top of the exact same OS system calls you just learned.

Best Practices

• Paths: Never hardcode paths like C:\Users\Name\file.txt. Windows uses \, Mac/Linux use /. Use standard libraries (like Python's os.path or pathlib) to safely build paths that work on any computer.
• Read Chunking: If a file is 10 Gigabytes, file.read() will try to load all 10GB into RAM at once, crashing your computer. Always read massive files line-by-line using a loop, processing one tiny chunk at a time.

Interview Questions

Revision Sheet

Connections

• Previous Chapter: You must combine File I/O with try/except (Chapter 10) because disks are inherently unreliable.
• Future Chapters: We are creating and destroying a lot of data. Next, we will learn how the computer cleans up the RAM mess we leave behind (Memory Management, Chapter 12).