Python File Handling

Why File Handling Matters — Programs That Remember

In Python, a text file is a sequence of characters stored on disk. Files come in two flavours — text (readable characters like source code, CSVs, logs, JSON, HTML) and binary (images, executables, PDFs). This tutorial focuses entirely on text files, which cover 90% of everyday scripting needs.

The open() Function — Your Gateway to Every File

Every file operation starts with the built-in open() function. It takes a path and a mode, and hands back a file object that you can read from or write to.

# Basic syntax
f = open("notes.txt", "r", encoding="utf-8")
content = f.read()
f.close()
print(content)

File Modes — Read, Write, Append (and the Ones That Bite)

The second argument to open() is the mode. Pick the wrong one and you either can't do what you want — or worse, silently destroy an existing file.

Text Mode vs Binary Mode

Add "b" to any mode (like "rb", "wb") and you're in binary mode — you read and write raw bytes instead of decoded strings. Use binary for images, PDFs, and archives. For everything human-readable, stay in text mode.

The with Statement — The Only Right Way to Open Files

Manually calling close() is fragile. If an exception happens between open and close, the file stays open until the garbage collector gets around to it — leaking file handles, locking files on Windows, and losing buffered writes. The with statement (a context manager) guarantees the file gets closed the moment the block ends, even if an error is raised.

# The old, error-prone pattern - avoid
f = open("notes.txt", "r", encoding="utf-8")
data = f.read()
f.close()                        # skipped if read() raises

# The correct pattern - use this everywhere
with open("notes.txt", "r", encoding="utf-8") as f:
    data = f.read()
# f is guaranteed closed here, even if read() raised
print(data)

Reading Text Files — Four Techniques

Python gives you four ways to pull text out of a file. Which one to pick depends entirely on file size and how you want to process the content.

Example — All Four Techniques Side by Side

# Assume notes.txt contains:
#   Buy milk
#   Finish tutorial
#   Call mom

# 1) read() - everything as one string
with open("notes.txt", encoding="utf-8") as f:
    text = f.read()
    print(repr(text))

# 2) readline() - one line, keeps \n
with open("notes.txt", encoding="utf-8") as f:
    first = f.readline()
    print(repr(first))

# 3) readlines() - list of lines
with open("notes.txt", encoding="utf-8") as f:
    lines = f.readlines()
    print(lines)

# 4) Iterate the file - best for large files
with open("notes.txt", encoding="utf-8") as f:
    for i, line in enumerate(f, 1):
        print(f"{i}: {line.rstrip()}")

Writing Text Files — write() and writelines()

Writing is symmetric to reading. Two methods handle it: write() takes one string, writelines() takes an iterable of strings. Neither adds a newline for you — that's your job.

# write() - one string at a time. Add \n yourself.
with open("todo.txt", "w", encoding="utf-8") as f:
    f.write("Buy milk\n")
    f.write("Finish tutorial\n")
    f.write("Call mom\n")

# writelines() - takes an iterable. Still no auto-newlines!
tasks = ["Buy milk\n", "Finish tutorial\n", "Call mom\n"]
with open("todo2.txt", "w", encoding="utf-8") as f:
    f.writelines(tasks)

# Idiomatic pattern - join then write once
with open("todo3.txt", "w", encoding="utf-8") as f:
    f.write("\n".join(["Buy milk", "Finish tutorial", "Call mom"]))

Reminder — Mode "w" Wipes First

# Before running: todo.txt contains 100 important lines.
with open("todo.txt", "w", encoding="utf-8") as f:
    f.write("just this one line\n")

# After running: todo.txt contains ONE line. The other 100 are gone forever.

Appending — Mode "a"

To add to a file without destroying what's already in it, use append mode. The file pointer starts at the end and every write extends the file. This is the mode you want for logs, audit trails, and any incremental data collection.

import datetime

def log_event(message):
    now = datetime.datetime.now().isoformat(timespec="seconds")
    with open("app.log", "a", encoding="utf-8") as f:
        f.write(f"[{now}] {message}\n")

log_event("server started")
log_event("user aarav signed in")
log_event("processed 42 requests")

# Read it back
with open("app.log", encoding="utf-8") as f:
    print(f.read())

Encoding — The One Argument You Should Never Skip

A text file on disk is really just bytes. An encoding is the rule that turns those bytes into characters and vice versa. UTF-8 is the modern universal choice — it handles every language and script, and it's the default on nearly every platform except older Windows.

# Always the right choice for text files
with open("names.txt", encoding="utf-8") as f:
    for line in f:
        print(line.rstrip())

# Writing with explicit encoding
names = ["Sofía", "Zoë", "Björn", "王小明"]
with open("names.txt", "w", encoding="utf-8") as f:
    f.write("\n".join(names))

File Paths — Working with pathlib

Hard-coding paths as strings works — until it doesn't. Slashes vs backslashes, joining segments, checking existence, extracting extensions — the standard pathlib module handles it all cleanly and portably.

from pathlib import Path

# Build paths without worrying about \ vs /
data_dir = Path("data")
notes = data_dir / "notes.txt"
print(notes)                     # data/notes.txt (or data\notes.txt on Windows)

# Handy inspection methods
print("exists?  ", notes.exists())
print("is file? ", notes.is_file())
print("suffix:  ", notes.suffix)
print("stem:    ", notes.stem)
print("parent:  ", notes.parent)

# pathlib plays nicely with open()
with open(notes, encoding="utf-8") as f:
    text = f.read()

# Even simpler - Path has shortcuts
text = notes.read_text(encoding="utf-8")
notes.write_text("a new note", encoding="utf-8")

Error Handling — The Errors You Will Hit

File I/O has more failure modes than almost any other operation: the file doesn't exist, you don't have permission, the disk is full, the encoding is wrong, someone else has it locked. Handle the specific exceptions you care about — don't blanket-catch everything.

def load_config(path):
    try:
        with open(path, encoding="utf-8") as f:
            return f.read()
    except FileNotFoundError:
        print(f"Config not found at {path}, using defaults")
        return ""
    except PermissionError:
        print(f"No permission to read {path}")
        raise
    except UnicodeDecodeError as e:
        print(f"Encoding problem in {path}: {e}")
        raise

config = load_config("settings.ini")

Common Pitfalls — Bugs Everyone Writes Once

A Complete Real-World Example — A Word-Frequency Counter

Let's put every technique together. This script reads a text file, counts how often each word appears (case-insensitive, ignoring punctuation), writes the top 10 to a report file, and prints them. Notice how it uses with, explicit encoding, line-by-line iteration, and targeted error handling.

import string
from collections import Counter
from pathlib import Path

def count_words(source_path, report_path, top_n=10):
    counter = Counter()
    translator = str.maketrans("", "", string.punctuation)

    # READ - line by line, so file size does not matter
    try:
        with open(source_path, encoding="utf-8") as f:
            for line in f:
                cleaned = line.lower().translate(translator)
                counter.update(cleaned.split())
    except FileNotFoundError:
        print(f"Source file not found: {source_path}")
        return

    top = counter.most_common(top_n)

    # WRITE - overwrite the report file each run
    with open(report_path, "w", encoding="utf-8") as f:
        f.write(f"Top {top_n} words in {source_path}\n")
        f.write("=" * 40 + "\n")
        for rank, (word, count) in enumerate(top, 1):
            f.write(f"{rank:2d}. {word:15s} {count}\n")

    # APPEND - log this run to an audit file
    with open("runs.log", "a", encoding="utf-8") as f:
        total = sum(counter.values())
        f.write(f"Processed {source_path}: {total} words, {len(counter)} unique\n")

    # Show summary to caller
    print(f"Report written to {report_path}")
    for rank, (word, count) in enumerate(top, 1):
        print(f"  {rank:2d}. {word:15s} {count}")


count_words("article.txt", "report.txt", top_n=10)

Golden Rules