Files and paths

Read and write files without leaking handles, without corrupting text, and without building paths out of strings.

13of 24
≈ 12Min read
5Parts

Always with

open without with leaks a handle until the garbage collector happens to get to it, which on some runtimes is never. The with block closes it at the end of the block, however the block ends.

Always pass encoding=. Without it Python uses the platform default, so the same code reads correctly on your machine and produces mojibake on the server. utf-8 unless you have a reason.

py
with open("notes.txt", encoding="utf-8") as f:
    text = f.read()
# closed here — even if the body raised

with open("out.txt", "w", encoding="utf-8") as f:
    f.write("first\n")
    f.writelines(["second\n", "third\n"])   # note: no newlines added for you

The modes

Text mode decodes bytes to str and translates line endings; binary mode gives you the bytes exactly as they are. Images, archives and anything you are going to hash must be binary — a stray decode will corrupt them silently.

ModeMeans
"r"Read, text. The default
"w"Write — truncates the file to nothing first
"a"Append
"x"Create, and fail if it already exists — the safe one
"rb" / "wb"Bytes. No encoding, no newline translation
"r+"Read and write, without truncating

Read lazily

A file object is an iterator over its lines. for line in f never holds more than one line, which is the difference between a script that handles a 2 GB log and one that stops.

py
with open("huge.log", encoding="utf-8") as f:
    for line in f:                 # one line at a time, constant memory
        handle(line.rstrip("\n"))

text = f.read()                    # the whole thing, as one string
lines = f.readlines()              # the whole thing, as a list — same memory cost

pathlib, not string concatenation

Path objects compare, sort and print sensibly, and every standard-library function that takes a filename takes one. Joining paths with + and "/" is a portability bug waiting for the first Windows user.

py
from pathlib import Path

p = Path("data") / "2026" / "sales.csv"    # the / operator joins, correctly
p.name          # 'sales.csv'
p.stem          # 'sales'
p.suffix        # '.csv'
p.parent        # Path('data/2026')

p.exists(), p.is_file(), p.stat().st_size
p.parent.mkdir(parents=True, exist_ok=True)

p.read_text(encoding="utf-8")      # open, read, close — one call
p.write_text(data, encoding="utf-8")
p.read_bytes()

list(Path("data").glob("**/*.csv"))        # recursive

Structured data

newline="" on a CSV file is not optional — the module does its own line-ending handling, and without it you get blank rows on some platforms. And DictReader never converts types for you: everything arrives as text, including the numbers.

py
import json, csv, io

json.loads(text)                   # text  -> objects
json.dumps(obj, indent=2)          # objects -> text
with open("d.json", encoding="utf-8") as f:
    obj = json.load(f)             # no "s": straight from a file

with open("rows.csv", newline="", encoding="utf-8") as f:
    for row in csv.DictReader(f):  # every value is a STRING
        total += int(row["qty"])

Practice