Working with Files and Paths

Programs become useful when they outlive the terminal session, and that means files: reading a CSV export, writing a JSON report, leaving scratch data behind safely. Chapter 15 of the Python for Beginners 24-chapter series covers the whole round trip — open() and its modes, why with is not optional, pathlib.Path as the modern way to talk about locations, JSON and CSV in both directions, encoding, and temporary files that clean up after themselves.

open(), modes and the with statement

open() returns a file object, and the mode decides what happens to a file that already exists:

Never leave a file open on the assumption the program will close it later. The with statement closes it the moment the indented block ends — even when an exception tears through it, which is why the context manager is guaranteed cleanup rather than a hopeful close() at the end. Text mode decodes bytes into str; binary mode hands back bytes, which is what images, audio and .zip files need. Use text for everything a human could read.

with open("notes.txt", "w", encoding="utf-8") as f:
    f.write("first line\n")
    f.write("second line\n")

with open("notes.txt", "a", encoding="utf-8") as f:
    f.write("third line\n")

with open("notes.txt", encoding="utf-8") as f:
    for line in f:
        print("|", line.rstrip())

Expected output:

| first line
| second line
| third line

The "a" block kept everything already in the file, and the loop shows the other essential pattern: iterating a file object yields one line at a time, a lazy read that costs almost no memory whatever the file’s size.

pathlib: paths as objects

String paths ("data/reports/" + name) break on the first folder separator you get wrong. The standard answer is pathlib.Path, an object that joins with the / operator and does the reading and writing itself:

from pathlib import Path

data = Path("data")
data.mkdir(exist_ok=True)
report = data / "report.txt"
report.write_text("generated by pathlib\n", encoding="utf-8")

print(report.exists())
print(report.read_text(encoding="utf-8").strip())
print(sorted(p.name for p in Path(".").glob("data/*.txt")))

Expected output:

True
generated by pathlib
['report.txt']

The useful surface is small: Path("a") / "b" joins, read_text() and write_text() open and close in one call, exists() and is_file() answer questions, glob("*.csv") pattern-matches, and mkdir(), rename() and unlink() manage the directory itself.

JSON and CSV round-trips

JSON is the lingua franca of web APIs: dumps() writes a string, loads() parses one, and dump() / load() do the same with file objects. Round-tripping a dictionary is three lines — and the file it leaves behind holds "active": true, because JSON has no True: keys are always strings, and the booleans and null are lowercase, in a syntax JavaScript developers will recognise from objects and JSON.

import json
from pathlib import Path

user = {"name": "Ada", "roles": ["admin", "editor"], "active": True}
Path("user.json").write_text(json.dumps(user, indent=2), encoding="utf-8")

loaded = json.loads(Path("user.json").read_text(encoding="utf-8"))
print(loaded == user, loaded["roles"][1])

Expected output: True editor. For spreadsheets, the standard csv module replaces manual splitting: pass newline="" so line endings are not doubled, and remember that everything comes back as a string.

import csv
from pathlib import Path

rows = [("name", "score"), ("Ada", 97), ("Linus", 88)]
with Path("scores.csv").open("w", newline="", encoding="utf-8") as f:
    csv.writer(f).writerows(rows)

with Path("scores.csv").open(newline="", encoding="utf-8") as f:
    for row in csv.DictReader(f):
        print(row["name"], row["score"])

Expected output:

Ada 97
Linus 88

writer accepts any iterable of values and DictReader turns each line into a dictionary keyed by the header row; convert with int(row["score"]) the moment you need arithmetic.

Encoding: utf-8 and UnicodeDecodeError

Files store bytes; text is an interpretation of them. Specify the encoding on both the write and the read and choose UTF-8, the default nearly everywhere. The failure mode is instructive — a file written in the old Latin-1 encoding, read as UTF-8:

# read_notes.py
with open("latin1.txt", encoding="utf-8") as f:
    print(f.read())

Expected output: the file was written with "café".encode("latin-1"), whose single byte \xe9 is not valid UTF-8 (the path shown is your own):

Traceback (most recent call last):
  File "/path/to/read_notes.py", line 3, in <module>
    print(f.read())
          ~~~~~~^^
  File "<frozen codecs>", line 325, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 3: unexpected end of data

The fix is to state the truth — Path("latin1.txt").read_text(encoding="latin-1") returns café — or, better, write UTF-8 from the start so no guessing is ever needed.

Temporary files and a worked example

Scratch files should never be your problem: tempfile creates them where the operating system expects throwaways and deletes them afterwards, even if your code raises.

import tempfile
from pathlib import Path

with tempfile.TemporaryDirectory() as tmp:
    scratch = Path(tmp) / "scratch.txt"
    scratch.write_text("temporary notes\n", encoding="utf-8")
    print(scratch.name, scratch.read_text(encoding="utf-8").strip())

print(scratch.exists())

Expected output: scratch.txt temporary notes then False — the folder is gone. Everything in this chapter, once, on a visits.csv of page,seconds rows:

# log_summary.py
import csv, json
from pathlib import Path

with Path("visits.csv").open(newline="", encoding="utf-8") as f:
    visits = list(csv.DictReader(f))

by_page = {}
for row in visits:
    by_page[row["page"]] = by_page.get(row["page"], 0) + 1

with Path("report.json").open("w", encoding="utf-8") as f:
    json.dump(by_page, f, indent=2, sort_keys=True)

print(Path("report.json").read_text(encoding="utf-8").strip())
print("pages:", len(by_page), "visits:", len(visits))

Expected output: a pretty-printed JSON object mapping "/" to 2, "/docs" to 2 and "/pricing" to 1, followed by pages: 3 visits: 5.

Common mistakes

• Opening files and never closing them — long-running scripts exhaust their file handle limit; use with, which closes on the way out even when an exception hits.
• Forgetting encoding="utf-8" — Python falls back to the platform default, so a file written on one machine can raise UnicodeDecodeError on another.
• Using "w" when you meant "a" — write mode silently truncates existing data to zero bytes; append mode preserves it.
• Joining paths with strings — "data" + "/" + name breaks on Windows and on folders that already end in a separator; use Path("data") / name.
• Calling read() on a huge file — it allocates the whole file in memory; iterate the file object line by line instead.

Practice checklist

1. Write three lines with open(..., "w"), append a fourth in append mode, then read the file back with a for loop.
2. Reproduce the pathlib example, then use Path(".").glob("**/*.py") to list every Python file beneath your working folder.
3. Round-trip a dictionary containing a nested list and a boolean through a JSON file, and check the file’s contents by eye.
4. Save a CSV with csv.writer, read it back with csv.DictReader, and print one column as an integer.
5. Recreate the latin1.txt traceback, then fix it by decoding with latin-1 and confirm the output is café.

What's next?

Data can now flow in and out of your programs. Next, give that data behaviour of its own — model real things with classes in Classes and Object-Oriented Programming.