Building Paths

Building Paths#

Working with Paths in Python: os module vs Shell Commands

by Silvia Mazzoni, DesignSafe, 2025

When writing scripts that manipulate files or navigate directories, it’s critical to use tools that are portable, robust, and easy to maintain across systems.

On DesignSafe (and generally on HPC systems), your scripts might run on JupyterHub, a virtual machine, or a batch environment—so writing path operations in pure Python ensures they behave consistently everywhere.

import os

Building Paths with `os.path.join()`#

When you’re working with files and folders in Python, you’ll often need to build complete paths from smaller parts — like combining a base directory with a filename.

Instead of writing paths as long strings, Python provides a better tool: os.path.join().

Why Use `os.path.join()`?#

Manually typing out full paths like this:

file_path = "/home/user/data/simulation/input.txt"

might seem fine, but it’s fragile and system-dependent:

You might mistype a slash (/ or \)
It won’t work across platforms (e.g., Windows vs Linux)
It’s hard to maintain or reuse if directory names change

Using os.path.join() solves all of that:

base_dir = "/home/user/data/simulation"
filename = "input.txt"

file_path = os.path.join(base_dir, filename)
print(file_path) # it even fixes all the / and \

/home/user/data/simulation/input.txt

project_path = os.path.join('/home/jupyter', 'CommunityData', 'OpenSees')
print("Project path:", project_path)

Project path: /home/jupyter/CommunityData/OpenSees

Using os.path.join() helps your scripts work reliably across platforms, and makes your code easier to maintain, debug, and reuse — especially in Jupyter notebooks, HPC workflows, or automation pipelines.

Troubleshooting common mistakes#

~ is not automatically expanded in os.path.join.
os.chdir only affects the current Python process. It doesn’t change your system’s terminal session.
Mixing hard-coded slashes (‘/’ or ‘') makes scripts fragile across systems. Always use os.path.join.

os.path.join('~', 'MyData')  # WRONG: stays literally as ~/MyData

'~/MyData'

os.path.exists(os.path.join('~', 'MyData'))

False

# Instead:
os.path.join(os.path.expanduser('~'), 'MyData')  # CORRECT

'/home/jupyter/MyData'

os.path.exists(os.path.join(os.path.expanduser('~'), 'MyData'))

True

Split Paths with os.path.split()#

When working with file paths in Python, it’s often useful to break them apart into their components—for example, separating a filename from its directory, or stripping off a file extension. This is especially important when building automated workflows that need to modify, move, or analyze files dynamically. The os.path module provides a set of reliable and portable tools for splitting paths into meaningful parts, so you can easily manage files without manually parsing strings. Below are some common functions that make this easy and consistent across different operating systems.

Use these when you want to programmatically handle parts of a path—e.g., checking extensions, extracting filenames, or navigating up directories.

`os.path.split(path)`#

Splits a path into two parts: the head (directory path) and the tail (final component like filename or last folder).

path = '/home/jupyter/Work/myfile.tcl'
head, tail = os.path.split(path)
print(head)  # → /home/jupyter/Work
print(tail)  # → myfile.tcl

/home/jupyter/Work
myfile.tcl

`os.path.splitext(path)`#

Splits the path into the filename and extension.

path = '/home/jupyter/Work/myfile.tcl'
head, tail = os.path.split(path)
print('head: ',head)  # → /home/jupyter/Work
print('tail: ',tail)  # → myfile.tcl

filename, ext = os.path.splitext(tail)
print('filename: ',filename)  # → model
print('ext: ',ext)       # → .inp

head:  /home/jupyter/Work
tail:  myfile.tcl
filename:  myfile
ext:  .tcl

`os.path.basename(path)`#

Returns just the last component of the path (like the filename).

os.path.basename('/home/jupyter/Work/myfile.tcl')  

'myfile.tcl'

`os.path.dirname(path)`#

Returns just the directory part, without the final file or folder.

os.path.dirname('/home/jupyter/Work/myfile.tcl')  

'/home/jupyter/Work'

os.path.normpath()#

Removes the trailing slash (/) if there is one, so basename() returns the actual last folder name rather than an empty string.

path = "/home/user/projects/myproject/"
normpath = os.path.normpath(path)
print('normpath: ',normpath)  # → myproject

normpath:  /home/user/projects/myproject

os.path.basename() (for strings)#

Returns the very last folder in a path (also called the basename of the directory path).

path = "/home/user/projects/myproject/"
last_folder = os.path.basename(os.path.normpath(path))
print('last_folder:',last_folder)  # → myproject

last_folder: myproject

pathlib path.name#

it is more modern & flexible

from pathlib import Path

path = Path("/home/user/projects/myproject/")
last_folder = path.name
print(last_folder)  # → myproject

myproject

Modern alternative: pathlib#

Python’s pathlib module (available since 3.4) provides an object-oriented way to handle paths. It does everything os.path does, but often more elegantly.

from pathlib import Path

# Home directory
home = Path.home()
print(home)  # /home/jupyter

# Build a path
project = home / 'CommunityData' / 'OpenSees'
print(project)  # /home/jupyter/MyData/OpenSees

# List files
for file in project.iterdir():
    print(file)

# Absolute path
print(project.resolve())

/home/jupyter
/home/jupyter/CommunityData/OpenSees
/home/jupyter/CommunityData/OpenSees/TrainingMaterial
/home/jupyter/CommunityData/OpenSees

Path.resolve() Method#

The resolve() method seen in the cell above is part of Python’s modern pathlib module, and it’s used to: Get the absolute, canonical path of a file or directory.

When you call .resolve() on a Path object:

It converts a relative path to an absolute path
It resolves any symbolic links (symlinks)
It cleans up .. and . from the path

Example#

from pathlib import Path

p = Path("myfolder/../data/file.txt")
print(p.resolve())

This might return:

/home/user/data/file.txt

Even though the original path included ../, resolve() normalizes it.

Why Use It?#

Ensures the path is absolute
Makes it easier to compare paths reliably
Ensures you’re working with the actual file location, especially if symlinks are involved

Note#

If the file doesn’t exist, resolve() may raise an error (in Python < 3.6), or simply return the cleaned-up path (in newer versions).
You can use resolve(strict=False) to avoid errors if the file doesn’t exist.

Summary on resolve()#

Use .resolve() when you want the true, absolute location of a file or folder, and want to clean up messy or relative path expressions in a reliable, cross-platform way.

Quick cheat sheet: os.path vs pathlib#

compact cheat sheet box comparing os.path to pathlib for the most common operations, so your readers can see exactly how they map line by line.

Make sure to add this at the top if you haven’t already:

from pathlib import Path
import os

Operation	os.path & os	pathlib
Get current directory	os.getcwd()	Path.cwd()
Get home directory	os.path.expanduser(‘~’)	Path.home()
Join paths	os.path.join(a, b, c)	Path(a) / b / c
Make absolute path	os.path.abspath(path)	Path(path).resolve()
List files in directory	os.listdir(path)	Path(path).iterdir()
Change directory	os.chdir(path)	(no direct change, use Path objects instead to keep code clean and explicit)
Check if path exists	os.path.exists(path)	Path(path).exists()
Create a directory	os.mkdir(‘new_dir’)	Path(‘new_dir’).mkdir()

Examples#

# os.path
import os
os.path.join('/home', 'jupyter', 'MyData')
os.path.expanduser('~')
os.path.abspath('some/file.txt')

'/home/jupyter/MyData/_ToCommunityData/OpenSees/TrainingMaterial/training-OpenSees-on-DesignSafe/Jupyter_Notebooks/some/file.txt'

# pathlib
from pathlib import Path
Path('/home') / 'jupyter' / 'MyData'
Path.home()
Path('some/file.txt').resolve()

PosixPath('/home/jupyter/MyData/_ToCommunityData/OpenSees/TrainingMaterial/training-OpenSees-on-DesignSafe/Jupyter_Notebooks/some/file.txt')

What is posixpath?#

posixpath is an internal Python module that implements the functions of os.path specifically for POSIX-style systems, like Linux and macOS (and broadly, anything UNIX-like).

When you do:

import os
os.path.join('a', 'b')

you’re actually using os.path, which is an alias that points to the appropriate module for your operating system.

On Linux or macOS, os.path is backed by posixpath.
On Windows, os.path is backed by ntpath.

So when you import os or os.path, Python quietly does:

import posixpath as os.path  # on Linux/macOS
import ntpath as os.path     # on Windows

This means os.path automatically uses the correct conventions for your system’s paths:

/ separators on Linux/macOS (via posixpath).
\ separators on Windows (via ntpath).

Should I ever import posixpath directly?#

No — almost never. You should always use os.path (or better, pathlib) so your code stays portable.

If you import posixpath directly, your code will break on Windows because posixpath hardcodes / conventions.

Summary#

Module	What it does	Typical use
os.path	Portable, automatically selects posixpath or ntpath based on your OS	Always recommended
posixpath	Hardcoded for POSIX paths (/)	Only for very specialized internal or cross-platform logic
ntpath	Hardcoded for Windows paths ()	Same, rarely used directly

Choosing the Right Library for Files and Paths in Python#

Python offers multiple libraries for handling files and paths — os, os.path, shutil, and pathlib — and it’s not always obvious which one to use.

Here’s a breakdown to help you understand when and why to use each:

Library	Best For	Style	Returns	Notes
`os`	Changing directories, running shell commands	Procedural (older)	Strings	Works well with `os.path`
`os.path`	Portable path operations (joining, expanding `~`)	Procedural (older)	Strings	Platform-independent path logic
`shutil`	Copying, moving, or deleting files/folders	Procedural (higher-level)	N/A	Useful for workflows and automation
`pathlib`	Modern, object-oriented path handling	Object-oriented	`Path` objects	Easier, cleaner syntax for many tasks

`os.path` vs `pathlib`#

Both are great, but pathlib is newer, more readable, and recommended for most new code. Here’s a quick comparison:

`os.path` Style:#

import os
path = os.path.join(os.path.expanduser('~'), 'myfolder', 'file.txt')
print(path)

/home/jupyter/myfolder/file.txt

`pathlib` Style:#

from pathlib import Path
path = Path.home() / 'myfolder' / 'file.txt'
print(path)

/home/jupyter/myfolder/file.txt

pathlib makes paths act like objects — you can read, write, check existence, etc., directly on the path.

So When Should You Use Each?#

Use Case	Recommended Library
Writing new scripts with paths	`pathlib` ✅
Interfacing with older code or libraries	`os.path`
Changing directories or accessing env vars	`os`
Running shell commands	`os.system()` or `subprocess`
Copying/moving/removing files and folders	`shutil` ✅
Creating file-processing workflows	`shutil` + `pathlib`