Disks and Mounts: df, du, lsblk and mount

"No space left on device" is one of the most common production emergencies, and adding a data disk is one of the most common setup tasks. Both require the same vocabulary: block devices, partitions, filesystems and mount points. This lesson shows how to see how full each filesystem is, find what is eating the space, identify disks, and attach a new one so it survives a reboot.

df: Free Space per Filesystem

df reports every mounted filesystem. Use -h for human-readable units and -T to include the filesystem type.

bash

df -h

bash

Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        40G   31G  8.2G  80% /
tmpfs           2.0G     0  2.0G   0% /dev/shm
/dev/sdb1       200G   45G  155G  23% /data

bash

df -h /var/log        # which filesystem holds this path, and its usage
df -i /               # inode usage; "disk full" with free space means inodes ran out
df -hT                # include type (ext4, xfs, tmpfs)

A filesystem at 100% stops services from writing logs and databases from committing. Anything above 80–85% deserves attention.

du: Where the Space Went

du walks directories and sums file sizes. The combination with sort is the standard way to find the culprit.

bash

du -sh /var/log                            # total for one directory
du -sh /var/* 2>/dev/null | sort -rh | head -10
du -h --max-depth=1 /home | sort -rh       # one level down
du -ah /var/log | sort -rh | head -20      # include files, not just directories
du -sh --exclude="*.cache" ~/projects

Usual suspects: /var/log (rotate or delete old logs), /var/cache/apt (apt clean), /var/lib/docker (docker system prune), journal files (journalctl --vacuum-size=200M), and old kernels (apt autoremove).

If df says the disk is full but du cannot find the data, a process is holding a deleted file open. Find it and restart the process:

bash

sudo lsof +L1 | head            # open files with zero links (deleted)

Block Devices, Partitions and Filesystems

A block device is a disk: /dev/sda, /dev/nvme0n1, /dev/vdb on cloud VMs. It is divided into partitions (/dev/sda1, /dev/nvme0n1p2), each formatted with a filesystem (ext4, xfs, btrfs) and mounted at a directory. lsblk shows the whole tree.

bash

lsblk -f

bash

NAME    FSTYPE LABEL UUID                                 MOUNTPOINTS
sda
├─sda1  ext4         3f1a9c2e-7b44-4c1e-9a0f-2d6a1e8b5f10 /
└─sda2  swap         8c0d...                              [SWAP]
sdb
└─sdb1  ext4   data  9e7b...                              /data

Other identification commands:

bash

sudo fdisk -l                 # partition tables of every disk
sudo blkid                    # UUIDs and types
findmnt                       # tree of current mounts

Adding a New Disk

Cloud providers attach a volume as a bare device such as /dev/sdb or /dev/nvme1n1. Partition (optional but conventional), format, mount.

bash

lsblk                                   # confirm the new device has no partitions or filesystem
sudo parted /dev/sdb --script mklabel gpt mkpart primary ext4 0% 100%
sudo mkfs.ext4 -L data /dev/sdb1        # formatting erases everything on the partition
sudo mkdir -p /data
sudo mount /dev/sdb1 /data
df -h /data

Making Mounts Permanent with /etc/fstab

A manual mount disappears at reboot. /etc/fstab lists filesystems to mount at boot. Reference disks by UUID, because device names such as /dev/sdb can change between boots.

bash

sudo blkid /dev/sdb1
# /dev/sdb1: LABEL="data" UUID="9e7b3c1d-..." TYPE="ext4"

bash

# /etc/fstab
# <device>                                   <mount>  <type>  <options>          <dump> <pass>
UUID=9e7b3c1d-1a2b-4c3d-8e9f-0a1b2c3d4e5f   /data    ext4    defaults,nofail    0      2

nofail lets the system boot even if the disk is missing, which matters on cloud VMs. Test with sudo mount -a; an error here would otherwise leave the machine stuck at boot.

Unmounting, Other Mounts and Swap

bash

sudo umount /data                  # note: umount, not unmount
sudo umount -l /data               # lazy: detach now, clean up when no longer busy
sudo fuser -mv /data               # who is using it, if umount says "target is busy"

sudo mount -o loop ubuntu.iso /mnt/iso          # mount an image file
sudo mount -t nfs 10.0.0.5:/exports/media /mnt/media
sudo mount -o remount,ro /                       # remount read-only (recovery)

Swap

Swap is disk space used when RAM runs out. Small cloud VMs often ship without it; a swap file is the quick fix.

bash

sudo fallocate -l 2G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab
free -h

Common Mistakes

Running mkfs on the wrong device; always confirm with lsblk and prefer the partition (sdb1) over the disk (sdb).
Using /dev/sdX in fstab instead of a UUID, then booting into recovery after a device is renumbered.
Editing fstab without mount -a to test.
Deleting a log that a process still has open, freeing nothing until the process restarts; truncate instead with : > file.
Confusing df (filesystem view) with du (directory view) when the two disagree.

Key Takeaways

df -h shows usage per filesystem; df -i catches inode exhaustion.
du -sh dir/* | sort -rh finds what is consuming space; lsof +L1 finds deleted-but-open files.
Disks are /dev/sdX or /dev/nvmeXnY, partitions add a number, and lsblk -f maps them to mount points.
New storage: partition, mkfs, mount, then a UUID line with nofail in /etc/fstab verified by mount -a.
Use umount (no "n"), fuser -mv for busy targets, and a swap file on small VMs.

Next lesson: Scheduling Tasks with cron and systemd Timers — run backups, cleanups and reports automatically on a schedule.