Resolving Linux Error: “No Space Left on Device” When Disk Space Shows 50% Free (Inodes Full)

Root Cause Diagnosis

Every ext2/ext3/ext4 filesystem allocates a fixed number of inodes at format time, calculated from a bytes-per-inode ratio — each inode tracks metadata (permissions, timestamps, block pointers) for exactly one file or directory, independent of that file’s actual size. When a workload creates enormous numbers of tiny files — session caches, mail queue files, log rotation artifacts, temp files from a runaway process — the inode table can be fully consumed while the underlying data blocks remain largely empty, because inode count and disk space are tracked as two completely separate resource pools. The kernel returns ENOSPC (“No space left on device”) the moment the inode table is exhausted, regardless of how much block storage remains free.

Quick Fix

Confirm inode exhaustion and free up entries immediately by clearing the most common culprit — stale session files:

df -i

If IUse% reads 100% on the affected filesystem, purge the highest-risk directory first:

find /tmp -type f -atime +1 -delete

This targets untouched temp files older than one day, which is safe on most systems and frequently recovers enough inodes to restore write access.

Step-by-Step Resolution

1. Confirm the problem is inodes, not blocks.

df -h    # shows block (space) usage
df -i    # shows inode usage

If df -h reports free space but df -i shows IUse% at or near 100%, inode exhaustion is confirmed. Note the mount point in the Filesystem column — you need this for later steps.

2. Identify which directories hold the highest file counts.

Run a per-directory file count sweep from the filesystem root (adjust the path to the affected mount):

for dir in /*; do
  echo -n "$dir: "
  find "$dir" -xdev -type f 2>/dev/null | wc -l
done | sort -t: -k2 -n -r

The -xdev flag prevents find from crossing into other mounted filesystems, keeping the count scoped to the affected volume.

3. Drill into the worst offender to find the exact source.

Common high-inode-consumption locations:

find /var/spool/postfix/maildrop -type f | wc -l
find /var/lib/php/sessions -type f | wc -l
find /var/log -type f | wc -l
find /home/*/cache -type f | wc -l

Mail queues, PHP/application session directories, and application-level cache directories (npm, pip, Docker layer cache) are the most frequent causes on production servers.

4. Delete or archive the excess files.

For stale PHP sessions older than the configured session lifetime:

find /var/lib/php/sessions -type f -mmin +1440 -delete

For a stuck mail queue:

postsuper -d ALL deferred

For old rotated logs not caught by logrotate:

find /var/log -name "*.gz" -mtime +30 -delete
find /var/log -name "*.log.[0-9]*" -mtime +30 -delete

5. Check for a deleted-but-still-open file holding inodes/space.

A process can hold a file descriptor open on a file that’s been unlinked from the directory tree, which won’t show up in a normal find but still consumes an inode until the process closes it or is restarted:

lsof +L1

+L1 lists open files with a link count of less than 1 (i.e., deleted files still held open). Restart the owning process, or truncate the file descriptor directly if restart isn’t immediately possible:

: > /proc/<pid>/fd/<fd_number>

6. Once inodes are freed, verify write access is restored.

touch /path/on/affected/filesystem/testfile && rm /path/on/affected/filesystem/testfile

7. Address the root cause to prevent recurrence.

• Configure tmpwatch/tmpfiles.d cleanup rules for /tmp and application temp directories:

cat /etc/tmpfiles.d/tmp.conf

• Set log rotation to be more aggressive if /var/log was the culprit — edit /etc/logrotate.d/ entries to reduce rotate count or add maxage.
• For directories with legitimately high file counts (e.g., a queue or cache directory by design), plan to reformat with a higher inode allocation ratio during the next maintenance window, since inode count cannot be increased on an existing ext2/3/4 filesystem without reformatting.

Why This Fails for Some Users

• XFS filesystems: XFS allocates inodes dynamically rather than at format time, so df -i on XFS often shows very high or unlimited-looking inode counts. Inode exhaustion on XFS is rarer but still possible under extreme small-file workloads, and the fix is identical — locate and remove the file-count source — but you cannot “run out” of inodes in the traditional ext4 sense unless the filesystem itself is nearly full on blocks too.
• Docker/container hosts: inode exhaustion frequently traces back to /var/lib/docker/overlay2, where accumulated dangling image layers and unused containers each contribute thousands of small metadata files. docker system prune -a --volumes resolves this, but standard find-based sweeps of /var often miss it because the overlay filesystem mounts obscure per-layer file counts from a simple recursive count.
• Encrypted or LVM-thin-provisioned volumes: df -i reports inode usage for the mounted filesystem correctly, but if the underlying LVM thin pool itself is separately exhausted, freeing inodes at the filesystem level won’t resolve ENOSPC — check lvs -a -o+data_percent,metadata_percent for thin pool metadata exhaustion, which produces an identical symptom from a different layer.
• Filesystems formatted with a low -i bytes-per-inode ratio ages ago: servers provisioned years earlier with default mke2fs settings sized for large media files, then repurposed for workloads generating millions of small files (mail, session caches), hit this ceiling much earlier than expected relative to their actual disk size — checking tune2fs -l /dev/sdXN | grep -i inode reveals the original allocation ratio.
• NFS-mounted filesystems: df -i on an NFS client reflects the exported filesystem’s inode table on the server, not a locally adjustable value — the fix must be applied on the NFS server itself, and client-side cleanup commands will silently fail to free server-side inodes if permissions don’t allow deletion from the client.
• Snapshot-holding filesystems (Btrfs, LVM snapshots, ZFS): deleted files may not actually free their inodes/blocks if a snapshot still references them; rm succeeds but df -i doesn’t improve until the referencing snapshot is also removed (btrfs subvolume delete, lvremove on the snapshot, or zfs destroy on the snapshot dataset).