Tmpfs

Okay, hear me out: tmpfs actually has some weird and surprising behavior sometimes and it's worth knowing how it actually works and what happens when it doesn't work or isn't used in the places you expect.

For those not in the know, tmpfs is used when you want to mount a directory to memory. It's an entire filesystem, but directly in memory. This has the added effect that, when you reboot your Linux machine, it's cleared out. You'll very, very often find that your /tmp directory is a tmpfs filesystem. While lesser-known, it's the same story with /dev/shm.

How much storage does tmpfs have? How big can it be?

One would assume that a tmpfs filesystem would have the same amount of storage space as you have RAM, but that's not quite correct.

By default, all tmpfs filesystems will have half of your available RAM set as their maximum size. If you need to manually set the size, however, you can use the size mount option. eg.

mount -o size=1G tmpfs /mnt/test

For an fstab entry, it would look like this:

tmpfs /tmp tmpfs defaults,size=1G 0 0

A single tmpfs filesystem can have a maximum size of +. This means you can have a tmpfs system larger than your maximum RAM, but doing this will absolutely cause you massive problems if it ever fills up.

Danger

Be careful about mounting too many tmpfs filesystems or setting their size too large - especially if your RAM is limited.

What happens when /tmp isn't a tmpfs filesystem?

In short: anything you put in /tmp will stay there until it's manually deleted or cleaned up by a sysemd-tmpfiles. The contents of /tmp don't get wiped automatically on reboot unless you clear it out manually or set up a systemd-tmpfiles rule to do so on a regular basis.

Not-so-temporary, eh? In hindsight, this should have been obvious for me, but I really only started figuring this out after working with CIS benchmarks and STIGs which tell you to use a "real" directory or volume for /tmp.

To set up systemd-tmpfs to automatically clean out /tmp on boot (the behavior you'd expect) do the following steps:

  1. Create a file,/etc/tmpfiles.d/tmp.conf, and add the following line to it:
D! /tmp 1777 root root 0
  • D!says two things: 1) that this is a directory (D) and 2) that it should only ever be acted on during boot (!)
  • /tmpshould be obvious. It's the location we want to affect
  • 1777sets the permissions for the directory - the standard for/tmp
  • root rootdefines the owner user and group
  • 0is the maximum age of the file before it's cleaned. It can be eg.1d,10m,2h, etc but we use0here to clean up everything
  1. Apply our custom ruleset immediately:
systemd-tmpfiles --clean /etc/tmpfiles.d/tmp.conf
Note

This isn't strictly necessary because it only removes files on boot, but it's a good habit to get into when editing your tmpfiles configs.

And you're done! It's easy enough to configure (the man page helps a lot) but it's just not obvious that you should configure it.

Bonus: Remove old files in /tmp regularly

The systemd-tmpfiles service is quite flexible. If, for example, you'd like to clean out yesterday's files on a regular basis, you can do something like this:

  1. Edit/etc/tmpfiles.d/tmp.confand add the following line below the existing one:
D /tmp 1777 root root 1d
Note

Ownership and permissions set here override the previous rule we had.

  • Dindicates a directory, but this time without the added!so it runs when it needs to rather than only during boot
  • The rest is the same, until..
  • 1donly removes files that are at least 1 day old so we don't accidentally clean out something that's being used
  1. Again, apply the custom ruleset immediately:
systemd-tmpfiles --clean /etc/tmpfiles.d/tmp.conf
Danger

You can do this even with a tmpfs-mounted filesystem to help keep long-running machines clean.

Now, not only will /tmp be completely cleared out on boot, but it'll also stay clean by managing itself.

But how often does systemd-tmpfiles even run? When are files cleaned out?

Good question! How regularly is "regularly", anyway? Well, since systemd-tmpfiles is part of systemd it's safe to assume it has a service or timer of some kind. A quick systemctl status systemd-tmpfiles<tab><tab> confirms this:

[root@ansible ~]# systemctl status systemd-tmpfiles
systemd-tmpfiles-clean.service      systemd-tmpfiles-clean.timer        systemd-tmpfiles.service            systemd-tmpfiles-setup-dev.service  systemd-tmpfiles-setup.service

And it looks like there is, indeed, a timer. Presumably specifically one for running the --clean operation.

[root@ansible ~]# systemctl status systemd-tmpfiles-clean.timer
● systemd-tmpfiles-clean.timer - Daily Cleanup of Temporary Directories
     Loaded: loaded (/usr/lib/systemd/system/systemd-tmpfiles-clean.timer; static)
     Active: active (waiting) since Thu 2024-10-17 01:01:29 UTC; 52min ago
      Until: Thu 2024-10-17 01:01:29 UTC; 52min ago
    Trigger: Fri 2024-10-18 01:16:40 UTC; 23h left
   Triggers: ● systemd-tmpfiles-clean.service
       Docs: man:tmpfiles.d(5)
             man:systemd-tmpfiles(8)

Oct 17 01:01:29 ansible.yggdrasil.home systemd[1]: Started Daily Cleanup of Temporary Directories.

We can look at the /usr/lib/systemd/system/systemd-tmpfiles-clean.timer referenced here to see what that looks like:

[root@ansible ~]# cat /usr/lib/systemd/system/systemd-tmpfiles-clean.timer
#  SPDX-License-Identifier: LGPL-2.1-or-later
#
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

[Unit]
Description=Daily Cleanup of Temporary Directories
Documentation=man:tmpfiles.d(5) man:systemd-tmpfiles(8)
ConditionPathExists=!/etc/initrd-release

[Timer]
OnBootSec=15min
OnUnitActiveSec=1d

This tells us that it will do a first run 15 minutes after a system boot. From that point, it runs once a day.

So, any of your tmpfiles.d files that tell systemd-tmpfiles to clean out old files will run after 15 minutes, and then from there every day. The exact time depends on when (the time of day) the system was booted.

Why use a tmpfs filesystem anyway?

Harkening back to the note on computers for a minute, here, I'll remind you that all data (eg. files) are just a long string of bytes. For permanent storage, those bytes are stored on mediums like disks. In order to use those bytes you first need to load them into memory. To work on a file, for instance, that file (or at least part of it) needs to be loaded into RAM.

Well, when your filesystem is entirely in RAM already, it's pretty easy for the computer to load it up. For systems like Plex, it's recommended to have the transcode directory be a ramdisk (a tmpfs filesystem) for this reason. It's fast. Like, really fast.

Another great reason is why systems have a /tmp directory in the first place: it's nice to have some temporary storage place that everyone can use for various files. Even systemd uses it for some of its stuff. Here's a RHEL system that's been freshly booted up:

tmpfs--60e4a15be8134c6852a0e1085e1630da_MD5.png

Ultimately it's up to you to decide when and how they're used and cleaned. Linux provides some reasonable defaults that most users are used to - so, personally, I'd stick with those. To each their own, though!