Automate a Nightly Git Snapshot of /etc for Change Tracking

I like knowing exactly what changed under /etc and when — after a package upgrade, after I "just tweaked one thing," after someone else touched a box. A nightly git snapshot gives me a dated, diffable trail I can read with git log and git diff. It is not a substitute for real config management, but it is cheap, honest history.

Before you run this

What it does: it initialises a git repository that tracks the contents of /etc, then installs a systemd timer that once a night stages every change and commits it if anything differs. It only reads your config files and records copies of them into a repository — it does not modify, move, or delete anything under /etc.

Privileges: the snapshot must run as root. /etc contains files only root can read (/etc/shadow, /etc/ssl/private/*), so the timer runs as root and the repository is owned by root.

This repository will contain your secrets. A full snapshot of /etc includes /etc/shadow, TLS private keys, and any credentials living in config files. Treat the repo directory as sensitive: keep it 0700, keep it local, and think hard before you ever push it to a remote. I cover excluding files below, but understand the default captures everything readable.

Test first. Read the script before you install it, and stand it up on a test VM or a spare box first. Watch one manual run and inspect the first commit before you trust the timer on anything production.

Undo is clean. Because this only adds a repository and a timer, backing it out is just disabling the timer and deleting the repo directory (shown at the end). Nothing in /etc is altered, so there is nothing irreversible here.

Assumptions: Debian 12 or Ubuntu 22.04 with systemd and git installed (sudo apt install git). On RHEL/Alma the paths and systemctl usage are identical; only the package manager differs. If you would rather use a packaged, purpose-built tool, etckeeper does essentially this and hooks into apt/dnf — I mention it so you know it exists, but this guide builds the plain-git version so you can see every moving part.

The design: a repo that lives outside /etc

I keep the git directory out of /etc and point it at /etc as its work tree. This is the well-known "bare repo plus explicit work-tree" pattern, and it keeps a .git folder from appearing inside /etc.

Create the repository once:

# Bare repo lives here; /etc is the work tree it tracks
sudo git init --bare /var/lib/config-tracker.git
sudo chmod 700 /var/lib/config-tracker.git

Every git command then names both directories explicitly, for example:

sudo git --git-dir=/var/lib/config-tracker.git --work-tree=/etc status

Optionally exclude files

git reads a .gitignore from the root of the work tree, so /etc/.gitignore controls what is tracked. If you would rather not capture private key material, list it here. This is a genuine trade-off: exclude /etc/shadow and you stop tracking user/password changes too. Decide deliberately.

# /etc/.gitignore — only if you choose to exclude secrets
ssl/private/

If you keep the repo local and 0700, I generally track everything so the history is complete.

The snapshot script

Save this as /usr/local/sbin/config-tracker.sh:

#!/usr/bin/env bash
set -euo pipefail

GIT_DIR=/var/lib/config-tracker.git
WORK_TREE=/etc

# One helper so we don't repeat the long invocation
git_cmd() {
  git --git-dir="$GIT_DIR" --work-tree="$WORK_TREE" "$@"
}

# Stage every addition, change, and deletion under /etc
git_cmd add -A

# Commit only if something actually changed (keeps the log meaningful)
if ! git_cmd diff --cached --quiet; then
  git_cmd \
    -c user.name="config-tracker" \
    -c user.email="config-tracker@$(hostname -f)" \
    commit -q -m "Config snapshot $(date '+%Y-%m-%d %H:%M:%S %z')"
fi

Make it executable:

sudo chmod 750 /usr/local/sbin/config-tracker.sh

A few notes on the non-obvious lines. set -euo pipefail stops the script on any error rather than committing a half-staged state. The diff --cached --quiet check exits non-zero when there are staged changes, so we only commit on nights where something actually moved — no empty daily commits. I set the identity inline with -c so the script does not depend on a global git config existing for root.

Take the first snapshot by hand and read it before automating anything:

sudo /usr/local/sbin/config-tracker.sh
sudo git --git-dir=/var/lib/config-tracker.git --work-tree=/etc log --oneline
sudo git --git-dir=/var/lib/config-tracker.git --work-tree=/etc show --stat HEAD

The --stat output should list the files that were captured. If that looks wrong — too much, too little, an unexpected path — fix it now, before the timer runs unattended.

Schedule it with a systemd timer

I use a systemd timer rather than cron because the logging and status reporting are better. Create the service unit at /etc/systemd/system/config-tracker.service:

[Unit]
Description=Nightly git snapshot of /etc

[Service]
Type=oneshot
ExecStart=/usr/local/sbin/config-tracker.sh

And the timer at /etc/systemd/system/config-tracker.timer:

[Unit]
Description=Run the nightly /etc git snapshot

[Timer]
OnCalendar=*-*-* 02:30:00
Persistent=true

[Install]
WantedBy=timers.target

OnCalendar=*-*-* 02:30:00 runs it every night at 02:30 local time. Persistent=true means that if the machine was off at 02:30, the snapshot runs at the next boot instead of being silently skipped. Adjust the time to suit; man systemd.time documents the OnCalendar format if you want a different cadence.

Enable and start the timer:

sudo systemctl daemon-reload
sudo systemctl enable --now config-tracker.timer

Verify it worked

Confirm the timer is registered and see when it will next fire:

systemctl list-timers config-tracker.timer

You should see a NEXT time around 02:30 and the unit listed as active. To prove the whole path works without waiting until tomorrow, trigger the service directly and check the log:

sudo systemctl start config-tracker.service
journalctl -u config-tracker.service --no-pager -n 20

A clean run exits with no error output. Then inspect the history — this is the payoff:

# Full commit history
sudo git --git-dir=/var/lib/config-tracker.git --work-tree=/etc log --oneline

# What changed in the most recent snapshot
sudo git --git-dir=/var/lib/config-tracker.git --work-tree=/etc show HEAD

To answer "what did this box's /etc look like a week ago," you now have git log, git diff, and git show against dated commits.

Undo / roll back

Because this only added a repo and a timer, removal is straightforward and touches nothing in /etc:

# Stop the schedule
sudo systemctl disable --now config-tracker.timer

# Remove the units and reload
sudo rm /etc/systemd/system/config-tracker.timer /etc/systemd/system/config-tracker.service
sudo systemctl daemon-reload

# Remove the script
sudo rm /usr/local/sbin/config-tracker.sh

# Remove the repository and, if you created one, the ignore file
sudo rm -rf /var/lib/config-tracker.git
sudo rm -f /etc/.gitignore

After that, systemctl list-timers will no longer show the unit, and /etc is exactly as it was — nothing this guide did ever modified a config file, only recorded copies of them.

For canonical reference on the timer syntax, see the systemd.timer and systemd.time man pages; for the git commands, the official git-add, git-commit, and git-log documentation on git-scm.com.