support@technetguy.site Remote IT Services · Worldwide · Get a Quote →
Linux Administration

From Kernel to Shell

Mastering the open source engine that runs this entire server — plus the snippets we actually reach for.

How Linux Works

Unlike Windows, Linux is modular. The Kernel talks to the hardware, the Shell (Bash, Zsh) talks to the Kernel via system calls, and you talk to the Shell — interactively, or through a script that does the same thing every time without getting bored on the 200th run.

Everything on a Linux box is either a process, a file, or a socket — including devices, which show up as files under /dev. That one design decision is why so many Linux tools compose so cleanly: a command that reads bytes from one thing can usually read bytes from anything.

Tux, the Linux mascot

Tux · CC0, Wikimedia Commons

Choosing Your Distro

Ubuntu

The industry standard for cloud and beginners. High compatibility, massive community support, and the distro this server itself runs.

RHEL / Fedora

Enterprise-grade stability. Used by large corporations for mission-critical apps, with a slower, more conservative package cadence.

Arch Linux

For power users. You build it from the ground up. "If you know Arch, you know Linux."

Real-World Snippets

Config patterns pulled straight from day-to-day server administration — copy-pasteable, not pseudocode. Adjust paths, usernames, and ports for your own environment.

A small VM fills up faster than you'd expect — package caches, old kernels, and container build layers are the usual culprits. This runs daily and logs what it finds.

bash
#!/usr/bin/env bash
# /usr/local/sbin/disk-cleanup.sh — run daily via cron
set -euo pipefail

THRESHOLD=80                      # warn if root partition crosses this %
LOGFILE=/var/log/disk-cleanup.log

usage=$(df --output=pcent / | tail -1 | tr -dc '0-9')

# Trim apt's package cache and orphaned dependencies
apt-get clean -y
apt-get autoremove --purge -y

# journald can quietly grow to gigabytes without this
journalctl --vacuum-time=7d

# Drop old Docker build cache and dangling images, if Docker is installed
if command -v docker >/dev/null 2>&1; then
    docker system prune -f --filter "until=168h" >> "$LOGFILE" 2>&1
fi

echo "$(date -Is) disk usage after cleanup: ${usage}%" >> "$LOGFILE"

if [ "$usage" -ge "$THRESHOLD" ]; then
    echo "$(date -Is) WARNING: root partition still at ${usage}%" >> "$LOGFILE"
    # wire this to a mail/webhook alert in production
fi
crontab
# crontab -e (root) — run at 03:17 every day
17 3 * * * /usr/local/sbin/disk-cleanup.sh

A unit file that runs your app without root, without a writable filesystem outside its own data directory, and without the syscalls a normal service never needs.

ini
# /etc/systemd/system/myapp.service
[Unit]
Description=My Application
After=network.target

[Service]
Type=simple
User=myapp
Group=myapp
WorkingDirectory=/opt/myapp
ExecStart=/opt/myapp/venv/bin/python -m myapp
Restart=on-failure
RestartSec=5

# --- Hardening ---
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/opt/myapp/data
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
RestrictSUIDSGID=true
RestrictRealtime=true
LockPersonality=true
MemoryDenyWriteExecute=true
SystemCallFilter=@system-service
SystemCallArchitectures=native

[Install]
WantedBy=multi-user.target

systemctl daemon-reload && systemctl enable --now myapp to apply. systemd-analyze security myapp scores how well-locked-down the unit actually is.

Default-deny inbound, explicit allow list, and a rate limit on the one port that's guaranteed to get scanned: SSH.

bash
# Reset to a known state (careful: this drops existing rules)
ufw --force reset

# Default posture: deny everything inbound, allow everything outbound
ufw default deny incoming
ufw default allow outgoing

# SSH — rate-limited so brute-force scanning gets throttled automatically
ufw limit 22/tcp comment 'SSH (rate-limited)'

# Web traffic
ufw allow 80/tcp comment 'HTTP'
ufw allow 443/tcp comment 'HTTPS'

# A monitoring agent that should only be reachable over the mesh VPN
ufw allow from 100.64.0.0/10 to any port 19999 comment 'Netdata via VPN only'

ufw enable
ufw status verbose

Watches auth and web logs, bans repeat offenders. See the Security hub for the reasoning behind these specific jails.

ini
# /etc/fail2ban/jail.local
[DEFAULT]
bantime  = 1h
findtime = 10m
maxretry = 5
backend  = systemd
ignoreip = 127.0.0.1/8 ::1

[sshd]
enabled  = true
port     = ssh
filter   = sshd
maxretry = 4
bantime  = 24h

[nginx-http-auth]
enabled  = true
port     = http,https
filter   = nginx-http-auth
logpath  = /var/log/nginx/error.log

[nginx-botsearch]
enabled  = true
port     = http,https
filter   = nginx-botsearch
logpath  = /var/log/nginx/access.log
maxretry = 2

fail2ban-client status sshd shows currently banned IPs for that jail.

Without this, application logs grow forever. With it, they rotate daily, compress, and get pruned after two weeks.

conf
# /etc/logrotate.d/myapp
/var/log/myapp/*.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    create 0640 myapp myapp
    sharedscripts
    postrotate
        systemctl reload myapp >/dev/null 2>&1 || true
    endscript
}

Troubleshooting One-Liners

Click each one for what it does and when it actually earns its place in your muscle memory.

See exactly what's listening on which port, and which process (with PID) owns it — before assuming a service is up, down, or bound to the wrong interface. Replaces the older, slower netstat -tulpn on modern distros.
Tail the last 100 lines of one systemd unit's logs without dropping into a pager — makes it scriptable and pipeable into grep. Swap -n 100 for -f to follow live, or add --since "10 min ago" to bound a time window.
Find what's actually eating disk space, top-level directory by top-level directory, largest first. Drop into the biggest offender and repeat (du -sh /var/* | sort -rh | head) until you hit an actual file. Faster than guessing.
Sorts every process by memory usage, descending. The first non-kernel line is usually your answer when a box feels sluggish or the OOM killer has been busy (check with dmesg | grep -i "killed process").
Watches an Nginx/Apache access log live while filtering out the boring 200 and 304 responses, so 4xx/5xx errors and redirects actually stand out on a busy log during a deploy or an incident.
Tests whether a TCP port is actually reachable, with no application-layer noise involved — the fastest way to separate "the firewall is blocking this" from "the app crashed." -z means scan without sending data, -v is verbose.
Lists every file over 100MB on the current filesystem, human-readable sizes. -xdev stops it from crossing into other mounted filesystems (like a mounted network share), which keeps the results relevant and the command fast.

Where This Connects

These are the same patterns used to keep a multi-node home lab and cloud fleet patched, backed up, and under 80% disk on every box — see the home lab case study for how it holds up at scale, or Security & Hardening for the ufw/fail2ban reasoning in more depth.