================================================================================ 05: UBUNTU HARDENING: SOLUTION WALKTHROUGH Ubuntu 24.04 LTS (GNOME), with 26.04 notes Training reference for virtual machines you own or are authorized to change. Three levels: L1 Doorman, L2 Watchman, L3 Hunter. Every item = COMMAND, WHY, NOTE, and the hardening AREA it belongs to; ALSO adds a second caution when an item needs one. No scripts by design: type it, read it, explain it. Item IDs match the Lockdown Drills trainer for this OS. Unofficial practice; not affiliated with or endorsed by Canonical. HARDENING AREAS (CIS Critical Security Controls v8.1) Account management ........ CIS Control 5 Access control ............ CIS Control 6 Secure configuration ...... CIS Control 4 (software inventory: Control 2) Patching .................. CIS Control 7 Malware defenses .......... CIS Control 10 Audit logging ............. CIS Control 8 Also tagged: Data protection (CIS Control 3); Investigation (forensics basics: answer questions and preserve evidence before you change things). ================================================================================ ------------------------------ DOORMAN ----------------------------------------- Lock the doors. Users, admins, passwords, lockout, firewall, updates, snaps and junk software, obvious services, GDM auto-login and screen lock. Learn it first: Defense Ladder s3, Debian-family hardening. https://freeapp4that.store/defense-ladder/#sophomore/s3 -------------------------------------------------------------------------------- [B-00] Scenario brief and evidence first AREA: Investigation (forensics basics) GUI: open the scenario brief (the documented requirements for this machine) and write down authorized users, admins, required services, and anything forbidden $ mkdir -p ~/audit/day1 && cd ~/audit/day1 && touch changes.txt && pwd (one folder for evidence and backups; log each change with what, why, and how to undo it) $ sudo cp -a /etc/login.defs /etc/login.defs.bak (copy each file before you edit it; -a keeps the owner, mode, and time) $ sudo rsync -a /home/ /mnt/backup/home/ (copies the home folders to a mounted backup drive; a backup counts once a test restore works) $ man login.defs (the manual for a command or a config file; / searches, n finds the next match, q quits) WHY: Authorized users, admins, and required services decide what is a fix and what breaks the business. Evidence first: answer the investigation questions before any change (Defense Ladder f1). NOTE: Unfamiliar differs from unauthorized: trace and ask before removing agents (Defense Ladder s1). Before PAM, sudoers, or SSH edits, keep a root shell (sudo -i) open to fix lockout typos. ALSO: Start B-08's upgrade now in another terminal. B-05 and B-09 wait for its apt lock, so do the apt-free items first and read its result near the end. [B-01] List human users, admins, and UID-0 accounts AREA: Account management (CIS Control 5) $ awk -F: '$3>=1000 && $3<65534 {print $1" uid="$3" shell="$7}' /etc/passwd $ awk -F: '$3==0 {print $1}' /etc/passwd $ getent group sudo admin adm wheel docker lxd disk $ awk -F: -v g="$(getent group sudo | cut -d: -f3)" '$4==g {print $1" has sudo as its primary group"}' /etc/passwd $ whoami; id (the account you work as, and its groups, before you judge anyone else's) GUI: Settings, System, Users (Unlock first). WHY: An extra UID-0 user is a second root. Members of sudo (or wheel) and admin (default sudoers grants %admin) hold root-level power; adm members read logs. NOTE: A user whose primary group is sudo (/etc/passwd field 4) holds admin rights yet hides from getent group sudo. ALSO: Treat docker, lxd, and disk members as root too: they can mount the host filesystem or read raw disks. [B-02] Remove or disable an unauthorized user AREA: Account management (CIS Control 5) $ id -u baduser (0 means a hidden root: skip the next two lines and follow the NOTE) $ sudo usermod -L -e 1 baduser; ps -o pid,unit,cmd -u baduser $ sudo loginctl disable-linger baduser; sudo loginctl terminate-user baduser; sudo pkill -KILL -u baduser; pgrep -u baduser $ sudo deluser --remove-home baduser $ sudo mkdir -p /root/removed-users && sudo deluser --remove-home --backup-to /root/removed-users baduser $ sudo userdel -r baduser $ sudo usermod -L -e 1 baduser && sudo usermod -s /usr/sbin/nologin baduser WHY: --remove-home clears planted files, deluser drops the crontab userdel leaves, and --backup-to keeps a root-only copy of the home as evidence. Lock, expiry, and nologin disable without destroying evidence: -e 1 stops SSH key sign-in, which -L alone leaves working. NOTE: The brief decides between delete and disable. Lock first, end the processes, then delete: when pgrep -u prints nothing, the account is idle. A service that runs as the user restarts its processes until you stop the unit ps names (sudo systemctl disable --now ). When id -u prints 0 the account is a hidden root: pkill -u on it ends every root process and deluser refuses it, so read its home (field 6 of getent passwd) and remove it with sudo userdel -f , adding -r only when that home is its own. [B-03] Fix admin group membership AREA: Account management (CIS Control 5) $ sudo gpasswd -d mallory sudo $ sudo loginctl terminate-user mallory (ends the open sessions, which keep the old groups) $ sudo usermod -aG sudo alice $ groups alice $ sudo adduser keola (when the brief lists an account this machine lacks: it sets a password and creates the home; the low-level form is sudo useradd -m -s /bin/bash keola, then sudo passwd keola) $ sudo addgroup designers && sudo gpasswd -M malia,keola designers (when the brief names a group and its members; -M sets the whole member list) GUI: Settings, System, Users: Administrator toggle (adds or removes the sudo group). WHY: Least privilege. -aG appends; -G alone replaces existing groups. Open sessions keep their starting groups: removed admins keep sudo until each session ends; new members gain it at next sign-in. [B-04] Passwords: strong, forced change, no empties, root locked AREA: Account management (CIS Control 5) $ sudo passwd alice $ sudo chage -d 0 alice $ awk -F: '$3>=1000 && $3<65534 {print $1}' /etc/passwd | grep -vx "${SUDO_USER:-$USER}" | xargs -r -n1 sudo chage -d 0 (every human account except yours) $ sudo awk -F: '$2=="" {print $1" HAS NO PASSWORD"}' /etc/shadow $ awk -F: '$2!="x" {print $1" has something other than x in /etc/passwd field 2"}' /etc/passwd $ sudo passwd -l root $ sudo passwd -S root (expect: root L) WHY: Empty field 2 in /etc/shadow or /etc/passwd means Enter logs in; pam_unix reads /etc/shadow only when /etc/passwd holds x (fix: sudo pwconv, which carries an empty field into /etc/shadow, then sudo passwd or sudo passwd -l ). Locked root forces accountable sudo over shared passwords. NOTE: Record changed passwords safely (on paper, in a lab). Keep your own sign-in password unless the brief says otherwise; leave service and key-only accounts out of the bulk chage line. [B-05] Password aging and quality AREA: Account management (CIS Control 5) $ sudo sed -i 's/^PASS_MAX_DAYS.*/PASS_MAX_DAYS\t60/' /etc/login.defs $ sudo sed -i 's/^PASS_MIN_DAYS.*/PASS_MIN_DAYS\t1/' /etc/login.defs $ sudo sed -i 's/^PASS_WARN_AGE.*/PASS_WARN_AGE\t7/' /etc/login.defs $ sudo sed -i 's/^UMASK.*/UMASK\t\t027/' /etc/login.defs $ diff /etc/login.defs.bak /etc/login.defs (compares the B-00 copy with your edit: < marks the old lines, > the new ones) $ sudo chage -M 60 -m 1 -W 7 alice $ for u in $(awk -F: '$3>=1000 && $3<=60000 {print $1}' /etc/passwd); do sudo chage -M 60 -m 1 -W 7 "$u"; done $ sudo passwd -S -a (min, max, and warn for every account) $ sudo apt install -y libpam-pwquality $ sudo nano /etc/security/pwquality.conf minlen = 12 difok = 3 maxrepeat = 3 enforce_for_root dcredit = -1 ucredit = -1 lcredit = -1 ocredit = -1 (only when the brief's policy asks for character classes) $ sudo nano /etc/pam.d/common-password password requisite pam_pwquality.so retry=3 (ensure present) ... pam_unix.so ... remember=5 (add to the pam_unix line) $ grep -n pam_pwquality /etc/pam.d/common-password $ sudo apt install -y libpwquality-tools && printf 'Summer2026\n' | pwscore WHY: login.defs covers only new users; chage (or the loop) sets existing ones. UMASK sets the starting permissions of new files through pam_umask; with a private group per user (USERGROUPS_ENAB yes) users see 0007. pwquality enforces complexity at each change, and remember= blocks reuse. Confirm with pwscore, which tests a password without changing one. NOTE: The scenario brief sets the numbers; these commands show one sample brief. The CIS Benchmark asks for a minlen of 14 or more, a history of 24 or more, a maximum age of 365 days or fewer, a minimum age of 1 day or more, and a UMASK of 027 or stricter; Defense Ladder s2 compares the CIS and NIST rulebooks, and s3 applies them on Linux. Leave obscure in place. Numbers on the pam_pwquality.so line override pwquality.conf, so keep them in the conf file and leave the PAM line at retry=3. Install libpam-pwquality before any hand edit to a common-* file (B-06): after one, pam-auth-update stops updating those files while apt still reports success, so the grep is the proof. Open a root shell before the loop: an account whose last change is older than the new maximum must change its password at its next sign-in, yours included. [B-06] Account lockout with pam_faillock AREA: Account management (CIS Control 5) $ sudo nano /etc/security/faillock.conf deny = 5 unlock_time = 1800 fail_interval = 900 audit $ sudo nano /etc/pam.d/common-auth (order matters) auth required pam_faillock.so preauth silent auth [success=1 default=ignore] pam_unix.so (existing line: set success=1, remove nullok) auth [default=die] pam_faillock.so authfail auth sufficient pam_faillock.so authsucc $ sudo nano /etc/pam.d/common-account (add first) account required pam_faillock.so $ su - alice (the lockout test, in a second terminal while your root shell stays open: each wrong password raises the count sudo faillock --user alice shows) $ sudo faillock --user alice $ sudo faillock --user alice --reset WHY: Stops online brute force. pam_faillock replaced pam_tally2, which current Ubuntu, Mint, and Debian releases have dropped. nullok allows empty passwords. faillock.conf holds the numbers; locking starts only once the PAM lines are in place, and numbers written on a PAM line override the file. Confirm with sudo faillock --user NAME after failed tries. NOTE: Test with su - alice in a second terminal before closing your root shell. Set the pam_unix line to [success=1 default=ignore] whatever number it had: some installs (Ubuntu desktop among them) ship [success=2] because a pam_sss line follows, and a 2 left in place sends every correct password past authsucc to pam_deny. unlock_time = 0 keeps a lock until sudo faillock --user NAME --reset or a reboot (the records live in /run/faillock); root stays exempt unless even_deny_root is set. Hand-added lines make pam-auth-update treat common-auth as locally modified; Hunter A-09 shows the managed route. On a desktop, the sufficient authsucc line ends the sign-in stack before pam_gnome_keyring runs, so the login keyring stays locked; the account line in common-account also resets the count, and A-09's managed route relies on it alone. [B-07] Firewall with ufw AREA: Secure configuration (CIS Control 4) $ sudo apt install -y ufw $ sudo ufw default deny incoming $ sudo ufw default allow outgoing $ sudo ufw allow 22/tcp $ sudo ufw enable $ sudo ufw logging on $ sudo ufw status verbose WHY: Default-deny inbound covers the service you have not found yet. NOTE: Only allow 22/tcp if the brief requires SSH. Add rules for required services (80/tcp, 443/tcp, 21/tcp). [B-08] Updates: now and automatic AREA: Patching (CIS Control 7) $ sudo apt update && sudo apt upgrade -y $ apt list --upgradable $ sudo apt full-upgrade (when upgrade lists packages kept back; read the REMOVED list) $ uname -r (the running kernel; after a kernel update, the new version runs once you reboot) $ sudo snap refresh $ snap list; apt-mark showhold (held in the Notes column, or a held package, freezes updates) $ sudo snap refresh --unhold ; sudo apt-mark unhold $ apt-config dump | grep -E '^APT::Periodic::(Update-Package-Lists|Unattended-Upgrade) ' APT::Periodic::Update-Package-Lists "1"; APT::Periodic::Unattended-Upgrade "1"; $ systemctl is-enabled apt-daily.timer apt-daily-upgrade.timer $ sudo apt install -y unattended-upgrades && sudo dpkg-reconfigure -plow unattended-upgrades $ sudo unattended-upgrade --dry-run -d 2>&1 | grep 'Allowed origins' (expect an Ubuntu noble-security origin) GUI: Software and Updates, Updates: Automatically check for updates Daily; When there are security updates: Download and install automatically WHY: unattended-upgrades is preinstalled, run by apt-daily and apt-daily-upgrade timers. Check for Update-Package-Lists or Unattended-Upgrade at 0, the package removed, or automatic checks set to Never; apt-config shows what apt uses. NOTE: apt reads every apt.conf.d file, so a later file can reset either to 0. After a kernel update, uname -r shows the old kernel until reboot; /var/run/reboot-required marks one due. ALSO: Snaps update themselves; snap refresh forces it now. Close Firefox first: running snaps update once closed. 'Deferred due to phasing' packages are Ubuntu's staged rollout; a later run installs them. [B-09] Remove unapproved software AREA: Secure configuration: software (CIS Controls 2 and 4) $ apt list --installed 2>/dev/null | grep -Ei 'nmap|zenmap|john|hydra|medusa|wireshark|tshark|aircrack|ophcrack|netcat|ncat|socat|telnetd|nikto|metasploit|hashcat|sqlmap|doona|torrent|deluge|amule|scanmem|gameconqueror|minetest|aisleriot|mahjongg|sudoku|freeciv|wesnoth' $ dpkg-query -W -f='${db:Status-Abbrev}\t${binary:Package}\t${Section}\t${binary:Summary}\n' | awk -F'\t' '$1 ~ /^ii/' | grep -iE 'games|crack|brute|sniff|exploit|scanner|torrent|peer-to-peer' $ zgrep -hE '^(Start-Date|Commandline)' /var/log/apt/history.log* $ dpkg -l | grep -i $ sudo apt purge --auto-remove nmap $ ls -la /usr/local/bin /usr/local/sbin /opt $ sudo snap list $ sudo snap remove GUI: App Center, Manage WHY: Hacking tools, games, and P2P clients are acceptable-use violations and attack surface. Description search finds tools by purpose, Section lists games, apt's history.log dates installs, and /usr/local holds hand-placed files. NOTE: Required vsftpd, apache2, samba, openssh-server, and mysql stay; Watchman I-12 hardens them. Keep pam_pwquality's dictionary (cracklib-runtime, libcrack2) and netcat-openbsd (purging removes ubuntu-minimal). Snaps like Firefox appear only in snap list. ALSO: Desktop-installed transmission-gtk is still P2P; the brief decides. Skip -y and read apt's REMOVED list: --auto-remove also removes packages marked unneeded. Purge only dpkg-listed names; apt stops at unknown ones. [B-10] Find unapproved media files AREA: Data protection (CIS Control 3) $ sudo find /home /root /tmp /var /dev/shm /srv /opt /mnt /media /usr/local -xdev -type f \( -iname '*.mp3' -o -iname '*.mp4' -o -iname '*.m4a' -o -iname '*.mov' -o -iname '*.avi' -o -iname '*.mkv' -o -iname '*.webm' -o -iname '*.wmv' -o -iname '*.flv' -o -iname '*.mpg' -o -iname '*.mpeg' -o -iname '*.wav' -o -iname '*.flac' -o -iname '*.ogg' -o -iname '*.aac' -o -iname '*.wma' \) -not -path '/var/lib/snapd/*' -not -path '/var/lib/flatpak/*' -printf '%u\t%s\t%p\n' 2>/dev/null $ sudo find /home /root /tmp /var/tmp /dev/shm /srv /opt -xdev -type f -size +32k -exec file --mime-type {} + 2>/dev/null | grep -E ': (audio|video)/' $ sudo apt install -y plocate && sudo updatedb && sudo locate -i '*.mp3' (a quick second look; the index skips /tmp, /dev/shm, and /media, so find stays the proof) $ sudo sha256sum ''; sudo rm -- '' $ sudo mkdir -p /root/quarantine && sudo mv -- '' /root/quarantine/ (keeps the file as evidence when the brief asks for it, out of the user's reach) WHY: Many acceptable-use policies ban personal media; the search also surfaces hidden folders in /home and /var/tmp. file reads first bytes, so a song renamed notes.txt still shows as audio/mpeg. NOTE: App-folder hits (~/.local/share/flatpak, ~/snap, /opt/) are usually the app's; images go only if banned. Before rm, confirm paths against the brief and, for hash questions, run sha256sum (Defense Ladder s6). ALSO: -printf records owner, size, and path for change logs and investigation questions. Skipping /usr except /usr/local avoids OS sounds in /usr/share. Trust find; locate's nightly index skips /tmp and /dev/shm. [B-11] Services: what runs, what listens AREA: Secure configuration (CIS Control 4) $ systemctl list-units --type=service --state=running $ sudo ss -tulpn $ systemctl list-sockets --no-pager $ sudo systemctl disable --now vsftpd $ sudo systemctl disable --now telnet.socket inetd xinetd 2>/dev/null WHY: Telnet and FTP are cleartext; avahi, cups, rpcbind, nfs, and smb are exposure when unneeded. ss -tulpn shows the PID and program for every open port. NOTE: Required services stay. Check what starts a service before trusting it is off: a .socket unit can hold the port after its .service stops; ss shows systemd holding it. ALSO: disable --now stops units and keeps them off at boot. systemctl list-sockets pairs each socket with its service; disable both. I-07 traces an unknown listener to its unit and package. [B-12] GDM auto-login and screen lock AREA: Secure configuration (CIS Control 4) $ sudo nano /etc/gdm3/custom.conf [daemon] AutomaticLoginEnable=false GUI: Settings, System, Users, Automatic Login Off $ gsettings set org.gnome.desktop.screensaver lock-enabled true $ gsettings set org.gnome.desktop.screensaver lock-delay 0 $ gsettings set org.gnome.desktop.session idle-delay 300 GUI: Settings, Privacy and Security, Screen Lock: Automatic Screen Lock On; blank screen after 5 minutes WHY: Auto-login means physical access is root in ten seconds; an unlocked idle desktop is the same thing a few minutes later. NOTE: gsettings changes the current user only; repeat for each authorized user or use the GUI as each. [B-13] Find plaintext credential files and unapproved archives AREA: Data protection (CIS Control 3) $ sudo grep -rIil --exclude-dir=.cache --exclude-dir=.mozilla --exclude-dir=snap --exclude-dir=flatpak -E 'passw(or)?d|credential|secret|api[_-]?key|BEGIN [A-Z ]*PRIVATE KEY' /home /root /tmp /var/tmp /dev/shm /srv /opt /var/www 2>/dev/null $ sudo find /home /root /tmp /var/tmp /dev/shm /srv /opt -xdev -type f \( -iname '*.zip' -o -iname '*.7z' -o -iname '*.rar' -o -iname '*.tar' -o -iname '*.tgz' -o -iname '*.tar.gz' -o -iname '*.tar.xz' -o -iname '*.txt' \) -printf '%u\t%TY-%Tm-%Td\t%p\n' 2>/dev/null $ sudo unzip -l ''; sudo python3 -m zipfile -l ''; sudo tar -tvf '' $ file '' WHY: Password lists open every named account; archives slip tools past package searches. grep matches content under any name; unzip -l and tar -tvf list without extracting; file unmasks renamed ones. NOTE: Judge each .txt before deleting it: the brief, the investigation questions, and required documents stay. ~/.ssh keys are expected for users the brief lets sign in with keys (I-01). ALSO: Record the path and hash before removing anything (Defense Ladder s6). python3 -m zipfile works where unzip is absent. ------------------------------ WATCHMAN ---------------------------------------- Check the windows. SSH, sudoers, cron and timers, shell persistence, permissions and SUID, processes, AppArmor, auditing, apt sources, hardening the services you must keep, and browser policies. Learn it first: Defense Ladder j3, Debian and deeper Linux. https://freeapp4that.store/defense-ladder/#junior/j3 -------------------------------------------------------------------------------- [I-01] SSH hardening, only when SSH is required AREA: Access control (CIS Control 6) $ sudo grep -rniE '^[[:space:]]*(Include|Match|PermitRootLogin|PermitEmptyPasswords|PermitUserEnvironment|PasswordAuthentication|AllowUsers)' /etc/ssh/sshd_config /etc/ssh/sshd_config.d/ $ sudo less /etc/ssh/sshd_config (read the whole file before you edit it; / searches, n finds the next match, q quits) $ ssh-keygen -t ed25519 -C '@'; ssh-copy-id @ (on the admin's own machine, before PasswordAuthentication no; then test a key login) $ sudo nano /etc/ssh/sshd_config.d/00-hardening.conf (or the main file, after reading every drop-in) PermitRootLogin no PermitEmptyPasswords no MaxAuthTries 3 LoginGraceTime 30 X11Forwarding no UsePAM yes PermitUserEnvironment no PasswordAuthentication yes or no per the brief AllowUsers alice bob $ sudo sshd -t && sudo systemctl restart ssh $ sudo sshd -T | grep -E '^(permitrootlogin|permitemptypasswords|permituserenvironment|maxauthtries|passwordauthentication|x11forwarding|allowusers) ' $ sudo sshd -T -C user=root,host=localhost,addr=127.0.0.1 | grep -i '^permitrootlogin' $ systemctl list-units 'ssh*' --no-pager (is ssh.socket the unit that listens on 22?) $ sudo systemctl daemon-reload && sudo systemctl restart ssh.socket (after a Port or ListenAddress change) $ sudo find / -name authorized_keys -exec ls -l {} \; -exec cat {} \; 2>/dev/null WHY: Root over SSH erases accountability and invites brute force; a planted authorized_keys backdoor survives every password fix. sshd -T, plus -C for Match blocks, confirms the values in force. NOTE: The && runs the restart only when sshd -t passes, so a config typo leaves the running server in place. sshd keeps the first value it reads, and Include reads sshd_config.d/*.conf in name order, so 00-hardening.conf wins over the shipped drop-ins and the main file; a drop-in that sorts before 00- still wins, and the grep and sshd -T show it. When ssh.socket owns port 22, a Port change or a disable aimed at ssh.service alone leaves the socket listening on the old port. Not required: sudo systemctl disable --now ssh.socket ssh.service && sudo apt purge -y openssh-server; confirm with sudo ss -tlpn that port 22 is closed. If sshd -t or sshd -T stops at 'Missing privilege separation directory: /run/sshd', run sudo mkdir -p /run/sshd and test again. [I-02] sudoers audit AREA: Access control (CIS Control 6) $ sudo grep -R -nE 'NOPASSWD|!authenticate|ALL=\(ALL' /etc/sudoers /etc/sudoers.d/ $ sudo visudo $ sudo visudo -f /etc/sudoers.d/ $ sudo -l -U mallory $ sudo visudo -c; sudo -k; sudo -v (expect a password prompt) $ sudoedit /etc/hosts (edits a copy as you, then installs it as root; grant sudoedit FILE in sudoers in place of an editor, which can open a root shell) WHY: A NOPASSWD ALL line in a sudoers.d drop-in is a silent root backdoor. Edit only with visudo, never nano: a syntax error locks everyone out of sudo. NOTE: Defaults !authenticate, global or Defaults:, acts like NOPASSWD on every rule it reaches. In the grep, single quotes keep bash from reading the ! as history. ALSO: sudo -l -U lists the user's rules, with Defaults under Matching Defaults entries. [I-03] Cron, at and systemd timers AREA: Malware defenses (CIS Control 10) $ cat /etc/crontab /etc/anacrontab 2>/dev/null; sudo ls -la /etc/cron.d /etc/cron.hourly /etc/cron.daily /etc/cron.weekly /etc/cron.monthly /var/spool/cron/crontabs $ sudo dpkg --verify cron cron-daemon-common $ dpkg -S /etc/cron.d/* /etc/cron.hourly/* /etc/cron.daily/* /etc/cron.weekly/* /etc/cron.monthly/* 2>&1 | grep 'no path found' $ for u in $(cut -d: -f1 /etc/passwd); do sudo crontab -l -u "$u" 2>/dev/null | sed "s/^/$u: /"; done $ systemctl list-timers --all $ sudo atq (command not found means at is absent, along with any at jobs) $ sudo atrm (removes one job that atq lists; read it first with sudo at -c ) $ sudo crontab -r -u baduser WHY: Cron re-creates users, re-opens ports, and re-installs tools every few minutes, so a fixed problem comes back. The stock /etc/crontab runs four run-parts jobs (hourly, daily, weekly, and monthly). NOTE: Before deleting a planted job, record and hash its program, end its process, and find other starters (I-07's grep -rlF). Cron rereads changed files every minute; fixes apply without restart. ALSO: cron-daemon-common owns /etc/crontab on 24.04, so dpkg --verify flags any edit (??5?????? c /etc/crontab) without a clean VM to compare against. dpkg -S marks hand-placed drop-ins with 'no path found'. [I-04] Shell and startup persistence AREA: Malware defenses (CIS Control 10) $ grep -R -nE 'alias|nc |ncat|bash -i|/dev/tcp|curl .*\| *sh|wget .*\| *sh' /etc/profile /etc/bash.bashrc /etc/profile.d/ /home/*/.bashrc /home/*/.profile /root/.bashrc 2>/dev/null $ cat /etc/rc.local 2>/dev/null; ls -la /etc/init.d/ /etc/systemd/system/ $ systemctl list-unit-files --state=enabled $ echo '' | base64 -d (decodes a blob found in a cron line or startup file; read what it would run before anything else) WHY: An alias on sudo or a reverse shell in .bashrc runs every time a terminal opens, including the one you use to fix things. [I-05] Critical file permissions, world-writable, SUID AREA: Access control (CIS Control 6) $ sudo chown root:shadow /etc/shadow /etc/gshadow && sudo chmod 640 /etc/shadow /etc/gshadow $ sudo chown root:root /etc/passwd /etc/group && sudo chmod 644 /etc/passwd /etc/group $ sudo chown root:root /etc/sudoers && sudo chmod 440 /etc/sudoers $ sudo chown root:root /boot/grub/grub.cfg && sudo chmod 600 /boot/grub/grub.cfg $ sudo stat -c '%a %U:%G %n' /etc/passwd /etc/group /etc/shadow /etc/gshadow /etc/passwd- /etc/group- /etc/shadow- /etc/gshadow- /boot/grub/grub.cfg $ sudo chmod 750 /home/* $ sudo find / -xdev -type f -perm -0002 -not -path '/proc/*' 2>/dev/null $ sudo find / -xdev -type d -perm -0002 ! -perm -1000 2>/dev/null $ sudo find / -xdev -type f \( -perm -4000 -o -perm -2000 \) 2>/dev/null $ sudo chmod u-s /path/to/evil $ sudo getcap -r / 2>/dev/null (cap_net_raw or cap_net_admin on a packaged network tool is normal; cap_setuid on python3 or perl gives root to anyone who runs it: sudo setcap -r ) $ sudo getfacl /etc/shadow /etc/sudoers (a + after the mode in ls -l means extra entries; sudo setfacl -x u: removes one, and sudo setfacl -b removes them all) WHY: Readable /etc/shadow allows offline cracking. Writable /etc/passwd gives root, as does SUID on bash, nmap, find, or vim (GTFOBins). grub.cfg holds the kernel command line and any GRUB password hash. NOTE: Investigate anything outside this expected list. SUID: passwd, su, sudo, mount, umount, chsh, chfn, gpasswd, newgrp, pkexec, fusermount3, ssh-keysign, dbus-daemon-launch-helper, and polkit-agent-helper-1; SGID: chage, expiry, unix_chkpwd, and crontab. ALSO: Compare unexpected SUID/SGID files with a clean VM of the same release. Backups ending in - (same data) take the main files' owner and mode. update-grub keeps your grub.cfg mode. [I-06] Hosts file and name resolution AREA: Secure configuration (CIS Control 4) $ cat /etc/hosts $ resolvectl status 2>/dev/null | grep -i 'DNS Servers' $ ip -br a; ip route (addresses and the default route; explain anything the brief leaves out) $ ip -br link (every interface; an unexpected tun0, wg0, or second NIC needs an explanation) $ ip neigh (the MAC address behind each neighbor; two addresses that share one MAC suggest ARP spoofing) $ traceroute -n security.ubuntu.com (hop 1 should be the gateway the brief names; a different first hop means a changed route; sudo apt install traceroute when it is missing, or use tracepath) $ dig +short ; dig -x
(what DNS answers, to compare with /etc/hosts; sudo apt install bind9-dnsutils when dig is missing) $ nmcli connection show (saved network profiles; an unknown VPN or Wi-Fi profile needs an explanation) WHY: Redirecting security.ubuntu.com or archive.ubuntu.com to 127.0.0.1 breaks updates silently. [I-07] Processes and connections: find the backdoor AREA: Malware defenses (CIS Control 10) $ ps -eo pid,ppid,user,cmd --sort=-%cpu | head -25 $ top (live CPU use; a miner sits at the top of the list; q quits) $ ps -ef | grep -Ei 'nc |ncat|socat|bash -i|python.* -c|perl -e|xmrig|miner' | grep -v grep $ pstree -p (who started whom: a shell under apache2 or cron is a lead) $ sudo ss -tulpn; sudo lsof -i -P -n | grep -Ei 'LISTEN|ESTABLISHED' $ sudo ls -l /proc//exe; sudo cat /proc//cmdline | tr '\0' ' ' $ systemctl status --no-pager; dpkg -S "$(sudo readlink -f /proc//exe)" $ systemctl cat (the unit file plus every drop-in; ExecStart names the program it runs) $ sudo grep -rlF '' /etc /var/spool/cron /home /root 2>/dev/null $ sudo kill -9 WHY: Killing a process is pointless unless you also remove its starter (cron, a systemd unit, rc.local, .bashrc): find the parent, then the file. systemctl status names the unit to disable; dpkg -S 'no path found' means no package installed it (on 24.04 retry with /bin in place of /usr/bin first, since they merge); grep -rlF lists every file that starts it. NOTE: When systemctl status names cron.service, rc-local.service, inetd.service, or a session scope, the process is that starter's child: remove the planted line, end the process, keep the starter. A packaged python3, nc, or bash in /proc//exe hides what it runs, so read cmdline; an exe path ending in (deleted) is a finding. [I-08] AppArmor enforcing AREA: Secure configuration (CIS Control 4) $ sudo apt install -y apparmor-utils $ sudo systemctl enable --now apparmor $ sudo aa-status $ sudo aa-enforce /etc/apparmor.d/ (only profiles aa-status lists in complain mode) $ sudo aa-logprof (reads AppArmor events from the logs and offers a rule for each; allow only what the program needs) WHY: Mandatory access control: a compromised service cannot read files outside its profile. Ubuntu and Mint enable it by default, so a stopped or disabled AppArmor is itself a finding. NOTE: Enforce profiles by name: aa-enforce /etc/apparmor.d/* also enforces shipped, deliberately unconfined profiles that only grant one permission (see Version notes), breaking those apps. Test the brief's required apps after. [I-09] Logging and audit AREA: Audit logging (CIS Control 8) $ systemctl status rsyslog --no-pager $ df -h /var (a full /var stops the logs; keep space free before you turn auditing up) $ sudo apt install -y auditd && sudo systemctl enable --now auditd $ sudo nano /etc/audit/rules.d/hardening.rules -w /etc/passwd -p wa -k identity -w /etc/shadow -p wa -k identity -w /etc/sudoers -p wa -k sudoers -w /etc/sudoers.d/ -p wa -k sudoers $ sudo augenrules --load && sudo auditctl -l $ sudo ausearch -k identity $ sudo aureport -x --summary (audit events per program; an unfamiliar path is a lead) $ sudo grep -rnE 'audit=(0|off)' /etc/default/grub /etc/default/grub.d/ 2>/dev/null $ sudo mkdir -p /etc/default/grub.d && echo 'GRUB_CMDLINE_LINUX="$GRUB_CMDLINE_LINUX audit=1 audit_backlog_limit=8192"' | sudo tee /etc/default/grub.d/99-audit.cfg && sudo update-grub $ grep -oE 'audit=[^ ]+' /proc/cmdline (after a reboot: audit=1) $ last; lastlog; sudo lastb | head; who; w $ sudo grep -E 'Accepted|Failed|useradd|userdel|usermod' /var/log/auth.log | tail -40 $ sudo grep 'Failed password' /var/log/auth.log | wc -l (one number to track between checks; grep -c counts the same lines) $ sudo grep 'Failed password' /var/log/auth.log | awk '{print $(NF-3)}' | sort | uniq -c | sort -rn | head (failed sign-ins counted per source address) $ journalctl -p err -b --no-pager | tail $ journalctl --list-boots (a single line after several reboots means the journal forgets each boot: set Storage=persistent in /etc/systemd/journald.conf) $ logger -t hardening 'baseline done'; journalctl -t hardening --no-pager (a test line that shows up proves logging works end to end) $ date (the local time the logs stamp; timedatectl status below shows whether NTP keeps it right) $ timedatectl status (System clock synchronized: yes and NTP service: active keep log times comparable; when either reads otherwise, run sudo timedatectl set-ntp true) WHY: Audit rules answer who changed /etc/passwd and when. auditd starts mid-boot, so audit=1 makes the kernel record from the first process; audit=0 anywhere on the kernel line keeps auditing off. NOTE: If rsyslog is not installed (journald only, as on a default Debian install), use journalctl. [I-10] Account hygiene: shells, system accounts, odd groups AREA: Account management (CIS Control 5) $ grep -vE 'nologin|/bin/false|/bin/sync' /etc/passwd $ sudo usermod -s /usr/sbin/nologin $ sudo chage -l alice $ sudo lslogins -u (every human account with its process count, lock state, and last login in one table) $ getent group root shadow adm systemd-journal sudo admin docker lxd disk $ for g in shadow disk adm systemd-journal docker lxd; do awk -F: -v g="$(getent group "$g" | cut -d: -f3)" -v n="$g" 'g!="" && $4==g {print $1" has primary group "n}' /etc/passwd; done $ sudo gpasswd -d shadow WHY: A service account with /bin/bash is a login. Planted group memberships grant quiet privilege: disk reads raw disks, including the shadow file; adm and systemd-journal read the whole journal. NOTE: A primary group of shadow reads every hash without appearing in the member list. ALSO: gpasswd keeps /etc/group and /etc/gshadow in step; a hand edit can split them. [I-11] Apt sources hygiene (deb822 .sources files) AREA: Patching (CIS Control 7) $ cat /etc/apt/sources.list.d/ubuntu.sources $ ls -l /etc/apt/sources.list.d/; cat /etc/apt/sources.list.d/*.sources /etc/apt/sources.list.d/*.list 2>/dev/null $ ls -l /etc/apt/trusted.gpg.d/ /etc/apt/keyrings/ /etc/apt/preferences.d/ 2>/dev/null $ apt-cache policy | grep -- '-security' (expect 500 lines for noble-security; apt policy prints the same table) $ sudo add-apt-repository --remove ppa:evil/ppa $ sudo apt update WHY: A malicious repository or key means apt upgrade installs the attacker's packages. Keep the official archive.ubuntu.com and security.ubuntu.com entries; remove unknown PPAs and the keys their Signed-By fields point to. NOTE: /etc/apt/sources.list may hold only a comment. Each repository is one stanza (Types, URIs, Suites, Components, and Signed-By) in a .sources file under /etc/apt/sources.list.d/. ALSO: An Enabled: no line in a stanza, or a pin in /etc/apt/preferences.d, quietly switches off the security pocket (noble-security). apt-cache policy reveals either one. [I-12] Harden required services instead of removing them AREA: Secure configuration (CIS Control 4) vsftpd /etc/vsftpd.conf: anonymous_enable=NO local_enable=YES write_enable=NO chroot_local_user=YES ssl_enable=YES $ sudo systemctl restart vsftpd && systemctl is-active vsftpd $ openssl s_client -connect 127.0.0.1:21 -starttls ftp -brief : Options FollowSymLinks $ sudo apache2ctl configtest && sudo systemctl restart apache2 (apachectl configtest is the same check) $ curl -sI http://127.0.0.1/ | grep -i '^server:'; curl -si -X TRACE http://127.0.0.1/ | head -1 $ openssl x509 -in /etc/ssl/certs/ssl-cert-snakeoil.pem -noout -subject -issuer -dates (issuer equal to subject means self-signed; notAfter is the expiry) nginx /etc/nginx/nginx.conf http block: server_tokens off; ssl_protocols TLSv1.2 TLSv1.3; $ sudo nginx -t && sudo systemctl reload nginx samba /etc/samba/smb.conf [global]: map to guest = Never restrict anonymous = 2 ; per share: guest ok = no $ testparm && sudo systemctl restart smbd $ sudo mysql_secure_installation (on MariaDB the same script is sudo mariadb-secure-installation) $ sudo mysql -e "SELECT user, host, plugin FROM mysql.user;" (an empty user name, or root with host %, is a finding) mysql /etc/mysql/mysql.conf.d/mysqld.cnf: bind-address = 127.0.0.1 $ sudo systemctl restart mysql $ systemd-analyze security (an exposure level from 0 to 10, where lower is safer; each flagged line names a sandbox setting to add with systemctl edit) $ sudo systemctl edit (a drop-in for sandboxing, such as NoNewPrivileges=yes and ProtectSystem=full; restart it and confirm it stays active, since ProtectSystem=strict also needs ReadWritePaths= for the service's data and logs; systemd-analyze security shows the change) WHY: Required services must keep running, so harden their configs instead; anonymous FTP is the classic finding. /etc/ftpusers lists the accounts FTP refuses, root included, and openssl s_client shows TLS working. NOTE: A bad line, like spaces around =, stops vsftpd; is-active proves the restart took. Apache refuses mixed signed and plain options (Options -Indexes FollowSymLinks); write Options FollowSymLinks or -Indexes +FollowSymLinks. ALSO: After the restarts, curl should print Server: Apache (or nginx) without a version, and TRACE should answer 405. If curl is missing, run sudo apt install curl. [I-13] Browser protections (Firefox and Chromium) AREA: Secure configuration (CIS Control 4) $ snap list firefox chromium 2>/dev/null; pgrep -a firefox $ sudo mkdir -p /etc/firefox/policies $ sudo cp -n /snap/firefox/current/usr/lib/firefox/distribution/policies.json /etc/firefox/policies/ 2>/dev/null $ sudo nano /etc/firefox/policies/policies.json (add these keys inside "policies") "DisableSecurityBypass": {"SafeBrowsing": true}, "PopupBlocking": {"Default": true, "Locked": true}, "HttpsOnlyMode": "force_enabled", "Preferences": {"browser.safebrowsing.malware.enabled": {"Value": true, "Status": "locked"}, "browser.safebrowsing.phishing.enabled": {"Value": true, "Status": "locked"}, "browser.safebrowsing.downloads.enabled": {"Value": true, "Status": "locked"}, "xpinstall.whitelist.required": {"Value": true, "Status": "locked"}} $ python3 -m json.tool /etc/firefox/policies/policies.json >/dev/null && echo valid GUI: Firefox, about:policies: the Active tab lists each policy, and the Errors tab stays empty $ sudo find /home /root -type f \( -name prefs.js -o -name user.js \) -path '*firefox*' -exec grep -HE 'safebrowsing|https_only_mode|disable_open_during_load|xpinstall' {} + 2>/dev/null $ sudo mkdir -p /etc/chromium-browser/policies/managed && echo '{"SafeBrowsingProtectionLevel": 2}' | sudo tee /etc/chromium-browser/policies/managed/hardening.json GUI: Chromium, chrome://policy, Reload policies: SafeBrowsingProtectionLevel 2, status OK WHY: Safe Browsing blocks known phishing and malware pages and downloads, HTTPS-Only Mode asks before loading an unencrypted page, and the add-on warning stops a site installing an extension on its own. A policy file locks a setting for every user; a Settings checkbox covers one profile. A user.js line restores its value at every start, so a protection turned off there is a persistence finding. NOTE: Firefox reads /etc/firefox/policies/policies.json in place of its own copy, so cp brings the shipped policies along first; a new file needs the wrapper {"policies": { ... }}. Write "enabled" rather than "force_enabled" when the brief names a site that works over HTTP alone. Restart the browser. The Chromium snap reads policies from /etc/chromium-browser/policies. Updates come from sudo snap refresh (B-08), so Help, About Firefox reports the version only; close the browser first, because snap updates a running app only after it closes. ------------------------------ HUNTER ------------------------------------------ Hunt and harden. Kernel and network sysctl, rootkits and tampered packages, the immutable-flag trap, PAM internals and managed PAM profiles, forensics tooling, firewall depth, version notes, and a Lynis self-audit. Learn it first: Defense Ladder se1, FreeBSD: Hunting on any Unix. https://freeapp4that.store/defense-ladder/#senior/se1 -------------------------------------------------------------------------------- [A-01] Kernel and network hardening with sysctl AREA: Secure configuration (CIS Control 4) $ sudo nano /etc/sysctl.d/99-hardening.conf net.ipv4.ip_forward = 0 net.ipv4.conf.all.accept_redirects = 0 net.ipv4.conf.default.accept_redirects = 0 net.ipv4.conf.all.send_redirects = 0 net.ipv4.conf.default.send_redirects = 0 net.ipv4.conf.all.accept_source_route = 0 net.ipv4.conf.default.accept_source_route = 0 net.ipv4.conf.all.rp_filter = 1 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.all.log_martians = 1 net.ipv4.tcp_syncookies = 1 net.ipv4.icmp_echo_ignore_broadcasts = 1 net.ipv4.icmp_ignore_bogus_error_responses = 1 net.ipv6.conf.all.accept_redirects = 0 net.ipv6.conf.default.accept_redirects = 0 net.ipv6.conf.all.accept_source_route = 0 net.ipv6.conf.default.accept_source_route = 0 kernel.randomize_va_space = 2 kernel.kptr_restrict = 2 kernel.dmesg_restrict = 1 kernel.sysrq = 0 fs.protected_hardlinks = 1 fs.protected_symlinks = 1 fs.suid_dumpable = 0 $ sudo sysctl --system $ sysctl net.ipv4.ip_forward kernel.randomize_va_space $ dmesg | tail (as a regular user, 'Operation not permitted' proves kernel.dmesg_restrict = 1) $ sysctl -a 2>/dev/null | grep -E 'rp_filter|accept_redirects' (an interface that still reads 2 keeps loose checking: sudo sysctl -w net.ipv4.conf..rp_filter=1) $ sudo grep -rnE 'ip_forward|randomize_va_space|accept_redirects|rp_filter' /etc/sysctl.conf /etc/sysctl.d/ /etc/ufw/sysctl.conf 2>/dev/null WHY: ip_forward=1 enables pivoting; syncookies and rp_filter blunt SYN floods and spoofing; ASLR 2 and suid_dumpable 0 raise exploit cost. sysctl --system loads /etc/sysctl.d/ too; sysctl -p reads /etc/sysctl.conf alone. NOTE: ip_forward goes first: changing it resets the interface settings and would undo lines above it. A bad change can leave ip_forward=1 or randomize_va_space=0 in /etc/sysctl.conf, a file in /etc/sysctl.d/, or /etc/ufw/sysctl.conf, which ufw applies at each start (its keys use slashes: net/ipv4/ip_forward). The stock ufw file sets log_martians=0, so with ufw on, set its two log_martians lines to 1 as well. On 24.04, /etc/sysctl.conf applies last and wins. Stock rp_filter is 2 (loose) in /etc/sysctl.d/10-network-security.conf; 1 is strict and can drop traffic on a host with two network paths. The kernel takes the higher of the all value and each interface value for rp_filter, so read both back. [A-02] Rootkits, tampered packages, malware AREA: Malware defenses (CIS Control 10) $ sudo apt install -y rkhunter chkrootkit debsums clamav $ sudo debsums -c; sudo debsums -ce $ sudo rkhunter --propupd && sudo rkhunter --check --sk $ sudo chkrootkit $ sudo systemctl stop clamav-freshclam; sudo freshclam; sudo systemctl start clamav-freshclam; sudo clamscan -r -i --log=/root/clamscan.log /home /tmp /var/tmp /opt $ clamscan --version (the signature date proves the database loaded) $ which nc ncat netcat socat; sudo find / -xdev -type f \( -name nc -o -name ncat -o -name netcat -o -name socat \) 2>/dev/null $ sudo apt install -y --no-install-recommends aide aide-common && sudo aideinit -y -f (last step of the session) $ sudo aide --check --config /etc/aide/aide.conf WHY: debsums catches a replaced /bin/ls or sshd; rkhunter knows rootkit paths; AIDE reports changes since its baseline. Netcat is the backdoor toolkit: /usr/bin/nc (netcat-openbsd) is stock; hunt copies outside packages. NOTE: Run debsums before rkhunter --propupd, or rkhunter's baseline trusts tampered files. debsums -c skips conffiles; -ce lists edited dpkg ones, /etc/crontab included. sshd_config is ucf-managed, so compare it with /usr/share/openssh/sshd_config. ALSO: clamscan's signatures need internet and about 1.2 GiB of RAM: with 2 GB, close other programs or scan folder by folder. Installing AIDE with --no-install-recommends skips postfix and port 25. [A-03] The lsattr trap, core dumps, Ctrl-Alt-Del AREA: Secure configuration (CIS Control 4) $ sudo lsattr /etc/passwd /etc/shadow /etc/sudoers /etc/ssh/sshd_config $ sudo chattr -i /etc/passwd $ sudo nano /etc/security/limits.conf * hard core 0 $ sudo systemctl mask ctrl-alt-del.target $ findmnt /tmp; findmnt /dev/shm (OPTIONS should read nodev and nosuid, plus noexec where the brief allows; empty output means /tmp lives on the root file system) $ lsblk -f (FSTYPE crypto_LUKS means full-disk encryption; record it, since it is chosen at install time) $ mokutil --sb-state (SecureBoot enabled means the firmware checks each signed step of the boot chain; a BIOS-booted VM prints 'EFI variables are not supported on this system') $ grub-mkpasswd-pbkdf2 (hashes a GRUB superuser password for /etc/grub.d/40_custom: set superusers="root" and password_pbkdf2 root ; mark the normal entry --unrestricted (CLASS line in /etc/grub.d/10_linux) so unattended reboots still start; then sudo update-grub) WHY: An immutable flag set by attackers is the most common reason a fix refuses to stick on a compromised machine, so run lsattr first. NOTE: With the flag set, useradd, passwd, and chage fail even as root, with misleading errors like 'useradd: cannot open /etc/passwd' or 'Authentication token manipulation error'. [A-04] PAM deep dive AREA: Access control (CIS Control 6) required: must pass, keep going. requisite: must pass, stop on fail. sufficient: pass means done. optional. $ grep -nE 'nullok|pam_permit|pam_succeed_if' /etc/pam.d/common-auth /etc/pam.d/sudo /etc/pam.d/login /etc/pam.d/sshd $ sudo sed -i 's/ nullok//g' /etc/pam.d/common-auth $ grep -n 'pam_unix' /etc/pam.d/common-password $ sudo awk -F: '{h=$2; sub(/^!+/,"",h); if (h ~ /^[$](1|5|6)[$]/) print $1, substr(h,1,3)}' /etc/shadow WHY: auth sufficient pam_permit.so atop common-auth lets any password in. Expect yescrypt (the Ubuntu and Mint default), obscure, and remember=5 on the pam_unix line. NOTE: Adding sha512 to the pam_unix line downgrades every new hash. A hash changes only when its password changes, so old-format hashes remain until reset. ALSO: The awk line lists accounts still on MD5 ($1$), SHA-256 ($5$), or SHA-512 ($6$); reset those passwords. [A-05] Forensics toolkit AREA: Investigation (forensics basics) $ sudo find / -xdev -type f -newermt "2026-10-01" -not -path '/proc/*' -not -path '/sys/*' -not -path '/var/lib/*' 2>/dev/null | head -60 $ sudo find / -xdev \( -nouser -o -nogroup \) 2>/dev/null $ sudo find / -xdev -uid -ls 2>/dev/null; getent passwd (empty getent output: the owner was deleted) $ sudo find /home /root /tmp /var/tmp /dev/shm -name '.*' -type f 2>/dev/null $ sudo find /home /root /tmp /var/tmp /dev/shm /opt /srv /usr/local -xdev -type f -size -1024k -not -path '*/.mozilla/*' -not -path '*/snap/*' -not -path '*/.local/share/flatpak/*' -not -path '*/.var/app/*' -exec file {} + 2>/dev/null | grep -iE 'python|perl|shell script|ELF' $ comm -13 <(dpkg-query -L $(dpkg-query -W -f='${binary:Package} ') 2>/dev/null | sed -E 's#^/(s?bin)/#/usr/\1/#' | sort -u) <(sudo find /usr/bin /usr/sbin /usr/local/bin /usr/local/sbin -xdev -type f | sort -u) $ stat ; file ; strings | head; md5sum ; sha1sum ; sha256sum ; sha512sum $ xxd | head -2 (the first bytes give the real type, whatever the name claims: 7f45 4c46 is ELF, 2550 4446 is PDF; sudo apt install xxd when it is missing) $ sudo journalctl _COMM=sudo --no-pager | tail -40 $ history; sudo sh -c 'tail -n 20 /home/*/.bash_history /root/.bash_history' 2>/dev/null (the last lines each account typed; the glob runs as root because home folders are 750; an empty history, or one linked to /dev/null, is itself a finding) $ sudo grep -E 'useradd|userdel|usermod|groupadd|passwd' /var/log/auth.log* 2>/dev/null $ sudo tcpdump -n -i any -c 20 port 4444 (who talks to a suspicious port; -c stops after 20 packets; sudo apt install tcpdump when it is missing) WHY: Investigation questions usually want a hash, a timestamp, a hidden path, or who ran what. find -newermt shows recent changes; the file command reads first bytes, exposing renamed scripts. NOTE: comm prints system programs missing from every package's file list. A few come from install scripts, so confirm with dpkg -S before calling one planted. ALSO: find -uid takes a number, so it still works after the account is deleted. [A-06] Firewall depth: ufw internals, IPv6, unneeded daemons AREA: Secure configuration (CIS Control 4) $ sudo ufw status numbered; sudo ufw delete $ sudo ufw show added; sudo ufw show raw | less (added lists your rules as typed; raw is the full loaded set, before.rules included) $ grep IPV6 /etc/default/ufw $ sudo iptables -S; sudo nft list ruleset 2>/dev/null | head -60 $ sudo grep -nE 'ACCEPT.*--dport' /etc/ufw/before.rules /etc/ufw/user.rules 2>/dev/null $ sudo systemctl disable --now avahi-daemon cups rpcbind nfs-server bluetooth 2>/dev/null WHY: A rule planted in before.rules never shows in ufw status. IPv6 off in ufw leaves half the stack unfiltered. [A-07] Version notes: Ubuntu 24.04 LTS, 26.04 LTS and Debian 13 This page targets Ubuntu 24.04 LTS (noble) with GNOME 46. 24.04 ssh: socket-activated. ssh.socket holds port 22, so disable --now ssh alone leaves it listening; after a Port or ListenAddress change run systemctl daemon-reload, then systemctl restart ssh.socket 24.04 apt: deb822 sources in /etc/apt/sources.list.d/ubuntu.sources; /etc/apt/sources.list is only a pointer 24.04 AppArmor: unprivileged user namespaces are restricted, and many shipped profiles are in unconfined mode just to grant userns; never aa-enforce /etc/apparmor.d/* 24.04 Settings (GNOME 46): Users under System; Screen Lock under Privacy and Security; App Center replaces Ubuntu Software 24.04 also: pam_faillock (no pam_tally2), yescrypt hashes, unattended-upgrades on by default, systemd-timesyncd, Firefox as a snap, rsyslog, last and lastlog 26.04 LTS: sudo-rs is the default sudo (original sudo kept as sudo.ws); Rust coreutils (GNU cp, mv, rm kept); chrony with NTS replaces systemd-timesyncd on new installs; apt 3 (apt-key removed); Software and Updates app no longer installed by default 26.04 time: chronyc tracking shows Leap status Normal and a small System time offset while chrony keeps the clock; on 24.04, timedatectl status shows the same 26.04 sysctl: /etc/sysctl.conf is absent and skipped at boot; kernel settings live in /etc/sysctl.d/*.conf, and Ubuntu's defaults move to /usr/lib/sysctl.d/55-*.conf Debian 13 (trixie): sudo group (not wheel), gdm3; may lack ufw, rsyslog, and unattended-upgrades (apt install them); no snaps; lastlog2 and wtmpdb replace lastlog and last; systemd-sysctl skips /etc/sysctl.conf All of them: AppArmor present, /etc/ssh/sshd_config.d/ includes override the main file WHY: Knowing the defaults stops two wastes: hunting for a tool never installed, and calling a normal default a finding. Every other item holds across these releases; the differences live here. [A-08] Verify loop, every thirty minutes $ sudo ss -tulpn; getent group sudo admin; awk -F: '$3==0' /etc/passwd $ sudo ufw status verbose; sudo aa-status | head -3; systemctl is-active auditd $ sudo awk -F: '$2==""' /etc/shadow; awk -F: '$2!="x"' /etc/passwd; sudo lsattr /etc/passwd /etc/shadow WHY: Cron and timers undo fixes. Re-checking catches the undo, and anything you broke, before a user does. [A-09] Lockout the managed way: pam-auth-update profiles AREA: Account management (CIS Control 5) $ ls /usr/share/pam-configs/; grep '^Module:' /var/lib/pam/auth $ sudo nano /usr/share/pam-configs/faillock Name: Enable pam_faillock to deny access Default: yes Priority: 0 Auth-Type: Primary Auth: [default=die] pam_faillock.so authfail $ sudo nano /usr/share/pam-configs/faillock_notify Name: Notify of failed login attempts and reset count upon success Default: yes Priority: 1024 Auth-Type: Primary Auth: requisite pam_faillock.so preauth Account-Type: Primary Account: required pam_faillock.so $ sudo pam-auth-update --enable faillock faillock_notify $ grep -n pam_faillock /etc/pam.d/common-auth /etc/pam.d/common-account WHY: pam-auth-update builds common-auth and common-account from /usr/share/pam-configs profiles, computing [success=N] jumps, so both files work on Ubuntu and Mint and survive package installs. Priority 1024 precedes pam_unix (256); 0 follows. NOTE: After B-06's hand edit, pam-auth-update prints 'Local modifications to /etc/pam.d/common-*, not updating.' yet exits 0: always reread common-auth. Restore your backup before rerunning, or add --force. faillock.conf keeps the numbers. ALSO: --force leaves /etc/pam.d/common-*.pam-old copies (move them out) and restores nullok (strip it, A-04). Retest su - (B-06) with a root shell open. Indent module lines under Auth: or Account: fields. [A-10] Self-audit with Lynis AREA: Secure configuration (CIS Control 4) $ sudo apt install -y lynis $ sudo lynis audit system --quick $ sudo grep -E '^(warning|suggestion)' /var/log/lynis-report.dat $ sudo grep '^hardening_index=' /var/log/lynis-report.dat $ sudo lynis show details WHY: Lynis reads the system without changing it, then reports warnings, suggestions, and a hardening index from 0 to 100. Run it before and after fixes; the index should climb. NOTE: A suggestion is a lead to check, since some would break a service the brief requires. show details prints the log lines behind one test ID from the report. [A-11] Investigate a live process: /proc, tracers, and hidden PIDs AREA: Malware defenses (CIS Control 10) $ sudo ls -l /proc//exe (a [kworker] or other kernel-thread name with a program file is a disguise) $ tr '\0' ' ' < /proc//cmdline; grep -E 'Name|PPid|Uid' /proc//status $ sudo cat /proc//environ | tr '\0' '\n' (LD_PRELOAD here loads a library ahead of all others) $ sudo ls -l /proc//fd; sudo cat /proc//maps $ sudo pmap $ ldd /path/to/program (a library in maps that ldd leaves out came from LD_PRELOAD or /etc/ld.so.preload) $ sudo strace -p -e trace=network (connect() shows the address it calls; Ctrl+C detaches) $ sudo strace -f -p -e trace=file -s 200 -o /root/trace.txt $ sudo strace -p -e trace=process $ sudo apt install -y ltrace inotify-tools unhide $ sudo ltrace -p -e connect $ sudo fuser -v /path/to/file; sudo fuser -v -n tcp $ sudo fuser -k -n tcp (kills every process on the port: read the -v list first) $ sudo inotifywait -m -r -e modify,create,delete /etc /var/spool/cron $ sudo unhide proc; sudo unhide sys; sudo unhide-tcp WHY: A process can rename itself, so read what the kernel records: its program file, open files, mapped libraries, and starting environment. A tracer then shows what it does right now, such as the address it calls and the files it reads. Record each answer, stop it, then remove its starter. NOTE: Yama's kernel.yama.ptrace_scope is 1 on Ubuntu, so a plain user may trace only its own children: attach with sudo. A real kernel thread shows PPid 2 and an empty exe link. A PID that unhide finds and ps leaves out points at a changed ps (sudo debsums -s) or a rootkit. ltrace, inotify-tools, and unhide install from universe. ------------------------------ FIX OR BREAK? (practice cards) ------------------ Brief: Authorized administrators: alice. Authorized users: bob, carol. Action: Run sudo gpasswd -d bob sudo because bob is in the sudo group. Call: FIX. The brief lists bob as a standard user. Removing that account from sudo is least privilege. Brief: This server hosts the company website with Apache. Action: Run sudo apt purge apache2 to remove the web server. Call: BREAKS IT. Required service. Keep it running and harden it: ServerTokens Prod, no directory listing, updates. Brief: Investigation 2: what is the SHA-256 hash of /home/bob/Music/track.mp3? Action: Delete all media files, then answer the investigation questions. Call: CHECK FIRST. Hash it first, then delete. Evidence before cleanup. Brief: Nothing in the brief mentions /opt/backup-agent. Action: Delete /opt/backup-agent because it is not a standard package. Call: CHECK FIRST. Unrecognized is distinct from unauthorized. Find what runs it (a systemd unit, a cron job, the file owner) and ask the system owner before removing a backup or monitoring agent. Brief: Only alice may log in with SSH; SSH must stay available. Action: Run sudo systemctl disable --now ssh.socket ssh.service. Call: BREAKS IT. SSH is required. Set PermitRootLogin no and AllowUsers alice, and keep it running; ssh.socket is what listens on port 22, so leave it enabled. Brief: No games are allowed on company machines. Action: Run sudo apt purge --auto-remove minetest after confirming no investigation question mentions it, and read the REMOVED list before answering Y. Call: FIX. Acceptable-use violation, and confirmed it is not evidence. Log the removal. Brief: The company uses Samba for the shared drive. Action: Set guest ok = no and map to guest = Never in smb.conf and restart smbd. Call: FIX. Hardening a required service without breaking it. Exactly the job. Brief: Standard users must not be able to run commands as root. Action: Delete the line '%sudo ALL=(ALL:ALL) ALL' from /etc/sudoers with nano. Call: BREAKS IT. That removes sudo for the real admins, and editing with nano risks a syntax error that breaks sudo for everyone. Use visudo and remove only the unauthorized entries. Brief: The machine must not forward traffic between its interfaces. Action: Set net.ipv4.ip_forward = 0 and run sudo sysctl --system. Call: FIX. Matches the brief and removes a pivot. Brief: Your useradd and passwd commands keep failing on this machine with confusing errors, even under sudo. Action: Reinstall the passwd package. Call: CHECK FIRST. Check sudo lsattr /etc/passwd /etc/shadow first. The immutable flag is the usual cause, and a reinstall leaves it in place. Brief: Approved software: Firefox is the company browser. Action: Run sudo snap remove firefox because snaps are not in the apt list of approved packages. Call: BREAKS IT. Firefox on Ubuntu is a snap. Required software stays; audit snap list for unapproved apps instead. Brief: Lock an account after three failed sign-ins until an admin releases it. Action: Set deny = 3 and unlock_time = 0 in /etc/security/faillock.conf, then move on. Call: CHECK FIRST. faillock.conf only holds the numbers. Confirm the pam_faillock lines are in common-auth (grep -n pam_faillock /etc/pam.d/common-auth); locking starts only with them. Brief: Ubuntu desktop: add account lockout to common-auth by hand. Action: Insert the authfail and authsucc lines after the existing [success=2 default=ignore] pam_unix.so line and leave the 2 as it is. Call: BREAKS IT. The 2 now jumps past authfail and authsucc onto the pam_sss and pam_deny lines, so every correct password fails. Set it to [success=1], or use pam-auth-update (A-09). Brief: No hacking tools are allowed on this machine. Action: Run sudo apt purge -y netcat-openbsd because netcat is on the banned list. Call: CHECK FIRST. netcat-openbsd is stock on Ubuntu and part of ubuntu-minimal, so the purge also removes that metapackage. Read the REMOVED list without -y, keep the stock nc unless the brief bans it by name, and hunt copies outside packages (A-02). Brief: Passwords must meet a quality policy. Action: Type password requisite pam_pwquality.so retry=3 into common-password by hand, then run sudo apt install -y libpam-pwquality. Call: CHECK FIRST. The hand-added line makes pam-auth-update treat common-password as locally modified, so the package adds nothing and apt still reports success. Install first, then read it back with grep -n pam_pwquality /etc/pam.d/common-password (B-05). Brief: Passwords must be at least 14 characters. Action: Set minlen = 14 in /etc/security/pwquality.conf while the pam_pwquality.so line still reads retry=3 minlen=8, then move on. Call: CHECK FIRST. Numbers on the pam_pwquality.so line override pwquality.conf, so 8 still applies. Remove minlen=8 from the PAM line and keep the number in the conf file (B-05). Brief: The brief requires SSH and the company website, behind a firewall. Action: Run sudo ufw disable because the website stopped loading after you enabled ufw. Call: BREAKS IT. Turning ufw off exposes every listening port. Allow the site instead: sudo ufw allow 80/tcp (and 443/tcp for HTTPS), then check sudo ufw status numbered. ------------------------------ SPOT THE FINDING (output snippets) -------------- Lines from /etc/passwd. Tap the finding. root:x:0:0:root:/root:/bin/bash alice:x:1000:1000:Alice:/home/alice:/bin/bash >> sysadm:x:0:0::/home/sysadm:/bin/bash www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin WHY: A second UID-0 account is a second root. Remove it (after noting it for the investigation). Output of getent group sudo. Brief: alice is the only admin. Tap the finding. >> sudo:x:27:alice,mallory adm:x:4:syslog,alice users:x:100: plugdev:x:46:alice WHY: mallory in the sudo group is an unauthorized admin: sudo gpasswd -d mallory sudo. Lines from /etc/shadow. Tap the finding. alice:$y$j9T$...:19600:0:99999:7::: >> bob::19600:0:99999:7::: carol:!:19600:0:99999:7::: daemon:*:19600:0:99999:7::: WHY: An empty second field means bob logs in with no password. ! and * mean locked, which is fine. More lines from /etc/passwd. Tap the finding. root:x:0:0:root:/root:/bin/bash alice:x:1000:1000:Alice:/home/alice:/bin/bash >> keola::1003:1003::/home/keola:/bin/bash syslog:x:104:110::/home/syslog:/usr/sbin/nologin WHY: An empty second field in /etc/passwd means keola signs in with no password, whatever /etc/shadow holds. Run sudo pwconv, then sudo passwd keola. Lines from /etc/ssh/sshd_config. Tap the finding. Port 22 >> PermitRootLogin yes PasswordAuthentication yes X11Forwarding no WHY: PermitRootLogin yes lets attackers brute-force root directly. Set it to no and restart ssh. Lines found in /etc/sudoers.d/. Tap the finding. %sudo ALL=(ALL:ALL) ALL Defaults env_reset >> mallory ALL=(ALL) NOPASSWD: ALL root ALL=(ALL:ALL) ALL WHY: Passwordless full sudo for an unlisted user is a root backdoor. Remove it with visudo -f. Output of sudo ss -tulpn. Brief: no FTP on this machine. Tap the finding. tcp LISTEN 0 128 0.0.0.0:22 users:(("sshd",pid=655)) >> tcp LISTEN 0 32 0.0.0.0:21 users:(("vsftpd",pid=812)) udp UNCONN 0 0 127.0.0.53:53 users:(("systemd-resolve",pid=400)) tcp LISTEN 0 128 127.0.0.1:631 users:(("cupsd",pid=701)) WHY: vsftpd on port 21 when the brief forbids FTP: disable it and remove the package. Output of sudo lsattr. Tap the finding. --------------e------- /etc/shadow >> ----i---------e------- /etc/passwd --------------e------- /etc/sudoers --------------e------- /etc/group WHY: The i flag makes /etc/passwd immutable, so user changes fail with a misleading error. Clear it with sudo chattr -i /etc/passwd. Lines from /etc/sysctl.conf. Tap the finding. net.ipv4.tcp_syncookies = 1 >> net.ipv4.ip_forward = 1 kernel.randomize_va_space = 2 fs.protected_symlinks = 1 WHY: ip_forward=1 makes a workstation route traffic, a pivot for attackers. Set it to 0 and apply with sysctl --system. ------------------------------ OFFICIAL SOURCES -------------------------------- Ubuntu security https://ubuntu.com/security Ubuntu 24.04 LTS release notes https://documentation.ubuntu.com/release-notes/24.04/ Ubuntu 26.04 LTS summary for LTS users https://documentation.ubuntu.com/release-notes/26.04/summary-for-lts-users/ Ubuntu Server documentation: firewall (ufw) https://ubuntu.com/server/docs/how-to/security/firewalls/ Ubuntu Server documentation: automatic updates https://ubuntu.com/server/docs/how-to/software/automatic-updates/ sshd socket-based activation (Ubuntu) https://discourse.ubuntu.com/t/sshd-now-uses-socket-based-activation-ubuntu-22-10-and-later/30189 pam_faillock manual (Ubuntu 24.04) https://manpages.ubuntu.com/manpages/noble/man8/pam_faillock.8.html CIS Critical Security Controls v8.1 https://www.cisecurity.org/controls/v8 CIS Benchmarks https://www.cisecurity.org/cis-benchmarks ================================================================================