Appendix A-Hardening and Performance Tuning Reference

Curriculum-wide consolidation of the hardening and tuning slices. By week 24 the reader's host-baseline/ template should contain working examples of each section.

---

A.1 The systemd Hardening Cheat Sheet

For every long-running service, evaluate each:

[Service]
# Identity
DynamicUser=yes
User=svc
Group=svc

# Filesystem isolation
ProtectSystem=strict
ProtectHome=yes
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectKernelLogs=yes
ProtectControlGroups=yes
ProtectClock=yes
ProtectHostname=yes
ProtectProc=invisible
PrivateTmp=yes
PrivateDevices=yes
PrivateUsers=yes
PrivateMounts=yes
ReadOnlyPaths=/
ReadWritePaths=/var/lib/svc

# Networking
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
IPAddressAllow=localhost
IPAddressDeny=any
RestrictNetworkInterfaces=lo eth0

# Capabilities & syscalls
NoNewPrivileges=yes
CapabilityBoundingSet=
AmbientCapabilities=
SystemCallArchitectures=native
SystemCallFilter=@system-service
SystemCallFilter=~@privileged @resources

# Resources
MemoryMax=512M
MemoryHigh=384M
CPUQuota=200%
TasksMax=128
LimitNOFILE=65536

# Other
LockPersonality=yes
RestrictNamespaces=yes
RestrictRealtime=yes
RestrictSUIDSGID=yes
RemoveIPC=yes
UMask=0077

Validate with `systemd-analyze security - aim for "exposure level: 1.0 OK" or below.

---

A.2 Sysctl Baseline (Server)

# Network-connection backlog
net.core.somaxconn = 4096
net.ipv4.tcp_max_syn_backlog = 4096
net.core.netdev_max_backlog = 16384

# Network-TCP behavior
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_keepalive_time = 60
net.ipv4.tcp_keepalive_intvl = 10
net.ipv4.tcp_keepalive_probes = 6

# Network-anti-spoof
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv6.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.icmp_echo_ignore_broadcasts = 1

# VM / memory
vm.swappiness = 10
vm.dirty_ratio = 10
vm.dirty_background_ratio = 3
vm.overcommit_memory = 0
vm.mmap_min_addr = 65536
vm.unprivileged_userfaultfd = 0

# File handles & inotify
fs.file-max = 2097152
fs.inotify.max_user_watches = 524288
fs.protected_hardlinks = 1
fs.protected_symlinks = 1

# Kernel hardening
kernel.dmesg_restrict = 1
kernel.kptr_restrict = 2
kernel.unprivileged_bpf_disabled = 1
kernel.yama.ptrace_scope = 1
kernel.perf_event_paranoid = 2
kernel.kexec_load_disabled = 1

Customize per workload; never copy blindly.

---

A.3 The perf Reference Card

Goal	Command
Top CPU consumers (live)	perf top -F 99 -g
Sample profile + call graph	perf record -F 99 -g -- sleep 30; perf report
Flamegraph	perf record -F 99 -ag -- sleep 30; perf script | stackcollapse-perf.pl | flamegraph.pl > out.svg
Counter snapshot	perf stat -e cycles,instructions,cache-misses ./prog
Tracepoints	perf trace -F (strace replacement)
Sched debug	perf sched record sleep 10; perf sched latency
Block I/O	perf trace -e block:* -- sleep 10

---

A.4 The bpftrace Reference Card

# top syscalls per process for 10 s
bpftrace -e 'tracepoint:raw_syscalls:sys_enter { @[comm] = count() }' -c 'sleep 10'

# file open audit
bpftrace -e 'tracepoint:syscalls:sys_enter_openat { printf("%s %s\n", comm, str(args->filename)) }'

# tcp connect latency
bpftrace -e 'kprobe:tcp_v4_connect { @start[tid] = nsecs }
              kretprobe:tcp_v4_connect /@start[tid]/ {
                  @lat = hist((nsecs - @start[tid])/1000); delete(@start[tid])
              }'

# run-queue latency histogram
bpftrace tools/runqlat.bt

# what's filling the page cache
bpftrace -e 'kprobe:add_to_page_cache_lru { @[comm] = count() }'

---

A.5 SystemTap (Legacy)

SystemTap predates eBPF and is still occasionally useful on RHEL 7-era kernels:

stap -e 'probe syscall.open { printf("%s %s\n", execname(), filename) }'

For new work, prefer eBPF / bpftrace. Only learn stap if your environment forces it.

---

A.6 The host-baseline/ Template

host-baseline/
  ansible/
    roles/
      common/        # hostname, time, base packages
      sshd/          # hardened sshd_config
      audit/         # auditd ruleset
      sysctl/        # the baseline above
      firewall/      # nftables ruleset
      lsm/           # SELinux or AppArmor profiles
      observability/ # node_exporter, journald-remote, eBPF tools
      cgroups/       # tenant cgroup template
  systemd-units/     # service templates
  ebpf-tools/        # in-house tracing tools
  RUNBOOK.md
  THREAT_MODEL.md

This is the artifact every host you bring up after week 24 should be provisioned from.

Comments