Project

General

Profile

Actions
Copy link

Bug #376

closed

Bug: DNS cluster up/down alerts in Uptime Kuma + Pi-hole hardening never committed to repo

Bug #376: Bug: DNS cluster up/down alerts in Uptime Kuma + Pi-hole hardening never committed to repo

Added by Charles N 9 days ago. Updated 3 days ago.

Status:
Resolved
Priority:
High
Assignee:
-
Target version:
TSYS Group - Potential to Kinetic Ready
Start date:
08/06/2026
Due date:
% Done:

100%

Estimated time:

Description

Symptom

Uptime Kuma reports intermittent up/down alerts for the DNS cluster
(pfvi-netinfra-01 192.168.3.252 + pfv-netinfra-02 192.168.3.253).

Investigation findings (2026-08-06)

DNS service is HEALTHY right now. Verified from localhost on both nodes:

  • knownelement.com, git.knownelement.com, google.com all resolve correctly
  • Both Pi-hole containers: state=running, health=healthy, RestartCount=0
  • No container restart/die events in the last 6h on either node
  • shm_size fix (1024M) IS live and working on both nodes
  • gravity-validate.sh, healthcheck, and autoheal container all present and active

Root problem: hardening config was never committed

Commit 33b5c76 ("feat(netinfra): rebuild netinfra-02 ... + harden Pi-hole")
claims to have hardened Pi-hole on both nodes, but git show --stat proves it
only modified TWO markdown files (STATUS.md, AUDIT-2026-08-05.md). Zero config
files, zero scripts were ever written to the repo.

The working config lives ONLY on the boxes at:
/home/localuser/services/pihole/docker-compose.yml
/home/localuser/services/pihole/gravity-validate.sh

This means the DNS hardening cannot be reproduced, audited, or re-deployed
from version control. Related to the operator's earlier production outage
(gravity.db corruption from undersized /dev/shm).

Flapping source (still under investigation)

DNS resolves correctly from the boxes. Off-box LAN testing from the dev host
is unreliable (dev host has a conflicting Docker bridge on 192.168.3.0/24).
Need to inspect the Uptime Kuma monitor target (LAN IP vs Tailscale IP vs
port type) to determine why the monitor sees flaps.

Next steps

  1. Commit the live Pi-hole compose + gravity-validate.sh into the repo
    (netinfra/pihole/) with secrets handled.
  2. Inspect Uptime Kuma monitor config to find the flapping trigger.
  3. If monitor targets LAN IP, consider switching to Tailscale IP for stability.

References

  • Related closed ticket: #357 (DNS cluster build)
  • Fraudulent commit: 33b5c76
  • Code location: PFVCluster/netinfra/
Actions
Copy link

Also available in: PDF Atom