Our server ignored SIGTERM for an hour
A rolling deploy hung. The container was told to stop and simply did not, because of a detail of how PID 1 works that almost every Dockerfile gets wrong.
A deploy stalled with the old pod stuck in Terminating. Kubernetes had sent SIGTERM, waited out the grace period, and eventually killed it. The next deploy did the same. Two outages and two deadlocked rollouts later, the cause turned out to be four characters of process semantics.
PID 1 is not an ordinary process
The kernel applies no default signal action to PID 1. For any other process, a SIGTERM with no handler installed means terminate. For PID 1, a SIGTERM with no handler means nothing at all — the signal is delivered and discarded.
In a container, your application is usually PID 1. So a process that has not explicitly installed a SIGTERM handler cannot be asked to stop. It can only be killed.
Our NestJS app had not called enableShutdownHooks(), so Nest never registered a listener, so nothing in the process was watching for the signal. The container ran until SIGKILL arrived at the end of the grace period.
// Without this, Nest registers no signal listeners at all. As PID 1 the
// kernel applies no default action either, so SIGTERM is delivered and
// discarded and the container runs until it is killed.
app.enableShutdownHooks();The part that cost money
The outage was the visible damage. The quieter damage was billing. Voice sessions hold a lease that the server renews in the background and releases on shutdown, and usage is recorded as the time between those two points. A process that never gets to run its shutdown path never releases anything.
One abandoned session accumulated 101 minutes against an account whose entire monthly allowance was smaller than that.
Nobody was on the call. The lease simply kept renewing until something else cleaned it up, and the usage record was written for the whole span.
What we changed
Shutdown hooks are enabled, and the drain is bounded — a shutdown that waits indefinitely for in-flight work is a different way to hang. Separately, the lease renewal now checks that a participant is actually present, so a session nobody is on releases rather than renews.
Time from SIGTERM to exit went from the full 3600-second grace period to 352 milliseconds.
The generalisable version: if your container takes the full grace period to stop, it is not slow. It is not listening.