Field note
What a scaling policy cannot see
A scaling rule is only as honest as its input. CPU-based expansion looks tidy until memory pressure or connection pool exhaustion stalls the application while the scaler still reports headroom.
Map each scaling trigger to a failure mode you have actually seen or can reasonably expect. If cold starts dominate latency, instance count alone is not enough. If a downstream queue fills first, scale the consumer against queue depth rather than host utilization.
Test policies with controlled load that mimics production mix, not synthetic all-read traffic. Mixed read/write patterns often expose lock contention and cache miss storms that pure throughput tests hide.
Document the assumptions behind every threshold. When someone changes the application’s resource profile, those assumptions should be revisited in the same change process—not months later during an outage.