Your Kubernetes health checks are accidentally waking your services. Here’s the fix.

Posted on July 29, 2026 by KubeElasti Maintainers

CNCF projects highlighted in this post

Scale-to-zero breaks when health checks scale you back up. Learn how KubeElasti’s ProbeResponse lets Kubernetes services stay genuinely idle — while keeping load balancers and uptime monitors happy.

Scale-to-zero sounds perfect on paper.

Idle service. No pods. No bill.

Then you deploy it in production and discover a problem nobody warned you about: your load balancer doesn’t know your pods are intentionally gone. It keeps sending health checks. Every health check hits your resolver. Your resolver, doing its job, interprets traffic as a scale-up signal. Your pods wake up.

You’re paying for idle pods again.

This isn’t a KubeElasti bug. It’s a fundamental tension in how modern infrastructure is stitched together. Load balancers need to know a service is alive. Kubernetes autoscalers need to know a service is dead. These two things have been at odds since the first team tried running zero-replica deployments behind a cloud load balancer.

KubeElasti’s new ProbeResponse feature resolves this tension directly.

The Problem Nobody Talks About

Here’s how scale-to-zero breaks in most production environments:

You configure your service to scale down after 10 minutes of inactivity.

Your load balancer has been told to health-check every 30 seconds.

The moment your last pod terminates, two things happen simultaneously:

• Your Kubernetes service is now pointing at KubeElasti’s resolver.
• Your load balancer sends a GET /healthz.

The resolver receives a request. To KubeElasti, any incoming request is a signal: “someone needs this service.” It notifies the operator. The operator begins scaling up.

You’ve never gone idle. You’ve just added a 30-second loop that continuously wakes your workload.

This problem appears in three distinct forms:

1. Cloud Load Balancers
   AWS ALB, GCP Load Balancer, Azure Application Gateway — they all require periodic health checks to route traffic safely. These checks are frequent, relentless, and don’t understand the concept of “intentionally idle.”
2. Kubernetes Liveness and Readiness Probes
   Even internal probes configured on your pods often get replicated at the ingress layer. A service that’s scaled to zero but registered with an uptime monitor will trigger requests.
3. Platform Monitoring
   Internal developer platforms, service meshes like Istio, and observability stacks like Prometheus Blackbox Exporter continuously probe endpoints to assert availability. None of them distinguish between a degraded service and a deliberately idle one.

The result: teams that configure scale-to-zero achieve exactly 0% of the potential savings because their monitoring stack never lets them stay at zero.

Why Conventional Wisdom Fails

The typical advice is to separate your health check endpoint routing from your application routing. “Put health checks on a different service,” people suggest.

That doesn’t work for three reasons:

First, you don’t always control the health check configuration. Cloud load balancers check the IP and port your service is registered on. You can’t tell an AWS ALB to use a different backend for /healthz.

Second, separating routing creates operational complexity that teams never maintain. Over time, drift happens. Health checks start hitting your application endpoints anyway.

Third, and most importantly: the health check endpoint IS your application endpoint. The /healthz path is the path. Rerouting it defeats the purpose.

Other tools solve this by keeping a proxy permanently in the request path — even when the service is running. You trade latency for the ability to intercept health checks at all times. It works, but it’s an architectural compromise that compounds over time.

How ProbeResponse Works

KubeElasti’s ProbeResponse feature adds a matching layer directly to the resolver.

When your service is scaled to zero, the resolver is already intercepting all incoming traffic. ProbeResponse lets you define rules that the resolver evaluates before deciding whether to trigger a scale-up.

If a request matches a probe rule, the resolver answers it directly — with the status code and body you define — and never notifies the operator. The workload stays at zero. The health check receives a 200 OK. Everyone is happy.

Here’s what the configuration looks like:

probeResponse:
  - method: GET
    path:
      type: PathPrefix
      value: /healthz
    response:
      status: 200
      body: '{"ok":true}'
  - method: HEAD
    path:
      type: Exact
      value: /ready
    response:
      status: 204
      body: '{}'

The matching rules support:

• HTTP method (GET, HEAD, POST, etc.)
• Path matching: Exact, PathPrefix, or RegularExpression
• Header matchers (all ANDed)
• Query parameter matchers (all ANDed)

Rules are evaluated top to bottom. First match wins. If no rule matches, normal behavior applies — the request queues and triggers a scale-up.

This design is intentional. ProbeResponse is a gate, not a replacement for scaling. Real traffic still wakes your service. Synthetic health traffic stays synthetic.

The Architecture Insight

What makes ProbeResponse different from hacks like “route /healthz to a separate always-on pod” is where the intelligence lives.

In KubeElasti’s architecture, the resolver is already the interception layer during proxy mode. Adding probe matching here costs nothing architecturally — no extra hop, no additional component, no always-on proxy overhead.

When your service has active pods, the resolver is completely out of the request path. KubeElasti switches to serve mode and routes traffic directly to your pods. ProbeResponse rules only apply when the resolver is active, which is only when your service is at zero replicas.

This means:

• No latency penalty when your service is running
• No infrastructure overhead to maintain probe handling
• No separate deployment or service for health check routing

The same resolver that queues real requests and triggers scale-up simply checks an in-memory rule set before deciding what to do with a request. It’s genuinely zero-cost when your service is active.

Real-World Implications

Consider a few scenarios where this changes the economics:

Internal Development Services

Teams often run internal tools — CI preview environments, internal dashboards, test services — that need to exist but only get real traffic sporadically. With ProbeResponse, these services stay at zero between actual use sessions, even if platform monitoring checks them every 30 seconds.

GPU Workloads

GPU pods are expensive. The primary reason teams don’t scale GPU services to zero is that they can’t tolerate cold starts triggered by health checks. ProbeResponse makes genuine GPU idle time achievable — answer the health check from the resolver, keep the GPU pod terminated until real inference traffic arrives.

Licensed Software

Enterprise software that charges per running instance needs to stay off until genuinely needed. Health check traffic is not “needed.” ProbeResponse enforces that distinction.

East-West Traffic

Internal microservices often call each other on health check paths before sending real traffic. A service mesh like Istio may run periodic connectivity checks. ProbeResponse can handle these without triggering premature scale-up.

Practical Recommendations

Before configuring ProbeResponse, audit your health check sources:

1. Check your cloud load balancer health check configuration — identify the path, method, and frequency.
2. Review your Kubernetes ingress annotations — NGINX, Traefik, and Istio all have configurable health check behaviors.
3. Check your uptime monitoring — if you’re using Prometheus Blackbox Exporter, Datadog synthetic monitors, or any external uptime tool, note which endpoints they probe.
4. Review your service mesh configuration — Istio, Linkerd, and similar tools often run health checks as part of their sidecar behavior.

For each health check source, create a corresponding ProbeResponse rule. Use PathPrefix for broad coverage (e.g., all paths under /health/) and Exact for specific endpoints.

Set conservative response bodies. Return what your real service would return for a healthy state. If your monitoring stack validates the response body, ProbeResponse needs to match exactly.

Use header and query parameter matchers to disambiguate health checks from real traffic when they share the same path. Some monitoring tools set identifying headers you can use for this purpose.

KubeElasti Perspective

Scale-to-zero has always been a two-part problem.

The first part — scaling down to zero, queuing requests, scaling back up — is the visible problem. It’s the one that shows up in talks, blog posts, and feature comparisons.

The second part — staying at zero despite ambient infrastructure noise — is the problem that actually determines whether scale-to-zero delivers its theoretical value.

ProbeResponse is KubeElasti’s answer to the second part.

Teams that deploy KubeElasti without ProbeResponse often see partial savings. Their service scales to zero, then immediately scales back up due to a health check, stays alive for the cooldown period, and the cycle repeats. The logs show scale-up events but no corresponding user traffic.

ProbeResponse closes that loop.

When combined with a properly configured cooldownPeriod, ProbeResponse enables what we’d call genuine idle time — intervals where your service is truly off, not just between health check cycles.

For teams with GPU workloads, licensed software, or high-frequency internal services, this is the difference between a proof of concept and production-grade cost reduction.

Conclusion

Health checks are not optional. Cloud load balancers require them. Service meshes rely on them. Platform monitoring depends on them.

Scale-to-zero is not optional either — not if you’re serious about infrastructure efficiency.

For years, the tension between these two requirements forced teams into compromises: always-on proxies, separate health check services, or abandoning scale-to-zero on services that mattered most.

ProbeResponse removes that compromise.

Your health checks get their 200 OK. Your workload stays at zero. Your infrastructure costs reflect actual usage, not monitoring overhead.

That’s what genuine scale-to-zero looks like.

Key Takeaways

• Health checks are the most common reason scale-to-zero fails to deliver savings in production
• ProbeResponse lets the KubeElasti resolver answer configured probe requests directly, without triggering scale-up
• Rules match on method, path (Exact/PathPrefix/Regex), headers, and query params
• ProbeResponse only applies in proxy mode — zero overhead when your service is running
• GPU workloads, licensed software, and internal services gain the most from this feature
• Audit your health check sources before configuring — cloud LBs, service meshes, uptime monitors, and ingress controllers all probe independently