Kubernetes makes it easy to define CPU and memory requests for our applications.
But there is a problem:
How do we know whether those resource requests are actually correct?
If an application requests:
yaml
resources:
requests:
cpu: "1000m"
memory: "1Gi"
but normally consumes only a few millicores of CPU and a few megabytes of memory, we may be reserving significantly more cluster capacity than the workload actually needs.
I wanted to understand how Kubernetes cost optimization platforms detect this situation, so I built a small hands-on lab using:
Google Kubernetes Engine (GKE)
CAST AI
Kubernetes
Docker
FastAPI
Google Artifact Registry
The goal wasn't simply to install CAST AI.
I wanted to observe the complete process:
Deploy workload
↓
Observe resource usage
↓
Compare requests vs usage
↓
Identify over-provisioning
↓
Generate recommendation
↓
Apply rightsizing
↓
Verify from Kubernetes
Architecture
The lab architecture was intentionally simple.
FastAPI Coffee API
|
v
Docker Image
|
v
Google Artifact Registry
|
v
GKE Cluster
|
v
Kubernetes Deployment
|
+----------------+
| |
v v
Pod #1 Pod #2
| |
+-------+--------+
|
v
ClusterIP Service
+
|
v
CAST AI
|
+-------+-------+
| |
v v
Cost Monitoring Workload Optimization
1. Building a Small Test Application
I created a very small FastAPI application for the experiment.
from fastapi import FastAPI
import socket
import os
import time
app = FastAPI()
@app.get("/")
def home():
return {
"message": "Coffee Shop API",
"hostname": socket.gethostname(),
"pod": os.getenv("HOSTNAME"),
"time": time.time()
}
@app.get("/coffee")
def coffee():
return {
"coffee": "Cappuccino",
"price": 120
}
The hostname in the response was useful later because I could see which Kubernetes Pod handled each request.
2. Containerizing the API
The application was packaged using Docker.
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app.py .
EXPOSE 8000
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]
One small issue appeared immediately.
I accidentally had:
uvicornc
inside requirements.txt.
The Docker build failed with:
ERROR: No matching distribution found for uvicornc
The correct package was:
uvicorn
A simple typo, but a useful reminder that a Docker build failure doesn't necessarily mean there is a Docker problem.
Sometimes the failing layer is simply the application dependency layer.
3. Testing the Container Locally
Port 8000 was already occupied on my machine.
Instead of modifying the application, I mapped another host port:
docker run -d \
--name coffee-api \
-p 9999:8000 \
coffee-api:v1
This means:
localhost:9999
|
v
Docker port mapping
|
v
container:8000
The API could then be tested with:
curl http://localhost:9999/
and:
curl http://localhost:9999/coffee
4. Pushing the Image to Artifact Registry
The local Docker image needed to be accessible by GKE.
I tagged it for Google Artifact Registry:
docker tag coffee-api:v1 \
us-central1-docker.pkg.dev/<PROJECT>/<REPOSITORY>/coffee-api:v1
and pushed it:
docker push \
us-central1-docker.pkg.dev/<PROJECT>/<REPOSITORY>/coffee-api:v1
Now the deployment flow became:
Local Source
↓
Docker Image
↓
Artifact Registry
↓
GKE pulls image
↓
Pod starts
5. Intentionally Over-Provisioning the Workload
This was the important part of the experiment.
I intentionally gave the application much more CPU and memory than it needed.
resources:
requests:
cpu: "1000m"
memory: "1Gi"
limits:
cpu: "1500m"
memory: "1500Mi"
I deployed two replicas.
Therefore Kubernetes was effectively being told:
Pod #1 → 1 CPU + 1 GiB
Pod #2 → 1 CPU + 1 GiB
Total requested:
2 CPU
2 GiB memory
The workload was deliberately oversized so I could observe how CAST AI would analyze it.
6. Deploying to GKE
Before creating the workload, I used a server-side dry run:
kubectl apply \
-f deployment.yaml \
--dry-run=server
Then deployed it:
kubectl apply -f deployment.yaml
Verification:
kubectl get pods -l app=coffee-api
showed two running Pods.
7. Testing Kubernetes Self-Healing
I manually deleted one of the Pods:
kubectl delete pod <coffee-api-pod>
Then watched the Deployment:
kubectl get pods -l app=coffee-api -w
Kubernetes immediately created another Pod.
Why?
Because the Deployment declared:
replicas: 2
The controller continuously compares:
Desired State
vs
Actual State
When the actual state dropped to one Pod, Kubernetes reconciled it back to two.
8. Adding a ClusterIP Service
I exposed the workload internally using a Kubernetes Service.
apiVersion: v1
kind: Service
metadata:
name: coffee-api-service
spec:
selector:
app: coffee-api
ports:
- protocol: TCP
port: 80
targetPort: 8000
type: ClusterIP
Because it is a ClusterIP, there is intentionally no external IP.
The Service is intended for communication inside the cluster.
9. Testing Kubernetes Service Discovery
From another Pod inside the cluster, I called:
curl http://coffee-api-service/
Kubernetes DNS resolved the Service name.
The complete DNS name was:
coffee-api-service.default.svc.cluster.local
The Service had two backend endpoints corresponding to the two Coffee API Pods.
Repeated requests returned different Pod hostnames.
That gave me a simple demonstration of:
Client Pod
|
v
coffee-api-service
|
+-------------+
| |
v v
Coffee Pod 1 Coffee Pod 2
10. Connecting the GKE Cluster to CAST AI
Next I connected the cluster to CAST AI.
This part produced some of the most useful troubleshooting lessons in the entire experiment.
I encountered issues around:
gke-gcloud-auth-plugin
Google Cloud CLI installation
CAST AI authentication
CAST AI organization selection
GCP IAM permissions
Kubernetes RBAC
One particularly interesting problem occurred during CAST AI onboarding.
CAST AI needed permissions including:
iam.roles.create
iam.roles.update
iam.roles.undelete
My Google Cloud account already had several IAM roles, but it still didn't have these specific permissions.
Adding the required IAM role-management capability allowed onboarding to continue.
This reinforced an important IAM lesson:
Having many roles doesn't necessarily mean you have the permission required for a specific operation.
11. CAST AI Components Inside Kubernetes
After onboarding, CAST AI installed several components in the cluster.
A new namespace appeared:
castai-agent
and components included things such as:
castai-agent
castai-cluster-controller
castai-kvisor-agent
castai-kvisor-controller
castai-pod-mutator
castai-workload-autoscaler
CAST AI also created Kubernetes RBAC resources.
This helped me see the difference between two authorization layers:
GCP IAM
|
+---- permissions against cloud infrastructure
Kubernetes RBAC
|
+---- permissions against Kubernetes resources
They solve different problems.
12. The Interesting Part: Requests vs Actual Usage
My Deployment requested:
1 CPU
1 GiB
per Pod.
With two Pods:
2 CPU
2 GiB
were requested.
But checking actual usage:
kubectl top pods -l app=coffee-api
showed approximately:
CPU Memory
4m 30Mi
4m 30Mi
This is where the experiment became interesting.
The application was requesting dramatically more resources than it was actually consuming.
13. Understanding What CAST AI Was Showing
Initially I saw something like:
Coffee API
CPU: 2 CPU
Memory: 2 GiB
in the cost monitoring view.
At first, it is easy to interpret this as actual usage.
It wasn't.
It represented the resources requested by the workload.
That distinction is extremely important:
Requested Resources
!=
Actual Usage
Kubernetes scheduling decisions are strongly influenced by resource requests.
So an application can consume very little CPU while still reserving significantly more capacity from the scheduler's perspective.
14. CAST AI Workload Recommendation
After observing the workload, CAST AI identified the Coffee API as over-provisioned.
The original configuration was:
CPU Request: 1 CPU
Memory Request: 1 GiB
CAST AI recommended approximately:
CPU: 750m
Memory: 768Mi
or:
Original Recommended
1 CPU --------> 750m
1 GiB --------> 768Mi
The dashboard estimated roughly a 25% reduction in workload cost for this test workload.
15. Why Didn't It Recommend 4m CPU?
This was one of the most interesting questions for me.
If the application was using only around:
3-4m CPU
why recommend:
750m
instead of something close to actual usage?
Because safe workload optimization cannot simply use:
request = current usage
A production workload may experience:
traffic spikes
bursts
startup activity
background jobs
temporary CPU demand
Optimization systems therefore have to consider safety margins, historical behavior, policies, stability, and available observations.
That is an important difference between measuring utilization and making a safe resource recommendation.
16. Applying Vertical Rightsizing
I enabled workload optimization for the Coffee API.
The interesting result was that the original Deployment template still showed:
CPU: 1
Memory: 1Gi
while the running Pods showed:
CPU: 750m
Memory: 768Mi
That initially looked confusing.
So instead of trusting only the dashboard, I inspected Kubernetes directly.
17. Verifying the Runtime Allocation
I checked the running Pod:
kubectl get pod <pod-name> \
-o jsonpath='{.spec.containers[0].resources}'
and then inspected the runtime allocation:
kubectl get pod <pod-name> \
-o jsonpath='{.status.containerStatuses[0].allocatedResources}'
Kubernetes reported:
{
"cpu": "750m",
"memory": "768Mi"
}
That was the point where the experiment clicked for me.
Instead of saying:
"CAST AI says the workload was optimized."
I could verify from Kubernetes itself that the running container had the optimized allocation.
18. The Mental Model I Took Away
Before this experiment, it is easy to think about Kubernetes resources as one number.
In reality, there are several concepts to separate:
Resource Request
|
v
What the workload asks Kubernetes to reserve
Actual Usage
|
v
What the application really consumes
Recommended Resources
|
v
What an optimizer believes is appropriate
Runtime Allocation
|
v
Resources currently allocated to the running container
Infrastructure Cost
|
v
What the underlying nodes ultimately cost
These values are related, but they are not the same thing.
19. Workload Optimization Is Only Part of the Story
Another important takeaway:
Reducing a Pod request does not automatically mean your cloud bill instantly decreases.
For example:
Pod requests reduced
↓
Node has more free capacity
↓
More workloads can fit on existing node
But if the same number of VMs continues running, the infrastructure cost may remain unchanged.
To translate workload efficiency into infrastructure savings, the cluster also needs effective node-level optimization.
So there are really two related layers:
Workload optimization
CPU / memory rightsizing
↓
Better pod packing
Node optimization
Consolidation / autoscaling
↓
Fewer or cheaper VMs
↓
Actual infrastructure savings
This distinction is especially important when thinking about Kubernetes FinOps.
20. Troubleshooting Was Half the Learning
The successful dashboard wasn't actually the most valuable part of this lab.
The failures taught me more.
During the project I encountered:
Google Cloud CLI installation issues
gke-gcloud-auth-plugin problems
stale Kubernetes contexts
CAST AI authentication expiration
CAST AI vs GCP identity differences
missing IAM permissions
Docker dependency typo
occupied host ports
ClusterIP networking
Kubernetes DNS behavior
deprecated Endpoints API warning
Deployment vs Pod resource differences
Each issue forced me to identify which layer was actually failing.
My debugging model became:
Application
↓
Docker
↓
Artifact Registry
↓
Pod
↓
Service / DNS
↓
Kubernetes
↓
GKE
↓
Google Cloud IAM
↓
CAST AI
Before changing anything, ask:
Which layer is actually failing?
That simple question prevents a lot of random troubleshooting.
What I Learned
This experiment helped me understand several Kubernetes concepts more clearly:
Resource requests are not actual resource usage.
Over-provisioned requests can waste cluster capacity.
Workload rightsizing and node optimization solve different problems.
Kubernetes Services provide stable networking over dynamic Pods.
GCP IAM and Kubernetes RBAC are separate authorization layers.
Cloud integrations often require permissions in both Kubernetes and the cloud provider.
Cost dashboards need to be interpreted carefully.
Optimization recommendations should be verified against the actual Kubernetes runtime.
Troubleshooting cloud-native systems becomes easier when you identify the failing layer first.
Most importantly, Kubernetes cost optimization isn't only:
"Find cheaper VMs."
It also starts with asking:
Are my workloads requesting the resources they actually need?
Source Code and Full Lab
I documented the complete experiment, including:
GKE setup
CAST AI onboarding
FastAPI deployment
Docker and Artifact Registry
Kubernetes networking
cost monitoring
workload optimization
troubleshooting notes
GitHub:
https://github.com/Upshivam786/castai-gke-workload-optimization-lab
If you're learning Kubernetes, MLOps, DevOps, or FinOps, reproducing a small experiment like this is a great way to understand what resource optimization actually means instead of only looking at architecture diagrams.
Enter fullscreen mode Exit fullscreen mode
0 Comments
Log in to join the conversation.No comments yet. Be the first to share your thoughts.