1. Understanding kubectl port-forward
By default, when you use kubectl port-forward, it only binds to 127.0.0.1 (localhost).
• That means only processes on your own machine can reach the forwarded port.
• This is safer because it prevents external access unless you explicitly allow it.
⸻
🔹 Example: Forwarding Pod Port to Local Machine
Suppose your Pod (my-app-pod) exposes port 80, and you want to access it locally on 8080:
kubectl port-forward pod/my-app-pod 8080:80
Maps Pod’s port 80 → localhost:8080
• Accessible only on your machine, not from the network.
⸻
🔹 Exposing Beyond Localhost
If you want to allow other machines on your network (LAN or cloud VM) to access it:
kubectl port-forward --address 0.0.0.0 pod/my-app-pod 8080:80
--address 0.0.0.0 → binds to all network interfaces.
• Now anyone who can reach <your-ip>:8080 can access the Pod’s service.
✅ Useful for:
• Testing webhooks from external services
• Allowing teammates to test your Pod’s API or UI
• Debugging cluster apps from another device
Security Considerations
• Opening port-forward to 0.0.0.0 is not secure on untrusted or public networks.
• Anyone with access to your IP can now hit your Pod directly.
• Mitigation options:
• Restrict with firewall rules (e.g., allow only certain IP ranges).
• Use SSH tunnels instead of --address 0.0.0.0.
• Prefer creating a Kubernetes Service + Ingress for controlled external access.
⸻
🔹 Pro Tip: Port-Forwarding a Service Instead of a Pod
Instead of binding to a specific Pod (which may restart or move), you can forward a Service:
2. Injecting an Ephemeral Debug Container in Kubernetes
Sometimes your Pod’s image is minimal (like nginx, httpd, or Google’s distroless images) and does not include debugging tools (e.g., no bash, sh, curl, ps).
Instead of rebuilding your image, Kubernetes lets you attach an ephemeral debug container into the running Pod, sharing its namespaces (network, process, filesystem).
Ephemeral containers were introduced in Kubernetes v1.18 (beta by v1.23, stable in v1.25).
Step 1: Create the Base Pod (without debug tools)
apiVersion: v1
kind: Pod
metadata:
name: debug-demo
spec:
containers:
- name: app
image: nginx
kubectl apply -f debug-demo.yaml
At this point, the Pod runs nginx but has no shell or debugging tools.
🔹 Step 2: Inject an Ephemeral Container
kubectl debug debug-demo \
--image=busybox \
--target=app \
--name=debug-shell \
-- bash
• --image=busybox → use BusyBox (has sh, curl, netcat, etc.). You could also use ubuntu, debian, or nicolaka/netshoot for richer debugging tools.
• --target=app → joins the namespaces (network, process, IPC, etc.) of the main container (app).
• --name=debug-shell → names the ephemeral container (must be unique in the Pod).
• bash → command to run (if the image has only sh, replace bash with sh).
You’ll be dropped into a live shell with full access to the Pod’s filesystem, network, and process space.
What Can You Do Inside?
Ephemeral containers let you observe and debug without modifying your production image:
• Inspect filesystems
ls /usr/share/nginx/html
cat /etc/nginx/nginx.conf
Test connectivity
curl localhost:80
nc -vz other-service 3306
Check processes
ps aux
strace -p 1
Debug networking with netshoot (traceroute, tcpdump, dig, etc.).
Key Points & Best Practices
• Ephemeral containers are temporary — they don’t survive Pod restarts or rescheduling.
• They don’t modify PodSpec in a way that affects restarts — they’re strictly for debugging.
• They are not restarted automatically if they exit.
• Useful for:
• Debugging distroless or minimal images
• Inspecting live network traffic inside a Pod
• Running diagnostics tools without polluting production images
• Security: RBAC permissions (ephemeralcontainers) must be enabled for your user.
sequenceDiagram
participant User
participant kubectl
participant KubeAPI
participant Pod[Pod: nginx app container]
participant DebugContainer[Ephemeral Container: busybox]
User->>kubectl: kubectl debug debug-demo --image=busybox --target=app
kubectl->>KubeAPI: Request to inject ephemeral container
KubeAPI->>Pod: Add temporary debug container
Pod->>DebugContainer: Start busybox (bash/sh)
User->>DebugContainer: Gets shell access
DebugContainer->>Pod: Shares namespaces (network, FS, processes)
User->>DebugContainer: Run tools (curl, ps, strace)
Summary:
Ephemeral debug containers are the safest, fastest way to troubleshoot live Pods without rebuilding or redeploying images. They let you inject debugging tools on demand, inspect traffic and files, and troubleshoot issues inside the Pod’s environment.
⸻
Would you like me to also show you a real-world example using nicolaka/netshoot (which comes preloaded with dozens of network debugging tools) instead of just busybox?
Parent ref
https://medium.com/@obaff/7-kubernetes-debugging-tricks-you-probably-didnt-know-832884bd90a1
No comments:
Post a Comment