Answer: The kube-scheduler is a control plane component that assigns unscheduled pods to nodes in a two-phase process:

1. Filtering (Predicates): Evaluates nodes to find eligible hosts. Checks resource requirements (CPU/RAM requests), taints and tolerations, node selectors, port conflicts, and storage availability.
2. Scoring (Priorities): Ranks the filtered nodes to find the best fit. Ranks nodes based on pod affinity/anti-affinity, topology spread constraints, resource balance, and image availability (nodes that already pulled the image score higher). The node with the highest score is selected. The scheduler then creates a Binding object, informing the API server, which updates the pod's spec. The Kubelet on the selected node detects the assignment and starts the containers.

Answer: etcd is a distributed, consistent, highly-available key-value store used as the backing database for all Kubernetes cluster data (configuration states, active nodes, pod statuses, secrets, endpoints).

Key Characteristics:#

• Consistency: Uses the Raft consensus algorithm to guarantee strong consistency across cluster nodes.
• Single Source of Truth: Only the kube-apiserver communicates directly with etcd; all other components update their state via the API server.
• Watch API: Allows the API server to watch keys for real-time state changes, powering controller loops.

Answer: kube-proxy is a network daemon running on every node that implements the Kubernetes Service abstraction. It manages network routing, port forwarding, and service load-balancing:

Modes of Operation:#

• IPTables Mode (Default): Writes iptables rules to route traffic directed at a Service IP to one of the backend pods randomly. Requires sequential parsing, which can slow down in large clusters.
• IPVS (IP Virtual Server) Mode: Uses hash tables to handle traffic routing and load balancing. Offers $O(1)$ lookup performance, making it highly scalable for clusters with thousands of services.

Answer: CoreDNS is a flexible, extensible DNS server that runs as a deployment in the cluster, handling name resolution and service discovery. Every service created in the cluster is assigned a DNS name (e.g., <service-name>.<namespace>.svc.cluster.local). CoreDNS translates these names to the matching service ClusterIP. Pods are configured to point their DNS resolver (/etc/resolv.conf) to the CoreDNS service IP.

Answer: The kube-apiserver is the central management gateway of the control plane. It exposes the Kubernetes API over HTTP/HTTPS, serving as the entry point for all administrative, user, and controller modifications:

• Authentication & Authorization: Validates client identity (tokens, certificates) and checks permissions via RBAC rules.
• Admission Control: Intercepts requests to mutate or validate objects before writing them to etcd.
• Data Validation: Ensures configurations conform to API schemas.

Answer: A CNI (Container Network Interface) plugin is responsible for Pod networking in Kubernetes. When a Pod is created, the CNI plugin assigns it an IP address, creates its network interface, and connects it to the cluster network. It also configures routing so that Pods can communicate with other Pods and Services across the cluster. Common CNI plugins include Calico, Flannel, and Cilium.

Answer: For most production environments, I would choose Calico because it provides reliable networking, strong network policy support, and proven scalability. If advanced observability, security, and eBPF-based networking are important, I would consider Cilium. For simple labs or learning environments, Flannel is often the easiest choice.

Answer: eBPF (Extended Berkeley Packet Filter) is a Linux kernel technology that allows programs to run directly inside the Linux kernel without modifying the kernel source code.

Answer: Kubernetes networking is based on the principle that every Pod gets a unique IP and can communicate directly with any other Pod in the cluster. The CNI plugin provides networking and routing, kube-proxy enables Service communication, and CoreDNS handles service discovery.

Answer: A Kubernetes Service IP is a virtual IP that represents a set of backend Pods. kube-proxy watches Services and their endpoints and creates routing rules on each node. When traffic is sent to the Service IP, kube-proxy intercepts it, selects one of the backend Pods, rewrites the destination address, and forwards the packet to the chosen Pod. Depending on the mode, this is implemented using iptables, IPVS, or eBPF.

Answer: Ingress is the older Kubernetes resource for exposing HTTP/HTTPS services. Gateway API is its modern successor, offering better flexibility, advanced traffic routing, cleaner separation of responsibilities, and standardized configuration across different implementations.

Answer: NGINX Ingress Controller watches Ingress resources and converts them into NGINX configuration. When traffic arrives, NGINX applies the configured host and path rules, forwards requests to the correct Service, and Kubernetes routes the traffic to the backend Pods.

Answer: IngressClass tells Kubernetes which Ingress Controller should manage a specific Ingress. Multiple ingress controllers are commonly used to separate internal and external traffic, support different environments, or provide different networking features.

Answer: When troubleshooting a 503 from an Ingress, I would follow the request path: Ingress → Service → Endpoints → Pods. First, I'd verify the Ingress configuration and ensure the Ingress Controller is healthy. Then I'd check that the backend Service exists and has valid Endpoints. Next, I'd confirm the Pods are running, ready, and correctly matched by the Service selectors. I'd also review Ingress Controller logs and test the Service directly to isolate whether the issue is in the Ingress layer or the application layer.

Answer: CSI is the standard storage interface in Kubernetes. It allows Kubernetes to communicate with external storage systems and manage volumes for Pods. Kubernetes moved away from in-tree plugins to CSI because CSI is easier to maintain, more flexible, and allows storage vendors to update their drivers independently of Kubernetes.

Answer: A PersistentVolume (PV) is the actual storage resource available in the cluster. A PersistentVolumeClaim (PVC) is a request for storage made by an application or Pod. A StorageClass defines how storage should be dynamically provisioned, such as the storage type and provisioner. When a PVC is created, Kubernetes either binds it to an existing PV or uses the StorageClass to dynamically create a new PV and bind it to the claim.

Answer: In dynamic provisioning, a developer creates a PVC, Kubernetes uses the StorageClass and CSI driver to automatically create the required storage volume, generates a PV, binds it to the PVC, and then mounts it to the Pod.

Answer: I would edit the PVC and increase the requested storage size. If the StorageClass supports volume expansion, Kubernetes and the CSI driver will automatically expand the underlying volume and filesystem.

Answer: STS (Security Token Service) is an AWS service that provides temporary security credentials. In IRSA, a Kubernetes Pod uses its Service Account to assume an IAM Role, and AWS STS issues temporary credentials that allow the Pod to securely access AWS services without storing long-term access keys.

Answer: EKS uses AWS IAM for authentication and Kubernetes RBAC for authorization. IAM verifies the user's identity, and that identity is mapped to Kubernetes users or groups. RBAC policies then determine which Kubernetes resources and actions are allowed.

Answer: AWS Load Balancer Controller is a Kubernetes controller that automatically creates and manages AWS Load Balancers. It acts as a bridge between Kubernetes and AWS. When we create an Ingress, it creates an ALB (Application Load Balancer). When we create a Service of type LoadBalancer, it creates an NLB (Network Load Balancer). This removes the need to manually create and manage load balancers in AWS.

Answer: For a production EKS upgrade, I would first review version compatibility and upgrade the EKS control plane. Next, I would upgrade cluster add-ons such as CoreDNS, kube-proxy, the VPC CNI, and CSI drivers. Then I would perform a rolling upgrade of worker nodes using Managed Node Groups or node draining. To ensure zero downtime, applications should have multiple replicas, readiness probes, and PodDisruptionBudgets configured so traffic continues flowing while nodes are being replaced. Finally, I would validate cluster health and application functionality after the upgrade.

Answer: A DaemonSet guarantees that a copy of a specific pod runs on all (or selected) nodes in the cluster. As nodes are added to the cluster, the DaemonSet scheduler automatically provisions matching pods on them.

Common Use Cases:#

• Log Shippers: Fluentbit or Logstash gathering container outputs.
• Resource Monitoring: node-exporter or Datadog agents reporting node metrics.
• Network Proxies: kube-proxy or Calico CNI daemons.

Answer:

• Job: Runs a specific task to completion. Once the container exits successfully (status code 0), the Job halts execution and retains the pod logs.
• CronJob: Runs jobs on a repeating schedule using a standard cron format (e.g., 0 2 * * * for daily at 2 AM). Useful for backups, database cleanup, or generating reports.

Answer: A ReplicaSet is a controller that ensures a specified number of identical pod replicas are running at any given time. It uses a pod selector to identify active pods and spins up or terminates replicas to match the desired count. Note: Deployments manage ReplicaSets under the hood, handling rolling updates and version history automatically.

Answer: Init Containers are specialized containers that run and complete before the main application containers start in a pod.

Characteristics:#

• Run sequentially; each init container must exit successfully (status 0) before the next starts.
• If an init container fails, Kubernetes restarts the pod until it succeeds.
• Use Cases: Waiting for a database service to become reachable, downloading configuration templates, or running migrations.

Answer: Node Affinity is a set of scheduling rules that constrains which nodes a pod can be scheduled on based on node labels.

Types of Affinity:#

• requiredDuringSchedulingIgnoredDuringExecution (Hard requirement): The scheduler will not place the pod unless the node meets the label criteria.
• preferredDuringSchedulingIgnoredDuringExecution (Soft requirement): The scheduler tries to find a node matching the criteria; if none exist, it schedules the pod on another node anyway.

Answer: Allows scheduling pods relative to other pods already running on nodes, based on labels:

• Pod Affinity: Co-locates pods in the same failure domain (e.g., scheduling a cache pod on the same node as a web server to reduce latency).
• Pod Anti-Affinity: Prevents placing identical pods on the same node or availability zone (e.g., spreading replicas across zones to ensure high availability).

Answer: Used to ensure pods are not scheduled on inappropriate nodes:

• Taints: Applied to nodes to repel pods (e.g., kubectl taint nodes node1 GPU=true:NoSchedule).
• Tolerations: Applied to pods to allow them to schedule on tainted nodes (e.g., a GPU-utilizing machine learning pod tolerating the GPU=true taint). Taints repel pods, while tolerations allow pods to bypass the taints.

Answer: A Pod Disruption Budget (PDB) defines the minimum number of replicas (or maximum unavailable replicas) that must remain healthy during voluntary disruptions (e.g., node draining, cluster upgrades, scaling down). This ensures that administrative upgrades do not compromise application availability by taking down too many replicas simultaneously.

Answer: Topology Spread Constraints (topologySpreadConstraints) control how pods are distributed across failure domains (nodes, zones, regions, or racks) in a cluster. This prevents pod replicas from clustering on the same node or in a single zone, mitigating risk if a specific zone goes down.

Answer: RBAC is a security mechanism used to regulate access to resources based on user roles and permissions:

• Role: Defines permission rules within a specific namespace (e.g., read pods in default).
• ClusterRole: Defines permissions across the entire cluster (e.g., list nodes, read persistent volumes).
• RoleBinding: Associates a Role to a user, group, or ServiceAccount within a namespace.
• ClusterRoleBinding: Associates a ClusterRole to a user, group, or ServiceAccount cluster-wide.

Answer: A ServiceAccount provides an identity for processes running in pods, allowing them to authenticate against the Kubernetes API Server to perform actions (e.g., a Prometheus pod querying the API to discover other pods).

Answer: A NetworkPolicy defines firewall rules at the pod level, specifying how groups of pods can communicate with each other and with external network endpoints. By default, pods are non-isolated and accept traffic from any source. Enfacing a NetworkPolicy restricts traffic based on labels, namespaces, and ports.

Answer: Pod Security Standards (PSS) define security profiles to restrict pod capabilities, replacing the deprecated PodSecurityPolicies (PSP):

• Privileged: Unrestricted permissions. Allows container escape, access to host namespaces, and running as root.
• Baseline: Default profile. Restricts host access but allows running standard container applications.
• Restricted: Highly hardened profile. Restricts root privileges, enforces read-only filesystems, and limits capabilities.

Answer: An Admission Controller is a plugin that intercepts API requests to the API Server after authentication and authorization, but before the object is written to etcd:

• Mutating Controllers: Modify the request payload (e.g., injecting a sidecar container).
• Validating Controllers: Validate the request payload; if validation fails, the request is rejected (e.g., rejecting pods that do not specify CPU/RAM resource limits).

Answer: The HPA queries metrics (usually CPU/RAM usage from the Metrics Server) at regular intervals (default 15s) and adjusts the replica count of a deployment or statefulset to match a target utilization: $$\text{Desired Replicas} = \lceil \text{Current Replicas} \times (\text{Current Metric Value} / \text{Target Metric Value}) \rceil$$

Answer: The VPA monitors the actual resource usage of pods over time and automatically adjusts their CPU/RAM requests and limits. This optimizes resource utilization and prevents Out-Of-Memory (OOM) crashes by allocating more resources to pods that need them. Note: Standard VPA requires restarting pods to apply changes. Using VPA and HPA on the same metrics (like CPU/Memory) causes scheduling conflicts because they work against each other (HPA attempts to scale pod counts, while VPA attempts to scale pod sizes).

Answer: The Cluster Autoscaler adjusts the number of virtual machine nodes in the cluster's node groups:

• Scale Up: Triggered when pods are in a Pending state because the cluster has insufficient CPU/RAM to schedule them.
• Scale Down: Triggered when nodes are underutilized for a sustained period, and all pods running on them can be rescheduled elsewhere.

Answer: KEDA (Kubernetes Event-driven Autoscaling) is a lightweight component that extends standard autoscaling by allowing pods to scale based on external event sources (e.g., message counts in Kafka, RabbitMQ, or AWS SQS), including scaling deployments down to 0 replicas when there are no events to process.

Answer: ResourceQuotas set constraints on the total resource consumption in a namespace, limiting total CPU requests, memory requests, or object counts (e.g., maximum of 10 Services or 50 Pods).

Answer: To achieve high availability:

1. Multi-AZ Node Groups: Distribute worker nodes across at least three Availability Zones.
2. Pod Topology Spread Constraints: Configure topologySpreadConstraints to force pod replicas to be distributed evenly across zones and nodes.
3. Horizontal Pod Autoscaler (HPA): Pair it with the Cluster Autoscaler (or Karpenter) to spin up new pods and nodes dynamically when load triggers CPU/Memory thresholds.
4. Pod Disruption Budgets (PDB): Enforce a PDB (e.g., minAvailable: 2) to prevent voluntary node drains from bringing down all app instances during cluster maintenance.

Answer:

1. RollingUpdate Strategy: Configure the deployment with maxUnavailable: 0 (keeps all instances active during release) and maxSurge: 25% (provisions extra temporary pods).
2. Readiness Probes: Configure a strict readiness probe (e.g., testing database connection and warmup) so traffic is routed to new pods only when they are fully initialized.
3. PreStop Lifecycle Hooks: Implement a preStop hook (e.g., sleep 15) to allow the container to finish inflight requests before receiving the SIGTERM signal.
4. Canary Deployments via Service Mesh / Ingress: Route 5% of traffic to the new version using Istio or Argo Rollouts, monitoring logs and error budgets before expanding the deployment.

Answer:

1. Set Accurate Requests and Limits: Analyze actual container consumption using Prometheus. Avoid over-provisioning requests.
2. Vertical Pod Autoscaler (VPA): Set VPA to Off (recommendation mode) to analyze resource utilization and suggest optimal CPU/RAM configurations, then apply those recommendations.
3. Karpenter Node Provisioning: Replace standard Cluster Autoscaler with Karpenter, which dynamically launches right-sized compute nodes (using cheaper Spot instances where applicable) and consolidates underutilized nodes automatically.
4. Namespace ResourceQuotas: Set resource quotas to prevent development teams from provisioning excessively large sandbox pods.

Answer:

1. Unified Control Plane: Deploy a management suite like Rancher, Google Anthos, or Azure Arc to manage API authentication, access controls, and cluster health centrally.
2. GitOps Synchronization: Use GitOps controllers (ArgoCD or Flux) running inside each cluster. Configure them to watch a single Git repository. Merging to the Git repository triggers agents to reconcile configurations locally.
3. Multi-Cluster Service Mesh: Deploy Istio in a multi-primary configuration to enable secure mTLS communication, unified service discovery, and routing across different cloud boundaries.

Answer:

1. etcd Encryption at Rest: Enable KMS envelope encryption in the API server so Kubernetes secrets are encrypted before they are written to the etcd database.
2. Secrets Store CSI Driver: Integrate Kubernetes with AWS Secrets Manager or HashiCorp Vault. The CSI driver mounts secrets as local files in container memory, avoiding storage in etcd altogether.
3. Strict RBAC: Restrict get and list operations on secrets to cluster admins and specific service accounts.
4. Scan Container Images: Run vulnerability checks on all images using Trivy inside the CI/CD pipeline to catch embedded secrets.

Answer:

1. Deploy Velero: Run Velero inside the cluster to manage backup and restore operations.
2. Backup Kubernetes Manifests: Schedule Velero to back up all cluster resources (CRDs, Deployments, Configs, Secrets) to a secure, encrypted S3 bucket.
3. CSI Storage Volume Snapshots: Integrate Velero with cloud CSI drivers to take snapshots of active Persistent Volumes (PVs).
4. Automated Restore Testing: Set up an automated script that periodically creates a sandbox cluster, restores the Velero backup, and runs smoke tests to verify backup validity.

Answer:

1. Check CPU Throttling: Query container metrics. Even if CPU utilization is low, high CPU throttling (visible via container_cpu_cfs_throttled_seconds_total) indicates CPU limits are too low, causing kernel pauses.
2. Audit Logs: Check log collectors (Loki/Fluentd) for database timeouts, slow database queries, or application stack traces.
3. Analyze Network Latency: Use a service mesh (Istio) dashboard to view latency charts between microservices and find the slow communication hop.
4. Distributed Tracing: Look at trace paths in Jaeger to locate exactly which downstream database query or API call is introducing latency.

Answer:

1. Deploy Parallel Deployments: Keep the stable deployment (v1.0) active and launch a small canary deployment (v2.0).
2. Weighted Ingress Routing: Configure your Ingress controller (e.g., NGINX Ingress annotations) or Service Mesh (Istio VirtualService) to split traffic, routing 95% to v1.0 and 5% to v2.0.
3. Monitor Metrics: Analyze HTTP error rates, latency, and logs on the canary pods.
4. Automated Promotion/Rollback: If error budgets are maintained, gradually increase the traffic weight to 100%. If error rates spike, instantly change the traffic weight back to 0% to route all users to the stable release.

Answer:

1. Node Labeling & Affinities: Label nodes located in EU datacenters (e.g., topology.kubernetes.io/region=eu-west-1). Configure nodeAffinity on GDPR-scoped pods to ensure they schedule only on EU nodes.
2. EU-Scoped StorageClasses: Provision PVs dynamically using StorageClasses bound to EU cloud zones.
3. Network Policies: Implement strict egress policies to prevent data from being forwarded to API endpoints located outside the EU.

Answer:

1. Decompose the Monolith: Split the monolith into logical services (e.g., cart, payment, inventory) using the Strangler Fig Pattern, wrapping monolithic endpoints in APIs.
2. Database-Per-Service Migration: Split the central database. Deploy CDC (Change Data Capture) pipelines to sync and transition database states gradually.
3. Deploy API Gateway: Route incoming client requests through an API Gateway (like Emissary-ingress or Kong) to route traffic dynamically to the monolith or newly migrated microservices.