Horizontal Scaling Node.js on Kubernetes

Scaling Node.js applications horizontally on Kubernetes allows you to handle increased traffic by distributing the load across multiple container instances, known as Pods. This article provides a straightforward guide on how to implement horizontal scaling for a Node.js application using both manual replica management and the Kubernetes Horizontal Pod Autoscaler (HPA), ensuring your application remains highly available and performant under heavy loads.

Understanding Node.js and Horizontal Scaling

Node.js operates on a single-threaded event loop. Because a single Node.js process cannot natively utilize multiple CPU cores, vertical scaling (adding more CPU/RAM to a single container) has limited benefits. Horizontal scaling—running multiple concurrent instances of your Node.js application behind a load balancer—is the standard approach to scaling Node.js in production. In Kubernetes, this is achieved by increasing the number of running Pods in a Deployment.

Step 1: Ensure Your Node.js App is Stateless

Before scaling horizontally, your Node.js application must be stateless. Since Kubernetes routes incoming traffic across any available Pod, you cannot rely on local in-memory storage for user sessions or application state. * Sessions: Store session data in an external, shared in-memory database like Redis. * File Uploads: Save uploaded files to cloud storage (e.g., AWS S3) rather than the local container filesystem.

Step 2: Define Resource Requests and Limits

For Kubernetes to scale your application effectively, you must define how much CPU and memory your Node.js containers require. Edit your deployment configuration file (deployment.yaml) to include resource definitions:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nodejs-app
spec:
  replicas: 2
  template:
    spec:
      containers:
      - name: nodejs-container
        image: your-node-app-image:latest
        resources:
          requests:
            memory: "256Mi"
            cpu: "200m"
          limits:
            memory: "512Mi"
            cpu: "500m"

Step 3: Manual Horizontal Scaling

If you anticipate a traffic spike (e.g., a marketing campaign), you can manually scale your Node.js deployment using the Kubernetes command-line tool, kubectl.

To scale your deployment to 5 replicas instantly, run:

kubectl scale deployment nodejs-app --replicas=5

Alternatively, you can update the replicas field in your deployment.yaml file to 5 and apply the changes:

kubectl apply -f deployment.yaml

Step 4: Automated Scaling with Horizontal Pod Autoscaler (HPA)

To automatically scale your Node.js application based on real-time traffic and CPU/Memory usage, you must deploy a Horizontal Pod Autoscaler (HPA).

Prerequisite

The Kubernetes Metrics Server must be installed in your cluster. You can verify if it is running by executing:

kubectl top nodes

Creating the HPA

Create an hpa.yaml file to automatically scale your Node.js application between 2 and 10 pods based on CPU utilization:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: nodejs-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: nodejs-app
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70

Apply the HPA to your cluster:

kubectl apply -f hpa.yaml

Under this configuration, when the average CPU utilization across all Pods exceeds 70%, Kubernetes will automatically spin up new Pods (up to 10) to distribute the load. When traffic decreases and CPU usage drops, the HPA will gradually terminate excess Pods down to the minimum limit of 2.

Step 5: Handling Graceful Shutdowns

When Kubernetes scales down your Node.js application, it terminates Pods. To prevent active user requests from being abruptly dropped, your Node.js application must handle the SIGTERM signal gracefully. Add the following handler to your Node.js server code:

const server = app.listen(3000);

process.on('SIGTERM', () => {
  console.log('SIGTERM signal received: closing HTTP server');
  server.close(() => {
    console.log('HTTP server closed');
    process.exit(0);
  });
});

This code ensures that the Node.js process stops accepting new requests but finishes processing existing connections before shutting down.