Tune Nginx Parameters

Procedures and recommendations for tuning Nginx parameters

Prerequisites

This assumes you’ve followed the Geodesic Module Usage guide which covers all the scaffolding necessary to get started.

Kubernetes Nginx Ingress controller is built around the Kubernetes Ingress resource that uses ConfigMap to store the Nginx configuration.

We optimized Nginx configuration settings to fine-tune Nginx performance in a Kubernetes cluster:

nginx.conf

# Set the worker processes based on the number of CPU cores
# Setting to `auto` will calculate it automatically
worker_processes auto;

# Number of file descriptors used for Nginx
# The limit for the maximum descriptors on the server is usually set by the OS.
# If you don't set descriptors, then OS settings will be used which is by default 2000
worker_rlimit_nofile 100000;

events {
    # Determines how many clients will be served by one worker
    # max clients = worker_connections * worker_processes
    # max clients is also limited by the number of socket connections available on the system (~64k)
    worker_connections 4000;

    # Optmized to serve many clients with each thread
    use epoll;

    # Accept as many connections as possible
    multi_accept on;
}

http {
    open_file_cache max=200000 inactive=20s;
    open_file_cache_valid 30s;
    open_file_cache_min_uses 2;
    open_file_cache_errors on;

    # To boost I/O on HDD, disable access logs
    access_log off;

    # Copy data between file descriptors within the kernel
    # Faster then read() + write()
    sendfile on;

    # Send all headers at once
    tcp_nopush on;

    # Don't buffer data when sending. Good for small data bursts in real time
    tcp_nodelay on;

    # Reduce the amount of data sent over network
    gzip on;
    gzip_min_length 10240;
    gzip_proxied expired no-cache no-store private auth;
    gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/json application/xml;
    gzip_disable msie6;

    # Allow the server to close connection on non responding client. This will free up memory
    reset_timedout_connection on;

    # Request timed out. Default 60
    client_body_timeout 60;

    send_timeout 60;

    # Server will close connection after this timeout
    keepalive_timeout 60;

    # Number of requests client can make
    keepalive_requests 100000;
}

As mentioned above, these settings are stored in the Kubernetes cluster as ConfigMap, and get deployed by nginx-ingress Helm chart.

The nginx-ingress chart is deployed from the geodesic shell using the Helmfile.

To apply the new Nginx parameters, modify controller.config settings in the Helmfile and then follow Install with Master Helmfile instructions to update the cluster with the new Nginx settings.

Read More

For more information, check out the following links: