AWS ElasticSearch — NGINX reverse Proxy for accessing Kibana.

One finds it difficult to access kibana if the Elasticsearch server is placed inside the private subnet of a VPC. The private subnet does not allow any kind of communication to the server via public networks thereby denying access to kibana and other elastic search applications. To overcome this problem we will create a Proxy server in the public subnet. Any request made to the proxy server will be redirected to the ElasticSearch.

What is NGINX?

NGINX is a free, open-source, high-performance HTTP server and reverse proxy, as well as an IMAP/POP3 proxy server. NGINX is known for its high performance, stability, rich feature set, simple configuration, and low resource consumption.

Creating an Nginx reverse proxy server in the same VPC public subnet

Nginx Reverse Proxy Architechture

Lets quickly look at the ElasticSearch server-

ES in the Private Subnet

Configuring Reverse Proxy server-

In computer networks, a reverse proxy is a type of proxy server that retrieves resources on behalf of a client from one or more servers. These resources are then returned to the client, appearing as if they originated from the proxy server itself.

step 1 — create an EC2 ubuntu instance

Choose the ubuntu server

configure the instance and make sure it attached to the public subnet of the VPC

You should have a server now running with the above configuration

step 2 -install NGINX webserver

sudo apt-get install nginx

step 3 — install apache2-utils

sudo apt-get install apache2-utils

step 4- use the Nginx config

user root;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;

# Load dynamic modules. See /usr/share/nginx/README.fedora.
include /usr/share/nginx/modules/*.conf;

events {
    worker_connections 1024;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;
    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    # Load modular configuration files from the /etc/nginx/conf.d directory.
    include /etc/nginx/conf.d/*.conf;

    server {
      listen 80 default_server;
      listen [::]:80 default_server ipv6only=on;
      server_name _;

      # Comment out the following two lines if you do not want to enable HTTP Basic Authentication
      auth_basic "Restricted Access";
      auth_basic_user_file /etc/nginx/conf.d/.htpasswd;

     location / {
        # Set Host to match ES endpoint
        proxy_set_header Host vpc-xxx-xxx-xxxx-xxxxxx.us-west-2.es.amazonaws.com;

        # We want to be sure that we are sending instance IP instead of browser's IP
        proxy_set_header X-Real-IP xx.xx.xx.136;

        # Various headers
        proxy_http_version 1.1;
        proxy_set_header Connection "Keep-Alive";
        proxy_set_header Proxy-Connection "Keep-Alive";
        proxy_set_header Authorization "";

        # Forward requests to ES, please use https
        proxy_pass https://vpc-xxx-xxx-xxxx-xxxxxx.us-west-2.es.amazonaws.com;
      }

        location /kibana {
        # Set Host to match ES endpoint
        proxy_set_header Host vpc-xxx-xxx-xxxx-xxxxxx.us-west-2.es.amazonaws.com;
        # We want to be sure that we are sending instance IP instead of browser's IP
        proxy_set_header X-Real-IP xx.xx.xx.136;
        # Various headers

        proxy_http_version 1.1;
        proxy_set_header Connection "Keep-Alive";
        proxy_set_header Proxy-Connection "Keep-Alive";
        proxy_set_header Authorization "";

        # Forward requests to Kibana, please use https and /_plugin/kibana/
        proxy_pass https://vpc-xxx-xxx-xxxx-xxxxxx.us-west-2.es.amazonaws.com/_plugin/kibana/;

        # Ensure that requests are coming back to /kibana/
        proxy_redirect https://vpc-xxx-xxx-xxxx-xxxxxx.us-west-2.es.amazonaws.com/_plugin/kibana/ http://xx.xx.xx.136/kibana/;
      }
    }
}

step 5 -create an HTTP auth

cd /etc/nginx/conf.d/.htpasswd
sudo htpasswd -c <username>

step 6- check whether the Nginx config is correct

sudo nginx -t

step 7 — restart NGINX

sudo service nginx restart

step 8 — access kibana

access your EC2 IP address and add suffix /_plugin/kibana