AWS ElasticSearch — NGINX reverse Proxy for accessing Kibana.
One finds it difficult to access kibana if the Elasticsearch server is placed inside the private subnet of a VPC. The private subnet does not allow any kind of communication to the server via public networks thereby denying access to kibana and other elastic search applications. To overcome this problem we will create a Proxy server in the public subnet. Any request made to the proxy server will be redirected to the ElasticSearch.
What is NGINX?
NGINX is a free, open-source, high-performance HTTP server and reverse proxy, as well as an IMAP/POP3 proxy server. NGINX is known for its high performance, stability, rich feature set, simple configuration, and low resource consumption.
Creating an Nginx reverse proxy server in the same VPC public subnet
Lets quickly look at the ElasticSearch server-
Configuring Reverse Proxy server-
In computer networks, a reverse proxy is a type of proxy server that retrieves resources on behalf of a client from one or more servers. These resources are then returned to the client, appearing as if they originated from the proxy server itself.
step 1 — create an EC2 ubuntu instance
step 2 -install NGINX webserver
sudo apt-get install nginx
step 3 — install apache2-utils
sudo apt-get install apache2-utils
step 4- use the Nginx config
user root;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
# Load dynamic modules. See /usr/share/nginx/README.fedora.
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 1024;
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Load modular configuration files from the /etc/nginx/conf.d directory.
include /etc/nginx/conf.d/*.conf;
server {
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
server_name _;
# Comment out the following two lines if you do not want to enable HTTP Basic Authentication
auth_basic "Restricted Access";
auth_basic_user_file /etc/nginx/conf.d/.htpasswd;
location / {
# Set Host to match ES endpoint
proxy_set_header Host vpc-xxx-xxx-xxxx-xxxxxx.us-west-2.es.amazonaws.com;
# We want to be sure that we are sending instance IP instead of browser's IP
proxy_set_header X-Real-IP xx.xx.xx.136;
# Various headers
proxy_http_version 1.1;
proxy_set_header Connection "Keep-Alive";
proxy_set_header Proxy-Connection "Keep-Alive";
proxy_set_header Authorization "";
# Forward requests to ES, please use https
proxy_pass https://vpc-xxx-xxx-xxxx-xxxxxx.us-west-2.es.amazonaws.com;
}
location /kibana {
# Set Host to match ES endpoint
proxy_set_header Host vpc-xxx-xxx-xxxx-xxxxxx.us-west-2.es.amazonaws.com;
# We want to be sure that we are sending instance IP instead of browser's IP
proxy_set_header X-Real-IP xx.xx.xx.136;
# Various headers
proxy_http_version 1.1;
proxy_set_header Connection "Keep-Alive";
proxy_set_header Proxy-Connection "Keep-Alive";
proxy_set_header Authorization "";
# Forward requests to Kibana, please use https and /_plugin/kibana/
proxy_pass https://vpc-xxx-xxx-xxxx-xxxxxx.us-west-2.es.amazonaws.com/_plugin/kibana/;
# Ensure that requests are coming back to /kibana/
proxy_redirect https://vpc-xxx-xxx-xxxx-xxxxxx.us-west-2.es.amazonaws.com/_plugin/kibana/ http://xx.xx.xx.136/kibana/;
}
}
}
step 5 -create an HTTP auth
cd /etc/nginx/conf.d/.htpasswd
sudo htpasswd -c <username>
step 6- check whether the Nginx config is correct
sudo nginx -t
step 7 — restart NGINX
sudo service nginx restart
step 8 — access kibana
access your EC2 IP address and add suffix /_plugin/kibana