Successful Implementation - But can't curl elasticsearch post search-guard installation?

Search Guard
1

There are 9 elasticsearch_nodes in my configurations in 3 virtual machines.

In each VM, have 3 ES_nodes comprising of master, data, search load balancer, kibana and logstash.

For example in one of the VMs:

4001: ES_Master, 4116: Data, 4276: SLB, 5804: Logstash

netstat -anp | grep LISTEN | grep java

tcp6 0 0 :::3304 :::* LISTEN 5804/java

tcp6 0 0 :::3305 :::* LISTEN 5804/java

tcp6 0 0 192.168.0.1:9200 :::* LISTEN 4116/java

tcp6 0 0 192.168.0.1:9201 :::* LISTEN 4001/java

tcp6 0 0 192.168.0.1:9202 :::* LISTEN 4276/java

tcp6 0 0 192.168.0.1:9300 :::* LISTEN 4001/java

tcp6 0 0 192.168.0.1:9301 :::* LISTEN 4116/java

tcp6 0 0 192.168.0.1:9302 :::* LISTEN 4276/java

tcp6 0 0 :::3300 :::* LISTEN 5804/java

tcp6 0 0 :::3301 :::* LISTEN 5804/java

tcp6 0 0 :::3302 :::* LISTEN 5804/java

tcp6 0 0 :::3303 :::* LISTEN 5804/java

I’m using same “root-ca.pem” which I configured in kibana.yml for SSL communication with elasticsearch, which worked absolutely fine for kibana to curl the elasticsearch data, health, indices etc… but returning nothing…

Few of my commands:

$curl --cacert “/etc/pki/root-ca.pem” -XGET ‘https://192.168.0.1:9200/_indices/v’
$curl --cacert “/etc/pki/root-ca.pem” -XGET ‘https://192.168.0.1:9200/_cluster/health’

No command returns anything output not even error, but if I miss --cacert flag:

$curl -XGET ‘https://192.168.0.1:9200/_indices/v’

returns: curl: (60) Peer’s certificate issuer has been marked as not trusted by the user.

More details here: curl - SSL CA Certificates

curl performs SSL certificate verification by default, using a “bundle”

of Certificate Authority (CA) public keys (CA certs). If the default

bundle file isn’t adequate, you can specify an alternate file

using the --cacert option.

If this HTTPS server uses a certificate signed by a CA represented in

the bundle, the certificate verification probably failed due to a

problem with the certificate (it might be expired, or the name might

not match the domain name in the URL).

If you’d like to turn off curl’s verification of the certificate, use

the -k (or --insecure) option.

This tells my es is secured, but how to curl it? I used example.sh in search-guard-ssl repository to generate the certs. Any help please…? How to curl search-guard secured ES?

But my ES do have indices, data and even health is green.

2

My https://localhost:9200/_plugin/head output

···

On Tuesday, 9 August 2016 18:13:07 UTC+2, SAI KRISHNA GHANTA wrote:

There are 9 elasticsearch_nodes in my configurations in 3 virtual machines.

In each VM, have 3 ES_nodes comprising of master, data, search load balancer, kibana and logstash.

For example in one of the VMs:

4001: ES_Master, 4116: Data, 4276: SLB, 5804: Logstash

netstat -anp | grep LISTEN | grep java

tcp6 0 0 :::3304 :::* LISTEN 5804/java

tcp6 0 0 :::3305 :::* LISTEN 5804/java

tcp6 0 0 192.168.0.1:9200 :::* LISTEN 4116/java

tcp6 0 0 192.168.0.1:9201 :::* LISTEN 4001/java

tcp6 0 0 192.168.0.1:9202 :::* LISTEN 4276/java

tcp6 0 0 192.168.0.1:9300 :::* LISTEN 4001/java

tcp6 0 0 192.168.0.1:9301 :::* LISTEN 4116/java

tcp6 0 0 192.168.0.1:9302 :::* LISTEN 4276/java

tcp6 0 0 :::3300 :::* LISTEN 5804/java

tcp6 0 0 :::3301 :::* LISTEN 5804/java

tcp6 0 0 :::3302 :::* LISTEN 5804/java

tcp6 0 0 :::3303 :::* LISTEN 5804/java

I’m using same “root-ca.pem” which I configured in kibana.yml for SSL communication with elasticsearch, which worked absolutely fine for kibana to curl the elasticsearch data, health, indices etc… but returning nothing…

Few of my commands:

$curl --cacert “/etc/pki/root-ca.pem” -XGET ‘https://192.168.0.1:9200/_indices/v’
$curl --cacert “/etc/pki/root-ca.pem” -XGET ‘https://192.168.0.1:9200/_cluster/health’

No command returns anything output not even error, but if I miss --cacert flag:

$curl -XGET ‘https://192.168.0.1:9200/_indices/v’

returns: curl: (60) Peer’s certificate issuer has been marked as not trusted by the user.

More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a “bundle”

of Certificate Authority (CA) public keys (CA certs). If the default

bundle file isn’t adequate, you can specify an alternate file

using the --cacert option.

If this HTTPS server uses a certificate signed by a CA represented in

the bundle, the certificate verification probably failed due to a

problem with the certificate (it might be expired, or the name might

not match the domain name in the URL).

If you’d like to turn off curl’s verification of the certificate, use

the -k (or --insecure) option.

This tells my es is secured, but how to curl it? I used example.sh in search-guard-ssl repository to generate the certs. Any help please…? How to curl search-guard secured ES?

But my ES do have indices, data and even health is green.

3

What if you use this?

$ curl -k -XGET ‘https://192.168.0.1:9200/_indices/v’

``

···

On Tuesday, August 9, 2016 at 12:13:07 PM UTC-4, SAI KRISHNA GHANTA wrote:

There are 9 elasticsearch_nodes in my configurations in 3 virtual machines.

In each VM, have 3 ES_nodes comprising of master, data, search load balancer, kibana and logstash.

For example in one of the VMs:

4001: ES_Master, 4116: Data, 4276: SLB, 5804: Logstash

netstat -anp | grep LISTEN | grep java

tcp6 0 0 :::3304 :::* LISTEN 5804/java

tcp6 0 0 :::3305 :::* LISTEN 5804/java

tcp6 0 0 192.168.0.1:9200 :::* LISTEN 4116/java

tcp6 0 0 192.168.0.1:9201 :::* LISTEN 4001/java

tcp6 0 0 192.168.0.1:9202 :::* LISTEN 4276/java

tcp6 0 0 192.168.0.1:9300 :::* LISTEN 4001/java

tcp6 0 0 192.168.0.1:9301 :::* LISTEN 4116/java

tcp6 0 0 192.168.0.1:9302 :::* LISTEN 4276/java

tcp6 0 0 :::3300 :::* LISTEN 5804/java

tcp6 0 0 :::3301 :::* LISTEN 5804/java

tcp6 0 0 :::3302 :::* LISTEN 5804/java

tcp6 0 0 :::3303 :::* LISTEN 5804/java

I’m using same “root-ca.pem” which I configured in kibana.yml for SSL communication with elasticsearch, which worked absolutely fine for kibana to curl the elasticsearch data, health, indices etc… but returning nothing…

Few of my commands:

$curl --cacert “/etc/pki/root-ca.pem” -XGET ‘https://192.168.0.1:9200/_indices/v’
$curl --cacert “/etc/pki/root-ca.pem” -XGET ‘https://192.168.0.1:9200/_cluster/health’

No command returns anything output not even error, but if I miss --cacert flag:

$curl -XGET ‘https://192.168.0.1:9200/_indices/v’

returns: curl: (60) Peer’s certificate issuer has been marked as not trusted by the user.

More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a “bundle”

of Certificate Authority (CA) public keys (CA certs). If the default

bundle file isn’t adequate, you can specify an alternate file

using the --cacert option.

If this HTTPS server uses a certificate signed by a CA represented in

the bundle, the certificate verification probably failed due to a

problem with the certificate (it might be expired, or the name might

not match the domain name in the URL).

If you’d like to turn off curl’s verification of the certificate, use

the -k (or --insecure) option.

This tells my es is secured, but how to curl it? I used example.sh in search-guard-ssl repository to generate the certs. Any help please…? How to curl search-guard secured ES?

But my ES do have indices, data and even health is green.

4

Nothing.

But when I use curl -k -XGET ‘https://192.168.0.1:9200/_cat/indices?pretty’ -sS -u admin:admin

I can see list of indices.

But again I can’t connect secured elasticsearch using curator.

Remember, leave a key empty if there is no value. None will be a string,

not a Python “NoneType”

client:

hosts:

  • 192.168.0.1

port: 9200

url_prefix:

use_ssl: True

certificate: “/usr/local/bin/root-ca.pem”

client_cert:

client_key:

aws_key:

aws_secret_key:

aws_region:

ssl_no_validate: True

http_auth: admin:admin

timeout: 30

master_only: False

logging:

loglevel: INFO

logfile: curator.log

logformat: default

blacklist: [‘elasticsearch’, ‘urllib3’]

[root@elk-micro-elk-27-jlr-i-7fc907f4 ~]$ curator “/root/action_file.yml”

/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:770: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: Advanced Usage - urllib3 2.1.0 documentation

InsecureRequestWarning)

/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:770: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: Advanced Usage - urllib3 2.1.0 documentation

InsecureRequestWarning)

/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:770: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: Advanced Usage - urllib3 2.1.0 documentation

InsecureRequestWarning)

/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:770: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: Advanced Usage - urllib3 2.1.0 documentation

action_file:

1:

action: open

description: >-

Open indices older than 30 days but younger than 60 days (based on index

name), for logstash- prefixed indices.

options:

timeout_override:

continue_if_exception: False

disable_action: False

filters:

  • filtertype: pattern

kind: prefix

value: default-

exclude:

  • filtertype: age

source: name

direction: older

timestring: ‘%Y.%m.%d’

unit: days

unit_count: 2

exclude:

  • filtertype: age

source: name

direction: younger

timestring: ‘%Y.%m.%d’

unit: days

unit_count: 1

exclude:

Curator version: 4.0.5

···

On Wednesday, 10 August 2016 20:06:15 UTC+2, Sam Mingolelli wrote:

What if you use this?

$ curl -k -XGET ‘https://192.168.0.1:9200/_indices/v’

``

On Tuesday, August 9, 2016 at 12:13:07 PM UTC-4, SAI KRISHNA GHANTA wrote:

There are 9 elasticsearch_nodes in my configurations in 3 virtual machines.

In each VM, have 3 ES_nodes comprising of master, data, search load balancer, kibana and logstash.

For example in one of the VMs:

4001: ES_Master, 4116: Data, 4276: SLB, 5804: Logstash

netstat -anp | grep LISTEN | grep java

tcp6 0 0 :::3304 :::* LISTEN 5804/java

tcp6 0 0 :::3305 :::* LISTEN 5804/java

tcp6 0 0 192.168.0.1:9200 :::* LISTEN 4116/java

tcp6 0 0 192.168.0.1:9201 :::* LISTEN 4001/java

tcp6 0 0 192.168.0.1:9202 :::* LISTEN 4276/java

tcp6 0 0 192.168.0.1:9300 :::* LISTEN 4001/java

tcp6 0 0 192.168.0.1:9301 :::* LISTEN 4116/java

tcp6 0 0 192.168.0.1:9302 :::* LISTEN 4276/java

tcp6 0 0 :::3300 :::* LISTEN 5804/java

tcp6 0 0 :::3301 :::* LISTEN 5804/java

tcp6 0 0 :::3302 :::* LISTEN 5804/java

tcp6 0 0 :::3303 :::* LISTEN 5804/java

I’m using same “root-ca.pem” which I configured in kibana.yml for SSL communication with elasticsearch, which worked absolutely fine for kibana to curl the elasticsearch data, health, indices etc… but returning nothing…

Few of my commands:

$curl --cacert “/etc/pki/root-ca.pem” -XGET ‘https://192.168.0.1:9200/_indices/v’
$curl --cacert “/etc/pki/root-ca.pem” -XGET ‘https://192.168.0.1:9200/_cluster/health’

No command returns anything output not even error, but if I miss --cacert flag:

$curl -XGET ‘https://192.168.0.1:9200/_indices/v’

returns: curl: (60) Peer’s certificate issuer has been marked as not trusted by the user.

More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a “bundle”

of Certificate Authority (CA) public keys (CA certs). If the default

bundle file isn’t adequate, you can specify an alternate file

using the --cacert option.

If this HTTPS server uses a certificate signed by a CA represented in

the bundle, the certificate verification probably failed due to a

problem with the certificate (it might be expired, or the name might

not match the domain name in the URL).

If you’d like to turn off curl’s verification of the certificate, use

the -k (or --insecure) option.

This tells my es is secured, but how to curl it? I used example.sh in search-guard-ssl repository to generate the certs. Any help please…? How to curl search-guard secured ES?

But my ES do have indices, data and even health is green.