Empty file path for searchguard.ssl.transport.truststore_filepath

Search Guard
21

yes, authentication succeeded! now its about the roles ...

···

Am 14.03.2018 um 21:39 schrieb ivy2@uchicago.edu:

I think, I am getting somewhere after changing usersearch.

Now in the web browser I am getting an error:

{"error":{"root_cause":[{"type":"security_exception","reason":"no permissions for [cluster:monitor/main] and User [name=ivy2, roles=, requestedTenant=null]"}],"type":"security_exception","reason":"no permissions for [cluster:monitor/main] and User [name=ivy2, roles=, requestedTenant=null]"},"status":403}

Does that mean that I finally got through LDAP but I need to configure the role somehow?

The logs and configuration are attached.

On Wednesday, March 14, 2018 at 2:45:59 PM UTC-5, iv...@uchicago.edu wrote:

On Wednesday, March 14, 2018 at 11:34:06 AM UTC-5, Search Guard wrote:
connection seems ok but the user is not found.
you need to set these props correctly:

userbase:

username_attribute:

usersearch:

see LDAP Authentication | Security for Elasticsearch | Search Guard

It does not really explain to me how to find out those strings.
Based on the fact that ldapsearch works:

ldapsearch -x -v -h ldap.rcc.uchicago.edu -p 389 -b "uid=ivy2,ou=People,dc=rcc,dc
=uchicago,dc=edu"

I tried the following:

authentication_backend:

bind_dn: 'ou=People,dc=rcc,dc=uchicago,dc=edu'
userbase: 'ou=People,dc=rcc,dc=uchicago,dc=edu'
usersearch: '(sAMAccountName={0})'
username_attribute: uid

authz:
bind_dn: 'ou=People,dc=rcc,dc=uchicago,dc=edu'
rolebase: 'ou=People,dc=rcc,dc=uchicago,dc=edu'
rolesearch: '(member={0})'
userbase: 'ou=People,dc=rcc,dc=uchicago,dc=edu'
usersearch: '(uid={0})'

On Wednesday, 14 March 2018 17:10:50 UTC+1,

On Wednesday, March 14, 2018 at 11:02:43 AM UTC-5, Search Guard wrote:
nothing attached

Oh, sorry, fixed.

How important are permissions on certificate files? Originally there were complaints about insecure permissions. I fixed that.

Also, to double check how changes from sg_config.yml propagate into Elastic Search: once I change this file, I run:

/home_local/elasticsearch/elasticsearch-6.2.1/plugins/search-guard-6/tools/sgadmin.sh -cd /home_local/elasticsearch/ela
sticsearch-6.2.1/plugins/search-guard-6/sgconfig -icl -key /home_local/elasticsearch/elasticsearch-6.2.1/config/HadoopC
ertificates/hadoop.rcc.uchicago.edu-key.pem -cert /home_local/elasticsearch/elasticsearch-6.2.1/config/HadoopCertificat
es/hadoop.rcc.uchicago.edu-chain.crt -cacert /home_local/elasticsearch/elasticsearch-6.2.1/config/HadoopCertificates/in
common.crt -nhnv

from tools directory and get messages that the changes were successful (provided that elastic search is running).
As far as I understand, elasticsearch does not have to be restarted after such changes?

On Wednesday, 14 March 2018 16:58:53 UTC+1

On Wednesday, March 14, 2018 at 6:44:38 AM UTC-5, Search Guard wrote:
If ldapsearch -x -v -h ldap.rcc.uchicago.edu -p 389 -b "uid=ivy2,ou=People,dc=rcc,dc=uchicago,dc=edu" works then you can also try

enable_ssl: false
enable_start_tls: false
enable_ssl_client_auth: false
verify_hostnames: false
hosts:
  - ldap.rcc.uchicago.edu:389
  - ldap.uchicago.edu:389

It still does not work. The logs and the configuration file is attached.

I have root access to ldap.rcc.uchicago.edu. Is there some log that would tell if my request reaches LDAP server and if it does, why the user is not found?
At first glance, I did not find such a log.

> Am 14.03.2018 um 02:16 schrieb Igor Yakushin <iv...@uchicago.edu>:
>
>
>
> On Tue, Mar 13, 2018 at 5:54 PM, SG <in...@search-guard.com> wrote:
> pls add this to the sg_config.yml (same indent level as enable_ssl)
>
> pemtrustedcas_filepath: HadoopCertificates/incommon.crt
>
>
> The error changed but still no luck. The files are attached.
>
> Is the user from which account Elastic Search is running supposed to be known to LDAP? I created a local elasticsearch account that LDAP does not know about. Does it matter?
>
> If I can run ldapsearch from this account successfully:
> ldapsearch -x -v -h ldap.rcc.uchicago.edu -p 389 -b "uid=ivy2,ou=People,dc=rcc,dc=uchicago,dc=edu"
> does it tell anything about SearchGuard's ability to use LDAP or this is not related?
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> To post to this group, send email to search...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/CALQ0CdW9ha_0AWe4ikAfunbO_eYCZv26D7J_2bbZai%3DmJD_PRA%40mail.gmail.com\.
> For more options, visit https://groups.google.com/d/optout\.
> <log.txt><sg_config.yml>

--
You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/81c285c6-6d66-4217-9752-7516288aa10d%40googlegroups.com\.
For more options, visit https://groups.google.com/d/optout\.
<log.txt><sg_config.yml>

22

The roles have nothing to do with LDAP but are internal to SearchGuard as far as I understand?

···

On Wednesday, March 14, 2018 at 4:00:04 PM UTC-5, Search Guard wrote:

yes, authentication succeeded! now its about the roles …

23

If that is working then try setting verify_hostnames to true (because
false is insecure)

If use multiple nodes in Elastic Search cluster and do
verify_hostnames: true
does it mean that I need different certificates on different hosts or can I
reuse the same certificates?
The complication is that the rest of the nodes are on internal network and
are not visible to the outside world and our university IT people would
only generate certificates for hosts with public ip addresses as far as I
know.

Another question: now that I have LDAP working, how can I use
pyelasticsearch to connect to the cluster without hardcoding my password?
One can obviously provide a password in a plain text in the code but that
is not a good idea:

from pyelasticsearch import ElasticSearch
es = ElasticSearch('http://localhost:9200/&#39;, username='', password='')

···

On Tue, Mar 13, 2018 at 4:42 PM, SG <info@search-guard.com> wrote:

> Am 13.03.2018 um 22:19 schrieb ivy2@uchicago.edu:
>
>
>
> On Tuesday, March 13, 2018 at 3:56:07 PM UTC-5, Search Guard wrote:
> pls post your complete sg_config.yml (beware of sensitive information
like passwords!)
>
>
> Attached.
>
>
>
>
> > Am 13.03.2018 um 21:50 schrieb iv...@uchicago.edu:
> >
> >
> >
> > On Tuesday, February 27, 2018 at 11:49:09 AM UTC-6, Search Guard wrote:
> > you can just omit the bind_dn and password, then a LDAP anonymous bind
will be performed which seems the way to go in your setup
> >
> > If this is still not working than pls enable debug logging and post
(or mail) the complete logfile. Pls refer to http://docs.search-guard.com/
latest/troubleshooting-tls#setting-the-log-level-to-debug
> >
> > Hello,
> > I am still stuck with LDAP. The log file is attached.
> > Thank you,
> > Igor
> >
> >
> >
> >
> >
> >
> > > Am 19.02.2018 um 23:00 schrieb Igor Yakushin <iv...@uchicago.edu>:
> > >
> > >
> > >
> > > On Mon, Feb 19, 2018 at 2:44 PM, Jochen Kressin < > jkre...@floragunn.com> wrote:
> > > Yes, usually you need a user (in LDAP terminology: bind_dn) and
password to talk to an LDAP server. But this depends on how your LDAP is
configured actually: If it allows anonymous (usually read-only) access then
you can skip the bind_dn and password section if the sg_config LDAP
configuration. Search Guard will then try to access the server without
bind_dn and password. But if your LDAP server requires authentication, then
you need to add the bind_dn and password. This depends on what server you
user and how it is configured.
> > >
> > >
> > > Is there a way to figure out how my server is configured by using
for example 'ldapsearch' or some other tool?
> > >
> > > I looked into a working java program that creates a web page where
users can type their credentials and get access via LDAP to our system and
at first glance I do not see that it is using any password to query LDAP
server (except for the credentials of the user that tries to login):
> > > ==========
> > > environment.put(javax.naming.Context.PROVIDER_URL,
"ldaps://ldap.uchicago.edu");
> > > environment.put(javax.naming.Context.SECURITY_AUTHENTICATION,
"simple");
> > > environment.put(javax.naming.Context.SECURITY_PRINCIPAL,
usernameFormatted);
> > > environment.put(javax.naming.Context.SECURITY_CREDENTIALS,
password);
> > > DirContext ctx = new InitialDirContext(environment);
> > > SearchControls controls = new SearchControls();
> > > controls.setSearchScope(SearchControls.SUBTREE_SCOPE);
> > > NamingEnumeration<SearchResult> results =
ctx.search("ou=people,dc=uchicago,dc=edu", "uid="+username, controls);
> > > ==========
> > > Here is also a working php code
> > > ==========
> > > <?php
> > > $ldap_dn = "uid=".$_POST["username"].",dc=uchicago,dc=edu";
> > > $ldap_password = $_POST["password"];
> > > $ldap_con = ldap_connect("ldap.uchicago.edu","636");
> > > ldap_set_option($ldap_con, LDAP_OPT_PROTOCOL_VERSION, 3);
> > >
> > > if(@ldap_bind($ldap_con,$ldap_dn,$ldap_password))
> > > echo "Authenticated";
> > > else
> > > echo "Invalid Credential";
> > > ?>
> > > ==========
> > > Apparently I should be using ldap.uchicago.edu:636. That probably
means that I should use
> > > enable_ssl_client_auth: true
> > > ?
> > >
> > > Can you tell me from above what bind_dn, userbase I should use?
> > >
> > > I tried the following settings for both authc and authz but still no
luck:
> > >
> > > ==========
> > > enable_ssl_client_auth: true
> > > hosts:
> > > - ldap.uchicago.edu:636
> > > bind_dn: 'dc=uchicago,dc=edu'
> > > #password: password
> > > userbase: 'ou=people,dc=uchicago,dc=edu'
> > >
> > > ==========
> > > LDAP response read timed out, timeout used:5000ms.,
providerException=javax.naming.
> > > NamingException: LDAP response read timed out, timeout
used:5000ms.]. Try next.
> > > [2018-02-19T15:57:52,005][WARN ][c.f.s.a.BackendRegistry ]
Authentication finally failed for ivy2
> > >
> > > ==========
> > >
> > >
> > > On Monday, February 19, 2018 at 6:38:23 PM UTC+1, iv...@uchicago.edu > wrote:
> > > Hi Jochen,
> > > After following your recommendations I am now getting a different
kind of error when trying to use my LDAP credentials to log into ES:
> > > =========
> > > No password given for bind_dn cn=admin,dc=rcc,dc=uchica
> > > go,dc=edu. Will try to authenticate anonymously to ldap
> > > [2018-02-19T11:20:13,156][WARN ][c.f.s.a.BackendRegistry ]
Authentication finally failed for ivy2
> > > =========
> > > Does one normally have to use something like a password or
certificate to talk to LDAP? What bind_dn is usually given?
> > > Thank you,
> > > Igor
> > >
> > >
> > > On Monday, February 19, 2018 at 4:54:05 AM UTC-6, Jochen Kressin > wrote:
> > > From the configs and screeshots and LDAP config you sent I see that
you enable TLS and also want to verify hostnames:
> > >
> > > enable_ssl: true
> > > verify_hostnames: true
> > >
> > > This means that the connection to LDAP will be TLS encrypted and
that Search Guard needs to verify the TLS certificate that the LDAP server
sends against a Root CA / trusted CA. So in order to use TLS, you need to
configure the trusted CAs in the LDAP module. This would be the root and/or
signing certificate of the certificate you installed on the LDAP server.
> > >
> > > However, in the LDAP config you specify port 389 for the LDAP
connection, which is usually used for plain / unencrypted connections. The
port for TLS would be 636. Are you sure your LDAP server listens in port
389 for TLS connections? What happens if you configure:
> > >
> > > enable_ssl: false
> > > verify_hostnames: false
> > >
> > >
> > >
> > > On Sunday, February 18, 2018 at 6:03:25 AM UTC+1, iv...@uchicago.edu > wrote:
> > > Hello,
> > >
> > > I have just started playing with Search Guard and got stuck
configuring LDAP.
> > >
> > > First I installed the demo. It worked OK except I do not see
"Tenants" tab on the left as the document says. See the screenshot. Is it a
problem? I do want each user to be able to store his Kibana dashboards both
privately and publicly.
> > >
> > > Next I tried to enable LDAP. When I try to connect to ES from a web
browser using userid/password from LDAP, I am getting in ES logs:
> > > ====
> > > Unable to connect to ldapserver 128.135.112.111:389 due
> > > to ElasticsearchException[Empty file path for
searchguard.ssl.transport.truststore_filepath]
> > > ====
> > >
> > > How to fix it?
> > > Do I have to use truststore? Currently I just have whatever
certificates were generated by demo.
> > >
> > > ldapsearch -x -v -h 128.135.112.111 -p 389 -b
"dc=rcc,dc=uchicago,dc=edu"
> > > from the host I am running ES on works fine.
> > >
> > > Once I'll manage to configure LDAP, besides Kibana, I want to be
able to use ES from python.
> > > What options do I need to specify to
> > > ===
> > > from elasticsearch import Elasticsearch
> > > es = Elasticsearch()
> > > ===
> > > to connect to the protected ES when running on the same host where
Kibana and ES are running from inside user account known to LDAP?
> > >
> > > Thank you,
> > > Igor
> > >
> > >
> > > =====================
> > > When asking questions, please provide the following information:
> > >
> > > * Search Guard and Elasticsearch version
> > >
> > > 6.2.1
> > >
> > > * Installed and used enterprise modules, if any
> > >
> > > Elasticsearch, Kibana, Search Guard
> > >
> > > * JVM version and operating system version
> > >
> > > 1.8.0_121
> > > CentOS 7.2
> > >
> > > * Search Guard configuration files
> > >
> > > Attached
> > >
> > > * Elasticsearch log messages on debug level
> > >
> > > Attached
> > >
> > > * Other installed Elasticsearch or Kibana plugins, if any
> > >
> > > None
> > > =======================
> > >
> > > --
> > > You received this message because you are subscribed to a topic in
the Google Groups "Search Guard Community Forum" group.
> > > To unsubscribe from this topic, visit https://groups.google.com/d/
topic/search-guard/jVjrVCh0yLY/unsubscribe.
> > > To unsubscribe from this group and all its topics, send an email to
search-guard...@googlegroups.com.
> > > To post to this group, send email to search...@googlegroups.com.
> > > To view this discussion on the web visit
https://groups.google.com/d/msgid/search-guard/8d481577-
a50a-4343-aaa2-9af7c39f4ed8%40googlegroups.com.
> > > For more options, visit https://groups.google.com/d/optout\.
> > >
> > >
> > >
> > > --
> > > Igor Yakushin, Ph.D.
> > > Computational Scientist
> > > Kavli Institute for Cosmological Physics, ERC #413
> > > Research Computing Center, room #2
> > > The University of Chicago
> > > 773-834-5313
> > > https://rcc.uchicago.edu
> > >
> > >
> > > --
> > > You received this message because you are subscribed to the Google
Groups "Search Guard Community Forum" group.
> > > To unsubscribe from this group and stop receiving emails from it,
send an email to search-guard...@googlegroups.com.
> > > To post to this group, send email to search...@googlegroups.com.
> > > To view this discussion on the web visit
https://groups.google.com/d/msgid/search-guard/
CALQ0CdWtMVG1cC328x3Fdu3NHtvKBb2DQRLxgh3jjrkUSXSrsA%40mail.gmail.com.
> > > For more options, visit https://groups.google.com/d/optout\.
> >
> >
> > --
> > You received this message because you are subscribed to the Google
Groups "Search Guard Community Forum" group.
> > To unsubscribe from this group and stop receiving emails from it, send
an email to search-guard...@googlegroups.com.
> > To post to this group, send email to search...@googlegroups.com.
> > To view this discussion on the web visit https://groups.google.com/d/
msgid/search-guard/4ebf3dfe-f450-4a0e-8bc4-959c4e367301%40googlegroups.com
.
> > For more options, visit https://groups.google.com/d/optout\.
> > <log.txt>
>
>
> --
> You received this message because you are subscribed to the Google
Groups "Search Guard Community Forum" group.
> To unsubscribe from this group and stop receiving emails from it, send
an email to search-guard+unsubscribe@googlegroups.com.
> To post to this group, send email to search-guard@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
msgid/search-guard/6f62d946-09ff-4512-867d-930551947e39%40googlegroups.com
.
> For more options, visit https://groups.google.com/d/optout\.
> <elasticsearch.yml><sg_config.yml>

--
You received this message because you are subscribed to a topic in the
Google Groups "Search Guard Community Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/search-guard/jVjrVCh0yLY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/search-guard/CB47BA77-DD0A-418A-9E01-8D259D9EB908%40search-guard.com
.
For more options, visit https://groups.google.com/d/optout\.

--
Igor Yakushin, Ph.D.
Computational Scientist
Kavli Institute for Cosmological Physics, ERC #413
Research Computing Center, room #2
The University of Chicago
773-834-5313

24

And another question: how to secure Kibana?
In the demo kibanaserver user is created that uses password given in plain text in Kibana configuration file. I changed that password both on elasticsearch and kibana sides but it is still quite insecure. Do I understand correctly that I can instead use the same certificates as I used for LDAP to get rid of password authentication (or at least not to have it in a plain text format) and enable SSL?

I found a way in elasticsearch-py to specify username and password in plain text when connecting to elasticsearch server. Is there a way to avoid hardcoding passwords in python program?

25

And another question: how to secure Kibana?
In the demo kibanaserver user is created that uses password given in plain text in Kibana configuration file. I changed that password both on elasticsearch and kibana sides but it is still quite insecure. Do I understand correctly that I can instead use the same certificates as I used for LDAP to get rid of password authentication (or at least not to have it in a plain text format) and enable SSL?

I enabled SSL and got rid of kibanaserver password in the configuration file.

At first glance Kibana is working. However, SearchGuard tab disappeared when I log as admin. How to get it back? Is admin now treated for some reason as a regular user account?

The kibana configuration and log files are attached.

In the log there were complaints about cookie password. I set it but what is it for? Is it necessary to specify it?

log.txt (180 KB)

kibana.yml (5.63 KB)

···

On Thursday, March 15, 2018 at 1:31:37 PM UTC-5, iv…@uchicago.edu wrote:

I found a way in elasticsearch-py to specify username and password in plain text when connecting to elasticsearch server. Is there a way to avoid hardcoding passwords in python program?