Configuring elasticsearch which is having search guard with spring data elasticsearch transport clie

Search Guard
1

  • Search Guard and Elasticsearch version: search guard 5.6.8-19.1 and elasticsearch 5.6.8

  • Installed and used enterprise modules: no enterprise modules

  • JVM version and operating system version: java version 1.8.0_161 , Ubuntu 16.04.4 LTS

  • Search Guard configuration files

ELASTICSEARCH CONF FILE
######## Start Search Guard Demo Configuration ########

WARNING: revise all the lines below before you go into production

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.transport.keystore_type: JKS
searchguard.ssl.transport.keystore_filepath: keystore.jks
searchguard.ssl.transport.keystore_alias: keystorealias
searchguard.ssl.transport.keystore_password: password
searchguard.ssl.transport.truststore_type: JKS
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.truststore_alias: truststorestorealias
searchguard.ssl.transport.truststore_password: password
searchguard.authcz.admin_dn:

  • CN=kirk,OU=client,O=client,L=test, C=de

cluster.routing.allocation.disk.threshold_enabled: false
node.max_local_storage_nodes: 3
xpack.security.enabled: false
######## End Search Guard Demo Configuration ########

  • Elasticsearch log messages on debug level :

[2018-10-23T09:41:20,431][INFO ][c.f.s.c.PrivilegesEvaluator] No
index-level perm match for User [name=_sg_internal, roles=] [IndexType[index=test, type=testType]] [Action [[indices:data/read/search]]][RolesChecked [sg_own_index, sg_public]]

  • Other installed Elasticsearch or Kibana plugins: None

I am trying to connect to elasticsearch using transport client in spring data and I have made the following configuration for the client

public Client client() throws Exception {

	Settings esSettings = Settings.builder()

			.put("cluster.name",esClusterName)

			.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_ENABLED, true)

			.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_ENFORCE_HOSTNAME_VERIFICATION,false)

			.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_KEYSTORE_FILEPATH,"/path/to/keystore.jks")

			.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_TRUSTSTORE_FILEPATH, "path/to/truststore.jks")

			.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_KEYSTORE_PASSWORD, "password")

			.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_TRUSTSTORE_PASSWORD, "password").

			build();

	TransportClient client = new PreBuiltTransportClient(esSettings,SearchGuardSSLPlugin.class)

.addTransportAddress(

			  new InetSocketTransportAddress(InetAddress.getByName(esHost), esPort))

.addTransportAddress(

			  new InetSocketTransportAddress(InetAddress.getByName(esHost2), esPort))

.addTransportAddress(

			  new InetSocketTransportAddress(InetAddress.getByName(esHost3), esPort));

	return client;

}

But when I started the application I got the following error

failed to load elasticsearch nodes : org.elasticsearch.ElasticsearchSecurityException: no permissions for [indices:admin/exists] and User [name=_sg_internal, roles=[]]

and

when I tried to access data I got these errors

org.elasticsearch.ElasticsearchSecurityException: no permissions for [indices:data/read/search] and User [name=_sg_internal, roles=[]].

I have added the elasticsearch logs above.

I don’t think I have specified the user _sg_internal and his roles anywhere in the configuration.

Please let me know how transport client is using these values and how we can configure the client with these users information.

Thanks in advance…

2

Hi,

you may want to refer to these two artivles on how to use TransportClients with Search Guard:

https://search-guard.com/searchguard-elasicsearch-transport-clients/
https://search-guard.com/transport-client-authentication-authorization/

The articles are a bit dated, but the concepts are still the same. Every request to Elasticsearch has to be authorized by Search Guard first, which means we need a user or principal that is mapped to one or more Search Guard roles, which then determine the users permissions.

With a TransportClient you have two options:

  1. Use the DN of the TLS certificate as username, and map it to one or more SG roles. This is described in the first article. This means that all requests from your TransportClient will be assigned to the same user and have the same roles and permissions.

  2. Add the user’s credentials as header fields in the TransportClient. This is described in the second article. This means that you can use different users for each request.

What the best approach is depends on your usecase. As far is I can see, you can connect with the TransportClient, but then there is no user SG can map the request to. Please have a look at the first article on how to set it up.

Thanks!

···

On Tuesday, October 23, 2018 at 11:55:58 AM UTC+2, chicku lal wrote:

  • Search Guard and Elasticsearch version: search guard 5.6.8-19.1 and elasticsearch 5.6.8
  • Installed and used enterprise modules: no enterprise modules
  • JVM version and operating system version: java version 1.8.0_161 , Ubuntu 16.04.4 LTS
  • Search Guard configuration files

ELASTICSEARCH CONF FILE
######## Start Search Guard Demo Configuration ########

WARNING: revise all the lines below before you go into production

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.transport.keystore_type: JKS
searchguard.ssl.transport.keystore_filepath: keystore.jks
searchguard.ssl.transport.keystore_alias: keystorealias
searchguard.ssl.transport.keystore_password: password
searchguard.ssl.transport.truststore_type: JKS
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.truststore_alias: truststorestorealias
searchguard.ssl.transport.truststore_password: password
searchguard.authcz.admin_dn:

  • CN=kirk,OU=client,O=client,L=test, C=de

cluster.routing.allocation.disk.threshold_enabled: false
node.max_local_storage_nodes: 3
xpack.security.enabled: false
######## End Search Guard Demo Configuration ########

  • Elasticsearch log messages on debug level :

[2018-10-23T09:41:20,431][INFO ][c.f.s.c.PrivilegesEvaluator] No
index-level perm match for User [name=_sg_internal, roles=] [IndexType[index=test, type=testType]] [Action [[indices:data/read/search]]][RolesChecked [sg_own_index, sg_public]]

  • Other installed Elasticsearch or Kibana plugins: None

I am trying to connect to elasticsearch using transport client in spring data and I have made the following configuration for the client

public Client client() throws Exception {

  Settings esSettings = Settings.builder()

  		.put("[cluster.name](http://cluster.name)",esClusterName)

  		.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_ENABLED, true)

  		.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_ENFORCE_HOSTNAME_VERIFICATION,false)

  		.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_KEYSTORE_FILEPATH,"/path/to/keystore.jks")

  		.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_TRUSTSTORE_FILEPATH, "path/to/truststore.jks")

  		.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_KEYSTORE_PASSWORD, "password")

  		.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_TRUSTSTORE_PASSWORD, "password").

  		build();

  TransportClient client = new PreBuiltTransportClient(esSettings,SearchGuardSSLPlugin.class)

.addTransportAddress(

  		  new InetSocketTransportAddress(InetAddress.getByName(esHost), esPort))

.addTransportAddress(

  		  new InetSocketTransportAddress(InetAddress.getByName(esHost2), esPort))

.addTransportAddress(

  		  new InetSocketTransportAddress(InetAddress.getByName(esHost3), esPort));

  return client;

}

But when I started the application I got the following error

failed to load elasticsearch nodes : org.elasticsearch.ElasticsearchSecurityException: no permissions for [indices:admin/exists] and User [name=_sg_internal, roles=[]]

and

when I tried to access data I got these errors

org.elasticsearch.ElasticsearchSecurityException: no permissions for [indices:data/read/search] and User [name=_sg_internal, roles=[]].

I have added the elasticsearch logs above.

I don’t think I have specified the user _sg_internal and his roles anywhere in the configuration.

Please let me know how transport client is using these values and how we can configure the client with these users information.

Thanks in advance…

3

Hi Jochen Kressin,

Thank you for your reply. I have tried the first method and it is working.

···

On Tuesday, October 23, 2018 at 9:04:35 PM UTC+5:30, Jochen Kressin wrote:

Hi,

you may want to refer to these two artivles on how to use TransportClients with Search Guard:

https://search-guard.com/searchguard-elasicsearch-transport-clients/
https://search-guard.com/transport-client-authentication-authorization/

The articles are a bit dated, but the concepts are still the same. Every request to Elasticsearch has to be authorized by Search Guard first, which means we need a user or principal that is mapped to one or more Search Guard roles, which then determine the users permissions.

With a TransportClient you have two options:

  1. Use the DN of the TLS certificate as username, and map it to one or more SG roles. This is described in the first article. This means that all requests from your TransportClient will be assigned to the same user and have the same roles and permissions.
  1. Add the user’s credentials as header fields in the TransportClient. This is described in the second article. This means that you can use different users for each request.

What the best approach is depends on your usecase. As far is I can see, you can connect with the TransportClient, but then there is no user SG can map the request to. Please have a look at the first article on how to set it up.

Thanks!

On Tuesday, October 23, 2018 at 11:55:58 AM UTC+2, chicku lal wrote:

  • Search Guard and Elasticsearch version: search guard 5.6.8-19.1 and elasticsearch 5.6.8
  • Installed and used enterprise modules: no enterprise modules
  • JVM version and operating system version: java version 1.8.0_161 , Ubuntu 16.04.4 LTS
  • Search Guard configuration files

ELASTICSEARCH CONF FILE
######## Start Search Guard Demo Configuration ########

WARNING: revise all the lines below before you go into production

searchguard.ssl.transport.enforce_hostname_verification: false

searchguard.ssl.transport.keystore_type: JKS
searchguard.ssl.transport.keystore_filepath: keystore.jks
searchguard.ssl.transport.keystore_alias: keystorealias
searchguard.ssl.transport.keystore_password: password
searchguard.ssl.transport.truststore_type: JKS
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.truststore_alias: truststorestorealias
searchguard.ssl.transport.truststore_password: password
searchguard.authcz.admin_dn:

  • CN=kirk,OU=client,O=client,L=test, C=de

cluster.routing.allocation.disk.threshold_enabled: false
node.max_local_storage_nodes: 3
xpack.security.enabled: false
######## End Search Guard Demo Configuration ########

  • Elasticsearch log messages on debug level :

[2018-10-23T09:41:20,431][INFO ][c.f.s.c.PrivilegesEvaluator] No
index-level perm match for User [name=_sg_internal, roles=] [IndexType[index=test, type=testType]] [Action [[indices:data/read/search]]][RolesChecked [sg_own_index, sg_public]]

  • Other installed Elasticsearch or Kibana plugins: None

I am trying to connect to elasticsearch using transport client in spring data and I have made the following configuration for the client

public Client client() throws Exception {

  Settings esSettings = Settings.builder()

  		.put("[cluster.name](http://cluster.name)",esClusterName)

  		.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_ENABLED, true)

  		.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_ENFORCE_HOSTNAME_VERIFICATION,false)

  		.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_KEYSTORE_FILEPATH,"/path/to/keystore.jks")

  		.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_TRUSTSTORE_FILEPATH, "path/to/truststore.jks")

  		.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_KEYSTORE_PASSWORD, "password")

  		.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_TRUSTSTORE_PASSWORD, "password").

  		build();

  TransportClient client = new PreBuiltTransportClient(esSettings,SearchGuardSSLPlugin.class)

.addTransportAddress(

  		  new InetSocketTransportAddress(InetAddress.getByName(esHost), esPort))

.addTransportAddress(

  		  new InetSocketTransportAddress(InetAddress.getByName(esHost2), esPort))

.addTransportAddress(

  		  new InetSocketTransportAddress(InetAddress.getByName(esHost3), esPort));

  return client;

}

But when I started the application I got the following error

failed to load elasticsearch nodes : org.elasticsearch.ElasticsearchSecurityException: no permissions for [indices:admin/exists] and User [name=_sg_internal, roles=[]]

and

when I tried to access data I got these errors

org.elasticsearch.ElasticsearchSecurityException: no permissions for [indices:data/read/search] and User [name=_sg_internal, roles=[]].

I have added the elasticsearch logs above.

I don’t think I have specified the user _sg_internal and his roles anywhere in the configuration.

Please let me know how transport client is using these values and how we can configure the client with these users information.

Thanks in advance…