Issues loading from searchguard index

Search Guard
1

Hi There,

I am trying to work with elasticseach 1.5.2 and searchguard 0.5 plugin. After following instructions from github, I still can’t seem to have searchguard take over the authentication/authorization activities. Following are some of my questions:

  1. Where in the code does SeachGuard register to be the control point for authentication activities for all ES requests?

  2. I see that plugin complains few times that it can’t find the index. But I also see the logs that represent successful ‘index’ read - “Security configuration reloaded”. Once the searchguard index config is read, how is it applied? What Threads are triggered to enforce the ACL rules?

  3. Can I leave the Authentication to ‘HTTP Basic Authentication’ and use SearchGuard for ACL rules only?

  4. Currently I have kibana/nginx combination for ‘HTTP Basic Authentication’. How can i make that authentication to reach SearchGuard?

I have shared my config and es logs below. I really appreciate your help.

Thanks,

Sumana

Following is my searchguard ACL def:

···

===========================

[root@testhost elasticsearch]# curl -XGET localhost:9200/searchguard/ac/ac?pretty=1

{

“_index” : “searchguard”,

“_type” : “ac”,

“_id” : “ac”,

“_version” : 2,

“found” : true,

“_source”:{

“acl”: [

{

“Comment”: “By default no filters are executed and no filters a by-passed. In such a case an exception is thrown and access will be denied.”,

“filters_bypass”: ,

“filters_execute”:

},

{

“Comment”: “For role admin all filters are bypassed (so none will be executed). This means unrestricted access.”,

“roles”: [

“admin”

],

“filters_bypass”: [“*”],

“filters_execute”:

},

{

“Comment”: “”,

“roles”: [“root”],

“filters_bypass”: [“*”],

“filters_execute”:

},

{

“Comment”: “”,

“roles”: [“logrepos”],

“indices”: [

“logrepos*”

],

“filters_bypass”: [“*”],

“filters_execute”:

},

{

“Comment”: “”,

“roles”: [“yesterday”],

“indices”: [

“survey-2015.12.09”

],

“filters_bypass”: [“*”],

“filters_execute”:

}

]

}

}

[root@testhost elasticsearch]#

Following is what I see after i follow the instructions from github:

===============================================

[2015-12-10 16:15:26,652][INFO ][node ] [es-test-node] version[1.5.2], pid[17900], build[62ff986/2015-04-27T09:21:06Z]

[2015-12-10 16:15:26,652][INFO ][node ] [es-test-node] initializing …

[2015-12-10 16:15:27,015][INFO ][com.floragunn.searchguard.SearchGuardPlugin] Class enhancements for DLS/FLS successful

[2015-12-10 16:15:27,025][INFO ][plugins ] [es-test-node] loaded [searchguard], sites [head]

[2015-12-10 16:15:31,239][DEBUG][com.floragunn.searchguard.service.SearchGuardService] Loaded key from /var/lib/elasticsearch/searchguard_node_key.key

[2015-12-10 16:15:31,397][WARN ][com.floragunn.searchguard.util.SecurityUtil] AES 256 not supported, max key length for AES is 128. To enable AES 256 install ‘Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files’

[2015-12-10 16:15:31,706][DEBUG][com.floragunn.searchguard.util.SecurityUtil] Usable SSL/TLS protocols: [TLSv1, TLSv1.1, TLSv1.2]

[2015-12-10 16:15:31,706][DEBUG][com.floragunn.searchguard.util.SecurityUtil] Usable SSL/TLS cipher suites: [TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA]

[2015-12-10 16:15:32,185][INFO ][node ] [es-test-node] initialized

[2015-12-10 16:15:32,186][INFO ][node ] [es-test-node] starting …

[2015-12-10 16:15:32,190][TRACE][com.floragunn.searchguard.service.SearchGuardService] With settings {path.logs=/var/log/elasticsearch, path.work=/tmp/elasticsearch, path.conf=/etc/elasticsearch, path.data=/var/lib/elasticsearch, path.home=/usr/share/elasticsearch, pidfile=/var/run/elasticsearch/elasticsearch.pid, searchguard.authentication.settingsdb.user.newadmin=admin, http.cors.enabled=false, searchguard.authentication.authorizer.impl=com.floragunn.searchguard.authorization.simple.SettingsBasedAuthorizator, bootstrap.mlockall=true, searchguard.authentication.settingsdb.digest=SHA1, searchguard.authentication.settingsdb.user.spock=vulcan, script.disable_dynamic=false, cluster.name=testcluster, searchguard.authentication.http_authenticator.impl=com.floragunn.searchguard.authentication.http.basic.HTTPBasicAuthenticator, searchguard.enabled=true, indices.breaker.total.limit=70%, searchguard.allow_all_from_loopback=true, searchguard.authentication.settingsdb.user.michaeljackson=neverland, searchguard.authentication.authorizer.cache.enable=true, indices.fielddata.cache.size=58%, index.number_of_replicas=0, searchguard.authentication.authentication_backend.impl=com.floragunn.searchguard.authentication.backend.simple.SettingsBasedAuthenticationBackend, searchguard.config_index_name=searchguard, searchguard.authentication.authentication_backend.cache.enable=true, searchguard.http.enable_sessions=true, indices.breaker.request.limit=40%, indices.breaker.fielddata.limit=60%, searchguard.key_path=/var/lib/elasticsearch/, searchguard.authentication.settingsdb.user.admin=secret, index.number_of_shards=3, searchguard.http.xforwardedfor.trustedproxies=localhost, node.name=es-test-node, discovery.zen.ping.multicast.enabled=false, name=es-test-node, client.type=node, searchguard.actionrequestfilter.names.0=readonly, searchguard.authentication.authorization.settingsdb.roles.newcust.0=logrepos, searchguard.authentication.authorization.settingsdb.roles.admin.0=root, searchguard.actionrequestfilter.readonly.forbidden_actions.1=indices:admin*, searchguard.actionrequestfilter.readonly.forbidden_actions.0=cluster:, searchguard.actionrequestfilter.readonly.allowed_actions.0=indices:data/read/, searchguard.actionrequestfilter.readonly.allowed_actions.1=monitor, searchguard.authentication.authorization.settingsdb.roles.spock.0=yesterday, transport.type=com.floragunn.searchguard.transport.SearchGuardNettyTransport, bulk.udp.enabled=false}

[2015-12-10 16:15:32,735][DEBUG][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] using profile[default], worker_count[6], port[9300-9400], bind_host[null], publish_host[null], compress[false], connect_timeout[30s], connections_per_node[2/3/6/1/1], receive_predictor[512kb->512kb]

[2015-12-10 16:15:32,949][DEBUG][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] Bound profile [default] to address [/0:0:0:0:0:0:0:0:9300]

[2015-12-10 16:15:32,964][INFO ][transport ] [es-test-node] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/192.168.0.35:9300]}

[2015-12-10 16:15:33,005][INFO ][discovery ] [es-test-node] testcluster/zc7weDlZQKakjyZdnBoelg

[2015-12-10 16:15:33,089][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0x673ce6f6, /192.168.0.35:35918 => /192.168.0.35:9300]

[2015-12-10 16:15:33,107][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0x64160215, /192.168.0.35:35919 => /192.168.0.35:9300]

[2015-12-10 16:15:33,108][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0x397889a3, /192.168.0.35:35920 => /192.168.0.35:9300]

[2015-12-10 16:15:33,109][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0x42028e04, /192.168.0.35:35921 => /192.168.0.35:9300]

[2015-12-10 16:15:33,109][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0x401ac2a9, /192.168.0.35:35922 => /192.168.0.35:9300]

[2015-12-10 16:15:33,114][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0x083a59e2, /192.168.0.35:35923 => /192.168.0.35:9300]

[2015-12-10 16:15:33,117][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0xba78fb17, /192.168.0.35:35924 => /192.168.0.35:9300]

[2015-12-10 16:15:33,120][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0xf4f759e3, /192.168.0.35:35925 => /192.168.0.35:9300]

[2015-12-10 16:15:33,120][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0x2c2ac24e, /192.168.0.35:35926 => /192.168.0.35:9300]

[2015-12-10 16:15:33,121][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0xf3bcf672, /192.168.0.35:35927 => /192.168.0.35:9300]

[2015-12-10 16:15:33,121][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0x35e7dfc1, /192.168.0.35:35928 => /192.168.0.35:9300]

[2015-12-10 16:15:33,122][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0xae3eb39c, /192.168.0.35:35929 => /192.168.0.35:9300]

[2015-12-10 16:15:33,122][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0x70705b2f, /192.168.0.35:35930 => /192.168.0.35:9300]

[2015-12-10 16:15:33,151][DEBUG][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] connected to node [[es-test-node][zc7weDlZQKakjyZdnBoelg][testhost][inet[/192.168.0.35:9300]]]

[2015-12-10 16:15:33,161][DEBUG][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] send internal:discovery/zen/unicast from es-test-node to es-test-node

[2015-12-10 16:15:34,524][DEBUG][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] send internal:discovery/zen/unicast from es-test-node to es-test-node

[2015-12-10 16:15:36,032][DEBUG][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] send internal:discovery/zen/unicast from es-test-node to es-test-node

[2015-12-10 16:15:36,076][INFO ][cluster.service ] [es-test-node] new_master [es-test-node][zc7weDlZQKakjyZdnBoelg][testhost][inet[/192.168.0.35:9300]], reason: zen-disco-join (elected_as_master)

[2015-12-10 16:15:36,100][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] REQUEST on node es-test-node: internal:gateway/local/meta_state (class org.elasticsearch.gateway.local.state.meta.TransportNodesListGatewayMetaState$Request) from INTRANODE

[2015-12-10 16:15:36,100][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] Context

[2015-12-10 16:15:36,100][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] Headers

[2015-12-10 16:15:36,101][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] TYPE: intra node request, skip filters

[2015-12-10 16:15:36,101][DEBUG][com.floragunn.searchguard.filter.DLSActionFilter] REQUEST on node es-test-node: internal:gateway/local/meta_state (class org.elasticsearch.gateway.local.state.meta.TransportNodesListGatewayMetaState$Request) from INTRANODE

[2015-12-10 16:15:36,101][DEBUG][com.floragunn.searchguard.filter.DLSActionFilter] Context

[2015-12-10 16:15:36,101][DEBUG][com.floragunn.searchguard.filter.DLSActionFilter] Headers

[2015-12-10 16:15:36,101][DEBUG][com.floragunn.searchguard.filter.DLSActionFilter] TYPE: intra node request, skip filters

[2015-12-10 16:15:36,102][DEBUG][com.floragunn.searchguard.filter.FLSActionFilter] REQUEST on node es-test-node: internal:gateway/local/meta_state (class org.elasticsearch.gateway.local.state.meta.TransportNodesListGatewayMetaState$Request) from INTRANODE

[2015-12-10 16:15:36,102][DEBUG][com.floragunn.searchguard.filter.FLSActionFilter] Context

[2015-12-10 16:15:36,102][DEBUG][com.floragunn.searchguard.filter.FLSActionFilter] Headers

[2015-12-10 16:15:36,102][DEBUG][com.floragunn.searchguard.filter.FLSActionFilter] TYPE: intra node request, skip filters

[2015-12-10 16:15:36,103][TRACE][com.floragunn.searchguard.filter.SearchGuardActionFilter] action internal:gateway/local/meta_state (class org.elasticsearch.gateway.local.state.meta.TransportNodesListGatewayMetaState$Request) from INTRANODE

[2015-12-10 16:15:36,103][TRACE][com.floragunn.searchguard.filter.SearchGuardActionFilter] INTRANODE request

[2015-12-10 16:15:36,287][INFO ][http ] [es-test-node] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/192.168.0.35:9200]}

[2015-12-10 16:15:36,289][INFO ][node ] [es-test-node] started

[2015-12-10 16:15:36,287][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] REQUEST on node es-test-node: internal:gateway/local/started_shards (class org.elasticsearch.gateway.local.state.shards.TransportNodesListGatewayStartedShards$Request) from INTRANODE

[2015-12-10 16:15:36,290][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] Context

[2015-12-10 16:15:36,290][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] Headers

[2015-12-10 16:15:36,291][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] TYPE: intra node request, skip filters

[2015-12-10 16:15:36,294][DEBUG][com.floragunn.searchguard.filter.DLSActionFilter] REQUEST on node es-test-node: internal:gateway/local/started_shards (class org.elasticsearch.gateway.local.state.shards.TransportNodesListGatewayStartedShards$Request) from INTRANODE

[2015-12-10 16:15:36,297][DEBUG][com.floragunn.searchguard.filter.DLSActionFilter] Context

[2015-12-10 16:15:36,297][DEBUG][com.floragunn.searchguard.filter.DLSActionFilter] Headers

[2015-12-10 16:15:36,297][DEBUG][com.floragunn.searchguard.filter.DLSActionFilter] TYPE: intra node request, skip filters

[2015-12-10 16:15:36,297][DEBUG][com.floragunn.searchguard.filter.FLSActionFilter] REQUEST on node es-test-node: internal:gateway/local/started_shards (class org.elasticsearch.gateway.local.state.shards.TransportNodesListGatewayStartedShards$Request) from INTRANODE

[2015-12-10 16:15:39,248][TRACE][com.floragunn.searchguard.filter.SearchGuardActionFilter] action indices:data/read/get (class org.elasticsearch.action.get.GetRequest) from INTRANODE

[2015-12-10 16:15:39,248][TRACE][com.floragunn.searchguard.filter.SearchGuardActionFilter] INTRANODE request

[2015-12-10 16:15:39,248][ERROR][com.floragunn.searchguard.service.SearchGuardConfigService] [es-test-node] Try to refresh security configuration but it failed due to org.elasticsearch.action.NoShardAvailableActionException: [searchguard][0] null

org.elasticsearch.action.NoShardAvailableActionException: [searchguard][0] null

at org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$AsyncSingleAction.perform(TransportShardSingleOperationAction.java:175)

at org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$AsyncSingleAction.start(TransportShardSingleOperationAction.java:155)

at org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction.doExecute(TransportShardSingleOperationAction.java:89)

at org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction.doExecute(TransportShardSingleOperationAction.java:55)

at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:167)

at com.floragunn.searchguard.filter.SearchGuardActionFilter.apply0(SearchGuardActionFilter.java:141)

at com.floragunn.searchguard.filter.SearchGuardActionFilter.apply(SearchGuardActionFilter.java:89)

at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:165)

at com.floragunn.searchguard.filter.AbstractActionFilter.apply(AbstractActionFilter.java:105)

at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:165)

at com.floragunn.searchguard.filter.AbstractActionFilter.apply(AbstractActionFilter.java:105)

at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:165)

at com.floragunn.searchguard.filter.AbstractActionFilter.apply(AbstractActionFilter.java:105)

at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:165)

at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:82)

at org.elasticsearch.client.node.NodeClient.execute(NodeClient.java:98)

at org.elasticsearch.client.support.AbstractClient.get(AbstractClient.java:193)

at org.elasticsearch.action.get.GetRequestBuilder.doExecute(GetRequestBuilder.java:201)

at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:91)

at com.floragunn.searchguard.service.SearchGuardConfigService.reloadConfig(SearchGuardConfigService.java:81)

at com.floragunn.searchguard.service.SearchGuardConfigService.access$500(SearchGuardConfigService.java:41)

at com.floragunn.searchguard.service.SearchGuardConfigService$Reload.run(SearchGuardConfigService.java:111)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

[2015-12-10 16:15:40,250][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] REQUEST on node es-test-node: indices:data/read/get (class org.elasticsearch.action.get.GetRequest) from INTRANODE

[2015-12-10 16:15:40,250][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] Context

[2015-12-10 16:15:40,250][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] Headers

[2015-12-10 16:15:40,251][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] TYPE: intra node request, skip filters

[2015-12-10 16:15:40,251][DEBUG][com.floragunn.searchguard.filter.DLSActionFilter] REQUEST on node es-test-node: indices:data/read/get (class org.elasticsearch.action.get.GetRequest) from INTRANODE

[2015-12-10 16:15:40,251][DEBUG][com.floragunn.searchguard.filter.FLSActionFilter] Context

[2015-12-10 16:15:40,251][DEBUG][com.floragunn.searchguard.filter.FLSActionFilter] Headers

[2015-12-10 16:15:40,251][DEBUG][com.floragunn.searchguard.filter.FLSActionFilter] TYPE: intra node request, skip filters

[2015-12-10 16:15:40,252][TRACE][com.floragunn.searchguard.filter.SearchGuardActionFilter] action indices:data/read/get (class org.elasticsearch.action.get.GetRequest) from INTRANODE