Hi There,
I am trying to work with elasticseach 1.5.2 and searchguard 0.5 plugin. After following instructions from github, I still can’t seem to have searchguard take over the authentication/authorization activities. Following are some of my questions:
-
Where in the code does SeachGuard register to be the control point for authentication activities for all ES requests?
-
I see that plugin complains few times that it can’t find the index. But I also see the logs that represent successful ‘index’ read - “Security configuration reloaded”. Once the searchguard index config is read, how is it applied? What Threads are triggered to enforce the ACL rules?
-
Can I leave the Authentication to ‘HTTP Basic Authentication’ and use SearchGuard for ACL rules only?
-
Currently I have kibana/nginx combination for ‘HTTP Basic Authentication’. How can i make that authentication to reach SearchGuard?
I have shared my config and es logs below. I really appreciate your help.
Thanks,
Sumana
Following is my searchguard ACL def:
···
===========================
[root@testhost elasticsearch]# curl -XGET localhost:9200/searchguard/ac/ac?pretty=1
{
“_index” : “searchguard”,
“_type” : “ac”,
“_id” : “ac”,
“_version” : 2,
“found” : true,
“_source”:{
“acl”: [
{
“Comment”: “By default no filters are executed and no filters a by-passed. In such a case an exception is thrown and access will be denied.”,
“filters_bypass”: ,
“filters_execute”:
},
{
“Comment”: “For role admin all filters are bypassed (so none will be executed). This means unrestricted access.”,
“roles”: [
“admin”
],
“filters_bypass”: [“*”],
“filters_execute”:
},
{
“Comment”: “”,
“roles”: [“root”],
“filters_bypass”: [“*”],
“filters_execute”:
},
{
“Comment”: “”,
“roles”: [“logrepos”],
“indices”: [
“logrepos*”
],
“filters_bypass”: [“*”],
“filters_execute”:
},
{
“Comment”: “”,
“roles”: [“yesterday”],
“indices”: [
“survey-2015.12.09”
],
“filters_bypass”: [“*”],
“filters_execute”:
}
]
}
}
[root@testhost elasticsearch]#
Following is what I see after i follow the instructions from github:
===============================================
[2015-12-10 16:15:26,652][INFO ][node ] [es-test-node] version[1.5.2], pid[17900], build[62ff986/2015-04-27T09:21:06Z]
[2015-12-10 16:15:26,652][INFO ][node ] [es-test-node] initializing …
[2015-12-10 16:15:27,015][INFO ][com.floragunn.searchguard.SearchGuardPlugin] Class enhancements for DLS/FLS successful
[2015-12-10 16:15:27,025][INFO ][plugins ] [es-test-node] loaded [searchguard], sites [head]
[2015-12-10 16:15:31,239][DEBUG][com.floragunn.searchguard.service.SearchGuardService] Loaded key from /var/lib/elasticsearch/searchguard_node_key.key
[2015-12-10 16:15:31,397][WARN ][com.floragunn.searchguard.util.SecurityUtil] AES 256 not supported, max key length for AES is 128. To enable AES 256 install ‘Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files’
[2015-12-10 16:15:31,706][DEBUG][com.floragunn.searchguard.util.SecurityUtil] Usable SSL/TLS protocols: [TLSv1, TLSv1.1, TLSv1.2]
[2015-12-10 16:15:31,706][DEBUG][com.floragunn.searchguard.util.SecurityUtil] Usable SSL/TLS cipher suites: [TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA]
[2015-12-10 16:15:32,185][INFO ][node ] [es-test-node] initialized
[2015-12-10 16:15:32,186][INFO ][node ] [es-test-node] starting …
[2015-12-10 16:15:32,190][TRACE][com.floragunn.searchguard.service.SearchGuardService] With settings {path.logs=/var/log/elasticsearch, path.work=/tmp/elasticsearch, path.conf=/etc/elasticsearch, path.data=/var/lib/elasticsearch, path.home=/usr/share/elasticsearch, pidfile=/var/run/elasticsearch/elasticsearch.pid, searchguard.authentication.settingsdb.user.newadmin=admin, http.cors.enabled=false, searchguard.authentication.authorizer.impl=com.floragunn.searchguard.authorization.simple.SettingsBasedAuthorizator, bootstrap.mlockall=true, searchguard.authentication.settingsdb.digest=SHA1, searchguard.authentication.settingsdb.user.spock=vulcan, script.disable_dynamic=false, cluster.name=testcluster, searchguard.authentication.http_authenticator.impl=com.floragunn.searchguard.authentication.http.basic.HTTPBasicAuthenticator, searchguard.enabled=true, indices.breaker.total.limit=70%, searchguard.allow_all_from_loopback=true, searchguard.authentication.settingsdb.user.michaeljackson=neverland, searchguard.authentication.authorizer.cache.enable=true, indices.fielddata.cache.size=58%, index.number_of_replicas=0, searchguard.authentication.authentication_backend.impl=com.floragunn.searchguard.authentication.backend.simple.SettingsBasedAuthenticationBackend, searchguard.config_index_name=searchguard, searchguard.authentication.authentication_backend.cache.enable=true, searchguard.http.enable_sessions=true, indices.breaker.request.limit=40%, indices.breaker.fielddata.limit=60%, searchguard.key_path=/var/lib/elasticsearch/, searchguard.authentication.settingsdb.user.admin=secret, index.number_of_shards=3, searchguard.http.xforwardedfor.trustedproxies=localhost, node.name=es-test-node, discovery.zen.ping.multicast.enabled=false, name=es-test-node, client.type=node, searchguard.actionrequestfilter.names.0=readonly, searchguard.authentication.authorization.settingsdb.roles.newcust.0=logrepos, searchguard.authentication.authorization.settingsdb.roles.admin.0=root, searchguard.actionrequestfilter.readonly.forbidden_actions.1=indices:admin*, searchguard.actionrequestfilter.readonly.forbidden_actions.0=cluster:, searchguard.actionrequestfilter.readonly.allowed_actions.0=indices:data/read/, searchguard.actionrequestfilter.readonly.allowed_actions.1=monitor, searchguard.authentication.authorization.settingsdb.roles.spock.0=yesterday, transport.type=com.floragunn.searchguard.transport.SearchGuardNettyTransport, bulk.udp.enabled=false}
[2015-12-10 16:15:32,735][DEBUG][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] using profile[default], worker_count[6], port[9300-9400], bind_host[null], publish_host[null], compress[false], connect_timeout[30s], connections_per_node[2/3/6/1/1], receive_predictor[512kb->512kb]
[2015-12-10 16:15:32,949][DEBUG][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] Bound profile [default] to address [/0:0:0:0:0:0:0:0:9300]
[2015-12-10 16:15:32,964][INFO ][transport ] [es-test-node] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/192.168.0.35:9300]}
[2015-12-10 16:15:33,005][INFO ][discovery ] [es-test-node] testcluster/zc7weDlZQKakjyZdnBoelg
[2015-12-10 16:15:33,089][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0x673ce6f6, /192.168.0.35:35918 => /192.168.0.35:9300]
[2015-12-10 16:15:33,107][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0x64160215, /192.168.0.35:35919 => /192.168.0.35:9300]
[2015-12-10 16:15:33,108][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0x397889a3, /192.168.0.35:35920 => /192.168.0.35:9300]
[2015-12-10 16:15:33,109][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0x42028e04, /192.168.0.35:35921 => /192.168.0.35:9300]
[2015-12-10 16:15:33,109][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0x401ac2a9, /192.168.0.35:35922 => /192.168.0.35:9300]
[2015-12-10 16:15:33,114][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0x083a59e2, /192.168.0.35:35923 => /192.168.0.35:9300]
[2015-12-10 16:15:33,117][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0xba78fb17, /192.168.0.35:35924 => /192.168.0.35:9300]
[2015-12-10 16:15:33,120][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0xf4f759e3, /192.168.0.35:35925 => /192.168.0.35:9300]
[2015-12-10 16:15:33,120][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0x2c2ac24e, /192.168.0.35:35926 => /192.168.0.35:9300]
[2015-12-10 16:15:33,121][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0xf3bcf672, /192.168.0.35:35927 => /192.168.0.35:9300]
[2015-12-10 16:15:33,121][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0x35e7dfc1, /192.168.0.35:35928 => /192.168.0.35:9300]
[2015-12-10 16:15:33,122][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0xae3eb39c, /192.168.0.35:35929 => /192.168.0.35:9300]
[2015-12-10 16:15:33,122][TRACE][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] channel opened: [id: 0x70705b2f, /192.168.0.35:35930 => /192.168.0.35:9300]
[2015-12-10 16:15:33,151][DEBUG][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] connected to node [[es-test-node][zc7weDlZQKakjyZdnBoelg][testhost][inet[/192.168.0.35:9300]]]
[2015-12-10 16:15:33,161][DEBUG][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] send internal:discovery/zen/unicast from es-test-node to es-test-node
[2015-12-10 16:15:34,524][DEBUG][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] send internal:discovery/zen/unicast from es-test-node to es-test-node
[2015-12-10 16:15:36,032][DEBUG][com.floragunn.searchguard.transport.SearchGuardNettyTransport] [es-test-node] send internal:discovery/zen/unicast from es-test-node to es-test-node
[2015-12-10 16:15:36,076][INFO ][cluster.service ] [es-test-node] new_master [es-test-node][zc7weDlZQKakjyZdnBoelg][testhost][inet[/192.168.0.35:9300]], reason: zen-disco-join (elected_as_master)
[2015-12-10 16:15:36,100][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] REQUEST on node es-test-node: internal:gateway/local/meta_state (class org.elasticsearch.gateway.local.state.meta.TransportNodesListGatewayMetaState$Request) from INTRANODE
[2015-12-10 16:15:36,100][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] Context
[2015-12-10 16:15:36,100][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] Headers
[2015-12-10 16:15:36,101][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] TYPE: intra node request, skip filters
[2015-12-10 16:15:36,101][DEBUG][com.floragunn.searchguard.filter.DLSActionFilter] REQUEST on node es-test-node: internal:gateway/local/meta_state (class org.elasticsearch.gateway.local.state.meta.TransportNodesListGatewayMetaState$Request) from INTRANODE
[2015-12-10 16:15:36,101][DEBUG][com.floragunn.searchguard.filter.DLSActionFilter] Context
[2015-12-10 16:15:36,101][DEBUG][com.floragunn.searchguard.filter.DLSActionFilter] Headers
[2015-12-10 16:15:36,101][DEBUG][com.floragunn.searchguard.filter.DLSActionFilter] TYPE: intra node request, skip filters
[2015-12-10 16:15:36,102][DEBUG][com.floragunn.searchguard.filter.FLSActionFilter] REQUEST on node es-test-node: internal:gateway/local/meta_state (class org.elasticsearch.gateway.local.state.meta.TransportNodesListGatewayMetaState$Request) from INTRANODE
[2015-12-10 16:15:36,102][DEBUG][com.floragunn.searchguard.filter.FLSActionFilter] Context
[2015-12-10 16:15:36,102][DEBUG][com.floragunn.searchguard.filter.FLSActionFilter] Headers
[2015-12-10 16:15:36,102][DEBUG][com.floragunn.searchguard.filter.FLSActionFilter] TYPE: intra node request, skip filters
[2015-12-10 16:15:36,103][TRACE][com.floragunn.searchguard.filter.SearchGuardActionFilter] action internal:gateway/local/meta_state (class org.elasticsearch.gateway.local.state.meta.TransportNodesListGatewayMetaState$Request) from INTRANODE
[2015-12-10 16:15:36,103][TRACE][com.floragunn.searchguard.filter.SearchGuardActionFilter] INTRANODE request
[2015-12-10 16:15:36,287][INFO ][http ] [es-test-node] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/192.168.0.35:9200]}
[2015-12-10 16:15:36,289][INFO ][node ] [es-test-node] started
[2015-12-10 16:15:36,287][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] REQUEST on node es-test-node: internal:gateway/local/started_shards (class org.elasticsearch.gateway.local.state.shards.TransportNodesListGatewayStartedShards$Request) from INTRANODE
[2015-12-10 16:15:36,290][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] Context
[2015-12-10 16:15:36,290][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] Headers
[2015-12-10 16:15:36,291][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] TYPE: intra node request, skip filters
[2015-12-10 16:15:36,294][DEBUG][com.floragunn.searchguard.filter.DLSActionFilter] REQUEST on node es-test-node: internal:gateway/local/started_shards (class org.elasticsearch.gateway.local.state.shards.TransportNodesListGatewayStartedShards$Request) from INTRANODE
[2015-12-10 16:15:36,297][DEBUG][com.floragunn.searchguard.filter.DLSActionFilter] Context
[2015-12-10 16:15:36,297][DEBUG][com.floragunn.searchguard.filter.DLSActionFilter] Headers
[2015-12-10 16:15:36,297][DEBUG][com.floragunn.searchguard.filter.DLSActionFilter] TYPE: intra node request, skip filters
[2015-12-10 16:15:36,297][DEBUG][com.floragunn.searchguard.filter.FLSActionFilter] REQUEST on node es-test-node: internal:gateway/local/started_shards (class org.elasticsearch.gateway.local.state.shards.TransportNodesListGatewayStartedShards$Request) from INTRANODE
[2015-12-10 16:15:39,248][TRACE][com.floragunn.searchguard.filter.SearchGuardActionFilter] action indices:data/read/get (class org.elasticsearch.action.get.GetRequest) from INTRANODE
[2015-12-10 16:15:39,248][TRACE][com.floragunn.searchguard.filter.SearchGuardActionFilter] INTRANODE request
[2015-12-10 16:15:39,248][ERROR][com.floragunn.searchguard.service.SearchGuardConfigService] [es-test-node] Try to refresh security configuration but it failed due to org.elasticsearch.action.NoShardAvailableActionException: [searchguard][0] null
org.elasticsearch.action.NoShardAvailableActionException: [searchguard][0] null
at org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$AsyncSingleAction.perform(TransportShardSingleOperationAction.java:175)
at org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$AsyncSingleAction.start(TransportShardSingleOperationAction.java:155)
at org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction.doExecute(TransportShardSingleOperationAction.java:89)
at org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction.doExecute(TransportShardSingleOperationAction.java:55)
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:167)
at com.floragunn.searchguard.filter.SearchGuardActionFilter.apply0(SearchGuardActionFilter.java:141)
at com.floragunn.searchguard.filter.SearchGuardActionFilter.apply(SearchGuardActionFilter.java:89)
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:165)
at com.floragunn.searchguard.filter.AbstractActionFilter.apply(AbstractActionFilter.java:105)
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:165)
at com.floragunn.searchguard.filter.AbstractActionFilter.apply(AbstractActionFilter.java:105)
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:165)
at com.floragunn.searchguard.filter.AbstractActionFilter.apply(AbstractActionFilter.java:105)
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:165)
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:82)
at org.elasticsearch.client.node.NodeClient.execute(NodeClient.java:98)
at org.elasticsearch.client.support.AbstractClient.get(AbstractClient.java:193)
at org.elasticsearch.action.get.GetRequestBuilder.doExecute(GetRequestBuilder.java:201)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:91)
at com.floragunn.searchguard.service.SearchGuardConfigService.reloadConfig(SearchGuardConfigService.java:81)
at com.floragunn.searchguard.service.SearchGuardConfigService.access$500(SearchGuardConfigService.java:41)
at com.floragunn.searchguard.service.SearchGuardConfigService$Reload.run(SearchGuardConfigService.java:111)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
[2015-12-10 16:15:40,250][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] REQUEST on node es-test-node: indices:data/read/get (class org.elasticsearch.action.get.GetRequest) from INTRANODE
[2015-12-10 16:15:40,250][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] Context
[2015-12-10 16:15:40,250][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] Headers
[2015-12-10 16:15:40,251][DEBUG][com.floragunn.searchguard.filter.RequestActionFilter] TYPE: intra node request, skip filters
[2015-12-10 16:15:40,251][DEBUG][com.floragunn.searchguard.filter.DLSActionFilter] REQUEST on node es-test-node: indices:data/read/get (class org.elasticsearch.action.get.GetRequest) from INTRANODE
[2015-12-10 16:15:40,251][DEBUG][com.floragunn.searchguard.filter.FLSActionFilter] Context
[2015-12-10 16:15:40,251][DEBUG][com.floragunn.searchguard.filter.FLSActionFilter] Headers
[2015-12-10 16:15:40,251][DEBUG][com.floragunn.searchguard.filter.FLSActionFilter] TYPE: intra node request, skip filters
[2015-12-10 16:15:40,252][TRACE][com.floragunn.searchguard.filter.SearchGuardActionFilter] action indices:data/read/get (class org.elasticsearch.action.get.GetRequest) from INTRANODE