SG 6.5.1 [security_exception] no permissions for [indices:admin/create] how to fix ?

To be honest updating ELK Stack with searchguard with each version is getting more irritating and troublesome instead of getting easier. Yet again errors after update that I have no idea how to fix. Hopefully someone can help and tell me what is going on here. Updating cluster was yet again not possible because sgadmin update does not work with red clusters even if forced so I tried to do it with 1 node setup as below. result is that elasticsearch throws SSL errors, cannot connect to other nodes in cluster and sometimes crashes. Kibana tires to lauch but crases every single time because of permissions issues.

ELK 6.5.1 | SG 6.5.1-23.2 & 6.5.1-16

sg_roles:

Allows everything, but no changes to searchguard configuration index

sg_all_access:
readonly: true
cluster:
- UNLIMITED
indices:
’:
'
’:
- UNLIMITED
tenants:
admin_tenant: RW

Read all, but no write permissions

sg_readall:
readonly: true
cluster:
- CLUSTER_COMPOSITE_OPS_RO
indices:
’:
'
’:
- READ

Read all and monitor, but no write permissions

sg_readall_and_monitor:
cluster:
- CLUSTER_MONITOR
- CLUSTER_COMPOSITE_OPS_RO
indices:
’:
'
’:
- READ

For users which use kibana, access to indices must be granted separately

sg_kibana_user:
readonly: true
cluster:
- INDICES_MONITOR
- CLUSTER_COMPOSITE_OPS
indices:
‘?kibana’:
’:
- MANAGE
- INDEX
- READ
- DELETE
‘?kibana-6’:
'
’:
- MANAGE
- INDEX
- READ
- DELETE
’:
'
’:
- indices:data/read/field_caps*

For the kibana server

sg_kibana_server:
readonly: true
cluster:
- CLUSTER_MONITOR
- CLUSTER_COMPOSITE_OPS
- cluster:admin/xpack/monitoring*
- indices:admin/template*
indices:
‘?kibana’:
’:
- INDICES_ALL
‘?kibana-6’:
'
’:
- INDICES_ALL
‘?reporting*’:
’:
- INDICES_ALL
'?monitoring
’:
‘*’:
- INDICES_ALL

For logstash and beats

sg_logstash:
cluster:
- CLUSTER_MONITOR
- CLUSTER_COMPOSITE_OPS
- indices:admin/template/get
- indices:admin/template/put
indices:
‘logstash-’:
'
’:
- CRUD
- CREATE_INDEX
beat’:
‘*’:
- CRUD
- CREATE_INDEX

Allows adding and modifying repositories and creating and restoring snapshots

sg_manage_snapshots:
cluster:
- MANAGE_SNAPSHOTS
indices:
’:
'
’:
- “indices:data/write/index”
- “indices:admin/create”

Allows each user to access own named index

sg_own_index:
cluster:
- CLUSTER_COMPOSITE_OPS
indices:
‘${user_name}’:
‘*’:
- INDICES_ALL

``

elasticsearch config:

···

Default Elasticsearch configuration from elasticsearch-docker.

from https://github.com/elastic/elasticsearch-docker/blob/master/build/elasticsearch/elasticsearch.yml

#cluster.name: “some-cluster”
network.host: 0.0.0.0
node.name: “client1”
#network.publish_host: 172.X.X.102

minimum_master_nodes need to be explicitly set when bound on a public IP

set to 1 to allow single node clusters

Details: https://github.com/elastic/elasticsearch/pull/17288

#discovery.zen.minimum_master_nodes: 1
#discovery.zen.ping.unicast.hosts: [“172.XX.XX.XX”,“172.XX.XX.XX”,“172.XX.XX.XX”]

Use single node discovery in order to disable production mode and avoid bootstrap checks

see https://www.elastic.co/guide/en/elasticsearch/reference/current/bootstrap-checks.html

discovery.type: single-node

Search Guard

#xpack.security.enabled: false // if enabled throws erros in SG6.5.1 worked fine in older version
searchguard.enterprise_modules_enabled: false

searchguard.ssl.transport.pemcert_filepath: sg/client1.pem
searchguard.ssl.transport.pemkey_filepath: sg/client1.key
searchguard.ssl.transport.pemkey_password: XXXXXXXXX
searchguard.ssl.transport.pemtrustedcas_filepath: sg/root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.transport.resolve_hostname: false

searchguard.nodes_dn:

  • CN=client1,OU=sigh,O=blabla,DC=blabla,DC=com
  • CN=client2,OU=sigh,O=blabla,DC=blabla,DC=com
  • CN=client3,OU=sigh,O=blabla,DC=blabla,DC=com
    searchguard.authcz.admin_dn:
  • CN=admin,OU=sigh,O=blabla,DC=blabla,DC=com
  • CN=admin2,OU=sigh,O=blabla,DC=blabla,DC=com

``

elasticsearch logs:

Kibana logs:

issue: [security_exception] no permissions for [indices:admin/create] , this also appears for other users and was working fine in older SG versions.

Unavailable",“prevState”:“red”,“prevMsg”:“Unable to connect to Elasticsearch at http://elasticsearch:9200/."}
{“type”:“log”,”@timestamp":“2018-12-03T12:53:41Z”,“tags”:[“status”,“plugin:elasticsearch@6.5.1”,“info”],“pid”:1,“state”:“green”,“message”:“Status changed from red to green - Ready”,“prevState”:“red”,“prevMsg”:“Service Unavailable”}
{“type”:“log”,"@timestamp":“2018-12-03T12:53:41Z”,“tags”:[“info”,“migrations”],“pid”:1,“message”:“Creating index .kibana_2.”}
{“type”:“error”,"@timestamp":“2018-12-03T12:53:41Z”,“tags”:[“fatal”,“root”],“pid”:1,“level”:“fatal”,“error”:{“message”:"[security_exception] no permissions for [indices:admin/create] and User [name=kibanaserver, roles=, requestedTenant=null]",“name”:“Error”,“stack”:"[security_exception] no permissions for [indices:admin/create] and User [name=kibanaserver, roles=, requestedTenant=null] :: {“path”:"/.kibana_2",“query”:{},“body”:"{\“mappings\”:{\“doc\”:{\“dynamic\”:\“strict\”,\“properties\”:{\“config\”:{\“dynamic\”:\“true\”,\“properties\”:{\“buildNum\”:{\“type\”:\“keyword\”}}},\“dashboard\”:{\“properties\”:{\“description\”:{\“type\”:\“text\”},\“hits\”:{\“type\”:\“integer\”},\“kibanaSavedObjectMeta\”:{\“properties\”:{\“searchSourceJSON\”:{\“type\”:\“text\”}}},\“optionsJSON\”:{\“type\”:\“text\”},\“panelsJSON\”:{\“type\”:\“text\”},\“refreshInterval\”:{\“properties\”:{\“display\”:{\“type\”:\“keyword\”},\“pause\”:{\“type\”:\“boolean\”},\“section\”:{\“type\”:\“integer\”},\“value\”:{\“type\”:\“integer\”}}},\“timeFrom\”:{\“type\”:\“keyword\”},\“timeRestore\”:{\“type\”:\“boolean\”},\“timeTo\”:{\“type\”:\“keyword\”},\“title\”:{\“type\”:\“text\”},\“uiStateJSON\”:{\“type\”:\“text\”},\“version\”:{\“type\”:\“integer\”}}},\“graph-workspace\”:{\“properties\”:{\“description\”:{\“type\”:\“text\”},\“kibanaSavedObjectMeta\”:{\“properties\”:{\“searchSourceJSON\”:{\“type\”:\“text\”}}},\“numLinks\”:{\“type\”:\“integer\”},\“numVertices\”:{\“type\”:\“integer\”},\“title\”:{\“type\”:\“text\”},\“version\”:{\“type\”:\“integer\”},\“wsState\”:{\“type\”:\“text\”}}},\“index-pattern\”:{\“properties\”:{\“fieldFormatMap\”:{\“type\”:\“text\”},\“fields\”:{\“type\”:\“text\”},\“intervalName\”:{\“type\”:\“keyword\”},\“notExpandable\”:{\“type\”:\“boolean\”},\“sourceFilters\”:{\“type\”:\“text\”},\“timeFieldName\”:{\“type\”:\“keyword\”},\“title\”:{\“type\”:\“text\”},\“type\”:{\“type\”:\“keyword\”},\“typeMeta\”:{\“type\”:\“keyword\”}}},\“search\”:{\“properties\”:{\“columns\”:{\“type\”:\“keyword\”},\“description\”:{\“type\”:\“text\”},\“hits\”:{\“type\”:\“integer\”},\“kibanaSavedObjectMeta\”:{\“properties\”:{\“searchSourceJSON\”:{\“type\”:\“text\”}}},\“sort\”:{\“type\”:\“keyword\”},\“title\”:{\“type\”:\“text\”},\“version\”:{\“type\”:\“integer\”}}},\“server\”:{\“properties\”:{\“uuid\”:{\“type\”:\“keyword\”}}},\“timelion-sheet\”:{\“properties\”:{\“description\”:{\“type\”:\“text\”},\“hits\”:{\“type\”:\“integer\”},\“kibanaSavedObjectMeta\”:{\“properties\”:{\“searchSourceJSON\”:{\“type\”:\“text\”}}},\“timelion_chart_height\”:{\“type\”:\“integer\”},\“timelion_columns\”:{\“type\”:\“integer\”},\“timelion_interval\”:{\“type\”:\“keyword\”},\“timelion_other_interval\”:{\“type\”:\“keyword\”},\“timelion_rows\”:{\“type\”:\“integer\”},\“timelion_sheet\”:{\“type\”:\“text\”},\“title\”:{\“type\”:\“text\”},\“version\”:{\“type\”:\“integer\”}}},\“type\”:{\“type\”:\“keyword\”},\“updated_at\”:{\“type\”:\“date\”},\“url\”:{\“properties\”:{\“accessCount\”:{\“type\”:\“long\”},\“accessDate\”:{\“type\”:\“date\”},\“createDate\”:{\“type\”:\“date\”},\“url\”:{\“type\”:\“text\”,\“fields\”:{\“keyword\”:{\“type\”:\“keyword\”,\“ignore_above\”:2048}}}}},\“visualization\”:{\“properties\”:{\“description\”:{\“type\”:\“text\”},\“kibanaSavedObjectMeta\”:{\“properties\”:{\“searchSourceJSON\”:{\“type\”:\“text\”}}},\“savedSearchId\”:{\“type\”:\“keyword\”},\“title\”:{\“type\”:\“text\”},\“uiStateJSON\”:{\“type\”:\“text\”},\“version\”:{\“type\”:\“integer\”},\“visState\”:{\“type\”:\“text\”}}},\“migrationVersion\”:{\“dynamic\”:\“true\”,\“type\”:\“object\”},\“namespace\”:{\“type\”:\“keyword\”},\“kql-telemetry\”:{\“properties\”:{\“optInCount\”:{\“type\”:\“long\”},\“optOutCount\”:{\“type\”:\“long\”}}}}}},\“settings\”:{\“number_of_shards\”:1,\“auto_expand_replicas\”:\“0-1\”}}",“statusCode”:403,“response”:"{\“error\”:{\“root_cause\”:[{\“type\”:\“security_exception\”,\“reason\”:\“no permissions for [indices:admin/create] and User [name=kibanaserver, roles=, requestedTenant=null]\”}],\“type\”:\“security_exception\”,\“reason\”:\“no permissions for [indices:admin/create] and User [name=kibanaserver, roles=, requestedTenant=null]\”},\“status\”:403}"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector. (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:165:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4949:19)\n at emitNone (events.js:111:20)\n at IncomingMessage.emit (events.js:208:7)\n at endReadableNT (_stream_readable.js:1064:12)\n at _combinedTickCallback (internal/process/next_tick.js:138:11)\n at process._tickCallback (internal/process/next_tick.js:180:9)"},“message”:"[security_exception] no permissions for [indices:admin/create] and User [name=kibanaserver, roles=, requestedTenant=null]"}

``

elasticsearch:

issue: SSL Problem Insufficient buffer remaining for AEAD cipher fragment (2). Needs to be more than tag size (16), was not occuring in older version of SG

[2018-12-03T15:17:21,471][ERROR][c.f.s.s.t.SearchGuardSSLNettyTransport] [client1] SSL Problem Insufficient buffer remaining for AEAD cipher fragment (2). Needs to be more than tag size (16)
javax.net.ssl.SSLHandshakeException: Insufficient buffer remaining for AEAD cipher fragment (2). Needs to be more than tag size (16)
at sun.security.ssl.Alert.createSSLException(Alert.java:128) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:321) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:264) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:259) ~[?:?]
at sun.security.ssl.SSLTransport.decode(SSLTransport.java:129) ~[?:?]
at sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:672) ~[?:?]
at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:627) ~[?:?]
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:443) ~[?:?]
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:422) ~[?:?]
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:634) ~[?:?]
at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:294) ~[netty-handler-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1297) ~[netty-handler-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1199) ~[netty-handler-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1243) ~[netty-handler-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) ~[netty-codec-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ~[netty-codec-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) ~[netty-codec-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897) [netty-common-4.1.30.Final.jar:4.1.30.Final]
at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: javax.crypto.BadPaddingException: Insufficient buffer remaining for AEAD cipher fragment (2). Needs to be more than tag size (16)
at sun.security.ssl.SSLCipher$T13GcmReadCipherGenerator$GcmReadCipher.decrypt(SSLCipher.java:1852) ~[?:?]
at sun.security.ssl.SSLEngineInputRecord.decodeInputRecord(SSLEngineInputRecord.java:240) ~[?:?]
at sun.security.ssl.SSLEngineInputRecord.decode(SSLEngineInputRecord.java:197) ~[?:?]
at sun.security.ssl.SSLEngineInputRecord.decode(SSLEngineInputRecord.java:160) ~[?:?]
at sun.security.ssl.SSLTransport.decode(SSLTransport.java:108) ~[?:?]
… 26 more

``

Regarding Kibana 6.5.1, please refer to the upgrade instructions:

https://docs.search-guard.com/latest/upgrading-560

There are changes in Kibana which require permissions changes for the kibana server and the kibana user roles. The shipped role definitions are up to date, you can use them as reference.

Regarding the TLS exception - do you use an older version of sgadmin by chance? Or OpenSSL? The netty version in Elasticsearch has also changed with 6.5.x, so maybe you are using an older version of tcnative?

···

On Monday, December 3, 2018 at 3:29:39 PM UTC+1, Grzegorz M wrote:

To be honest updating ELK Stack with searchguard with each version is getting more irritating and troublesome instead of getting easier. Yet again errors after update that I have no idea how to fix. Hopefully someone can help and tell me what is going on here. Updating cluster was yet again not possible because sgadmin update does not work with red clusters even if forced so I tried to do it with 1 node setup as below. result is that elasticsearch throws SSL errors, cannot connect to other nodes in cluster and sometimes crashes. Kibana tires to lauch but crases every single time because of permissions issues.

ELK 6.5.1 | SG 6.5.1-23.2 & 6.5.1-16

sg_roles:

Allows everything, but no changes to searchguard configuration index

sg_all_access:
readonly: true
cluster:
- UNLIMITED
indices:
’:
'
’:
- UNLIMITED
tenants:
admin_tenant: RW

Read all, but no write permissions

sg_readall:
readonly: true
cluster:
- CLUSTER_COMPOSITE_OPS_RO
indices:
’:
'
’:
- READ

Read all and monitor, but no write permissions

sg_readall_and_monitor:
cluster:
- CLUSTER_MONITOR
- CLUSTER_COMPOSITE_OPS_RO
indices:
’:
'
’:
- READ

For users which use kibana, access to indices must be granted separately

sg_kibana_user:
readonly: true
cluster:
- INDICES_MONITOR
- CLUSTER_COMPOSITE_OPS
indices:
‘?kibana’:
’:
- MANAGE
- INDEX
- READ
- DELETE
‘?kibana-6’:
'
’:
- MANAGE
- INDEX
- READ
- DELETE
’:
'
’:
- indices:data/read/field_caps*

For the kibana server

sg_kibana_server:
readonly: true
cluster:
- CLUSTER_MONITOR
- CLUSTER_COMPOSITE_OPS
- cluster:admin/xpack/monitoring*
- indices:admin/template*
indices:
‘?kibana’:
’:
- INDICES_ALL
‘?kibana-6’:
'
’:
- INDICES_ALL
‘?reporting*’:
’:
- INDICES_ALL
'?monitoring
’:
‘*’:
- INDICES_ALL

For logstash and beats

sg_logstash:
cluster:
- CLUSTER_MONITOR
- CLUSTER_COMPOSITE_OPS
- indices:admin/template/get
- indices:admin/template/put
indices:
‘logstash-’:
'
’:
- CRUD
- CREATE_INDEX
beat’:
‘*’:
- CRUD
- CREATE_INDEX

Allows adding and modifying repositories and creating and restoring snapshots

sg_manage_snapshots:
cluster:
- MANAGE_SNAPSHOTS
indices:
’:
'
’:
- “indices:data/write/index”
- “indices:admin/create”

Allows each user to access own named index

sg_own_index:
cluster:
- CLUSTER_COMPOSITE_OPS
indices:
‘${user_name}’:
‘*’:
- INDICES_ALL

``

elasticsearch config:


Default Elasticsearch configuration from elasticsearch-docker.

from https://github.com/elastic/elasticsearch-docker/blob/master/build/elasticsearch/elasticsearch.yml

#cluster.name: “some-cluster”
network.host: 0.0.0.0
node.name: “client1”
#network.publish_host: 172.X.X.102

minimum_master_nodes need to be explicitly set when bound on a public IP

set to 1 to allow single node clusters

Details: https://github.com/elastic/elasticsearch/pull/17288

#discovery.zen.minimum_master_nodes: 1
#discovery.zen.ping.unicast.hosts: [“172.XX.XX.XX”,“172.XX.XX.XX”,“172.XX.XX.XX”]

Use single node discovery in order to disable production mode and avoid bootstrap checks

see https://www.elastic.co/guide/en/elasticsearch/reference/current/bootstrap-checks.html

discovery.type: single-node

Search Guard

#xpack.security.enabled: false // if enabled throws erros in SG6.5.1 worked fine in older version
searchguard.enterprise_modules_enabled: false

searchguard.ssl.transport.pemcert_filepath: sg/client1.pem
searchguard.ssl.transport.pemkey_filepath: sg/client1.key
searchguard.ssl.transport.pemkey_password: XXXXXXXXX
searchguard.ssl.transport.pemtrustedcas_filepath: sg/root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.transport.resolve_hostname: false

searchguard.nodes_dn:

  • CN=client1,OU=sigh,O=blabla,DC=blabla,DC=com
  • CN=client2,OU=sigh,O=blabla,DC=blabla,DC=com
  • CN=client3,OU=sigh,O=blabla,DC=blabla,DC=com
    searchguard.authcz.admin_dn:
  • CN=admin,OU=sigh,O=blabla,DC=blabla,DC=com
  • CN=admin2,OU=sigh,O=blabla,DC=blabla,DC=com

``

elasticsearch logs:

Kibana logs:

issue: [security_exception] no permissions for [indices:admin/create] , this also appears for other users and was working fine in older SG versions.

Unavailable",“prevState”:“red”,“prevMsg”:“Unable to connect to Elasticsearch at http://elasticsearch:9200/."}
{“type”:“log”,”@timestamp":“2018-12-03T12:53:41Z”,“tags”:[“status”,“plugin:elasticsearch@6.5.1”,“info”],“pid”:1,“state”:“green”,“message”:“Status changed from red to green - Ready”,“prevState”:“red”,“prevMsg”:“Service Unavailable”}
{“type”:“log”,"@timestamp":“2018-12-03T12:53:41Z”,“tags”:[“info”,“migrations”],“pid”:1,“message”:“Creating index .kibana_2.”}
{“type”:“error”,"@timestamp":“2018-12-03T12:53:41Z”,“tags”:[“fatal”,“root”],“pid”:1,“level”:“fatal”,“error”:{“message”:"[security_exception] no permissions for [indices:admin/create] and User [name=kibanaserver, roles=, requestedTenant=null]",“name”:“Error”,“stack”:"[security_exception] no permissions for [indices:admin/create] and User [name=kibanaserver, roles=, requestedTenant=null] :: {“path”:"/.kibana_2",“query”:{},“body”:"{\“mappings\”:{\“doc\”:{\“dynamic\”:\“strict\”,\“properties\”:{\“config\”:{\“dynamic\”:\“true\”,\“properties\”:{\“buildNum\”:{\“type\”:\“keyword\”}}},\“dashboard\”:{\“properties\”:{\“description\”:{\“type\”:\“text\”},\“hits\”:{\“type\”:\“integer\”},\“kibanaSavedObjectMeta\”:{\“properties\”:{\“searchSourceJSON\”:{\“type\”:\“text\”}}},\“optionsJSON\”:{\“type\”:\“text\”},\“panelsJSON\”:{\“type\”:\“text\”},\“refreshInterval\”:{\“properties\”:{\“display\”:{\“type\”:\“keyword\”},\“pause\”:{\“type\”:\“boolean\”},\“section\”:{\“type\”:\“integer\”},\“value\”:{\“type\”:\“integer\”}}},\“timeFrom\”:{\“type\”:\“keyword\”},\“timeRestore\”:{\“type\”:\“boolean\”},\“timeTo\”:{\“type\”:\“keyword\”},\“title\”:{\“type\”:\“text\”},\“uiStateJSON\”:{\“type\”:\“text\”},\“version\”:{\“type\”:\“integer\”}}},\“graph-workspace\”:{\“properties\”:{\“description\”:{\“type\”:\“text\”},\“kibanaSavedObjectMeta\”:{\“properties\”:{\“searchSourceJSON\”:{\“type\”:\“text\”}}},\“numLinks\”:{\“type\”:\“integer\”},\“numVertices\”:{\“type\”:\“integer\”},\“title\”:{\“type\”:\“text\”},\“version\”:{\“type\”:\“integer\”},\“wsState\”:{\“type\”:\“text\”}}},\“index-pattern\”:{\“properties\”:{\“fieldFormatMap\”:{\“type\”:\“text\”},\“fields\”:{\“type\”:\“text\”},\“intervalName\”:{\“type\”:\“keyword\”},\“notExpandable\”:{\“type\”:\“boolean\”},\“sourceFilters\”:{\“type\”:\“text\”},\“timeFieldName\”:{\“type\”:\“keyword\”},\“title\”:{\“type\”:\“text\”},\“type\”:{\“type\”:\“keyword\”},\“typeMeta\”:{\“type\”:\“keyword\”}}},\“search\”:{\“properties\”:{\“columns\”:{\“type\”:\“keyword\”},\“description\”:{\“type\”:\“text\”},\“hits\”:{\“type\”:\“integer\”},\“kibanaSavedObjectMeta\”:{\“properties\”:{\“searchSourceJSON\”:{\“type\”:\“text\”}}},\“sort\”:{\“type\”:\“keyword\”},\“title\”:{\“type\”:\“text\”},\“version\”:{\“type\”:\“integer\”}}},\“server\”:{\“properties\”:{\“uuid\”:{\“type\”:\“keyword\”}}},\“timelion-sheet\”:{\“properties\”:{\“description\”:{\“type\”:\“text\”},\“hits\”:{\“type\”:\“integer\”},\“kibanaSavedObjectMeta\”:{\“properties\”:{\“searchSourceJSON\”:{\“type\”:\“text\”}}},\“timelion_chart_height\”:{\“type\”:\“integer\”},\“timelion_columns\”:{\“type\”:\“integer\”},\“timelion_interval\”:{\“type\”:\“keyword\”},\“timelion_other_interval\”:{\“type\”:\“keyword\”},\“timelion_rows\”:{\“type\”:\“integer\”},\“timelion_sheet\”:{\“type\”:\“text\”},\“title\”:{\“type\”:\“text\”},\“version\”:{\“type\”:\“integer\”}}},\“type\”:{\“type\”:\“keyword\”},\“updated_at\”:{\“type\”:\“date\”},\“url\”:{\“properties\”:{\“accessCount\”:{\“type\”:\“long\”},\“accessDate\”:{\“type\”:\“date\”},\“createDate\”:{\“type\”:\“date\”},\“url\”:{\“type\”:\“text\”,\“fields\”:{\“keyword\”:{\“type\”:\“keyword\”,\“ignore_above\”:2048}}}}},\“visualization\”:{\“properties\”:{\“description\”:{\“type\”:\“text\”},\“kibanaSavedObjectMeta\”:{\“properties\”:{\“searchSourceJSON\”:{\“type\”:\“text\”}}},\“savedSearchId\”:{\“type\”:\“keyword\”},\“title\”:{\“type\”:\“text\”},\“uiStateJSON\”:{\“type\”:\“text\”},\“version\”:{\“type\”:\“integer\”},\“visState\”:{\“type\”:\“text\”}}},\“migrationVersion\”:{\“dynamic\”:\“true\”,\“type\”:\“object\”},\“namespace\”:{\“type\”:\“keyword\”},\“kql-telemetry\”:{\“properties\”:{\“optInCount\”:{\“type\”:\“long\”},\“optOutCount\”:{\“type\”:\“long\”}}}}}},\“settings\”:{\“number_of_shards\”:1,\“auto_expand_replicas\”:\“0-1\”}}",“statusCode”:403,“response”:"{\“error\”:{\“root_cause\”:[{\“type\”:\“security_exception\”,\“reason\”:\“no permissions for [indices:admin/create] and User [name=kibanaserver, roles=, requestedTenant=null]\”}],\“type\”:\“security_exception\”,\“reason\”:\“no permissions for [indices:admin/create] and User [name=kibanaserver, roles=, requestedTenant=null]\”},\“status\”:403}"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector. (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:165:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4949:19)\n at emitNone (events.js:111:20)\n at IncomingMessage.emit (events.js:208:7)\n at endReadableNT (_stream_readable.js:1064:12)\n at _combinedTickCallback (internal/process/next_tick.js:138:11)\n at process._tickCallback (internal/process/next_tick.js:180:9)"},“message”:"[security_exception] no permissions for [indices:admin/create] and User [name=kibanaserver, roles=, requestedTenant=null]"}

``

elasticsearch:

issue: SSL Problem Insufficient buffer remaining for AEAD cipher fragment (2). Needs to be more than tag size (16), was not occuring in older version of SG

[2018-12-03T15:17:21,471][ERROR][c.f.s.s.t.SearchGuardSSLNettyTransport] [client1] SSL Problem Insufficient buffer remaining for AEAD cipher fragment (2). Needs to be more than tag size (16)
javax.net.ssl.SSLHandshakeException: Insufficient buffer remaining for AEAD cipher fragment (2). Needs to be more than tag size (16)
at sun.security.ssl.Alert.createSSLException(Alert.java:128) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:321) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:264) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:259) ~[?:?]
at sun.security.ssl.SSLTransport.decode(SSLTransport.java:129) ~[?:?]
at sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:672) ~[?:?]
at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:627) ~[?:?]
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:443) ~[?:?]
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:422) ~[?:?]
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:634) ~[?:?]
at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:294) ~[netty-handler-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1297) ~[netty-handler-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1199) ~[netty-handler-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1243) ~[netty-handler-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) ~[netty-codec-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ~[netty-codec-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) ~[netty-codec-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897) [netty-common-4.1.30.Final.jar:4.1.30.Final]
at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: javax.crypto.BadPaddingException: Insufficient buffer remaining for AEAD cipher fragment (2). Needs to be more than tag size (16)
at sun.security.ssl.SSLCipher$T13GcmReadCipherGenerator$GcmReadCipher.decrypt(SSLCipher.java:1852) ~[?:?]
at sun.security.ssl.SSLEngineInputRecord.decodeInputRecord(SSLEngineInputRecord.java:240) ~[?:?]
at sun.security.ssl.SSLEngineInputRecord.decode(SSLEngineInputRecord.java:197) ~[?:?]
at sun.security.ssl.SSLEngineInputRecord.decode(SSLEngineInputRecord.java:160) ~[?:?]
at sun.security.ssl.SSLTransport.decode(SSLTransport.java:108) ~[?:?]
… 26 more

``