Starting Elasticsearch 6.4.2 Changing ownership of /data folder [2018-10-24T17:32:53,133][INFO ][o.e.n.Node ] [es-client-7d46689dff-wnbpj] initializing ... [2018-10-24T17:32:53,501][INFO ][o.e.e.NodeEnvironment ] [es-client-7d46689dff-wnbpj] using [1] data paths, mounts [[/data (/dev/sda1)]], net usable_space [90.3gb], net total_space [94.2gb], types [ext4] [2018-10-24T17:32:53,501][INFO ][o.e.e.NodeEnvironment ] [es-client-7d46689dff-wnbpj] heap size [247.6mb], compressed ordinary object pointers [true] [2018-10-24T17:32:53,505][INFO ][o.e.n.Node ] [es-client-7d46689dff-wnbpj] node name [es-client-7d46689dff-wnbpj], node ID [JuQ6Yu2oTES8q943-HZlbw] [2018-10-24T17:32:53,505][INFO ][o.e.n.Node ] [es-client-7d46689dff-wnbpj] version[6.4.2], pid[1], build[default/tar/04711c2/2018-09-26T13:34:09.098244Z], OS[Linux/4.14.56+/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_171/25.171-b11] [2018-10-24T17:32:53,506][INFO ][o.e.n.Node ] [es-client-7d46689dff-wnbpj] JVM arguments [-XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+DisableExplicitGC, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Xms256m, -Xmx256m, -Des.path.home=/elasticsearch, -Des.path.conf=/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=tar] [2018-10-24T17:33:02,917][INFO ][c.f.s.SearchGuardPlugin ] ES Config path is /elasticsearch/config [2018-10-24T17:33:03,205][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] OpenSSL not available (this is not an error, we simply fallback to built-in JDK SSL) because of java.lang.ClassNotFoundException: io.netty.internal.tcnative.SSL [2018-10-24T17:33:03,235][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Config directory is /elasticsearch/config/, from there the key- and truststore files are resolved relatively [2018-10-24T17:33:03,811][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS Transport Client Provider : JDK [2018-10-24T17:33:03,820][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS Transport Server Provider : JDK [2018-10-24T17:33:03,820][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] TLS HTTP Provider : JDK [2018-10-24T17:33:03,820][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Enabled TLS protocols for transport layer : [TLSv1.1, TLSv1.2] [2018-10-24T17:33:03,821][INFO ][c.f.s.s.DefaultSearchGuardKeyStore] Enabled TLS protocols for HTTP layer : [TLSv1.1, TLSv1.2] [2018-10-24T17:33:05,318][INFO ][c.f.s.SearchGuardPlugin ] Clustername: streaming-utilization-qual [2018-10-24T17:33:05,593][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [aggs-matrix-stats] [2018-10-24T17:33:05,593][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [analysis-common] [2018-10-24T17:33:05,593][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [ingest-common] [2018-10-24T17:33:05,593][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [lang-expression] [2018-10-24T17:33:05,593][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [lang-mustache] [2018-10-24T17:33:05,594][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [lang-painless] [2018-10-24T17:33:05,594][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [mapper-extras] [2018-10-24T17:33:05,594][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [parent-join] [2018-10-24T17:33:05,594][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [percolator] [2018-10-24T17:33:05,595][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [rank-eval] [2018-10-24T17:33:05,595][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [reindex] [2018-10-24T17:33:05,596][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [repository-url] [2018-10-24T17:33:05,596][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [transport-netty4] [2018-10-24T17:33:05,596][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [tribe] [2018-10-24T17:33:05,596][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [x-pack-core] [2018-10-24T17:33:05,596][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [x-pack-deprecation] [2018-10-24T17:33:05,596][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [x-pack-graph] [2018-10-24T17:33:05,597][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [x-pack-logstash] [2018-10-24T17:33:05,597][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [x-pack-monitoring] [2018-10-24T17:33:05,597][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [x-pack-rollup] [2018-10-24T17:33:05,597][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [x-pack-security] [2018-10-24T17:33:05,597][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [x-pack-sql] [2018-10-24T17:33:05,597][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [x-pack-upgrade] [2018-10-24T17:33:05,598][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded module [x-pack-watcher] [2018-10-24T17:33:05,602][INFO ][o.e.p.PluginsService ] [es-client-7d46689dff-wnbpj] loaded plugin [search-guard-6] [2018-10-24T17:33:16,825][WARN ][o.e.d.s.ScriptModule ] Script: returning default values for missing document values is deprecated. Set system property '-Des.scripting.exception_for_missing_value=true' to make behaviour compatible with future major versions. [2018-10-24T17:33:19,805][WARN ][o.e.d.c.s.Settings ] [http.enabled] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version. [2018-10-24T17:33:27,032][INFO ][c.f.s.a.i.AuditLogImpl ] Configured categories on rest layer to ignore: [AUTHENTICATED, GRANTED_PRIVILEGES] [2018-10-24T17:33:27,033][INFO ][c.f.s.a.i.AuditLogImpl ] Configured categories on transport layer to ignore: [AUTHENTICATED, GRANTED_PRIVILEGES] [2018-10-24T17:33:27,033][INFO ][c.f.s.a.i.AuditLogImpl ] Configured Users to ignore: [kibanaserver] [2018-10-24T17:33:27,033][INFO ][c.f.s.a.i.AuditLogImpl ] Configured Users to ignore for read compliance events: [kibanaserver] [2018-10-24T17:33:27,034][INFO ][c.f.s.a.i.AuditLogImpl ] Configured Users to ignore for write compliance events: [kibanaserver] [2018-10-24T17:33:27,103][INFO ][c.f.s.a.i.AuditLogImpl ] Message routing enabled: true [2018-10-24T17:33:27,108][WARN ][c.f.s.c.ComplianceConfig ] If you plan to use field masking pls configure searchguard.compliance.salt to be a random string of 16 chars length identical on all nodes [2018-10-24T17:33:27,116][INFO ][c.f.s.c.ComplianceConfig ] PII configuration [auditLogPattern=org.joda.time.format.DateTimeFormatter@7962a746, auditLogIndex=null]: {} [2018-10-24T17:33:28,714][INFO ][o.e.d.DiscoveryModule ] [es-client-7d46689dff-wnbpj] using discovery type [zen] [2018-10-24T17:33:32,597][INFO ][c.f.s.SearchGuardPlugin ] 4 Search Guard modules loaded so far: [Module [type=REST_MANAGEMENT_API, implementing class=com.floragunn.searchguard.dlic.rest.api.SearchGuardRestApiActions], Module [type=DLSFLS, implementing class=com.floragunn.searchguard.configuration.SearchGuardFlsDlsIndexSearcherWrapper], Module [type=AUDITLOG, implementing class=com.floragunn.searchguard.auditlog.impl.AuditLogImpl], Module [type=MULTITENANCY, implementing class=com.floragunn.searchguard.configuration.PrivilegesInterceptorImpl]] [2018-10-24T17:33:32,601][INFO ][o.e.n.Node ] [es-client-7d46689dff-wnbpj] initialized [2018-10-24T17:33:32,605][INFO ][o.e.n.Node ] [es-client-7d46689dff-wnbpj] starting ... [2018-10-24T17:33:33,314][INFO ][o.e.t.TransportService ] [es-client-7d46689dff-wnbpj] publish_address {10.16.7.17:9300}, bound_addresses {10.16.7.17:9300} [2018-10-24T17:33:33,340][INFO ][o.e.b.BootstrapChecks ] [es-client-7d46689dff-wnbpj] bound or publishing to a non-loopback address, enforcing bootstrap checks [2018-10-24T17:33:33,432][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Check if searchguard index exists ... [2018-10-24T17:33:38,711][INFO ][o.e.c.s.ClusterApplierService] [es-client-7d46689dff-wnbpj] detected_master {es-master-5858d4896f-6mdws}{g3RBbFECRAOOnJSFrop56A}{Y4eM8EM9ROGz56TLJA3U7w}{10.16.3.18}{10.16.3.18:9300}{xpack.installed=true}, added {{es-master-5858d4896f-6wxvd}{qZdbB5uIQJOa9aLUfOJ85Q}{J-rMXi3oSvi4TMlvsq_9fg}{10.16.4.14}{10.16.4.14:9300}{xpack.installed=true},{es-master-5858d4896f-6mdws}{g3RBbFECRAOOnJSFrop56A}{Y4eM8EM9ROGz56TLJA3U7w}{10.16.3.18}{10.16.3.18:9300}{xpack.installed=true},}, reason: apply cluster state (from master [master {es-master-5858d4896f-6mdws}{g3RBbFECRAOOnJSFrop56A}{Y4eM8EM9ROGz56TLJA3U7w}{10.16.3.18}{10.16.3.18:9300}{xpack.installed=true} committed version [15]]) [2018-10-24T17:33:41,897][INFO ][o.e.l.LicenseService ] [es-client-7d46689dff-wnbpj] license [bdb8590f-bb71-4a32-b531-5b3eb6a1fe46] mode [basic] - valid [2018-10-24T17:33:42,332][INFO ][c.f.s.h.SearchGuardHttpServerTransport] [es-client-7d46689dff-wnbpj] publish_address {10.16.7.17:9200}, bound_addresses {10.16.7.17:9200} [2018-10-24T17:33:42,333][INFO ][o.e.n.Node ] [es-client-7d46689dff-wnbpj] started [2018-10-24T17:33:42,423][ERROR][c.f.s.c.ConfigurationLoader] Failure No shard available for [org.elasticsearch.action.get.MultiGetShardRequest@22e5b165] retrieving configuration for [config, roles, rolesmapping, internalusers, actiongroups] (index=searchguard) [2018-10-24T17:33:42,425][ERROR][c.f.s.c.ConfigurationLoader] Failure No shard available for [org.elasticsearch.action.get.MultiGetShardRequest@22e5b165] retrieving configuration for [config, roles, rolesmapping, internalusers, actiongroups] (index=searchguard) [2018-10-24T17:33:42,425][ERROR][c.f.s.c.ConfigurationLoader] Failure No shard available for [org.elasticsearch.action.get.MultiGetShardRequest@22e5b165] retrieving configuration for [config, roles, rolesmapping, internalusers, actiongroups] (index=searchguard) [2018-10-24T17:33:42,427][ERROR][c.f.s.c.ConfigurationLoader] Failure No shard available for [org.elasticsearch.action.get.MultiGetShardRequest@22e5b165] retrieving configuration for [config, roles, rolesmapping, internalusers, actiongroups] (index=searchguard) [2018-10-24T17:33:42,428][ERROR][c.f.s.c.ConfigurationLoader] Failure No shard available for [org.elasticsearch.action.get.MultiGetShardRequest@22e5b165] retrieving configuration for [config, roles, rolesmapping, internalusers, actiongroups] (index=searchguard) [2018-10-24T17:33:45,767][INFO ][o.e.c.s.ClusterApplierService] [es-client-7d46689dff-wnbpj] added {{es-data-0}{fGgREP6cS3yylPDDG-7NNQ}{8_ZsHfppRXWJWTTy6zHDnw}{10.16.2.15}{10.16.2.15:9300}{xpack.installed=true},}, reason: apply cluster state (from master [master {es-master-5858d4896f-6mdws}{g3RBbFECRAOOnJSFrop56A}{Y4eM8EM9ROGz56TLJA3U7w}{10.16.3.18}{10.16.3.18:9300}{xpack.installed=true} committed version [16]]) [2018-10-24T17:33:50,911][INFO ][c.f.s.c.ComplianceConfig ] Compliance features are enabled [2018-10-24T17:33:50,912][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Search Guard License Info: SearchGuardLicense [uid=00000000-0000-0000-0000-000000000000, type=TRIAL, features=[COMPLIANCE], issueDate=2018-10-24, expiryDate=2018-12-24, issuedTo=The world, issuer=floragunn GmbH, startDate=2018-10-24, majorVersion=6, clusterName=*, allowedNodeCount=2147483647, msgs=[], expiresInDays=60, isExpired=false, valid=true, action=, prodUsage=Yes, one cluster with all commercial features and unlimited nodes per cluster., clusterService=org.elasticsearch.cluster.service.ClusterService@60f63c35, getMsgs()=[], getExpiresInDays()=60, isExpired()=false, isValid()=true, getAction()=, getProdUsage()=Yes, one cluster with all commercial features and unlimited nodes per cluster.] [2018-10-24T17:33:50,912][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Search Guard License Type: TRIAL, valid [2018-10-24T17:33:50,913][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Node 'es-client-7d46689dff-wnbpj' initialized [2018-10-24T17:34:18,773][INFO ][o.e.c.s.ClusterApplierService] [es-client-7d46689dff-wnbpj] added {{es-data-1}{w80EMEo2Swac2gin8Np7aw}{0YE7vVAVQT-3AP5AAiLHXw}{10.16.1.16}{10.16.1.16:9300}{xpack.installed=true},}, reason: apply cluster state (from master [master {es-master-5858d4896f-6mdws}{g3RBbFECRAOOnJSFrop56A}{Y4eM8EM9ROGz56TLJA3U7w}{10.16.3.18}{10.16.3.18:9300}{xpack.installed=true} committed version [24]]) [2018-10-24T17:37:24,338][INFO ][c.f.s.c.ComplianceConfig ] Compliance features are enabled [2018-10-24T17:37:24,341][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Search Guard License Info: SearchGuardLicense [uid=00000000-0000-0000-0000-000000000000, type=TRIAL, features=[COMPLIANCE], issueDate=2018-10-24, expiryDate=2018-12-24, issuedTo=The world, issuer=floragunn GmbH, startDate=2018-10-24, majorVersion=6, clusterName=*, allowedNodeCount=2147483647, msgs=[], expiresInDays=60, isExpired=false, valid=true, action=, prodUsage=Yes, one cluster with all commercial features and unlimited nodes per cluster., clusterService=org.elasticsearch.cluster.service.ClusterService@60f63c35, getMsgs()=[], getExpiresInDays()=60, isExpired()=false, isValid()=true, getAction()=, getProdUsage()=Yes, one cluster with all commercial features and unlimited nodes per cluster.] [2018-10-24T17:37:24,341][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Search Guard License Type: TRIAL, valid [2018-10-24T17:37:43,337][WARN ][o.e.d.a.a.i.t.p.PutIndexTemplateRequest] Deprecated field [template] used, replaced by [index_patterns] [2018-10-24T17:39:45,388][WARN ][o.e.d.a.a.i.t.p.PutIndexTemplateRequest] Deprecated field [template] used, replaced by [index_patterns] [2018-10-24T17:39:46,273][WARN ][o.e.d.a.a.i.t.p.PutIndexTemplateRequest] Deprecated field [template] used, replaced by [index_patterns] [2018-10-24T17:39:47,190][WARN ][o.e.d.a.a.i.t.p.PutIndexTemplateRequest] Deprecated field [template] used, replaced by [index_patterns] [2018-10-24T17:40:07,691][WARN ][o.e.d.a.a.i.t.p.PutIndexTemplateRequest] Deprecated field [template] used, replaced by [index_patterns] [2018-10-24T17:40:08,305][WARN ][o.e.d.a.a.i.t.p.PutIndexTemplateRequest] Deprecated field [template] used, replaced by [index_patterns]