We’ve recently upgraded elasticsearch and kibana from 5.6.5 to 6.1.1. We also got the Search Guard plugins for elastic and kibana upgraded to the latest 6.1.1 versions.
Everything is currently up and running, including Kibana. However, when attempting to modify an existing Visualization, we get the following error (logged in as an admin user):
When we remove the Search Guard plugins, we have no problem creating/editing visualizations or creating index patterns. Any idea what the issue might be?
Does this only happen when you click on Visualizations?
What happens when you e.g. click on Management, Dev Tools or Timelion?
Do you see anything in the KI logs?
···
On Thursday, January 11, 2018 at 12:30:30 AM UTC+1, Tanner Kinkead wrote:
We’ve recently upgraded elasticsearch and kibana from 5.6.5 to 6.1.1. We also got the Search Guard plugins for elastic and kibana upgraded to the latest 6.1.1 versions.
Everything is currently up and running, including Kibana. However, when attempting to modify an existing Visualization, we get the following error (logged in as an admin user):
When we remove the Search Guard plugins, we have no problem creating/editing visualizations or creating index patterns. Any idea what the issue might be?
I have upgraded my test environment from 5.6.x to 6.1 but I didnt experience this issue. May I know whether you followed the steps for upgrading the kibana index and other index( including tenant index) to version 6 compatible?
Does this only happen when you click on Visualizations?
– It happens when saving a visualization, a change to an existing visualization, or creating a new index pattern (there may be other circumstances, I have not fully tested)
What happens when you e.g. click on Management, Dev Tools or Timelion?
– All of these are fully accessible
Do you see anything in the KI logs?
– Sorry, which logs are these?
Kibana index upgrade
– We had a lot of old visualizations and dashboards that were invalid because we had changed our index patterns to support cross-cluster search. We deleted the kibana index from version 5 and allowed a new one to be created after we upgraded to 6.
I found this in the elasticsearch logs when re-creating the issue:
[2018-01-11T18:46:07,372][INFO ][c.f.s.c.PrivilegesEvaluator] No cluster-level perm match for User [name=kibanaserver, roles=, requestedTenant=null] [IndexType [index=.kibana, type=*]] [Action [[indices:admin/template/put]]] [RolesChecked [sg_kibana_server, sg_own_index, sg_public]]
[2018-01-11T18:46:07,372][INFO ][c.f.s.c.PrivilegesEvaluator] No permissions for {}
Kibana connects to elasticsearch using the account kibanaserver. The kibanaserver account is given the sg_kibana_server role in sg_roles_mapping.yml. sg_kibana_server has the following permissions:
This config was unchanged from our SearchGuard 5 deployment. Are these incorrect?
···
On Wednesday, January 10, 2018 at 3:30:30 PM UTC-8, Tanner Kinkead wrote:
We’ve recently upgraded elasticsearch and kibana from 5.6.5 to 6.1.1. We also got the Search Guard plugins for elastic and kibana upgraded to the latest 6.1.1 versions.
Everything is currently up and running, including Kibana. However, when attempting to modify an existing Visualization, we get the following error (logged in as an admin user):
When we remove the Search Guard plugins, we have no problem creating/editing visualizations or creating index patterns. Any idea what the issue might be?
[2018-01-11T18:46:07,372][INFO ][c.f.s.c.PrivilegesEvaluator] No cluster-level perm match for User [name=kibanaserver, roles=, requestedTenant=null] [IndexType [index=.kibana, type=*]] [Action [[indices:admin/template/put]]] [RolesChecked [sg_kibana_server, sg_own_index, sg_public]]
the kibana server user is lacking the indices:admin/template/put permission on cluster level, and this is because you used role definitions for ES5, not for ES6:
The permission schema can change from Elasticsearch version to Elasticsearch version, and especially for major upgrades this can be a source of error. In other words, if you use the permission schema we ship for Elasticsearch 5.x, this will cause issues on Elasticsearch 6.x
Please revise your role definitions and action groups and compare them with the ones shipped with SG 6. For example, the role for the kibana server user we ship with SG 6 is:
As you can see it contains the missing indices:admin/template/put permissions.
···
On Thursday, January 11, 2018 at 7:53:11 PM UTC+1, Tanner Kinkead wrote:
Do you see anything in the Elasticsearch logs?
– Yes (see below)
Does this only happen when you click on Visualizations?
– It happens when saving a visualization, a change to an existing visualization, or creating a new index pattern (there may be other circumstances, I have not fully tested)
What happens when you e.g. click on Management, Dev Tools or Timelion?
– All of these are fully accessible
We’ve recently upgraded elasticsearch and kibana from 5.6.5 to 6.1.1. We also got the Search Guard plugins for elastic and kibana upgraded to the latest 6.1.1 versions.
Everything is currently up and running, including Kibana. However, when attempting to modify an existing Visualization, we get the following error (logged in as an admin user):
When we remove the Search Guard plugins, we have no problem creating/editing visualizations or creating index patterns. Any idea what the issue might be?
Do you see anything in the KI logs?
– Sorry, which logs are these?
Kibana index upgrade
– We had a lot of old visualizations and dashboards that were invalid because we had changed our index patterns to support cross-cluster search. We deleted the kibana index from version 5 and allowed a new one to be created after we upgraded to 6.
I found this in the elasticsearch logs when re-creating the issue:
[2018-01-11T18:46:07,372][INFO ][c.f.s.c.PrivilegesEvaluator] No cluster-level perm match for User [name=kibanaserver, roles=, requestedTenant=null] [IndexType [index=.kibana, type=*]] [Action [[indices:admin/template/put]]] [RolesChecked [sg_kibana_server, sg_own_index, sg_public]]
[2018-01-11T18:46:07,372][INFO ][c.f.s.c.PrivilegesEvaluator] No permissions for {}
Kibana connects to elasticsearch using the account kibanaserver. The kibanaserver account is given the sg_kibana_server role in sg_roles_mapping.yml. sg_kibana_server has the following permissions:
On Saturday, January 13, 2018 at 1:05:25 PM UTC-8, Jochen Kressin wrote:
According to this entry in the log:
[2018-01-11T18:46:07,372][INFO ][c.f.s.c.PrivilegesEvaluator] No cluster-level perm match for User [name=kibanaserver, roles=, requestedTenant=null] [IndexType [index=.kibana, type=*]] [Action [[indices:admin/template/put]]] [RolesChecked [sg_kibana_server, sg_own_index, sg_public]]
the kibana server user is lacking the indices:admin/template/put permission on cluster level, and this is because you used role definitions for ES5, not for ES6:
The permission schema can change from Elasticsearch version to Elasticsearch version, and especially for major upgrades this can be a source of error. In other words, if you use the permission schema we ship for Elasticsearch 5.x, this will cause issues on Elasticsearch 6.x
Please revise your role definitions and action groups and compare them with the ones shipped with SG 6. For example, the role for the kibana server user we ship with SG 6 is:
sg_kibana_server:
readonly: true
cluster:
- CLUSTER_MONITOR
- CLUSTER_COMPOSITE_OPS
- cluster:admin/xpack/monitoring*
- indices:admin/template*
indices:
'?kibana':
'*':
- INDICES_ALL
'?reporting*':
'*':
- INDICES_ALL
'?monitoring*':
'*':
- INDICES_ALL
As you can see it contains the missing indices:admin/template/put permissions.
On Thursday, January 11, 2018 at 7:53:11 PM UTC+1, Tanner Kinkead wrote:
Do you see anything in the Elasticsearch logs?
– Yes (see below)
Does this only happen when you click on Visualizations?
– It happens when saving a visualization, a change to an existing visualization, or creating a new index pattern (there may be other circumstances, I have not fully tested)
What happens when you e.g. click on Management, Dev Tools or Timelion?
– All of these are fully accessible
We’ve recently upgraded elasticsearch and kibana from 5.6.5 to 6.1.1. We also got the Search Guard plugins for elastic and kibana upgraded to the latest 6.1.1 versions.
Everything is currently up and running, including Kibana. However, when attempting to modify an existing Visualization, we get the following error (logged in as an admin user):
When we remove the Search Guard plugins, we have no problem creating/editing visualizations or creating index patterns. Any idea what the issue might be?
Do you see anything in the KI logs?
– Sorry, which logs are these?
Kibana index upgrade
– We had a lot of old visualizations and dashboards that were invalid because we had changed our index patterns to support cross-cluster search. We deleted the kibana index from version 5 and allowed a new one to be created after we upgraded to 6.
I found this in the elasticsearch logs when re-creating the issue:
[2018-01-11T18:46:07,372][INFO ][c.f.s.c.PrivilegesEvaluator] No cluster-level perm match for User [name=kibanaserver, roles=, requestedTenant=null] [IndexType [index=.kibana, type=*]] [Action [[indices:admin/template/put]]] [RolesChecked [sg_kibana_server, sg_own_index, sg_public]]
[2018-01-11T18:46:07,372][INFO ][c.f.s.c.PrivilegesEvaluator] No permissions for {}
Kibana connects to elasticsearch using the account kibanaserver. The kibanaserver account is given the sg_kibana_server role in sg_roles_mapping.yml. sg_kibana_server has the following permissions: