Since upgrading to Elastic Stack 7.11.2 with Search Guard 50.0.0, all users (including admin users) have read-only access in Kibana in Discover (unable to save searches, see screenshot):
I am also unable to access dev tools (see second screen shot):
Finally, as soon as I click on the space selector icon (the TS in the above screenshot), the icon vanishes. I can still access all spaces by entering the URL.
I am assuming that these are all permissions issues, and are all related, since the first two appear to be permissions issues, and I haven’t been able to find any information about this for the Elastic Stack 7.11.2 in general.
I have logs in DEBUG mode, but they are too large to upload, please let me know how I can get them to you and what other information you need.
srgbnd
March 22, 2021, 7:27pm
2
Hi. In 7.11, we have a bug, the Kibana plugin doesn’t work properly without Multitenancy. Do you have Multitenancy disabled? Please share kibana.yml. Do you have any error in the web browser console log?
Could you please try the following build? https://maven.search-guard.com//search-guard-kibana-plugin-snapshot/my-test-SNAPSHOT/search-guard-kibana-plugin-7.11.2-master-SNAPSHOT_351-the-kibana-7-11-is-broken-if-multitenancy-is-disabled.zip
Yes. Multitenancy is disabled.
srgbnd:
Please share kibana.yml.
Here you go (note: we deploy to EKS via the Elasticsearch Helm charts, so many config options are handled outside kibana.yml):
########################################################################
# Kibana configuration
#
# Kibana configuration is primarily handled through environment
# variables. This file is for items not configured by environment
# variables.
#
# Enable/Disable self-monitoring.
monitoring.kibana.collection.enabled: false
# Xpack configuration.
xpack:
# Must be false when using Search Guard
security.enabled: false
server:
xsrf:
whitelist:
- '/searchguard/saml/acs/idpinitiated'
- '/searchguard/saml/acs'
- '/searchguard/saml/logout'
# Require TLS for connections to Kibana
ssl:
enabled: true
certificate: '/usr/share/kibana/config/certs/cert.pem'
key: '/usr/share/kibana/config/certs/key.pem'
# Configure connection to Elasticsearch
elasticsearch:
username: '${ELASTICSEARCH_USERNAME}'
password: '${ELASTICSEARCH_PASSWORD}'
requestHeadersWhitelist:
- 'Authorization'
- 'sgtenant'
ssl:
alwaysPresentCertificate: false
certificate: '/usr/share/kibana/config/certs/cert.pem'
key: '/usr/share/kibana/config/certs/key.pem'
certificateAuthorities:
- '/usr/share/kibana/config/certs/ca_bundle.pem'
verificationMode: 'none'
# Configure authentication
searchguard:
# See documentation at:
# https://docs.search-guard.com/latest/saml-authentication#activating-saml
allow_client_certificates: true
auth:
type: '{{ ["basicauth", "saml"][elk_sg_auth_saml|default(true)|int] }}'
debug: {{ elk_kibana_sg_auth_debug|default(false)|string|lower }}
basicauth:
forbidden_usernames:
- 'kibanaserver'
- 'logstash'
cookie:
isSameSite: 'None'
secure: true
Here’s what I’m getting from the console log:
Some messages have been moved to the Issues panel.
discover:1 Unchecked runtime.lastError: Could not establish connection. Receiving end does not exist.
discover:1 Unchecked runtime.lastError: Could not establish connection. Receiving end does not exist.
discover:342 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
bootstrap.js:43 ^ A single error about an inline script not firing due to content security policy is expected!
content.js:2927 Uncaught Error: Attempting to use a disconnected port object
at postMessage (content.js:2927)
at Init.window.onload (content.js:1171)
postMessage @ content.js:2927
Init.window.onload @ content.js:1171
load (async)
Init @ content.js:1168
(anonymous) @ content.js:1161
content.js:2927 Uncaught Error: Attempting to use a disconnected port object
at postMessage (content.js:2927)
at handleNewScriptError (content.js:1316)
at HTMLDocument.<anonymous> (content.js:1321)
postMessage @ content.js:2927
handleNewScriptError @ content.js:1316
(anonymous) @ content.js:1321
error (async)
errorDetectCodeToInject @ VM130:31
(anonymous) @ VM130:64
(anonymous) @ content.js:1326
VM130:26 TypeError: Cannot read property 'home_tutorial_directory' of undefined
at home.plugin.js:1
at Array.filter (<anonymous>)
at FeatureCatalogueRegistry.get (home.plugin.js:1)
at _callee$ (home.chunk.1.js:10)
at l (kbn-ui-shared-deps.js:321)
at Generator._invoke (kbn-ui-shared-deps.js:321)
at Generator.forEach.e.<computed> [as next] (kbn-ui-shared-deps.js:321)
at application_asyncGeneratorStep (home.chunk.1.js:10)
at _next (home.chunk.1.js:10)
at home.chunk.1.js:10
window.console.error @ VM130:26
_callee$ @ core.entry.js:13
l @ kbn-ui-shared-deps.js:321
(anonymous) @ kbn-ui-shared-deps.js:321
forEach.e.<computed> @ kbn-ui-shared-deps.js:321
asyncGeneratorStep @ core.entry.js:13
_throw @ core.entry.js:13
Promise.then (async)
asyncGeneratorStep @ core.entry.js:13
_next @ core.entry.js:13
(anonymous) @ core.entry.js:13
(anonymous) @ core.entry.js:13
mount @ core.entry.js:13
(anonymous) @ core.entry.js:13
fs @ kbn-ui-shared-deps.js:375
vl @ kbn-ui-shared-deps.js:375
t.unstable_runWithPriority @ kbn-ui-shared-deps.js:383
Hr @ kbn-ui-shared-deps.js:375
bl @ kbn-ui-shared-deps.js:375
ol @ kbn-ui-shared-deps.js:375
(anonymous) @ kbn-ui-shared-deps.js:375
t.unstable_runWithPriority @ kbn-ui-shared-deps.js:383
Hr @ kbn-ui-shared-deps.js:375
$r @ kbn-ui-shared-deps.js:375
Gr @ kbn-ui-shared-deps.js:375
el @ kbn-ui-shared-deps.js:375
enqueueSetState @ kbn-ui-shared-deps.js:375
y.setState @ kbn-ui-shared-deps.js:353
(anonymous) @ kbn-ui-shared-deps.js:283
e @ core.entry.js:6
(anonymous) @ core.entry.js:6
notifyListeners @ core.entry.js:6
p @ core.entry.js:6
(anonymous) @ core.entry.js:6
confirmTransitionTo @ core.entry.js:6
push @ core.entry.js:6
navigate @ core.entry.js:13
_callee2$ @ core.entry.js:13
l @ kbn-ui-shared-deps.js:321
(anonymous) @ kbn-ui-shared-deps.js:321
forEach.e.<computed> @ kbn-ui-shared-deps.js:321
application_service_asyncGeneratorStep @ core.entry.js:13
_next @ core.entry.js:13
Promise.then (async)
application_service_asyncGeneratorStep @ core.entry.js:13
_next @ core.entry.js:13
(anonymous) @ core.entry.js:13
(anonymous) @ core.entry.js:13
navigateToApp @ core.entry.js:13
redirectOnMissingCapabilities @ devTools.chunk.1.js:1
renderApp @ devTools.chunk.1.js:1
_callee$ @ devTools.plugin.js:1
l @ kbn-ui-shared-deps.js:321
(anonymous) @ kbn-ui-shared-deps.js:321
forEach.e.<computed> @ kbn-ui-shared-deps.js:321
asyncGeneratorStep @ devTools.plugin.js:1
_next @ devTools.plugin.js:1
Promise.then (async)
asyncGeneratorStep @ devTools.plugin.js:1
_next @ devTools.plugin.js:1
Promise.then (async)
asyncGeneratorStep @ devTools.plugin.js:1
_next @ devTools.plugin.js:1
(anonymous) @ devTools.plugin.js:1
(anonymous) @ devTools.plugin.js:1
mount @ devTools.plugin.js:1
_callee$ @ core.entry.js:13
l @ kbn-ui-shared-deps.js:321
(anonymous) @ kbn-ui-shared-deps.js:321
forEach.e.<computed> @ kbn-ui-shared-deps.js:321
application_service_asyncGeneratorStep @ core.entry.js:13
_next @ core.entry.js:13
(anonymous) @ core.entry.js:13
(anonymous) @ core.entry.js:13
(anonymous) @ core.entry.js:13
_callee$ @ core.entry.js:13
l @ kbn-ui-shared-deps.js:321
(anonymous) @ kbn-ui-shared-deps.js:321
forEach.e.<computed> @ kbn-ui-shared-deps.js:321
asyncGeneratorStep @ core.entry.js:13
_next @ core.entry.js:13
(anonymous) @ core.entry.js:13
(anonymous) @ core.entry.js:13
mount @ core.entry.js:13
(anonymous) @ core.entry.js:13
fs @ kbn-ui-shared-deps.js:375
vl @ kbn-ui-shared-deps.js:375
t.unstable_runWithPriority @ kbn-ui-shared-deps.js:383
Hr @ kbn-ui-shared-deps.js:375
bl @ kbn-ui-shared-deps.js:375
ol @ kbn-ui-shared-deps.js:375
(anonymous) @ kbn-ui-shared-deps.js:375
t.unstable_runWithPriority @ kbn-ui-shared-deps.js:383
Hr @ kbn-ui-shared-deps.js:375
$r @ kbn-ui-shared-deps.js:375
Gr @ kbn-ui-shared-deps.js:375
el @ kbn-ui-shared-deps.js:375
enqueueSetState @ kbn-ui-shared-deps.js:375
y.setState @ kbn-ui-shared-deps.js:353
(anonymous) @ kbn-ui-shared-deps.js:283
e @ core.entry.js:6
(anonymous) @ core.entry.js:6
notifyListeners @ core.entry.js:6
p @ core.entry.js:6
(anonymous) @ core.entry.js:6
confirmTransitionTo @ core.entry.js:6
push @ core.entry.js:6
navigate @ core.entry.js:13
_callee2$ @ core.entry.js:13
l @ kbn-ui-shared-deps.js:321
(anonymous) @ kbn-ui-shared-deps.js:321
forEach.e.<computed> @ kbn-ui-shared-deps.js:321
application_service_asyncGeneratorStep @ core.entry.js:13
_next @ core.entry.js:13
Promise.then (async)
application_service_asyncGeneratorStep @ core.entry.js:13
_next @ core.entry.js:13
(anonymous) @ core.entry.js:13
(anonymous) @ core.entry.js:13
navigateToApp @ core.entry.js:13
onClick @ core.entry.js:13
m @ kbn-ui-shared-deps.js:375
S @ kbn-ui-shared-deps.js:375
(anonymous) @ kbn-ui-shared-deps.js:375
x @ kbn-ui-shared-deps.js:375
M @ kbn-ui-shared-deps.js:375
O @ kbn-ui-shared-deps.js:375
w @ kbn-ui-shared-deps.js:375
Tn @ kbn-ui-shared-deps.js:375
ce @ kbn-ui-shared-deps.js:375
Mn @ kbn-ui-shared-deps.js:375
Ln @ kbn-ui-shared-deps.js:375
wn @ kbn-ui-shared-deps.js:375
t.unstable_runWithPriority @ kbn-ui-shared-deps.js:383
Hr @ kbn-ui-shared-deps.js:375
se @ kbn-ui-shared-deps.js:375
On @ kbn-ui-shared-deps.js:375
Show 65 more frames
content.js:2927 Uncaught Error: Attempting to use a disconnected port object
at postMessage (content.js:2927)
at handleNewScriptError (content.js:1316)
at HTMLDocument.<anonymous> (content.js:1321)
postMessage @ content.js:2927
handleNewScriptError @ content.js:1316
(anonymous) @ content.js:1321
Promise.then (async)
asyncGeneratorStep @ core.entry.js:13
_next @ core.entry.js:13
(anonymous) @ core.entry.js:13
(anonymous) @ core.entry.js:13
mount @ core.entry.js:13
(anonymous) @ core.entry.js:13
fs @ kbn-ui-shared-deps.js:375
vl @ kbn-ui-shared-deps.js:375
t.unstable_runWithPriority @ kbn-ui-shared-deps.js:383
Hr @ kbn-ui-shared-deps.js:375
bl @ kbn-ui-shared-deps.js:375
ol @ kbn-ui-shared-deps.js:375
(anonymous) @ kbn-ui-shared-deps.js:375
t.unstable_runWithPriority @ kbn-ui-shared-deps.js:383
Hr @ kbn-ui-shared-deps.js:375
$r @ kbn-ui-shared-deps.js:375
Gr @ kbn-ui-shared-deps.js:375
el @ kbn-ui-shared-deps.js:375
enqueueSetState @ kbn-ui-shared-deps.js:375
y.setState @ kbn-ui-shared-deps.js:353
(anonymous) @ kbn-ui-shared-deps.js:283
e @ core.entry.js:6
(anonymous) @ core.entry.js:6
notifyListeners @ core.entry.js:6
p @ core.entry.js:6
(anonymous) @ core.entry.js:6
confirmTransitionTo @ core.entry.js:6
push @ core.entry.js:6
navigate @ core.entry.js:13
_callee2$ @ core.entry.js:13
l @ kbn-ui-shared-deps.js:321
(anonymous) @ kbn-ui-shared-deps.js:321
forEach.e.<computed> @ kbn-ui-shared-deps.js:321
application_service_asyncGeneratorStep @ core.entry.js:13
_next @ core.entry.js:13
Promise.then (async)
application_service_asyncGeneratorStep @ core.entry.js:13
_next @ core.entry.js:13
(anonymous) @ core.entry.js:13
(anonymous) @ core.entry.js:13
navigateToApp @ core.entry.js:13
redirectOnMissingCapabilities @ devTools.chunk.1.js:1
renderApp @ devTools.chunk.1.js:1
_callee$ @ devTools.plugin.js:1
l @ kbn-ui-shared-deps.js:321
(anonymous) @ kbn-ui-shared-deps.js:321
forEach.e.<computed> @ kbn-ui-shared-deps.js:321
asyncGeneratorStep @ devTools.plugin.js:1
_next @ devTools.plugin.js:1
Promise.then (async)
asyncGeneratorStep @ devTools.plugin.js:1
_next @ devTools.plugin.js:1
Promise.then (async)
asyncGeneratorStep @ devTools.plugin.js:1
_next @ devTools.plugin.js:1
(anonymous) @ devTools.plugin.js:1
(anonymous) @ devTools.plugin.js:1
mount @ devTools.plugin.js:1
_callee$ @ core.entry.js:13
l @ kbn-ui-shared-deps.js:321
(anonymous) @ kbn-ui-shared-deps.js:321
forEach.e.<computed> @ kbn-ui-shared-deps.js:321
application_service_asyncGeneratorStep @ core.entry.js:13
_next @ core.entry.js:13
(anonymous) @ core.entry.js:13
(anonymous) @ core.entry.js:13
(anonymous) @ core.entry.js:13
_callee$ @ core.entry.js:13
l @ kbn-ui-shared-deps.js:321
(anonymous) @ kbn-ui-shared-deps.js:321
forEach.e.<computed> @ kbn-ui-shared-deps.js:321
asyncGeneratorStep @ core.entry.js:13
_next @ core.entry.js:13
(anonymous) @ core.entry.js:13
(anonymous) @ core.entry.js:13
mount @ core.entry.js:13
(anonymous) @ core.entry.js:13
fs @ kbn-ui-shared-deps.js:375
vl @ kbn-ui-shared-deps.js:375
t.unstable_runWithPriority @ kbn-ui-shared-deps.js:383
Hr @ kbn-ui-shared-deps.js:375
bl @ kbn-ui-shared-deps.js:375
ol @ kbn-ui-shared-deps.js:375
(anonymous) @ kbn-ui-shared-deps.js:375
t.unstable_runWithPriority @ kbn-ui-shared-deps.js:383
Hr @ kbn-ui-shared-deps.js:375
$r @ kbn-ui-shared-deps.js:375
Gr @ kbn-ui-shared-deps.js:375
el @ kbn-ui-shared-deps.js:375
enqueueSetState @ kbn-ui-shared-deps.js:375
y.setState @ kbn-ui-shared-deps.js:353
(anonymous) @ kbn-ui-shared-deps.js:283
e @ core.entry.js:6
(anonymous) @ core.entry.js:6
notifyListeners @ core.entry.js:6
p @ core.entry.js:6
(anonymous) @ core.entry.js:6
confirmTransitionTo @ core.entry.js:6
push @ core.entry.js:6
navigate @ core.entry.js:13
_callee2$ @ core.entry.js:13
l @ kbn-ui-shared-deps.js:321
(anonymous) @ kbn-ui-shared-deps.js:321
forEach.e.<computed> @ kbn-ui-shared-deps.js:321
application_service_asyncGeneratorStep @ core.entry.js:13
_next @ core.entry.js:13
Promise.then (async)
application_service_asyncGeneratorStep @ core.entry.js:13
_next @ core.entry.js:13
(anonymous) @ core.entry.js:13
(anonymous) @ core.entry.js:13
navigateToApp @ core.entry.js:13
onClick @ core.entry.js:13
m @ kbn-ui-shared-deps.js:375
S @ kbn-ui-shared-deps.js:375
(anonymous) @ kbn-ui-shared-deps.js:375
x @ kbn-ui-shared-deps.js:375
M @ kbn-ui-shared-deps.js:375
O @ kbn-ui-shared-deps.js:375
w @ kbn-ui-shared-deps.js:375
Tn @ kbn-ui-shared-deps.js:375
ce @ kbn-ui-shared-deps.js:375
Mn @ kbn-ui-shared-deps.js:375
Ln @ kbn-ui-shared-deps.js:375
wn @ kbn-ui-shared-deps.js:375
t.unstable_runWithPriority @ kbn-ui-shared-deps.js:383
Hr @ kbn-ui-shared-deps.js:375
se @ kbn-ui-shared-deps.js:375
On @ kbn-ui-shared-deps.js:375
Show 61 more frames
I’ll have to rebuild my Kibana image, but certainly. Does this build require that multitenancy be enabled?
I’ve rebuilt and deployed to my non-production cluster and it functions as expected. Am I safe to deploy this snapshot to the production cluster, or will there be a fix pushed soon?
srgbnd
March 22, 2021, 8:40pm
5
Have you always used the xpack spaces? I don’t see xpack.spaces.enabled: true in kibana.yml. Kibana 7.11 made spaces enabled by default. I have just realised that the test build which I provided works only if the spaces plugin is disabled. We provide the proper fix and the patch release soon. I’ll ping you. The issue is The Kibana 7.11 is broken if multitenancy is disabled (#351) · Issues · search-guard / Search Guard Kibana Plugin · GitLab
You mean the workspaces? Yes - we’ve always used workspaces, we don’t use multitenancy; many of our applications are intertwined, so it makes more sense to use workspaces to segregate out teams, but still allow them access to other teams’ applications, when necessary.
I only have the default workspace available in my non-prod environment, so this might be the reason that it’s working as expected in non-prod.
srgbnd
March 23, 2021, 7:27am
7
You mean the workspaces?
Hi. I mean Spaces: Spaces | Kibana Guide [8.11] | Elastic In the non-prod, where you installed my test build, could you please create a couple of spaces and tell me if it behaves as you expect? I test it too.
srgbnd:
Hi. I mean Spaces: Spaces | Kibana Guide [7.11] | Elastic In the non-prod, where you installed my test build, could you please create a couple of spaces and tell me if it behaves as you expect? I test it too.
OK. We’re talking about the same thing, then. In an earlier version of the Elastics documentation they were referred to as workspaces, that’s what I’ve always referred to them as. I’ve never had spaces explicitly enabled in my kibana.yml, AFAIK, they’ve always been implicitly enabled, even prior to 7.11. I’ve been using them since sometime in early 7.x without issues.
As far as creating a few in my non-prod, I’ve created them and they work as expected; I haven’t tried restarting the cluster with them enabled yet, however.
I’ve restarted the non-prod cluster, and everything is continuing to work as expected.
1 Like
srgbnd
March 23, 2021, 10:47am
10
Thank you for your tests! We are working on the integration tests for this fix now.
1 Like
In the meantime, I’ll repeat my earlier question - given these results, should I be OK at the moment to push this to my prod cluster?
Thanks!
Any updates on this? We’re in a situation right now, where it would be helpful to be able to save Searches and run ad hoc queries through the console…
srgbnd
March 24, 2021, 3:40pm
13
Hi. The development of the integration tests takes more time than I expected. We do the patch this week. As far as I can see now, the fix, which is included in the test build, which I sent you, works as expected.
1 Like
srgbnd:
Hi. The development of the integration tests takes more time than I expected. We do the patch this week. As far as I can see now, the fix, which is included in the test build, which I sent you, works as expected.
I just deployed the image to my prod cluster, and it’s also working as expected there. I’ll rebuild once you push the updated version, so that I can get any last-minute changes.
Thanks!
srgbnd:
Hi. The development of the integration tests takes more time than I expected. We do the patch this week. As far as I can see now, the fix, which is included in the test build, which I sent you, works as expected.
Just deployed the SG Kibana 50.1.0 patch to my non-prod cluster, and everything is working as expected. Thanks for the quick response and turnaround.
1 Like
srgbnd
March 26, 2021, 2:44pm
17
Hi @Doug_Renze Thank you for the feedback.
1 Like
system
April 16, 2021, 2:44pm
18
This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.
