msearch error handling

Hello,
I have a Search Guard user which has access to all indices in the cluster except “investor”; if I submit the following msearch request:

curl -k -u noinvestor:password -XPOST https://localhost:9235/_msearch?pretty -d ’

{“index”:“company”, “ignore_unavailable”: true}

{“size”:0,“query”:{“bool”:{“must”:{“match_all”:{}},“must_not”:,“filter”:{“bool”:{“must”:[{“range”:{“founded_date”:{“gte”:-3256530944000,“lte”:1477065856000,“format”:“epoch_millis”}}}]}}}}}

{“index”:[“investor”], “ignore_unavailable”: true}

{“size”:0,“query”:{“bool”:{“must”:{“match_all”:{}},“must_not”:,“filter”:{“bool”:{“must”:}}}}}

I get the following response:

{

“error” : {

“root_cause” : [ {

“type” : “security_exception”,

“reason” : “no permissions for indices:data/read/msearch”

} ],

“type” : “security_exception”,

“reason” : “no permissions for indices:data/read/msearch”

},

“status” : 403

}

Would it be feasible/sensible to return an error response only for the searches on unauthorized indices? e.g.:

{

“responses” : [ {

“took” : 9,

“timed_out” : false,

“_shards” : {

“total” : 5,

“successful” : 5,

“failed” : 0

},

“hits” : {

“total” : 619008,

“max_score” : 0.0,

“hits” :

}

}, {

“error” : {

“root_cause” : [ {

“type” : “security_exception”,

“reason” : “no permissions for indices:data/read/msearch”

} ],

“type” : “security_exception”,

“reason” : “no permissions for indices:data/read/msearch”

},

“status” : 403

} ]

}

This kind of behaviour would be especially useful in Kibi/Kibana where all the searches generated by a dashboard are submitted as a single msearch request.

For comparison, Shield behaves in the same way, returning a single error response if at least one of the searches is not authorized.

Note that if you send an msearch containing an invalid query, you will still get the valid responses in the output, e.g.:

curl -k -u noinvestor:password -XPOST https://localhost:9235/_msearch?pretty -d ’

{“index”:“company”, “ignore_unavailable”: true}

{“size”:0, “query”:{“bool”:{“must”:{“match_all”:{}}}}}

{“index”:[“company”], “ignore_unavailable”: true}

{“size”:-1, “query”:{“bool”:{“must”:{“match_all”:{}}}}}

{

“responses” : [ {

“took” : 1,

“timed_out” : false,

“_shards” : {

“total” : 5,

“successful” : 5,

“failed” : 0

},

“hits” : {

“total” : 160106,

“max_score” : 0.0,

“hits” :

}

}, {

“error” : {

“root_cause” : [ {

“type” : “search_parse_exception”,

“reason” : “size is set to [-1] and is expected to be higher or equal to 0”,

“line” : 1,

“col” : 2

} ],

“type” : “search_phase_execution_exception”,

“reason” : “all shards failed”,

“phase” : “query”,

“grouped” : true,

“failed_shards” : [ {

“shard” : 0,

“index” : “company”,

“node” : “TCBl84UMSiq6aiEq8lUSOA”,

“reason” : {

“type” : “search_parse_exception”,

“reason” : “size is set to [-1] and is expected to be higher or equal to 0”,

“line” : 1,

“col” : 2

}

} ]

}

} ]

}

Cheers,

Fabio

I made this small change to test the desired behaviour:

With that change and this configuration:

noinvestor:

cluster:

  • indices:data/read/msearch

indices:

‘?kibi’:

‘*’:

  • KIBI_READONLY

‘/(article|company|investment)/’:

‘*’:

  • KIBI_READONLY

I get this response:

{

“responses” : [ {

“took” : 22,

“timed_out” : false,

“_shards” : {

“total” : 5,

“successful” : 5,

“failed” : 0

},

“hits” : {

“total” : 160032,

“max_score” : 0.0,

“hits” :

}

}, {

“error” : {

“root_cause” : [ {

“type” : “security_exception”,

“reason” : “no permissions for indices:data/read/search”

} ],

“type” : “security_exception”,

“reason” : “no permissions for indices:data/read/search”

}

} ]

}

If the permission is not granted in the cluster context, the request will just fail as before (and as expected).

Do you see any potential security/compatibility issue with this change? Tests were passing on 2.3.5 but I haven’t checked the coverage.

Thanks,

Fabio Corneti

···

On Friday, October 21, 2016 at 6:01:26 PM UTC+1, fa…@siren.solutions wrote:

Hello,
I have a Search Guard user which has access to all indices in the cluster except “investor”; if I submit the following msearch request:

Nevermind, while reading again the code it just occurred to me that we can use this configuration to have the expected result:

noinvestor:

indices:

‘?kibi’:

‘*’:

  • KIBI_READONLY

‘/(article|company|investment)/’:

‘*’:

  • KIBI_READONLY

‘*’:

‘*’:

  • indices:data/read/msearch*

Cheers,

Fabio Corneti

···

On Monday, October 24, 2016 at 1:50:32 PM UTC+1, fa…@siren.solutions wrote:

I made this small change to test the desired behaviour:

We’ve added this as feature request in our backlog anyways. Implementing it seems rather straight forward, and we think it’s generally a great addition to SG.

···

On Monday, 24 October 2016 15:24:11 UTC+2, fabio@siren.solutions wrote:

Nevermind, while reading again the code it just occurred to me that we can use this configuration to have the expected result:

noinvestor:

indices:

‘?kibi’:

‘*’:

  • KIBI_READONLY

‘/(article|company|investment)/’:

‘*’:

  • KIBI_READONLY

‘*’:

‘*’:

  • indices:data/read/msearch*

Cheers,

Fabio Corneti

On Monday, October 24, 2016 at 1:50:32 PM UTC+1, fa…@siren.solutions wrote:

I made this small change to test the desired behaviour: