Records are not being filtered when applying indices in sg_roles.yml

Search Guard
1

Hi,

I’m currently facing an issue with your service, while added new indices to Elasticsearch and apply them under Searchguard it seems that no records are being eliminated from Elasticsearch to Kibana, although other indices are seems to do the job.

I’m currently running Elaticsearch 2.4.4 and searchguard-2 2.4.4.12. My sg_roles.yml Goes as follows:

sg_all_access:
  cluster:
    - '*'
  indices:
    '*':
      '*':
        - '*'
#CRM roles
regular:
  cluster:
    - KIBI_CLUSTER
  indices:
    '*':
      '*':
        - KIBI_READONLY
    'customers':
      'customers':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
      _fls_:
        - '~email'
        - '~phone'
    'open_withdrawals':
      'open_withdrawals':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'customer_summaries':
      'customer_summaries':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'customer_trading_accounts':
      'customer_trading_accounts':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'deposits':
      'deposits':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'deposits_failed':
      'deposits_failed':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'deposits_withdrawals':
      'deposits_withdrawals':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'customers_compliance':
      'customers_compliance':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
      _fls_:
        - '~email'
    'positions':
      'positions':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'positions_forex':
      'positions_forex':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'withdrawals':
      'withdrawals':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'bonuses':
      'bonuses':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'attachments':
      'attachments':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'fees':
      'fees':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'users':
      'users':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'user_brand_desks':
      'user_brand_desks':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'user_follow_ups':
      'user_follow_ups':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
      _fls_:
        - '~email'
    '?kibi':
      'null':
        - 'indices:data/read/search'
        - 'indices:data/read/coordinate-search'

admin:
  cluster:
    - KIBI_CLUSTER
  indices:
    '*':
      '*':
        - KIBI_READONLY
    'customers':
      'customers':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
      _fls_:
        - '~email'
        - '~phone'
    'open_withdrawals':
      'open_withdrawals':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'customer_summaries':
      'customer_summaries':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'customer_trading_accounts':
      'customer_trading_accounts':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'deposits':
      'deposits':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'deposits_failed':
      'deposits_failed':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'deposits_withdrawals':
      'deposits_withdrawals':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'customers_compliance':
      'customers_compliance':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
      _fls_:
        - '~email'
    'positions':
      'positions':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'positions_forex':
      'positions_forex':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'withdrawals':
      'withdrawals':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'bonuses':
      'bonuses':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'attachments':
      'attachments':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'fees':
      'fees':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'users':
      'users':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'user_brand_desks':
      'user_brand_desks':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'user_follow_ups':
      'user_follow_ups':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
      _fls_:
        - '~email'
    '?kibi':
      'null':
        - 'indices:data/read/search'
        - 'indices:data/read/coordinate-search'
manager:
  cluster:
    - KIBI_CLUSTER
  indices:
    '*':
      '*':
        - KIBI_READONLY
    'customers':
      'customers':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
      _fls_:
        - '~email'
        - '~phone'
    'open_withdrawals':
      'open_withdrawals':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'customer_summaries':
      'customer_summaries':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'customer_trading_accounts':
      'customer_trading_accounts':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'deposits':
      'deposits':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'deposits_failed':
      'deposits_failed':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'deposits_withdrawals':
      'deposits_withdrawals':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'customers_compliance':
      'customers_compliance':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
      _fls_:
        - '~email'
    'positions':
      'positions':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'positions_forex':
      'positions_forex':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'withdrawals':
      'withdrawals':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'bonuses':
      'bonuses':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'attachments':
      'attachments':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'fees':
      'fees':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'users':
      'users':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'user_brand_desks':
      'user_brand_desks':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'user_follow_ups':
      'user_follow_ups':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
      _fls_:
        - '~email'
    '?kibi':
      'null':
        - 'indices:data/read/search'
        - 'indices:data/read/coordinate-search'
# Allows reading data from all indices.
sg_readall:
  indices:
    '*':
      '*':
        - READ

# Permissions for a Logstash client.
logstash:
  cluster:
    - indices:admin/template/get
    - indices:admin/template/put
    - indices:data/write/bulk
  indices:
    '*':
      '*':
        - CRUD
        - CREATE_INDEX
    '*beat*':
      '*':
        - CRUD
        - CREATE_INDEX

# Permissions for the Kibi transport client.
kibitransport:
  cluster:
      - cluster:monitor/nodes/liveness

# Permissions for the Kibi server process.
kibiserver:
  cluster:
      - cluster:monitor/nodes/info
      - cluster:monitor/health
      - cluster:monitor/state
      - cluster:monitor/nodes/stats
      - KIBI_CLUSTER
  indices:
    '*':
      '*':
        - indices:admin/get
    '?kibi':
      '*':
        - ALL
    '?kibiaccess':
      '*':
        - ALL
# Permissions for a Sentinl user.
sentinl:
  cluster:
    - KIBI_CLUSTER
  indices:
    '*':
      '*':
        - KIBI_READONLY
    'watcher_alarms*':
      '*':
        - KIBI_READWRITE
        - indices:admin/template/get
        - indices:admin/template/put
        - CREATE_INDEX
    '/(watcher|watcher_alarms)/':
      '*':
        - KIBI_READWRITE
        - indices:admin/template/get
        - indices:admin/template/put
        - CREATE_INDEX

# Permissions for a Kibi administrator (read-write access to the .kibi index).
kibiadmin:
  cluster:
    - KIBI_CLUSTER
    - cluster:admin/plugin/siren/license/put
  indices:
    '*':
      '*':
        - KIBI_READONLY
    '?kibi':
      '*':
        - KIBI_READWRITE
    'watcher':
      '*':
        - KIBI_READWRITE

Permissions for a Kibi user (read only access to the .kibi index).code here…

``

The problematic indices are; open_withdrawals, fees and attachments

My sg_roles_mappings goes as follows :

Enter code here…sg_all_access:

``

users:
    - admin
    - adminSuper9
  backendroles:
    - super
kibiserver:
  users:
    - kibiserver
    - adminSuper9
  backendroles:
    - super
kibiadmin:
  users:
    - kibiadmin
    - adminSuper9
  backendroles:
    - super
regular:
  users:
    - u5
  backendroles:
    - regular
admin:
  users:
    - admin
  backendroles:
    - admin
manager:
  users:
    - manager
  backendroles:
    - manager
kibitransport:
  users:
    - 'CN=demouser'
logstash:
  users:
    - logstash
    - adminSuper9
sentinl:
  users:
    - sentinl
    - adminSuper9

sg_config.yml:

Enter code here.searchguard:

dynamic:
    http:
      anonymous_auth_enabled: false
      xff:
        enabled: false
    authc:
      jwt_auth_domain:
        enabled: true
        order: 1
        http_authenticator:
          type: jwt
          challenge: false
          config:
            signing_key: "MldQUUZXc04yTEw0OVZvVTBGV2NDeExIdU1Qcll1ekc="
            jwt_header: "Authorization"
            roles_key: "role"
        authentication_backend:
          type: noop
      basic_internal_auth_domain:
        enabled: true
        order: 2
        http_authenticator:
          type: basic
          challenge: true
        authentication_backend:


    type: internal..

``

sg_actiongroups_yml:

Enter code here…ALL:

``

- "indices:*"
MANAGE:
  - "indices:monitor/*"
  - "indices:admin/*"
CREATE_INDEX:
  - "indices:admin/create"
MANAGE_ALIASES:
  - "indices:admin/aliases*"
MONITOR:
  - "indices:monitor/*"
DATA_ACCESS:
  - "indices:data/*"
WRITE:
  - "indices:data/write*"
READ:
  - "indices:data/read*"
DELETE:
  - "indices:data/write/delete*"
CRUD:
  - READ
  - WRITE
SEARCH:
  - "indices:data/read/search*"
  - "indices:data/read/msearch*"
  - SUGGEST
SUGGEST:
  - "indices:data/read/suggest*"
INDEX:
  - "indices:data/write/index*"
  - "indices:data/write/update*"
GET:
  - "indices:data/read/get*"
  - "indices:data/read/mget*"

# CLUSTER
CLUSTER_ALL:
  - cluster:*
CLUSTER_MONITOR:
  - cluster:monitor/*

KIBI_CLUSTER:
  - indices:data/read/scroll
  - cluster:admin/plugin/siren/license/get

KIBI_MSEARCH:
  - indices:data/read/coordinate-msearch*
  - indices:data/read/msearch*

KIBI_READONLY:
  - indices:data/read/field_stats*
  - indices:data/read/get*
  - indices:data/read/mget*
  - indices:data/read/search*
  - indices:data/read/coordinate-search*
  - indices:admin/mappings/get*
  - indices:admin/mappings/fields/get*
  - indices:admin/validate/query*
  - indices:admin/get*
  - indices:admin/version/get*
  - KIBI_MSEARCH

KIBI_READWRITE:
  - indices:admin/exists*
  - indices:admin/mapping/put*
  - indices:admin/refresh*
  - indices:data/write/delete*
  - indices:data/write/index*
  - indices:data/write/update*
  - KIBI_READONLY

sg_internal_users.yml:

ALL:
  - "indices:*"
MANAGE:
  - "indices:monitor/*"
  - "indices:admin/*"
CREATE_INDEX:
  - "indices:admin/create"
MANAGE_ALIASES:
  - "indices:admin/aliases*"
MONITOR:
  - "indices:monitor/*"
DATA_ACCESS:
  - "indices:data/*"
WRITE:
  - "indices:data/write*"
READ:
  - "indices:data/read*"
DELETE:
  - "indices:data/write/delete*"
CRUD:
  - READ
  - WRITE
SEARCH:
  - "indices:data/read/search*"
  - "indices:data/read/msearch*"
  - SUGGEST
SUGGEST:
  - "indices:data/read/suggest*"
INDEX:
  - "indices:data/write/index*"
  - "indices:data/write/update*"
GET:
  - "indices:data/read/get*"
  - "indices:data/read/mget*"

# CLUSTER
CLUSTER_ALL:
  - cluster:*
CLUSTER_MONITOR:
  - cluster:monitor/*

KIBI_CLUSTER:
  - indices:data/read/scroll
  - cluster:admin/plugin/siren/license/get

KIBI_MSEARCH:
  - indices:data/read/coordinate-msearch*
  - indices:data/read/msearch*

KIBI_READONLY:
  - indices:data/read/field_stats*
  - indices:data/read/get*
  - indices:data/read/mget*
  - indices:data/read/search*
  - indices:data/read/coordinate-search*
  - indices:admin/mappings/get*
  - indices:admin/mappings/fields/get*
  - indices:admin/validate/query*
  - indices:admin/get*
  - indices:admin/version/get*
  - KIBI_MSEARCH

KIBI_READWRITE:
  - indices:admin/exists*
  - indices:admin/mapping/put*
  - indices:admin/refresh*
  - indices:data/write/delete*
  - indices:data/write/index*
  - indices:data/write/update*
  - KIBI_READONLY

``

Thanks for your help.

2

Elasticsearch 2.x is End of Life (EOL) and therefore not longer supported, please upgrade to at least 5.6.14 or even better to 6.6.0

···

Am 05.02.2019 um 12:02 schrieb galb@proftit.com:

Hi,
I'm currently facing an issue with your service, while added new indices to Elasticsearch and apply them under Searchguard it seems that no records are being eliminated from Elasticsearch to Kibana, although other indices are seems to do the job.
I'm currently running Elaticsearch 2.4.4 and searchguard-2 2.4.4.12. My sg_roles.yml Goes as follows:

sg_all_access:
  cluster:
    - '*'
  indices:
    '*':
      '*':
        - '*'
#CRM roles
regular:
  cluster:
    - KIBI_CLUSTER
  indices:
    '*':
      '*':
        - KIBI_READONLY
    'customers':
      'customers':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
      _fls_:
        - '~email'
        - '~phone'
    'open_withdrawals':
      'open_withdrawals':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'customer_summaries':
      'customer_summaries':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'customer_trading_accounts':
      'customer_trading_accounts':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'deposits':
      'deposits':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'deposits_failed':
      'deposits_failed':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'deposits_withdrawals':
      'deposits_withdrawals':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'customers_compliance':
      'customers_compliance':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
      _fls_:
        - '~email'
    'positions':
      'positions':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'positions_forex':
      'positions_forex':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'withdrawals':
      'withdrawals':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'bonuses':
      'bonuses':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'attachments':
      'attachments':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'fees':
      'fees':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'users':
      'users':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'user_brand_desks':
      'user_brand_desks':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
    'user_follow_ups':
      'user_follow_ups':
        - '*'
      _dls_: '{"term" : {"username" : "${user.name}"}}'
      _fls_:
        - '~email'
    '?kibi':
      'null':
        - 'indices:data/read/search'
        - 'indices:data/read/coordinate-search'

admin:
  cluster:
    - KIBI_CLUSTER
  indices:
    '*':
      '*':
        - KIBI_READONLY
    'customers':
      'customers':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
      _fls_:
        - '~email'
        - '~phone'
    'open_withdrawals':
      'open_withdrawals':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'customer_summaries':
      'customer_summaries':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'customer_trading_accounts':
      'customer_trading_accounts':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'deposits':
      'deposits':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'deposits_failed':
      'deposits_failed':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'deposits_withdrawals':
      'deposits_withdrawals':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'customers_compliance':
      'customers_compliance':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
      _fls_:
        - '~email'
    'positions':
      'positions':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'positions_forex':
      'positions_forex':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'withdrawals':
      'withdrawals':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'bonuses':
      'bonuses':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'attachments':
      'attachments':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'fees':
      'fees':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'users':
      'users':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'user_brand_desks':
      'user_brand_desks':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'user_follow_ups':
      'user_follow_ups':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
      _fls_:
        - '~email'
    '?kibi':
      'null':
        - 'indices:data/read/search'
        - 'indices:data/read/coordinate-search'
manager:
  cluster:
    - KIBI_CLUSTER
  indices:
    '*':
      '*':
        - KIBI_READONLY
    'customers':
      'customers':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
      _fls_:
        - '~email'
        - '~phone'
    'open_withdrawals':
      'open_withdrawals':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'customer_summaries':
      'customer_summaries':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'customer_trading_accounts':
      'customer_trading_accounts':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'deposits':
      'deposits':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'deposits_failed':
      'deposits_failed':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'deposits_withdrawals':
      'deposits_withdrawals':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'customers_compliance':
      'customers_compliance':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
      _fls_:
        - '~email'
    'positions':
      'positions':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'positions_forex':
      'positions_forex':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'withdrawals':
      'withdrawals':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'bonuses':
      'bonuses':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'attachments':
      'attachments':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'fees':
      'fees':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'users':
      'users':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'user_brand_desks':
      'user_brand_desks':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
    'user_follow_ups':
      'user_follow_ups':
        - '*'
      _dls_: '{"term" : {"authorized_users" : "${user.name}"}}'
      _fls_:
        - '~email'
    '?kibi':
      'null':
        - 'indices:data/read/search'
        - 'indices:data/read/coordinate-search'
# Allows reading data from all indices.
sg_readall:
  indices:
    '*':
      '*':
        - READ

# Permissions for a Logstash client.
logstash:
  cluster:
    - indices:admin/template/get
    - indices:admin/template/put
    - indices:data/write/bulk
  indices:
    '*':
      '*':
        - CRUD
        - CREATE_INDEX
    '*beat*':
      '*':
        - CRUD
        - CREATE_INDEX

# Permissions for the Kibi transport client.
kibitransport:
  cluster:
      - cluster:monitor/nodes/liveness

# Permissions for the Kibi server process.
kibiserver:
  cluster:
      - cluster:monitor/nodes/info
      - cluster:monitor/health
      - cluster:monitor/state
      - cluster:monitor/nodes/stats
      - KIBI_CLUSTER
  indices:
    '*':
      '*':
        - indices:admin/get
    '?kibi':
      '*':
        - ALL
    '?kibiaccess':
      '*':
        - ALL
# Permissions for a Sentinl user.
sentinl:
  cluster:
    - KIBI_CLUSTER
  indices:
    '*':
      '*':
        - KIBI_READONLY
    'watcher_alarms*':
      '*':
        - KIBI_READWRITE
        - indices:admin/template/get
        - indices:admin/template/put
        - CREATE_INDEX
    '/(watcher|watcher_alarms)/':
      '*':
        - KIBI_READWRITE
        - indices:admin/template/get
        - indices:admin/template/put
        - CREATE_INDEX

# Permissions for a Kibi administrator (read-write access to the .kibi index).
kibiadmin:
  cluster:
    - KIBI_CLUSTER
    - cluster:admin/plugin/siren/license/put
  indices:
    '*':
      '*':
        - KIBI_READONLY
    '?kibi':
      '*':
        - KIBI_READWRITE
    'watcher':
      '*':
        - KIBI_READWRITE

# Permissions for a Kibi user (read only access to the .kibi index).code here...

The problematic indices are; open_withdrawals, fees and attachments

My sg_roles_mappings goes as follows :
Enter code here...sg_all_access:
  users:
    - admin
    - adminSuper9
  backendroles:
    - super
kibiserver:
  users:
    - kibiserver
    - adminSuper9
  backendroles:
    - super
kibiadmin:
  users:
    - kibiadmin
    - adminSuper9
  backendroles:
    - super
regular:
  users:
    - u5
  backendroles:
    - regular
admin:
  users:
    - admin
  backendroles:
    - admin
manager:
  users:
    - manager
  backendroles:
    - manager
kibitransport:
  users:
    - 'CN=demouser'
logstash:
  users:
    - logstash
    - adminSuper9
sentinl:
  users:
    - sentinl
    - adminSuper9

sg_config.yml:
Enter code here.searchguard:
  dynamic:
    http:
      anonymous_auth_enabled: false
      xff:
        enabled: false
    authc:
      jwt_auth_domain:
        enabled: true
        order: 1
        http_authenticator:
          type: jwt
          challenge: false
          config:
            signing_key: "MldQUUZXc04yTEw0OVZvVTBGV2NDeExIdU1Qcll1ekc="
            jwt_header: "Authorization"
            roles_key: "role"
        authentication_backend:
          type: noop
      basic_internal_auth_domain:
        enabled: true
        order: 2
        http_authenticator:
          type: basic
          challenge: true
        authentication_backend:
type: internal..

sg_actiongroups_yml:
Enter code here...ALL:
  - "indices:*"
MANAGE:
  - "indices:monitor/*"
  - "indices:admin/*"
CREATE_INDEX:
  - "indices:admin/create"
MANAGE_ALIASES:
  - "indices:admin/aliases*"
MONITOR:
  - "indices:monitor/*"
DATA_ACCESS:
  - "indices:data/*"
WRITE:
  - "indices:data/write*"
READ:
  - "indices:data/read*"
DELETE:
  - "indices:data/write/delete*"
CRUD:
  - READ
  - WRITE
SEARCH:
  - "indices:data/read/search*"
  - "indices:data/read/msearch*"
  - SUGGEST
SUGGEST:
  - "indices:data/read/suggest*"
INDEX:
  - "indices:data/write/index*"
  - "indices:data/write/update*"
GET:
  - "indices:data/read/get*"
  - "indices:data/read/mget*"

# CLUSTER
CLUSTER_ALL:
  - cluster:*
CLUSTER_MONITOR:
  - cluster:monitor/*

KIBI_CLUSTER:
  - indices:data/read/scroll
  - cluster:admin/plugin/siren/license/get

KIBI_MSEARCH:
  - indices:data/read/coordinate-msearch*
  - indices:data/read/msearch*

KIBI_READONLY:
  - indices:data/read/field_stats*
  - indices:data/read/get*
  - indices:data/read/mget*
  - indices:data/read/search*
  - indices:data/read/coordinate-search*
  - indices:admin/mappings/get*
  - indices:admin/mappings/fields/get*
  - indices:admin/validate/query*
  - indices:admin/get*
  - indices:admin/version/get*
  - KIBI_MSEARCH

KIBI_READWRITE:
  - indices:admin/exists*
  - indices:admin/mapping/put*
  - indices:admin/refresh*
  - indices:data/write/delete*
  - indices:data/write/index*
  - indices:data/write/update*
  - KIBI_READONLY

sg_internal_users.yml:
ALL:
  - "indices:*"
MANAGE:
  - "indices:monitor/*"
  - "indices:admin/*"
CREATE_INDEX:
  - "indices:admin/create"
MANAGE_ALIASES:
  - "indices:admin/aliases*"
MONITOR:
  - "indices:monitor/*"
DATA_ACCESS:
  - "indices:data/*"
WRITE:
  - "indices:data/write*"
READ:
  - "indices:data/read*"
DELETE:
  - "indices:data/write/delete*"
CRUD:
  - READ
  - WRITE
SEARCH:
  - "indices:data/read/search*"
  - "indices:data/read/msearch*"
  - SUGGEST
SUGGEST:
  - "indices:data/read/suggest*"
INDEX:
  - "indices:data/write/index*"
  - "indices:data/write/update*"
GET:
  - "indices:data/read/get*"
  - "indices:data/read/mget*"

# CLUSTER
CLUSTER_ALL:
  - cluster:*
CLUSTER_MONITOR:
  - cluster:monitor/*

KIBI_CLUSTER:
  - indices:data/read/scroll
  - cluster:admin/plugin/siren/license/get

KIBI_MSEARCH:
  - indices:data/read/coordinate-msearch*
  - indices:data/read/msearch*

KIBI_READONLY:
  - indices:data/read/field_stats*
  - indices:data/read/get*
  - indices:data/read/mget*
  - indices:data/read/search*
  - indices:data/read/coordinate-search*
  - indices:admin/mappings/get*
  - indices:admin/mappings/fields/get*
  - indices:admin/validate/query*
  - indices:admin/get*
  - indices:admin/version/get*
  - KIBI_MSEARCH

KIBI_READWRITE:
  - indices:admin/exists*
  - indices:admin/mapping/put*
  - indices:admin/refresh*
  - indices:data/write/delete*
  - indices:data/write/index*
  - indices:data/write/update*
  - KIBI_READONLY

Thanks for your help.

--
You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.
To post to this group, send email to search-guard@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/c1aa8bdb-f663-4f5e-ac4a-7baf28a5925c%40googlegroups.com\.
For more options, visit https://groups.google.com/d/optout\.

3

Can you at least advise about the files i sent you, regardless to the Elasticsearch version?

···

On Tue, Feb 5, 2019 at 1:14 PM SG info@search-guard.com wrote:

Elasticsearch 2.x is End of Life (EOL) and therefore not longer supported, please upgrade to at least 5.6.14 or even better to 6.6.0

Am 05.02.2019 um 12:02 schrieb galb@proftit.com:

Hi,

I’m currently facing an issue with your service, while added new indices to Elasticsearch and apply them under Searchguard it seems that no records are being eliminated from Elasticsearch to Kibana, although other indices are seems to do the job.

I’m currently running Elaticsearch 2.4.4 and searchguard-2 2.4.4.12. My sg_roles.yml Goes as follows:

sg_all_access:

cluster:

- '*'

indices:

'*':

  '*':

    - '*'

#CRM roles

regular:

cluster:

- KIBI_CLUSTER

indices:

'*':

  '*':

    - KIBI_READONLY

'customers':

  'customers':

    - '*'

  _dls_: '{"term" : {"username" : "${[user.name](http://user.name)}"}}'

  _fls_:

    - '~email'

    - '~phone'

'open_withdrawals':

  'open_withdrawals':

    - '*'

  _dls_: '{"term" : {"username" : "${[user.name](http://user.name)}"}}'

'customer_summaries':

  'customer_summaries':

    - '*'

  _dls_: '{"term" : {"username" : "${[user.name](http://user.name)}"}}'

'customer_trading_accounts':

  'customer_trading_accounts':

    - '*'

  _dls_: '{"term" : {"username" : "${[user.name](http://user.name)}"}}'

'deposits':

  'deposits':

    - '*'

  _dls_: '{"term" : {"username" : "${[user.name](http://user.name)}"}}'

'deposits_failed':

  'deposits_failed':

    - '*'

  _dls_: '{"term" : {"username" : "${[user.name](http://user.name)}"}}'

'deposits_withdrawals':

  'deposits_withdrawals':

    - '*'

  _dls_: '{"term" : {"username" : "${[user.name](http://user.name)}"}}'

'customers_compliance':

  'customers_compliance':

    - '*'

  _dls_: '{"term" : {"username" : "${[user.name](http://user.name)}"}}'

  _fls_:

    - '~email'

'positions':

  'positions':

    - '*'

  _dls_: '{"term" : {"username" : "${[user.name](http://user.name)}"}}'

'positions_forex':

  'positions_forex':

    - '*'

  _dls_: '{"term" : {"username" : "${[user.name](http://user.name)}"}}'

'withdrawals':

  'withdrawals':

    - '*'

  _dls_: '{"term" : {"username" : "${[user.name](http://user.name)}"}}'

'bonuses':

  'bonuses':

    - '*'

  _dls_: '{"term" : {"username" : "${[user.name](http://user.name)}"}}'

'attachments':

  'attachments':

    - '*'

  _dls_: '{"term" : {"username" : "${[user.name](http://user.name)}"}}'

'fees':

  'fees':

    - '*'

  _dls_: '{"term" : {"username" : "${[user.name](http://user.name)}"}}'

'users':

  'users':

    - '*'

  _dls_: '{"term" : {"username" : "${[user.name](http://user.name)}"}}'

'user_brand_desks':

  'user_brand_desks':

    - '*'

  _dls_: '{"term" : {"username" : "${[user.name](http://user.name)}"}}'

'user_follow_ups':

  'user_follow_ups':

    - '*'

  _dls_: '{"term" : {"username" : "${[user.name](http://user.name)}"}}'

  _fls_:

    - '~email'

'?kibi':

  'null':

    - 'indices:data/read/search'

    - 'indices:data/read/coordinate-search'

admin:

cluster:

- KIBI_CLUSTER

indices:

'*':

  '*':

    - KIBI_READONLY

'customers':

  'customers':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

  _fls_:

    - '~email'

    - '~phone'

'open_withdrawals':

  'open_withdrawals':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'customer_summaries':

  'customer_summaries':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'customer_trading_accounts':

  'customer_trading_accounts':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'deposits':

  'deposits':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'deposits_failed':

  'deposits_failed':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'deposits_withdrawals':

  'deposits_withdrawals':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'customers_compliance':

  'customers_compliance':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

  _fls_:

    - '~email'

'positions':

  'positions':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'positions_forex':

  'positions_forex':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'withdrawals':

  'withdrawals':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'bonuses':

  'bonuses':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'attachments':

  'attachments':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'fees':

  'fees':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'users':

  'users':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'user_brand_desks':

  'user_brand_desks':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'user_follow_ups':

  'user_follow_ups':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

  _fls_:

    - '~email'

'?kibi':

  'null':

    - 'indices:data/read/search'

    - 'indices:data/read/coordinate-search'

manager:

cluster:

- KIBI_CLUSTER

indices:

'*':

  '*':

    - KIBI_READONLY

'customers':

  'customers':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

  _fls_:

    - '~email'

    - '~phone'

'open_withdrawals':

  'open_withdrawals':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'customer_summaries':

  'customer_summaries':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'customer_trading_accounts':

  'customer_trading_accounts':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'deposits':

  'deposits':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'deposits_failed':

  'deposits_failed':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'deposits_withdrawals':

  'deposits_withdrawals':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'customers_compliance':

  'customers_compliance':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

  _fls_:

    - '~email'

'positions':

  'positions':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'positions_forex':

  'positions_forex':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'withdrawals':

  'withdrawals':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'bonuses':

  'bonuses':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'attachments':

  'attachments':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'fees':

  'fees':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'users':

  'users':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'user_brand_desks':

  'user_brand_desks':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

'user_follow_ups':

  'user_follow_ups':

    - '*'

  _dls_: '{"term" : {"authorized_users" : "${[user.name](http://user.name)}"}}'

  _fls_:

    - '~email'

'?kibi':

  'null':

    - 'indices:data/read/search'

    - 'indices:data/read/coordinate-search'

Allows reading data from all indices.

sg_readall:

indices:

'*':

  '*':

    - READ

Permissions for a Logstash client.

logstash:

cluster:

- indices:admin/template/get

- indices:admin/template/put

- indices:data/write/bulk

indices:

'*':

  '*':

    - CRUD

    - CREATE_INDEX

'*beat*':

  '*':

    - CRUD

    - CREATE_INDEX

Permissions for the Kibi transport client.

kibitransport:

cluster:

  - cluster:monitor/nodes/liveness

Permissions for the Kibi server process.

kibiserver:

cluster:

  - cluster:monitor/nodes/info

  - cluster:monitor/health

  - cluster:monitor/state

  - cluster:monitor/nodes/stats

  - KIBI_CLUSTER

indices:

'*':

  '*':

    - indices:admin/get

'?kibi':

  '*':

    - ALL

'?kibiaccess':

  '*':

    - ALL

Permissions for a Sentinl user.

sentinl:

cluster:

- KIBI_CLUSTER

indices:

'*':

  '*':

    - KIBI_READONLY

'watcher_alarms*':

  '*':

    - KIBI_READWRITE

    - indices:admin/template/get

    - indices:admin/template/put

    - CREATE_INDEX

'/(watcher|watcher_alarms)/':

  '*':

    - KIBI_READWRITE

    - indices:admin/template/get

    - indices:admin/template/put

    - CREATE_INDEX

Permissions for a Kibi administrator (read-write access to the .kibi index).

kibiadmin:

cluster:

- KIBI_CLUSTER

- cluster:admin/plugin/siren/license/put

indices:

'*':

  '*':

    - KIBI_READONLY

'?kibi':

  '*':

    - KIBI_READWRITE

'watcher':

  '*':

    - KIBI_READWRITE

Permissions for a Kibi user (read only access to the .kibi index).code here…

The problematic indices are; open_withdrawals, fees and attachments

My sg_roles_mappings goes as follows :

Enter code here…sg_all_access:

users:

- admin

- adminSuper9

backendroles:

- super

kibiserver:

users:

- kibiserver

- adminSuper9

backendroles:

- super

kibiadmin:

users:

- kibiadmin

- adminSuper9

backendroles:

- super

regular:

users:

- u5

backendroles:

- regular

admin:

users:

- admin

backendroles:

- admin

manager:

users:

- manager

backendroles:

- manager

kibitransport:

users:

- 'CN=demouser'

logstash:

users:

- logstash

- adminSuper9

sentinl:

users:

- sentinl

- adminSuper9

sg_config.yml:

Enter code here.searchguard:

dynamic:

http:

  anonymous_auth_enabled: false

  xff:

    enabled: false

authc:

  jwt_auth_domain:

    enabled: true

    order: 1

    http_authenticator:

      type: jwt

      challenge: false

      config:

        signing_key: "MldQUUZXc04yTEw0OVZvVTBGV2NDeExIdU1Qcll1ekc="

        jwt_header: "Authorization"

        roles_key: "role"

    authentication_backend:

      type: noop

  basic_internal_auth_domain:

    enabled: true

    order: 2

    http_authenticator:

      type: basic

      challenge: true

    authentication_backend:

type: internal…

sg_actiongroups_yml:

Enter code here…ALL:

  • “indices:*”

MANAGE:

  • “indices:monitor/*”
  • “indices:admin/*”

CREATE_INDEX:

  • “indices:admin/create”

MANAGE_ALIASES:

  • “indices:admin/aliases*”

MONITOR:

  • “indices:monitor/*”

DATA_ACCESS:

  • “indices:data/*”

WRITE:

  • “indices:data/write*”

READ:

  • “indices:data/read*”

DELETE:

  • “indices:data/write/delete*”

CRUD:

  • READ
  • WRITE

SEARCH:

  • “indices:data/read/search*”
  • “indices:data/read/msearch*”
  • SUGGEST

SUGGEST:

  • “indices:data/read/suggest*”

INDEX:

  • “indices:data/write/index*”
  • “indices:data/write/update*”

GET:

  • “indices:data/read/get*”
  • “indices:data/read/mget*”

CLUSTER

CLUSTER_ALL:

  • cluster:*

CLUSTER_MONITOR:

  • cluster:monitor/*

KIBI_CLUSTER:

  • indices:data/read/scroll
  • cluster:admin/plugin/siren/license/get

KIBI_MSEARCH:

  • indices:data/read/coordinate-msearch*
  • indices:data/read/msearch*

KIBI_READONLY:

  • indices:data/read/field_stats*
  • indices:data/read/get*
  • indices:data/read/mget*
  • indices:data/read/search*
  • indices:data/read/coordinate-search*
  • indices:admin/mappings/get*
  • indices:admin/mappings/fields/get*
  • indices:admin/validate/query*
  • indices:admin/get*
  • indices:admin/version/get*
  • KIBI_MSEARCH

KIBI_READWRITE:

  • indices:admin/exists*
  • indices:admin/mapping/put*
  • indices:admin/refresh*
  • indices:data/write/delete*
  • indices:data/write/index*
  • indices:data/write/update*
  • KIBI_READONLY

sg_internal_users.yml:

ALL:

  • “indices:*”

MANAGE:

  • “indices:monitor/*”
  • “indices:admin/*”

CREATE_INDEX:

  • “indices:admin/create”

MANAGE_ALIASES:

  • “indices:admin/aliases*”

MONITOR:

  • “indices:monitor/*”

DATA_ACCESS:

  • “indices:data/*”

WRITE:

  • “indices:data/write*”

READ:

  • “indices:data/read*”

DELETE:

  • “indices:data/write/delete*”

CRUD:

  • READ
  • WRITE

SEARCH:

  • “indices:data/read/search*”
  • “indices:data/read/msearch*”
  • SUGGEST

SUGGEST:

  • “indices:data/read/suggest*”

INDEX:

  • “indices:data/write/index*”
  • “indices:data/write/update*”

GET:

  • “indices:data/read/get*”
  • “indices:data/read/mget*”

CLUSTER

CLUSTER_ALL:

  • cluster:*

CLUSTER_MONITOR:

  • cluster:monitor/*

KIBI_CLUSTER:

  • indices:data/read/scroll
  • cluster:admin/plugin/siren/license/get

KIBI_MSEARCH:

  • indices:data/read/coordinate-msearch*
  • indices:data/read/msearch*

KIBI_READONLY:

  • indices:data/read/field_stats*
  • indices:data/read/get*
  • indices:data/read/mget*
  • indices:data/read/search*
  • indices:data/read/coordinate-search*
  • indices:admin/mappings/get*
  • indices:admin/mappings/fields/get*
  • indices:admin/validate/query*
  • indices:admin/get*
  • indices:admin/version/get*
  • KIBI_MSEARCH

KIBI_READWRITE:

  • indices:admin/exists*
  • indices:admin/mapping/put*
  • indices:admin/refresh*
  • indices:data/write/delete*
  • indices:data/write/index*
  • indices:data/write/update*
  • KIBI_READONLY

Thanks for your help.

–

You received this message because you are subscribed to the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.

To post to this group, send email to search-guard@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/c1aa8bdb-f663-4f5e-ac4a-7baf28a5925c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

–

You received this message because you are subscribed to the Google Groups “Search Guard Community Forum” group.

To unsubscribe from this group and stop receiving emails from it, send an email to search-guard+unsubscribe@googlegroups.com.

To post to this group, send email to search-guard@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/6D3FA5B5-5A61-49FD-8E6E-165FA04B645B%40search-guard.com.

For more options, visit https://groups.google.com/d/optout.

4

If I may, why at least 5.6.14?
I’m running with version 5.4.0, is that not good enough? Can you please clarify?
Thanks and sorry for jumping in outta nowhere

5

The complete 5.x series of Elasticsearch, including 5.4.0 and also 5.6.x is end of life, and thus unsupported:

Which means that also Search Guard for Elastic 5.x is end of life. So you should really upgrade to a recent / supported version of Elasticsearch. The current version of ES is 7.1.1.

6

pls refer to our EOL policy End of life | Security for Elasticsearch | Search Guard

7

I understand now! Thanks for your detailed reply!