Shield integration #399
Comments
|
@electrical any update? |
|
@cdenneen I'm afraid I left the company a few weeks ago and someone else is taking over. |
|
@electrical thanks. Do we know who this will be? Anyway we can get even the basics of these modules for shield and kibana published and public contribution help to work on them?(even if not a published module right away) Having these re-opened as public modules like the original electrical ones were might not be a bad idea if @elastic doesn't have the cycles to solely maintain? |
|
Hi @cdenneen, sorry for the delayed response here. The short story here is that @Jarpy and I will be helping to further the excellent work @electrical has already done. As a prerequisite to diving into the guts of the module, we'd really like to give the CI/testing environment some love to get it back up to speed running through the thorough test suite for some confidence moving forward. Once that's done (and ideally in a place the community can access and interact with) we'll be in a great place to start actively accepting community PRs. The community work is greatly appreciated and if we can put some infrastructure in place to lower the barrier to entry with good testing infra, we can make community contributions happen more quickly and confidently. I'll try and provide as much transparency as possible as we transition some things, please let us know about additional needs as well! 💯 |
|
@tylerjl thanks for the update
(Put shield at top because figured would be quick win to get initial release out) I left filebeats off this list for now because @pcfens has done great job of creating one and maintaining. So think besides the above list of missing modules the backlog of issues would b next of importance before filebeats module. Maybe creating a bullet list as issue to track order of what we can expect? |
|
@cdenneen yep, that aligns pretty closely to the pulse I've gotten from others as well. Hopefully we can pick up the shield work efficiently to keep it moving. I'm currently knee-deep in CI environment work, but hopefully we'll have a fully functioning CI pipeline Soon™. |
|
@tylerjl Any luck in releasing the beta for puppet-shield plugin? |
|
@cdenneen I'm happy to report I've been making lots of progress on Shield integration. I still need to put some work in on TLS management, but if anyone wants to offer feedback regarding design choices, please to check out the branch diff and let me know if the design pattern looks good from and end-user perspective. I'm first and foremost interested to know if this is usable for people. I've gotten most of the CI shakiness sorted out, so Shield support is the primary focus right now. |
|
@tylerjl awesome. Can you add realm support to that fork as well? |
|
@cdenneen assuming you mean Active Directory realms, standing up a Windows test environments is obviously less straightforward than a Linux one. That said, I think at the very least managing the role mappings yaml file should be very doable now that I've written the native puppet provider for the yaml roles file. Again, I'm not sure what the most preferable way would be for people to define those mappings in puppet, so feel free to comment on the branch regarding how it's shaping up. |
|
Any update when the PR is going to be merged ? |
|
No ETA yet, but I'll be pushing updates regularly if anyone is anxious to watch progress. |
|
@tylerjl sort of... the AD is just one option of a realm, you could use just about anything (LDAP, AD, OAuth, SAML, etc)... it's basically because of the custom realms (https://www.elastic.co/guide/en/shield/current/custom-realms.html) so esusers is just one realm... So for example most of my auth will be with esusers for applications but I would probably secure Kibana with shield and then use AD/LDAP to auth users to Kibana. |
|
Would love to see this too, so that we can easily manage local Shield users and LDAP configuration with Puppet 😄 For now, I'll write my own little module to manage these files. Keep up the great work guys! |
|
@tylerjl I see the mapping support in your fork... I think once the realm support is added to the elasticsearch.yml.erb this should be working, right? |
|
@cdenneen from what I understand, configuring realms is just a matter of defining the configuration in |
|
Yeah I'm guessing custom config block would work and result should look something like this: shield.authc:
realms:
esusers:
type: esusers
order: 0
ldap1:
type: ldap
order: 1
enabled: false
url: 'url_to_ldap1'
...
ad1:
type: active_directory
order: 3
url: 'url_to_ad'Otherwise maybe an if condition and loop in the erb to do hiera lookup for all |
|
I'd prefer to avoid hiera lookups in the erb per Puppet Lab's recommendation, but defining realm configuration in hiera should be as simple as something like the following in a hiera yaml config: ---
elasticsearch::config:
shield.authc:
realms:
esusers:
type: users
...The config on the |
|
Understood... makes sense... so what's left before your fork gets merged? Documentation updates? |
|
Just getting the tests clear, still some failures on different OS/puppet combinations... close, though! 😅 |
|
PR open at #624, do provide feedback if there are any parts that need changing. |
|
@tylerjl using 0.11.0 and getting ==> eslog: Error: Could not find a suitable provider for elasticsearch_shield_user_roles |
|
@tylerjl no it doesn't exist there: but does here: |
|
@tylerjl this is a fresh vagrant box (so no cruft there). |
|
So, my guess is that the In version 0.11.0 of the module, any plugin installations should drop configuration files into In your case you could either remove the module and then define it in puppet to manage, or, if you just want to start from scratch without any of the pre-packaged roles/other files, you could always create It's a bit of an ugly workaround, but it's a compromise between respecting the security manager's need to only read config files within the CONF_DIR and keeping |
|
@tylerjl thanks for that last comment. In our environment we installed the shield plugin with an older version of this Puppet module and we needed to run this before running Puppet with the newer module: |
|
@bflad thanks, I haven't had time to test. |
|
@tylerjl There must still be a ordering issue here. Here is the output of those directories as before: Only large difference is I changed the default Here is all my data, in case it helps tell me where I made a stupid mistake ;-): ---
profile::elasticsearch::es_instances:
- 'es-01'
- 'es-02'
profile::elasticsearch::data::config:
cluster.name: 'loggingtest'
node.name: "%{::hostname}"
index.number_of_shards: 5
index.number_of_replicas: 1
bootstrap.mlockall: true
discovery.zen.ping.unicast.hosts:
- 'eslogmd'
- 'eslogm'
discovery.zen.ping.multicast.enabled: false
node.data: true
node.master: false
http.port: '9200'
profile::elasticsearch::master::config:
cluster.name: 'loggingtest'
node.name: "%{::hostname}"
index.number_of_shards: 5
index.number_of_replicas: 1
bootstrap.mlockall: true
discovery.zen.ping.unicast.hosts:
- 'eslogmd'
- 'eslogm'
discovery.zen.ping.multicast.enabled: false
node.data: false
node.master: true
http.port: '9201'
elasticsearch::init_defaults:
ES_HOME: '/usr/share/elasticsearch'
MAX_OPEN_FILES: '65535'
MAX_MAP_COUNT: '262144'
LOG_DIR: '/var/log/elasticsearch'
DATA_DIR: '/data/elasticsearch'
WORK_DIR: '/tmp/elasticsearch'
ES_USER: 'elasticsearch'
ES_GROUP: 'elasticsearch'
ES_JAVA_OPTS: '-Djava.net.preferIPv4Stack=true'
ES_HEAP_SIZE: '128m'
MAX_LOCKED_MEMORY: 'unlimited'
elasticsearch::java: true
elasticsearch::datadir: '/data/elasticsearch'
elasticsearch::status: 'enabled'class profile::elasticsearch(
$es_instances = 'es-01'
) {
class {'::elasticsearch':
manage_repo => true,
repo_version => '2.x',
java_package => 'java-1.8.0-openjdk-headless',
package_pin => true,
version => '2.3.3',
}
::elasticsearch::plugin{'mobz/elasticsearch-head':
instances => $es_instances
}
::elasticsearch::plugin{'lmenezes/elasticsearch-kopf':
instances => $es_instances
}
::elasticsearch::plugin{'royrusso/elasticsearch-hq':
instances => $es_instances
}
::elasticsearch::plugin{'license':
instances => $es_instances
}
::elasticsearch::plugin{'shield':
instances => $es_instances
}
::elasticsearch::plugin{'watcher':
instances => $es_instances
}
::elasticsearch::plugin{'marvel-agent':
instances => $es_instances
}
elasticsearch::shield::user { 'admin':
password => 'changeme',
roles => ['admin'],
}
}class profile::elasticsearch::master(
$config = {}
) {
include ::profile::elasticsearch
::elasticsearch::instance {'es-02':
config => $config
}
firewall { '101 ES MASTER HTTP':
proto => 'tcp',
action => 'accept',
dport => ['9201','9301'],
}
}class profile::elasticsearch::data(
$config = {}
) {
include ::profile::elasticsearch
::elasticsearch::instance {'es-01':
config => $config
}
firewall { '100 ES HTTP':
proto => 'tcp',
action => 'accept',
dport => ['9200','9300'],
}
} |
|
Regarding the assumption about previous installations, you're right - we should handle that case, that may be tricky, have to think on that one. Regarding the ordering issue: that's strange; as seen here, the resource ordering explicitly enforces that the plugin should be installed before any shield users/roles are handled: ...
-> Elasticsearch::Plugin <| |>
-> Elasticsearch::Shield::Role <| |>
-> Elasticsearch::Shield::User <| |>
...One ordering issue that may come up in that manifest is that shield will fail to install unless the license plugin is there first, but it sounds like that isn't happening. I wonder if any other errors would show up if you got rid of the Agreed about the hiera config; I don't think that would be related to the issue you're seeing. |
|
From the clean run here are the errors in order received: Only other thing I can think of is the $es_instances APL for the profiles? |
|
Ah, so the plugins aren't getting installed. That'll definitely cause problems. The specific error comes from this I'd dig into how that call behaves (inside and outside puppet), it may be a Java issue of some sort. |
|
@tylerjl I think the issue might be i have some class parameters in |
|
@tylerjl merging wasn't happening because of a typo... so that is fixed and now java is installing so rest is working... just getting odd start error and nothing getting logged to any logs: |
|
Progress! 🏁 What shows up in the Elasticsearch instance's logs in |
|
All the logs are 0 bytes. Need to manually execute to debug I'm afraid. Update: So starting through systemctl isn't working but manually works. Is there something wrong with the systemd service file? Adding the init defaults just for good measure: |
|
@tylerjl looks like it's this init_defaults parameter: MAX_LOCKED_MEMORY: 'unlimited' Any idea why? |
|
I found some discussion about this in the past. It looks like the unit file itself explains what you need to set if you've configured mlockall to be true and instruct ES to lock all memory - basically, you need to set the env var to "infinity" (note that I haven't tried this, just reading the documentation comments in the unit file). |
|
@tylerjl are you sure? |
|
@cdenneen do you have a distro and version so I can try and emulate what you're seeing? I tried setting up an instance with the same mlockall and MAX_LOCKED_MEMORY settings in CentOS 7 but didn't see the same behavior. |
|
@tylerjl sysconfig systemd Looks like it's doing what it needs to but when the MAX_LOCKED_MEMORY is set in the init_defaults the systemctl status shows: and the logs are all 0 bytes: I've created a gist with all puppet and hieradata if it helps: https://gist.github.com/cdenneen/e43be80ed3c429c94e2f30767ef603b0 |
|
I got an instance up and running with the sysconfig and yaml config you've got, so I'm not really sure what could be happening. I'd suggest maybe bumping up the logging verbosity level to see if you can get Elasticsearch to spit out any additional information, as from the Puppet side everything seems to be writing configurations correctly. |
|
Setting instance to DEBUG leaves me with these in the log file: Does your instance have SELINUX enabled? |
|
I doubt it - maybe try disabling it across the board and seeing if that fixes things; if so, we can try and determine which access violations are hindering service startup. |
|
I've narrowed down the issue to the following lines: These are what cause If I leave everything else but remove those 2 lines... it works fine. |
|
@cdenneen this is probably outside the scope of this ticket at this point - if you want to post a summary of what you've got so far on https://discuss.elastic.co/c/elasticsearch, we can probably get more eyes on it and pull in some ES devs if we need to. |
|
@tylerjl thanks... opened support case |
|
@TylerLJ ticket yielded memory issue was the cause. Need at least 512mb. Logging should be corrected to show something. |
|
AFAIK, the only API access that requires username/password auth credentials is template management, and with the merge of #663 the top-level parameters If there are additional API options that need to be exposed aside from the API protocol, host, port, basic auth username and password, and whether or not to validate SSL certs, do raise an issue to address it. |
Shield enables authentication for API calls.
Since we do a few of those with the puppet module we need to expose options to support that.
The text was updated successfully, but these errors were encountered: