Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(SIMP-6238) Update to puppet-gitlab v3.0.2 #28

Merged
merged 4 commits into from
Mar 22, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .fixtures.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ fixtures:
concat: https://github.com/simp/puppetlabs-concat
gitlab:
repo: https://github.com/simp/puppet-gitlab.git
ref: v1.13.3
ref: v3.0.2
iptables: https://github.com/simp/pupmod-simp-iptables
ntpd: https://github.com/simp/pupmod-simp-ntpd
pam: https://github.com/simp/pupmod-simp-pam
Expand Down
10 changes: 7 additions & 3 deletions CHANGELOG
Original file line number Diff line number Diff line change
@@ -1,10 +1,14 @@
* Thu Mar 07 2019 Liz Nemsick <lnemsick.simp@gmail.com> - 0.3.5-0
* Thu Mar 07 2019 Liz Nemsick <lnemsick.simp@gmail.com> - 0.4.0-0
- Added missing simp-pki dependency to metadata.json
- Use ``generate_pem_hash_links`` option of pki_cert_sync to allow
the gitlab application to manage the certificate hash links.
``gitlab reconfigure`` generates those hash links.
- Update the upper bound of stdlib to < 6.0.0
- Update a URL in the README.md
- Added manage_package and package_ensure parameters to simp_gitlab
- Switched from vshn/gitlab 1.13.3 to its replacement puppet/gitlab
3.0.2 and reworked the manifests, as appropriate, to work with the
newer module.
- Updated the upper bound of stdlib to < 6.0.0
- Updated a URL in the README.md

* Tue Nov 06 2018 Chris Tessmer <chris.tessmer@onyxpoint.com> - 0.3.4-0
- Add explicit support for Oracle Linux
Expand Down
20 changes: 10 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
* [Beginning with simp_gitlab](#beginning-with-simp_gitlab)
* [Usage](#usage)
* [A basic GitLab setup using PKI](#a-basic-gitlab-setup-using-pki)
* [Passing custom parameters to `vshn-gitlab`](#passing-custom-parameters-to-vshn-gitlab)
* [Passing custom parameters to `puppet-gitlab`](#passing-custom-parameters-to-puppet-gitlab)
* [Configuring Nginx](#configuring-nginx)
* [Reference](#reference)
* [Further Reference for munging GitLab Omnibus](#further-reference-for-munging-gitlab-omnibus)
Expand Down Expand Up @@ -44,7 +44,7 @@ with SIMP.

[gitlab_omnibus]: https://docs.gitlab.com/omnibus/ "GitLab Omnibus"
[gitlab_rb_template]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template
[vshn_gitlab]: https://github.com/vshn/puppet-gitlab
[puppet_gitlab]: https://github.com/voxpupuli/puppet-gitlab
[simp_simp_options]: https://github.com/simp/pupmod-simp-simp_options


Expand Down Expand Up @@ -72,8 +72,8 @@ https://simp-project.atlassian.net/.
### What `simp_gitlab` affects

This module is a profile that integrates Gitlab with SIMP. It makes extensive
use of the component module [`vshn/gitlab`][vshn_gitlab]. It is important to
note that `vshn-gitlab` configures the GitLab Omnibus's `/etc/gitlab/gitlab.rb`
use of the component module [`puppet/gitlab`][puppet_gitlab]. It is important to
note that `puppet-gitlab` configures the GitLab Omnibus's `/etc/gitlab/gitlab.rb`
and runs `gitlab-ctl reconfigure`.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

puppet-gitlab no longer executes gitlab-ctl reconfigure explicitly unless you configure it to do so. Your package manager will do this whenever you install or upgrade the omnibus-gitlab package, as that's part of the omnibus's post-install script.

This same script in the omnibus also starts the gitlab supervisors for the relevant roles defined in the nodes gitlab.rb, and as such the puppet_gitlab module no longer manages the service explicitly by default.

I left some links to relevant info in the issue you opened in our repo. Let me know if you need any other information.


![Relationships between components](assets/simp_gitlab_components.png)
Expand Down Expand Up @@ -168,16 +168,16 @@ class { 'simp_gitlab':
```


### Passing custom parameters to `vshn-gitlab`
### Passing custom parameters to `puppet-gitlab`

Parameters for [`vshn/gitlab`][vshn_gitlab] can be passed in directly using the
Parameters for [`puppet/gitlab`][puppet_gitlab] can be passed in directly using the
(Hash) `$simp_gitlab::gitlab_options` parameter.

**Warning:** `$simp_gitlab::gitlab_options` parameters that conflict with the
rest of `simp_gitlab` may lead to undefined behavior / broken systems.


**Hint:** Many of the data structures used by [`vshn/gitlab`][vshn_gitlab]'s
**Hint:** Many of the data structures used by [`puppet/gitlab`][puppet_gitlab]'s
parameters are documented at
https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template

Expand Down Expand Up @@ -228,8 +228,8 @@ module's generated YARD documentation for reference material.
- Mattermost (chat): https://docs.gitlab.com/omnibus/gitlab-mattermost/README.html
- Prometheus (monitoring): https://docs.gitlab.com/ce/administration/monitoring/prometheus/index.html
- GitLab Docker images: https://docs.gitlab.com/omnibus/docker/README.html
* vshn/gitlab component module:
* https://github.com/vshn/puppet-gitlab
* puppet/gitlab component module:
* https://github.com/voxpupuli/puppet-gitlab
- Security & compliance
- https://www.stigviewer.com/stig/web_server/

Expand Down Expand Up @@ -271,7 +271,7 @@ configurations legitimately vary.
These services are internal to the GitLab Omnibus installation and are not
registered with the host OS or the Puppet providers for `service`. It is
possible for the services to take so long to start in the background that
dependencies within `vshn-gitlab` will fail.
dependencies within `puppet-gitlab` will fail.
<!-- this line is indented to continue the preceding bullet -->
In these cases, it should be sufficient to simply run Puppet again after the
services finish starting (status with `gitlab-ctl status`).
Expand Down
30 changes: 17 additions & 13 deletions functions/omnibus_config/gitlab.pp
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Compile a hash of settings for the gitlab module's parameters, using SIMP settings
# @return Hash of `vshn/gitlab` parameters
# @return Hash of `puppet/gitlab` parameters
function simp_gitlab::omnibus_config::gitlab() {

# For HTTPS, non-standard ports *must* be included in the external_url:
Expand All @@ -9,18 +9,22 @@ function simp_gitlab::omnibus_config::gitlab() {
}

$_gitlab_default_parameters = {
'external_url' => $_external_url,
'external_port' => $simp_gitlab::tcp_listen_port,
'nginx' => simp_gitlab::omnibus_config::nginx(),
'gitlab_rails' => simp_gitlab::omnibus_config::gitlab_rails(),
'shell' => simp_gitlab::omnibus_config::gitlab_shell(),
'mattermost' => simp_gitlab::omnibus_config::mattermost(),
'mattermost_nginx' => { 'enable' => false },
'prometheus' => { 'enable' => false },
'node_exporter' => { 'enable' => false },
'redis_exporter' => { 'enable' => false },
'postgres_exporter' => { 'enable' => false },
'gitlab_monitor' => { 'enable' => false },
# If simp_gitlab is managing the GitLab package, disable that capability in
# the gitlab class
'manage_package' => !$simp_gitlab::manage_package,
'manage_upstream_edition' => $simp_gitlab::edition,
'external_url' => $_external_url,
'external_port' => $simp_gitlab::tcp_listen_port,
'nginx' => simp_gitlab::omnibus_config::nginx(),
'gitlab_rails' => simp_gitlab::omnibus_config::gitlab_rails(),
'shell' => simp_gitlab::omnibus_config::gitlab_shell(),
'mattermost' => simp_gitlab::omnibus_config::mattermost(),
'mattermost_nginx' => { 'enable' => false },
'prometheus' => { 'enable' => false },
'node_exporter' => { 'enable' => false },
'redis_exporter' => { 'enable' => false },
'postgres_exporter' => { 'enable' => false },
'gitlab_monitor' => { 'enable' => false },
#'mattermost_nginx_eq_nginx' => true,
}
}
81 changes: 51 additions & 30 deletions manifests/init.pp
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,8 @@
# Full path of the the SSL CA certificate.
#
# @param edition
# Edition of GitLab to manage (`'ce'` or `'ee'`)
# The Gitlab Omnibus edition to install.
# Used to set gitlab::manage_upstream_edition.
#
# @param two_way_ssl_validation
# When `true`, server and clients will require mutual TLS authentication.
Expand All @@ -83,7 +84,7 @@
# Array of Nginx-comptaible SSL/TLS protocols for the web server to accept.
#
# @param gitlab_options
# Hash of manually-customized parameters for `vshn/gitlab`.
# Hash of manually-customized parameters for `puppet/gitlab`.
#
# These parameters will be deep-merged with settings generated by this
# profile. During the deep merge, the settings in `$gitlab_options` will
Expand Down Expand Up @@ -122,37 +123,52 @@
# @param ldap_group_base
# EE only
#
# @author simp
# @param manage_package
# Whether simp_gitlab will manage the gitlab-[ce,ee] package.
#
# - Set to true if you want simp_gitlab to set the GitLab root password,
# during the initial install and configuration of GitLab. This is
# **HIGHLY** recommended, as the root password is not secured during
# install otherwise. Anyone can navigate the the GitLab URL and set
# the root password.
#
# @param package_ensure
# The ensure status of the gitlab-[ce,ee] package, when managed by
# `$manage_gitlab` is true.
#
# @author https://github.com/simp/pupmod-simp-simp_gitlab/graphs/contributors
#
class simp_gitlab (
Simplib::Netlist $trusted_nets = simplib::lookup('simp_options::trusted_nets', {'default_value' => ['127.0.0.1/32'] }),
Simp_gitlab::Stroolean $pki = simplib::lookup('simp_options::pki', { 'default_value' => false }),
Simplib::Uri $external_url = $pki ? { true => "https://${facts['fqdn']}", 'simp' => "https://${facts['fqdn']}", default => "http://${facts['fqdn']}" },
Simplib::Netlist $denied_nets = [],
Simplib::Port $tcp_listen_port = $pki ? { true => 443, 'simp' => 443, default => 80},
Boolean $firewall = simplib::lookup('simp_options::firewall', {'default_value' => false}),
Boolean $ldap = simplib::lookup('simp_options::ldap', {'default_value' => false}),
Boolean $ldap_active_directory = false,
Array[Simplib::URI] $ldap_uri = simplib::lookup('simp_options::ldap::uri', {'default_value' => []}),
String[3] $ldap_base_dn = simplib::lookup('simp_options::ldap::base_dn', {'default_value' => simplib::ldap::domain_to_dn()}),
String[3] $ldap_bind_dn = simplib::lookup('simp_options::ldap::bind_dn', {'default_value' => "cn=hostAuth,ou=Hosts,${ldap_base_dn}"}),
String[1] $ldap_bind_pw = simplib::lookup('simp_options::ldap::bind_pw', {'default_value' => "cn=LDAPAdmin,ou=People,${ldap_base_dn}"}),
Optional[String[3]] $ldap_group_base = undef,
Optional[String[1]] $ldap_user_filter = undef,
Hash $gitlab_options = {},
String $app_pki_external_source = simplib::lookup('simp_options::pki::source', { 'default_value' => '/etc/pki/simp/x509' }),
Stdlib::Absolutepath $app_pki_dir = '/etc/pki/simp_apps/gitlab/x509',
Stdlib::Absolutepath $app_pki_key = "${app_pki_dir}/private/${facts['fqdn']}.pem",
Stdlib::Absolutepath $app_pki_cert = "${app_pki_dir}/public/${facts['fqdn']}.pub",
Stdlib::Absolutepath $app_pki_ca = "${app_pki_dir}/cacerts/cacerts.pem",
Boolean $two_way_ssl_validation = false,
Boolean $ldap_verify_certificates = true,
Integer[1] $ssl_verify_depth = 2,
Array[String[1]] $ssl_protocols = ['TLSv1.1','TLSv1.2'],
Array[String[1]] $cipher_suite = simplib::lookup( 'simp_options::openssl::cipher_suite', {
Simplib::Netlist $trusted_nets = simplib::lookup('simp_options::trusted_nets', {'default_value' => ['127.0.0.1/32'] }),
Simp_gitlab::Stroolean $pki = simplib::lookup('simp_options::pki', { 'default_value' => false }),
Simplib::Uri $external_url = $pki ? { true => "https://${facts['fqdn']}", 'simp' => "https://${facts['fqdn']}", default => "http://${facts['fqdn']}" },
Simplib::Netlist $denied_nets = [],
Simplib::Port $tcp_listen_port = $pki ? { true => 443, 'simp' => 443, default => 80},
Boolean $firewall = simplib::lookup('simp_options::firewall', {'default_value' => false}),
Boolean $ldap = simplib::lookup('simp_options::ldap', {'default_value' => false}),
Boolean $ldap_active_directory = false,
Array[Simplib::URI] $ldap_uri = simplib::lookup('simp_options::ldap::uri', {'default_value' => []}),
String[3] $ldap_base_dn = simplib::lookup('simp_options::ldap::base_dn', {'default_value' => simplib::ldap::domain_to_dn()}),
String[3] $ldap_bind_dn = simplib::lookup('simp_options::ldap::bind_dn', {'default_value' => "cn=hostAuth,ou=Hosts,${ldap_base_dn}"}),
String[1] $ldap_bind_pw = simplib::lookup('simp_options::ldap::bind_pw', {'default_value' => "cn=LDAPAdmin,ou=People,${ldap_base_dn}"}),
Optional[String[3]] $ldap_group_base = undef,
Optional[String[1]] $ldap_user_filter = undef,
Hash $gitlab_options = {},
String $app_pki_external_source = simplib::lookup('simp_options::pki::source', { 'default_value' => '/etc/pki/simp/x509' }),
Stdlib::Absolutepath $app_pki_dir = '/etc/pki/simp_apps/gitlab/x509',
Stdlib::Absolutepath $app_pki_key = "${app_pki_dir}/private/${facts['fqdn']}.pem",
Stdlib::Absolutepath $app_pki_cert = "${app_pki_dir}/public/${facts['fqdn']}.pub",
Stdlib::Absolutepath $app_pki_ca = "${app_pki_dir}/cacerts/cacerts.pem",
Boolean $two_way_ssl_validation = false,
Boolean $ldap_verify_certificates = true,
Integer[1] $ssl_verify_depth = 2,
Array[String[1]] $ssl_protocols = ['TLSv1.1','TLSv1.2'],
Array[String[1]] $cipher_suite = simplib::lookup( 'simp_options::openssl::cipher_suite', {
'default_value' => ['DEFAULT', '!MEDIUM']
}),
Enum['ce','ee'] $edition = 'ce',
Enum['ce','ee'] $edition = 'ce',
Boolean $manage_package = true,
String $package_ensure = simplib::lookup('simp_options::package_ensure', { 'default_value' => 'installed' }),
) {

simplib::assert_metadata( $module_name )
Expand Down Expand Up @@ -197,11 +213,16 @@
purge => true,
# ``gitlab reconfigure`` generates PEM hash links
generate_pem_hash_links => false,
notify => Class['gitlab'],
notify => Class['gitlab::service'],
}

Pki::Copy['gitlab'] -> Pki_cert_sync['/etc/gitlab/trusted-certs']
File['/etc/gitlab/trusted-certs'] -> Pki_cert_sync['/etc/gitlab/trusted-certs']

if $manage_package {
# Certs need to be in place for initial gitlab-ctl reconfigure
Pki_cert_sync['/etc/gitlab/trusted-certs'] -> Exec['initial_gitlab_reconfigure']
}
}

if $firewall {
Expand Down
64 changes: 58 additions & 6 deletions manifests/install.pp
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# ## Class simp_gitlab::install
#
# This class is called from simp_gitlab to configure & run the GitLab Omnibus
# installer. It uses the `vshn/gitlab` module to manage the Omnibus config.
# installer. It uses the `puppet/gitlab` module to manage the Omnibus config.
#
class simp_gitlab::install {
assert_private()
Expand All @@ -14,13 +14,14 @@

# If you need to configure the main NGINX server, you can use a `file`
# resource to drop a `.conf` file in `/etc/gitlab/nginx/conf.d/`
file {['/etc/gitlab', '/etc/gitlab/nginx', '/etc/gitlab/nginx/conf.d']:
# (Parent directory, /etc/gitlab, is managed by the gitlab class.
file {['/etc/gitlab/nginx', '/etc/gitlab/nginx/conf.d']:
ensure => directory,
}

file { '/etc/gitlab/nginx/conf.d/http_access_list.conf':
content => $_http_access_list,
notify => Class['gitlab'],
notify => Class['gitlab::service'],
}

# Make sure the standard authorized keys file path is used for the
Expand All @@ -43,8 +44,59 @@
* => deep_merge(simp_gitlab::omnibus_config::gitlab(), $::simp_gitlab::gitlab_options),
}

# This ill-advised hootenany is a hack until vshn/gitlab exposes ENV for the reconfigure
Exec <| title == 'gitlab_reconfigure' |> {
environment => [ "GITLAB_ROOT_PASSWORD=${simp_gitlab::gitlab_root_passwd}" ],
# The package resource and exec below are explicitly designed to ensure the
# root GitLab password is set as part of the initial gitlab install (very
# first reconfigure action). We can't do this with the gitlab module and, in
# fact, have to painfully work around assumptions built into that module's
# design to make it happen. Unfortunately, this makes simp_gitlab highly
# coupled with the internals of the gitlab module.
#
# To set the root password, we need to ensure the gitlab.rb generated by
# gitlab::omnibus_config is laid down **AFTER** the gitlab-[ce,ee] RPM is
# installed and that an initial reconfigure with the GITLAB_ROOT_PASSWORD
# environment variable set is executed.
#
# The reason the gitlab.rb/RPM ordering is important is as follows:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The details in this explanation would help a lot at voxpupuli/puppet-gitlab#301

# - The **ONLY** time GITLAB_ROOT_PASSWORD can be used to set the
# root password is during the absolutely **FIRST** gitlab reconfigure
# operation.
# - The %posttrans of the gitlab-ce RPM runs `gitlab-ctl upgrade`,
# which, if the external_url configuration parameter in gitlab.rb
# does not match the dummy value present in the file during a fresh
# RPM install ('http://gitlab.example.com'), executes a reconfigure
# operation under the hood.
# So, as the gitlab module is written, it precludes the GitLab root
# password from being set, unless the environment variable can be
# set during the package install.
#
# Another problem with the gitlab module that we have to work around is
# that the gitlab::service class is written as if at least 1 reconfigure
# has been executed. Specifically, the Exec['gitlab_reconfigure']
# requires Service['gitlab-runsvdir'], when, in fact the service files
# for that service are created during the initial gitlab reconfigure.
#
if $simp_gitlab::manage_package {
# This resource intentionally has a different title from the package
# title used by gitlab::install ('gitlab-omnibus') to aide debugging.
package { "gitlab-${simp_gitlab::edition}":
ensure => $simp_gitlab::package_ensure,
require => Class['gitlab::omnibus_package_repository'],
before => Class['gitlab::omnibus_config'],
notify => Exec['initial_gitlab_reconfigure'],
}

exec { 'initial_gitlab_reconfigure':
command => '/usr/bin/gitlab-ctl reconfigure',
environment => [ "GITLAB_ROOT_PASSWORD=${simp_gitlab::gitlab_root_passwd}" ],
logoutput => true,
# If gitlab-runsvdir service file is not present, initial gitlab
# reconfigure has not been executed. That file is create by Chef.
onlyif => '/usr/bin/test ! -e /usr/lib/systemd/system/gitlab-runsvdir.service',
refreshonly => true,
timeout => 1800,
tries => 5,
}

Class['gitlab::omnibus_config'] -> Exec['initial_gitlab_reconfigure'] -> Class['gitlab::service']
}
}
18 changes: 9 additions & 9 deletions metadata.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "simp-simp_gitlab",
"version": "0.3.5",
"version": "0.4.0",
"author": "SIMP Team",
"summary": "SIMP profiles for GitLab",
"license": "Apache-2.0",
Expand All @@ -19,6 +19,14 @@
"name": "herculesteam/augeasproviders_ssh",
"version_requirement": ">= 2.5.0 < 4.0.0"
},
{
"name": "puppet/gitlab",
"version_requirement": ">= 3.0.2 < 4.0.0"
},
{
"name": "puppetlabs/stdlib",
"version_requirement": ">= 4.13.1 < 6.0.0"
},
{
"name": "simp/iptables",
"version_requirement": ">= 6.0.1 < 7.0.0"
Expand Down Expand Up @@ -50,14 +58,6 @@
{
"name": "simp/svckill",
"version_requirement": ">= 3.2.1 < 4.0.0"
},
{
"name": "puppetlabs/stdlib",
"version_requirement": ">= 4.13.1 < 6.0.0"
},
{
"name": "vshn/gitlab",
"version_requirement": ">= 1.13.3 < 2.0.0"
}
],
"operatingsystem_support": [
Expand Down
Loading