Purge does not work #182
Comments
|
this seems likely related to #155. Do you have any branches that don't have their name corrected and if so are they purged? |
|
@tomposmiko Can you confirm that it only affects branches that are renamed/autocorrected by g10k? |
|
Oops, sorry, did not see your answer. |
|
hi @xorpaul , I cannot confirm. Go to the environments directory. The copied directory remained in place. Am I correct, it should be removed? |
|
Yes, it should be removed and I can't reproduce your problem: $ ./g10k -version
g10k v0.8.15 Build time: 0.8.15 UTC
$ cat test.yaml
---
:cachedir: '/tmp/g10k'
use_cache_fallback: false
deploy:
purge_levels: [ 'deployment', 'environment', 'puppetfile' ]
sources:
example:
remote: 'https://github.com/xorpaul/g10k-environment.git'
basedir: '/tmp/example/'
prefix: true
force_forge_versions: false
$ ./g10k -config test.yaml -branch benchmark -info
Need to sync /tmp/example/example_benchmark
WARN: Forge module puppetlabs-azure has been deprecated by its author since 2020-04-15 01:31:33 -0700
WARN: Forge module puppetlabs-corosync has been deprecated by its author since 2017-03-21 11:28:39 -0700 The author has suggested puppet-corosync as its replacement
WARN: Forge module vshn-gitlab has been deprecated by its author since 2018-02-08 09:25:47 -0800 The author has suggested puppet-gitlab as its replacement
WARN: Forge module locp-cassandra has been deprecated by its author since 2018-01-05 10:00:09 -0800 The author has suggested puppet-cassandra as its replacement
WARN: Forge module arioch-redis has been deprecated by its author since 2019-03-15 14:56:45 -0700 The author has suggested puppet-redis as its replacement
WARN: Forge module camptocamp-kmod has been deprecated by its author since 2021-06-14 03:15:07 -0700 The author has suggested puppet-kmod as its replacement
WARN: Forge module puppetlabs-dsc has been deprecated by its author since 2021-05-19 16:09:30 -0700
WARN: Forge module example42-logstash has been deprecated by its author since 2019-10-28 03:29:25 -0700 The author has suggested elastic-logstash as its replacement
Need to sync /tmp/example/example_benchmark/external_modules/stdlib
Need to sync /tmp/example/example_benchmark/external_modules/gitlab
Need to sync /tmp/example/example_benchmark/external_modules/supervisord
Need to sync /tmp/example/example_benchmark/external_modules/kmod
Need to sync /tmp/example/example_benchmark/external_modules/accounts
Need to sync /tmp/example/example_benchmark/external_modules/redis
Need to sync /tmp/example/example_benchmark/external_modules/openssl
Need to sync /tmp/example/example_benchmark/external_modules/ntp
Need to sync /tmp/example/example_benchmark/external_modules/unattended_upgrades
Need to sync /tmp/example/example_benchmark/external_modules/corosync
Need to sync /tmp/example/example_benchmark/external_modules/cassandra
Need to sync /tmp/example/example_benchmark/external_modules/xinetd
Need to sync /tmp/example/example_benchmark/external_modules/locales
Need to sync /tmp/example/example_benchmark/external_modules/vcsrepo
Need to sync /tmp/example/example_benchmark/external_modules/resolv_conf
Need to sync /tmp/example/example_benchmark/external_modules/lvm
Need to sync /tmp/example/example_benchmark/external_modules/vmwaretools
Need to sync /tmp/example/example_benchmark/external_modules/apt
Need to sync /tmp/example/example_benchmark/external_modules/firewall
Need to sync /tmp/example/example_benchmark/external_modules/aws
Need to sync /tmp/example/example_benchmark/external_modules/azure
Need to sync /tmp/example/example_benchmark/external_modules/apache
Need to sync /tmp/example/example_benchmark/external_modules/dsc
Need to sync /tmp/example/example_benchmark/external_modules/inifile
Need to sync /tmp/example/example_benchmark/external_modules/concat
Need to sync /tmp/example/example_benchmark/external_modules/postfix
Need to sync /tmp/example/example_benchmark/external_modules/logstash
Need to sync /tmp/example/example_benchmark/external_modules/haproxy
Need to sync /tmp/example/example_benchmark/external_modules/mysql
Synced test.yaml with branch benchmark with 5 git repositories and 25 Forge modules in 4.8s with git (1.4s sync, I/O 0.2s) and Forge (3.2s query+download, I/O 6.5s) using 50 resolve and 20 extract workers
$ ./g10k -config test.yaml -branch master -info
Need to sync /tmp/example/example_master
Need to sync /tmp/example/example_master/external_modules/postfix
Need to sync /tmp/example/example_master/external_modules/stdlib
Need to sync /tmp/example/example_master/external_modules/concat
Need to sync /tmp/example/example_master/external_modules/inifile
Need to sync /tmp/example/example_master/external_modules/ntp
Need to sync /tmp/example/example_master/external_modules/resolv_conf
Need to sync /tmp/example/example_master/external_modules/firewall
Need to sync /tmp/example/example_master/external_modules/aws
Need to sync /tmp/example/example_master/external_modules/apt
Need to sync /tmp/example/example_master/external_modules/apache
Synced test.yaml with branch master with 5 git repositories and 6 Forge modules in 2.7s with git (0.4s sync, I/O 0.1s) and Forge (2.2s query+download, I/O 1.1s) using 50 resolve and 20 extract workers
$ cp -pr /tmp/example/example_benchmark/ /tmp/example/example_master/
$ ls -l /tmp/example/example_master/
total 12
drwxrwxr-x 3 xorpaul xorpaul 4096 Jun 25 13:52 example_benchmark
drwxrwxr-x 12 xorpaul xorpaul 4096 Jun 25 13:52 external_modules
-rw-rw-r-- 1 xorpaul xorpaul 611 Aug 21 2020 Puppetfile
$ ./g10k -config test.yaml -branch master -info
Removing unmanaged path /tmp/example/example_master/example_benchmark
Removing unmanaged path /tmp/example/example_master/example_benchmark/.g10k-deploy.json
Removing unmanaged path /tmp/example/example_master/example_benchmark/Puppetfile
Removing unmanaged path /tmp/example/example_master/example_benchmark/external_modules
Synced test.yaml with branch master with 5 git repositories and 6 Forge modules in 0.9s with git (0.5s sync, I/O 0.0s) and Forge (0.0s query+download, I/O 0.0s) using 50 resolve and 20 extract workers
$ ls -l /tmp/example/example_master/
total 8
drwxrwxr-x 12 xorpaul xorpaul 4096 Jun 25 13:52 external_modules
-rw-rw-r-- 1 xorpaul xorpaul 611 Aug 21 2020 PuppetfileCan you provide more information or try to reproduce with my test.yaml and g10k commands? |
|
@xorpaul |
|
@tomposmiko Have you also tried a more recent version like https://github.com/xorpaul/g10k/releases/tag/v0.9.7 ? |
|
Yes, I tested it with this version: |
|
Hi @xorpaul. Also having issues with getting purge to work. Unless I'm screwing it up, I'm managing to reproduce this on one of our Rocky 8 puppet masters. Happy to test more on this end. Thanks. |
|
g10k checks the It does not detect if some folder was copied into the Puppet environment, when neither the control repository commit hash or the Puppetfile checksum was changed. To force a check/sync which will detect the copied folder you need to force a sync by deleting or modifying the g10k then does delete the copied folder as you can see with the |
|
Gotcha. Above I was trying to mimic a situation where r10k would delete environments that don't have a corresponding remote branch. Going to test things a bit more on my side and will repost with hopefully a good repo case if I can't get things working. Thanks. |
|
OK, I dug into this a good bit more. For the life of me, I can't get g10k to purge environments. Can you please let me know if I'm doing something wrong? Config g10k_test.yaml: Repro Steps:
Result: The I have to assume I'm doing something wrong here, but damned if I can figure out what it is. The equivalent setup worked for us with r10k previously. Please let me know if you need any more detail or testing done. Thanks. |
|
I mean g10k is telling you why it does not delete the non-existing branch: Why not sync all branches of your control repository and not using For me if a user specifies that g10k should only sync one specific branch either by using I'm not sure if I want to copy the behaviour of r10k here 🤔 |
|
Thanks for the response. I was able to now get this to work by having my compile masters all sync the entire control repo every time. That's not optimal - the build, while still fast, is 2-3x slower now. If I'm understanding how g10k works correctly, the current g10k behavior is a big, undocumented divergence from how r10k works. When deploying via r10k with
The r10k behavior seems excellent to me, since if you are creating dynamic environments on the regular and not deleting them, your build masters will quickly get loaded up with old, unnecessary environments. If you can confirm for me that the current behavior is what you are looking for, I'd be happy to put in a PR with a change to the readme. |
Would it be possible to make this behaviour configurable? I've got around 500 environments (partially because users are really bad at housekeeping), and checking/syncing them all just to get the purge behaviour isn't going to work well for me. |
hi,
I probably do something wrong with my environment, but do not find the reason.
g10k.yaml:
command:
My understanding is that this should purge the basedir by removing stale directories.
version:
Any suggestion, what's wrong with my setup?
Thanks in advance,
The text was updated successfully, but these errors were encountered: