Poetry Take 2

[tarkatronic]

After trying to update the pre-existing Poetry PR, and discovering numerous conflicts, I dediced it would be wise to start fresh. Luckily I was able to copy over much of my
work from previously, and enough things have been fixed now that this transition has been much easier this time around, so far.

I will need to go back through the other PR to find the full list of work that was necessary for this transition, but so far:

• Tox passes for all environments
• Docs build
• Make commands work

So I'd call this a pretty good start so far!

PR Checklist

• Wrote a good commit message & description [see guide below].
• Commit message starts with [Resolve #issue-number].
• Added/Updated unit tests.
• Added/Updated integration tests (if applicable).
• All unit tests (make test) are passing.
• Used the same coding conventions as the rest of the project.
• The new code passes pre-commit validations (pre-commit run --all-files).
• The PR relates to only one subject with a clear title.
  and description in grammatically correct, complete sentences.

Approver/Reviewer Checklist

• Before merge squash related commits.

Other Information

Guide to writing a good commit

[zaro0508]

Generally looks good however this PR is failing in CI. Also I would like to see some documentation added in the Contributions doc to at least tell users that they need to setup poetry to run builds and tests.

[zaro0508]

do we need to add some configs in pyproject.toml for tox?

[tarkatronic]

Whatever config currently exists for tox will work fine for now! I'm trying to limit the scope of this PR as much as possible.

[jfalkenstein]

This looks like it is meant to replace setup.py. Is that True? If so, why is setup.py not removed in this PR?

[tarkatronic]

Yes that is true. And I simply hadn't gotten to that just yet. I'm realizing I should have opened this as a draft PR for the time being!

[jfalkenstein]

What's the point of the quotes here?

[tarkatronic]

You know, I asked myself the very same question when I was copying this over from the original PR I had 😆

I know I had a reason for them previously, but I couldn't tell you what that is now. My guess would be, something didn't work right without them. But I can do some experimentation to figure that out.

[jfalkenstein]

What does this do?

[tarkatronic]

This basically makes diffs easier to deal with. It's not absolutely necessary, and I'd be fine removing it, but diffs of this file are essentially useless since it is entirely machine generated.

The main thing this does is to tell git to not even bother trying to show a diff.

[jfalkenstein]

I get that... but then the counter-argument is that if somebody updates dependencies but forgets to update the lock file, I want to see on the PR that they haven't modified the lock file. Perhaps we can add a pre-commit check that the lock file is unchanged after an install or something? In cases with lock files, I don't actually care what the changes are, only that there is a change when someone has modified dependencies.

[tarkatronic]

After looking this over and reading the comments so far, I realized it makes the most sense to convert this to a draft.

I'm also thinking that it might be prudent to make some other changes before implementing this. I'm happy to take feedback on these proposed changes.

1. Use importlib.metadata to remove the need for the version in sceptre/__init__.py (which is a pattern that's known to have potential gotchas)
2. Switch from CircleCI to GitHub Actions. This is something that was discussed previously and people seemed to be in favor of it way back when. This isn't an absolutely necessary pre-requisite, but it would make this transition massively easier and prevent having to get this setup working in one environment, only to have to then switch it to the new.
3. I'm certain there's more but nothing is occurring off the top of my head...

[jfalkenstein]

Use importlib.metadata to remove the need for the version in sceptre/init.py (which is a pattern that's known to have potential gotchas)

Unless there's a backport to 3.7 we can install, we cannot use that. That library was introduced in 3.8. It might be installable as a backport, in which case I'm fine doing that. But if we can't, then we shouldn't do that until 3.7 reaches end of life.

Switch from CircleCI to GitHub Actions. This is something that was discussed previously and people seemed to be in favor of it way back when. This isn't an absolutely necessary pre-requisite, but it would make this transition massively easier and prevent having to get this setup working in one environment, only to have to then switch it to the new.

Regarding GH Actions vs. circleci... I don't really have a dog in that fight. I have no real experience in either. I don't care that much... though I don't see why a transition to Poetry would need to require a transition of CI/CD tool at the same time.

I did have one other question: When Poetry creates a lock-file... that's just used for local development, right? It doesn't actually lock dependencies for those that install Sceptre as a library, right? There needs to be some "wiggle room" to install Sceptre into a virtual environment with other packages without entering a dependency hellscape.

[tarkatronic]

Use importlib.metadata to remove the need for the version in sceptre/init.py (which is a pattern that's known to have potential gotchas)

Unless there's a backport to 3.7 we can install, we cannot use that. That library was introduced in 3.8. It might be installable as a backport, in which case I'm fine doing that. But if we can't, then we shouldn't do that until 3.7 reaches end of life.

As a matter of fact, there is! https://pypi.org/project/importlib-metadata/

Switch from CircleCI to GitHub Actions. This is something that was discussed previously and people seemed to be in favor of it way back when. This isn't an absolutely necessary pre-requisite, but it would make this transition massively easier and prevent having to get this setup working in one environment, only to have to then switch it to the new.

Regarding GH Actions vs. circleci... I don't really have a dog in that fight. I have no real experience in either. I don't care that much... though I don't see why a transition to Poetry would need to require a transition of CI/CD tool at the same time.

The main reasoning is that CircleCI, especially with Sceptre's current environment, is very hard to get right. You can see my previous attempt for some idea on just how much effort that takes.

In contrast, a lot of what is being done manually with CircleCI just sorta "works" with GitHub Actions. For an example of a Poetry-based project that I set up a while back, see here: https://github.com/godaddy/tartufo/blob/main/.github/workflows/ci.yml

And IIRC since then, GitHub Actions has even added some Poetry-specific setup into their actions/setup-python workflow, which makes it even easier!

So the tl;dr is, CircleCI is hard, GitHub Actions is easy, and if the transition is going to happen at some point I think it makes sense to do before Poetry, to avoid even more re-work.

I did have one other question: When Poetry creates a lock-file... that's just used for local development, right? It doesn't actually lock dependencies for those that install Sceptre as a library, right? There needs to be some "wiggle room" to install Sceptre into a virtual environment with other packages without entering a dependency hellscape.

The lock file is used for local development, and not for end-user installations, yes, that is correct. And there is no reason at all that wiggle room can't be present with Poetry! In fact you may notice, I had to do a bit of massaging of allowed versions already. For example: https://github.com/Sceptre/sceptre/pull/1239/files#diff-50c86b7ed8ac2cf95bd48334961bf0530cdc77b5a56f852c5c61b89d735fd711R60

Poetry has got all kinds of ability to specify those dependencies with whatever amount of wiggle room is necessary.

[zaro0508]

So the tl;dr is, CircleCI is hard, GitHub Actions is easy, and if the transition is going to happen at some point I think it makes sense to do before Poetry, to avoid even more re-work.

As much as I would like to see this transition happen I don't envision it to happen anytime soon, or maybe at all. All of the repos in the GH Sceptre org is using circleci right now. I would hate to see us be in a state where we need to use two separate CI systems. In other words, unless there's someone that really want to put a lot of effort into moving Sceptre to GH actions I would just prefer to stay on one CI system.

[zaro0508]

closing this PR in favor of #1323