-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Atlantis 1.0.0 Release #2496
Comments
I think atlantis has been in production for years, I believe the 1.0.0 release is just a number, not an actual representation of where atlantis is. What do you think is missing? |
As mentioned in Slack, I personally think that making atlantis, hmm, cloud-native would be a such missing thing for version 1.0. |
locking db as you mentioned was merged yesterday and that will enable autoscaling. I guess I'm trying to understand what cloud-native means?, because Atlantis needs to be cloud provider agnostic BUT that does not mean that they could be an implementation that uses S3/blob storage to save the clones etc. I think we will hear back soon from people using the external locking and enabling things like EFS for the AtlantiData dir and we will see how that behaves, my guess is that it might work just fine. |
Oh, I missed the part that it's possible to use EFS or similar. We will try it with google filestore and see how it works! |
For a v1 release Atlantis should be able to run parallel plans for different workspaces that are in separate directories. |
@jamengual - we attempted to use EFS in AWS but it was far too slow. EFS is not very good at handling lots of small file reads / writes (e.g |
you need to test in the next tier of EFS:
Q. How much throughput can a file system support?
Bursting throughput mode, which is the default throughput mode for Amazon
EFS file systems, scales the throughput available to a file system as that
file system grows. Because file-based workloads are typically spiky -
requiring high levels of throughput at certain periods of time and lower
levels of throughput otherwise - Amazon EFS is designed to "burst" which
allows high throughput levels for periods of time. Also, because many
workloads are read-heavy, read operations are metered at a 1:3 ratio to
other NFS operations (such as write). All file systems deliver a consistent
baseline performance of 50 MB/second per TB of EFS Standard or EFS One Zone
storage, while all file systems (regardless of size) can burst to 100
MB/second, and file systems with more than 1 TB of EFS Standard or EFS One
Zone storage can burst to 100 MB/second per TB. Since read operations are
metered at a 1:3 ratio, you can drive up to 300 MiB/second per TiB of read
throughput. As you add data to your file system, the maximum throughput
available to the file system scales linearly and automatically with your
storage in the Amazon EFS Standard storage class. If you need more
throughput than you can achieve with the amount of data you have stored,
you can configure Provisioned Throughput to the specific amount your
workload requires.
https://aws.amazon.com/efs/faq/
create a dummy 1 TB file to jump to the next tier.
…On Thu, Sep 22, 2022 at 10:49 AM snorlaX-sleeps ***@***.***> wrote:
enabling things like EFS for the AtlantiData dir
@jamengual <https://github.com/jamengual> - we attempted to use EFS in
AWS but it was far too slow. EFS is not very good at handling lots of small
file reads / writes (e.g terraform init)
—
Reply to this email directly, view it on GitHub
<#2496 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAQ3ERG2L6BK2VJI4CAW64DV7SLZ3ANCNFSM6AAAAAAQGGHRHQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Could do! But a 50Gb EBS is a fair bit cheaper than a 1TB EFS (or so it was when we were originally deploying Atlantis). |
Off topic from V1.0, but I used EFS with provisioned IOPS of a reasonably high number, and the performance was still garbage. Looking at clones taking 2+ minutes compared to ~10 seconds to an EBS volume. I wonder if it'd be possible to split up the atlantis directory so that the plan files are stored on EBS, and the git check out is stored on ebs or instance storage
Curiously, we're using multiple plans in different directories, and it runs quite quickly, excusing EFS issues mentioned above |
@pauloconnor - Yes, it will run plans in parallel in different directories if the workspace names are different. |
We don't use different workspace names - just |
Apologies @pauloconnor - I realised I linked the wrong PR we were using for the workspaces, it was this: #2180 So yes, different directories can run in parallel, but if they use workspaces with the same name they don't run in parallel |
That's completely different to what I've observed on my Atlantis node - using multiple different directories with the same workspace name which all have changes, each directory will spin up a Terraform process in parallel. I have seen more than 16 concurrent tasks running |
Ok, thanks - wanted to confirm my linking of the wrong PR didn't make a difference. I will try this out on Monday |
running
|
It's probably worth opening a separate issue, since that's not related to releasing a v1.0.0. |
The question isn't whether this project is used in production. It's used quite a bit in production. The idea of this ticket is to legitimize it's production readiness. Some potential users and existing users that I've spoken to have shared feedback that they are hesitant in adopting/continuing with atlantis due to the lack of a 1.0 release. They assume that this project is in "beta". Therefore a 1.0 release would legitimize it. Terraform and other projects have done the same. They were in a "forever beta" and then when 0.15.x hit, they focused on stability, and then renamed it to 1.0 and continued with their minor (1.x.y) releases.
There are a few open source projects that I'm aware of that are cloud specific and cloud agnostic
Let's try to keep this issue related to 1.0 release and move those discussions to a separate ticket or github discussion.
Parallel plan issues are documented. Let's keep that discussion in those respective tickets. A new PR #2921 that was submitted recently seeks to use TF_DATA_DIR to reduce clones and allow better parallel workflows for workspaces. I think Nish (the first person to mention a 1.0 release, quoted above) was correct that that the VCS logic separated from this repo would be essential as that would
I created a separate issue to track it #2991 |
I think we should pick from the PRs we have in flight ( which are
considerable improvements/features) and set them for milestone 1.0 and
release 1.0.
I think we have been already in a good place to release 1.0 for a long time.
…On Mon, Jan 16, 2023 at 7:10 AM nitrocode ***@***.***> wrote:
I think atlantis has been in production for years, I believe the 1.0.0
release is just a number, not an actual representation of where atlantis is.
The question isn't whether this project is used in production. It's used
quite a bit in production. The idea of this ticket is to legitimize it's
production readiness. Some potential users and existing users that I've
spoken to have shared feedback that they are hesitant in
adopting/continuing with atlantis due to the lack of a 1.0 release. They
assume that this project is in "beta". Therefore a 1.0 release would
legitimize it.
Terraform and other projects have done the same. They were in a "forever
beta" and then when 0.15.x hit, they focused on stability, and then renamed
it to 1.0 and continued with their minor (1.x.y) releases.
As mentioned in Slack, I personally think that making atlantis, hmm,
cloud-native would be a such missing thing for version 1.0.
By cloud-native here I mean having a separate repository and plan storage,
and locking db (which is already implemented) so that Atlantis can be
horizontally scaled in Kubernetes or otherwise usable with Google Cloud Run
or similar tools from other cloud providers.
There are 2 open source projects that I'm aware of that are cloud specific.
- aws - https://github.com/terraform-aws-modules/terraform-aws-atlantis
- gce -
https://registry.terraform.io/modules/bschaatsbergen/atlantis/gce/latest
EFS discussions
Let's try to keep this issue related to 1.0 release and move those
discussions to a separate ticket or github discussion.
running atlantis:v0.20.1, parallel_plan: true works for me without locking
issues, but i get the below error when trying to run atlantis apply -d
environments/myorg-preprd/preprd. all environments use the default workspace
Parallel plan issues are documented. Let's keep that discussion in those
respective tickets.
A new PR #2921 <#2921> that
was submitted recently seeks to use TF_DATA_DIR to reduce clones and allow
better parallel workflows for workspaces.
------------------------------
I think Nish (the first person to mention a 1.0 release, quoted above) was
correct that that the VCS logic separated from this repo would be essential
as that would
- reduce code in atlantis repo
- faster builds and tests of atlantis
- allow maintainers to split github, gitlab, bitbucket, azuredevops,
etc into its own repos
- the repo may not even be managed by maintainers since most of the
maintainers are on github
- allow users and devs to contribute their own custom VCS providers
—
Reply to this email directly, view it on GitHub
<#2496 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAQ3ERBTAO4XKRR3XD5ICGTWSVQHBANCNFSM6AAAAAAQGGHRHQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
I'd be okay with renaming the upcoming 0.22.3 release as 1.0.0 if you and other maintainers are ok with that. cc @runatlantis/maintainers |
@krrrr38 brought up a good point in slack and mentioned that we should probably deprecate older flags and remove them from documentation.
Perhaps we should release the upcoming 0.22.3 as planned and save 1.0 once we get PRs for the above removal. There might be some other easy ideas that we may want to add prior to 1.0 since it is a major release. Created #2992 |
Community Note
Describe the user story
It would be nice to legitimize Atlantis for production by putting together a list of necessary items to create a 1.0 release.
This was also mentioned in the Atlantis slack community on Aug 9 2021 by @nishkrishnan
Describe the solution you'd like
Come up with a list of items that need to be complete for a 1.0 release
Future release
Describe the drawbacks of your solution
None that I can think of
Describe alternatives you've considered
None
The text was updated successfully, but these errors were encountered: