-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
pkg/asset/machines: Convert Master to a WriteableAsset #1211
pkg/asset/machines: Convert Master to a WriteableAsset #1211
Conversation
Is there anywhere in the code you could slip in a unit test that covered the installconfig on disk use case without requiring an actual provision? |
#552 has a framework for something like that, although the issue here is manifests on disk, not install-config on disk. |
7aaa902
to
7f61d57
Compare
I've pushed 7aaa902 -> 7f61d57 to fix the golint issue. I'm not sure how to handle the unit error:
More on that issue here, but I'm currently distinguishing between "set to the empty list" and "unset" when consuming the data while we have both |
I don't have a better solution for you. I had the same problem. installer/pkg/asset/machines/master.go Lines 99 to 102 in ad65a3a
|
7f61d57
to
d5c01c0
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with this approach. I was planning on doing this same thing to close the loop on the https://bugzilla.redhat.com/show_bug.cgi?id=1662119.
d5c01c0
to
1c0796a
Compare
955cf95
to
1e32e76
Compare
Wondering why hive was having problems with this panic led me to the realization that this PR does not completely solve the problem. The following will still panic, because loading from the state file will keep the provider config as raw json.
I am disheartened that this isn't caught by the unit tests that verify round-trip loading. And I am not sure why it isn't caught. |
Do we have hooks to deserialize when we load from the state file? If not, maybe we just need to throw in the towel on structured asset properties (which will make @abhinavdahiya sad ;). |
No hooks. |
What if we remove the |
I don't have strong opinions on whether the deserializing code lives in |
1e32e76
to
3fd7e2b
Compare
Ok, I've pushed 1e32e76 -> 3fd7e2b shifting to unmarshalling |
3fd7e2b
to
6b0bd4b
Compare
I've just pushed 3fd7e2b -> 6b0bd4b with the pivot to |
6b0bd4b
to
e8cead9
Compare
3e19bea
to
2f16fa5
Compare
/test e2e-aws |
will #1223 affect the Decode ? |
Sigh. Maybe? I guess for that I need to bump our vendor around openshift/cluster-api-provider-aws#152. |
Since 3326b6b (pkg/tfvars: Pull from []Machine instead of InstallConfig, 2018-11-28, openshift#792), we've been consuming structured master configurations to generate Terraform variables. But before this commit, the Openshift asset was the WriteableAsset responsible for the master configs, giving you the following issue: 1. openshift-install create manifests i. Master asset populated with structured data ii. Openshift asset pulls in Master, flattens to YAML, and writes to disk. 2. openshift-install create cluster i. Openshift asset reads master YAML from disk. ii. TerraformVariables asset pulls in Master, but it's empty. iii. Panic [1]. With this commit, responsibility for reading and writing master manifests to disk gets shifted to the Master asset itself. I've dropped the structured Master properties (Machines and MachineDeprecated) in favor of new methods on the asset. There are three possible paths towards recovering the asset that we feed in to TerraformVariables: a. After rendering it with Generate. This information had been structured before this commit, so no problems here. b. After reading from the disk. This information could be unmarshalled in Master.Load(), but... c. After loading it from the state file. There are no hooks for custom unmarshalling in this pathway, so while the generic YAML unmarshal would give us a structured machine object, it would not unmarshal the RawExtension that holds the provider spec. The lack of custom-unmarshal hooks for (c) means there's no reliable way to use Master properties to feed TerraformVariables structured provider information. With this commit, we use Master methods instead. That's just as efficient for the (b) and (c) cases, and while it's an uneccessary (de)serialize cycle for (a), that's not too expensive. Unpacking the YAML data into the appropriate structures is a bit hairy. I'm not familiar with this code though, maybe there's a better way. It will help once we complete the shift to the OpenShift machine-API types started in cf60daa (Pivot aws from cluster.k8s.io into machine.openshift.io, 2019-02-01, openshift#1175). I've also taken the opportunity to drop the list. It saves us a few lines of code for (de)serializing, and there's no upside to collecting the Machine configs together in a single file. The "glob, but not the master files" logic in the Openshift loader is also a bit ugly. Moving forward, I expect us to push the remaining Openshift assets out into their own assets as well, which would allow us to tighten down on the wildcard there. There's also a bit of dancing to ensure our Machine filenames are ordered by increasing index. I'm padding the filenames with %02d (for example) to get ...-01.yaml, etc. so they come back in the right order on load (filepath.Glob sorts its output [2]). To avoid hard-coding a pad size, I format the largest index, measure its length, and use that length to construct padFormat. [1]: openshift#1205 [2]: https://github.com/golang/go/blob/go1.11.5/src/path/filepath/match.go#L325
And shift the AWS provider's commit to land after they fixed registration [1] and also after their pivot to v1beta1 [2] (because 3a125a6, Change awsproviderconfig.k8s.io/v1alpha1 API version to awsproviderconfig.openshift.io/v1beta1, 2019-02-11, openshift#1223 already started us along that pivot). Generated (after the Gopkg.toml bump) with: $ dep ensure using: $ dep version dep: version : v0.5.0-31-g73b3afe build date : 2019-02-08 git hash : 73b3afe go version : go1.10.3 go compiler : gc platform : linux/amd64 features : ImportDuringSolve=false [1]: openshift/cluster-api-provider-aws#149 [2]: openshift/cluster-api-provider-aws#152
2f16fa5
to
d983eae
Compare
api requests getting rate-limited level=error msg="Error: Error applying plan:"
level=error
level=error msg="5 errors occurred:"
level=error msg="\t* module.vpc.aws_lb.api_external: 1 error occurred:"
level=error msg="\t* aws_lb.api_external: timeout while waiting for state to become 'active' (last state: 'provisioning', timeout: 10m0s)"
level=error
level=error
level=error msg="\t* module.vpc.aws_lb.api_internal: 1 error occurred:"
level=error msg="\t* aws_lb.api_internal: timeout while waiting for state to become 'active' (last state: 'provisioning', timeout: 10m0s)"
level=error
level=error
level=error msg="\t* module.vpc.aws_route_table_association.route_net[2]: 1 error occurred:"
level=error msg="\t* aws_route_table_association.route_net.2: timeout while waiting for state to become 'success' (timeout: 5m0s)"
level=error
level=error
level=error msg="\t* module.vpc.aws_route_table_association.route_net[3]: 1 error occurred:"
level=error msg="\t* aws_route_table_association.route_net.3: timeout while waiting for state to become 'success' (timeout: 5m0s)"
level=error
level=error
level=error msg="\t* module.vpc.aws_route.igw_route: 1 error occurred:"
level=error msg="\t* aws_route.igw_route: Error creating route: timeout while waiting for state to become 'success' (timeout: 2m0s)" will retest in some time. |
/test e2e-aws Hive CI still totally blocked on this. :( |
/retest |
The tests are green.. /lgtm |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: abhinavdahiya, wking The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Like the earlier 7a396d9 (pkg/asset/machines: Convert Master to a WriteableAsset, 2019-02-07, openshift#1211), but for Worker. This is groundwork for extracting configured availability zones when generating AWS Terraform variables, which will in turn allow us to avoid provisioning availability zones unless we are going to populate them with machines.
Like the earlier 7a396d9 (pkg/asset/machines: Convert Master to a WriteableAsset, 2019-02-07, openshift#1211), but for Worker. This is groundwork for extracting configured availability zones when generating AWS Terraform variables, which will in turn allow us to avoid provisioning availability zones unless we are going to populate them with machines.
Like the earlier 7a396d9 (pkg/asset/machines: Convert Master to a WriteableAsset, 2019-02-07, openshift#1211), but for Worker. This is groundwork for extracting configured availability zones when generating AWS Terraform variables, which will in turn allow us to avoid provisioning availability zones unless we are going to populate them with machines.
Like the earlier 7a396d9 (pkg/asset/machines: Convert Master to a WriteableAsset, 2019-02-07, openshift#1211), but for Worker. This is groundwork for extracting configured availability zones when generating AWS Terraform variables, which will in turn allow us to avoid provisioning availability zones unless we are going to populate them with machines.
Like the earlier 7a396d9 (pkg/asset/machines: Convert Master to a WriteableAsset, 2019-02-07, openshift#1211), but for Worker. This is groundwork for extracting configured availability zones when generating AWS Terraform variables, which will in turn allow us to avoid provisioning availability zones unless we are going to populate them with machines.
Like the earlier 7a396d9 (pkg/asset/machines: Convert Master to a WriteableAsset, 2019-02-07, openshift#1211), but for Worker. This is groundwork for extracting configured availability zones when generating AWS Terraform variables, which will in turn allow us to avoid provisioning availability zones unless we are going to populate them with machines.
Like the earlier 7a396d9 (pkg/asset/machines: Convert Master to a WriteableAsset, 2019-02-07, openshift#1211), but for Worker. This is groundwork for extracting configured availability zones when generating AWS Terraform variables, which will in turn allow us to avoid provisioning availability zones unless we are going to populate them with machines.
Like the earlier 7a396d9 (pkg/asset/machines: Convert Master to a WriteableAsset, 2019-02-07, openshift#1211), but for Worker. This is groundwork for extracting configured availability zones when generating AWS Terraform variables, which will in turn allow us to avoid provisioning availability zones unless we are going to populate them with machines.
Like the earlier 7a396d9 (pkg/asset/machines: Convert Master to a WriteableAsset, 2019-02-07, openshift#1211), but for Worker. This is groundwork for extracting configured availability zones when generating AWS Terraform variables, which will in turn allow us to avoid provisioning availability zones unless we are going to populate them with machines.
Since 3326b6b (#792), we've been consuming structured master configurations to generate Terraform variables. But before this commit, the
Openshift
asset was theWriteableAsset
responsible for the master configs, giving you the following issue:openshift-install create manifests
Master
asset populated with structured dataOpenshift
asset pulls inMaster
, flattens to YAML, and writes to disk.openshift-install create cluster
Openshift
asset reads master YAML from disk.TerraformVariables
asset pulls inMaster
, but it's empty.With this commit, responsibility for reading and writing master manifests to disk gets shifted to the
Master
asset itself.Unpacking the YAML data into the appropriate structures is a bit hairy. I'm not familiar with this code though, maybe there's a better way. It will help once we complete the shift to the OpenShift machine-API types started in cf60daa (#1175).
I've also taken the opportunity to drop the list. It saves us a few lines of code for (de)serializing, and there's no upside to collecting the
Machine
configs together in a single file.The "glob, but not the master files" logic in the
Openshift
loader is also a bit ugly. Moving forward, I expect us to push the remainingOpenshift
assets out into their own assets as well, which would allow us to tighten down on the wildcard there.@staebler, can you check the asset logic here? I think I have it right, but I'm doing more pattern-matching than understanding ;).
Fixes #1205