Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add devfile parser #2500

Merged

Conversation

kanchwala-yusuf
Copy link
Contributor

@kanchwala-yusuf kanchwala-yusuf commented Jan 8, 2020

What type of PR is this?

/kind feature

What does does this PR do / why we need it:
This PR implements the basic components devfile parser.

Which issue(s) this PR fixes**:

Fixes #2477

@openshift-ci-robot openshift-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. Required by Prow. size/XXL labels Jan 8, 2020
func (d *DevfileCtx) Validate() error {

// Validate devfile
if err := d.ValidateDevfileSchema(); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't we just return d.ValidateDevfileSchema()?

Copy link
Contributor Author

@kanchwala-yusuf kanchwala-yusuf Jan 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially, I was expecting more validations as a part of Validate() func, but currently there is only one, so this can be done.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

RegistryUrl *string `yaml:"registryUrl,omitempty" json:"registryUrl,omitempty"`
}

type DevfileComponentOpenshift struct {
Copy link
Contributor

@girishramnani girishramnani Jan 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we reuse all these strings from 1.0.0, maybe even extract them into a common package? and at places where the 1.0.1 structs have more parameters, then can we embed the 1.0.0 struct in the respective 1.0.1 struct and add the new parameter in the 1.0.1 struct?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This approach will help use use a common interface and not have branches in the layers of code which uses devfiles

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be actually a good idea. Not sure about a common package, but reusing structs from the previous version if they are not changes make actually sense. It will also nicely highlight the changes between versions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have refactored the code to have common structs and common functions between different versions of devfile.

As per our discussion with the devfile team, since the 1.0.1-beta version has been merged into 1.0.0 version, I have removed the 1.0.1-beta files from the code as well.

}

if !isDockerImageComponentPresent {
errMsg := fmt.Sprintf("odo requires atleast one component of type '%s' in devfile", DevfileComponentsTypeDockerimage)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we inline this error message?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taken a different approach for this.


// isProjectTypeSupported checks if the given project type is supported in ODO
func isProjectTypeSupported(typ DevfileProjectsType) bool {
for _, supportedType := range SupportedDevfileProjectTypes {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really, to use In() we may need to convert/typecast every element of SupportedDevfileProjectTypes of custom type DevfileProjectType to string.

@kanchwala-yusuf kanchwala-yusuf force-pushed the devfile-parser branch 7 times, most recently from fa1255a to 4c3fc64 Compare January 22, 2020 10:16
@kanchwala-yusuf
Copy link
Contributor Author

/retest

@kanchwala-yusuf
Copy link
Contributor Author

/restest

@kanchwala-yusuf kanchwala-yusuf changed the title [WIP] Add devfile parser Add devfile parser Jan 24, 2020
@openshift-ci-robot openshift-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. Required by Prow. label Jan 24, 2020
@kanchwala-yusuf
Copy link
Contributor Author

/retest

@kanchwala-yusuf
Copy link
Contributor Author

/test v4.1-integration-e2e-benchmark

@kadel
Copy link
Member

kadel commented Jan 29, 2020

@kanchwala-yusuf Please don't amend old commits. It makes it really hard to review changes :-(

Just push new one, the PRs get automatically squashed before it is merged to master, so number of commits in PR doesn't matter.

Copy link
Member

@kadel kadel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@openshift-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kadel

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. Required by Prow. label Jan 29, 2020
t.Helper()

// Create tempfile
f, err := ioutil.TempFile(os.TempDir(), TempJsonDevfilePrefix)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's use the fake package here and write the stuff in memory instead of the disk. That will speed up the testing.

func (d *DevfileCtx) SetDevfileContent() error {

// Read devfile
data, err := ioutil.ReadFile(d.absPath)
Copy link
Contributor

@mik-dass mik-dass Jan 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not use the Fs Filesystem which is part of the DevfileCtx? What will help us with unit tests using the fake system.


var (
tempDevfile = createTempDevfile(t, validJsonRawContent100())
d = DevfileCtx{absPath: tempDevfile.Name()}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can pass the fake filesystem here in DevfileCtx

t.Run("run command not present", func(t *testing.T) {

commands := []DevfileCommand{
{Name: "run app"},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since the description says run command not present, should this be build app?

emptyApiVersionJson = `{"apiVersion":}`
)

t.Run("valid apiVersion", func(t *testing.T) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we convert these tests to the tabular format we currently use in odo?

@kadel
Copy link
Member

kadel commented Jan 31, 2020

@mik-dass Good points on testing. Are you ok if @kanchwala-yusuf addresses this in followup PR?
I've opened #2553 so it doesn't get lost.

@kadel kadel mentioned this pull request Jan 31, 2020
5 tasks
Copy link
Contributor

@amitkrout amitkrout left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kanchwala-yusuf As this s a feature request, can you please define the acceptance criteria. it would be helpful for us to find out missing integration tests if any.

@mik-dass
Copy link
Contributor

mik-dass commented Feb 3, 2020

@mik-dass Good points on testing. Are you ok if @kanchwala-yusuf addresses this in followup PR?
I've opened #2553 so it doesn't get lost.

@kadel @kanchwala-yusuf Sure. Since #2553 is created to track the issue, I don't mind.

@kanchwala-yusuf
Copy link
Contributor Author

@kanchwala-yusuf As this s a feature request, can you please define the acceptance criteria. it would be helpful for us to find out missing integration tests if any.

@amitkrout , I am not sure if we have a user story for this. User stories started from this sprint. This issue was opened a couple of sprints back.

@kanchwala-yusuf
Copy link
Contributor Author

/reopen

@openshift-ci-robot
Copy link
Collaborator

@kanchwala-yusuf: Reopened this PR.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@kanchwala-yusuf
Copy link
Contributor Author

Closed this by mistake, hence reopend it.

@kadel
Copy link
Member

kadel commented Feb 3, 2020

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. Required by Prow. label Feb 3, 2020
@openshift-merge-robot openshift-merge-robot merged commit 8725bb0 into redhat-developer:master Feb 3, 2020
@cdrage cdrage added the kind/feature Categorizes issue as a feature request. For PRs, that means that the PR is the implementation label Mar 12, 2020
@rm3l rm3l added the estimated-size/XXL (60 - ??) Rough sizing for Epics. More than 3 sprints of work for one person label Jun 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. Required by Prow. estimated-size/XXL (60 - ??) Rough sizing for Epics. More than 3 sprints of work for one person kind/feature Categorizes issue as a feature request. For PRs, that means that the PR is the implementation lgtm Indicates that a PR is ready to be merged. Required by Prow.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement devfile parser
10 participants