Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipeline uses passedConstraints to provide correct inputs and outputs #65

Closed
bobcatfish opened this issue Sep 22, 2018 · 2 comments
Closed
Labels
meaty-juicy-coding-work This task is mostly about implementation!!! And docs and tests of course but that's a given

Comments

@bobcatfish
Copy link
Collaborator

bobcatfish commented Sep 22, 2018

Expected Behavior

If a user creates a Pipeline which has passedConstraints specified, this should be used to construct a DAG of tasks (including fan in and fan out)
2. Ensure that the version of the Resource used by the task is the same as the version used by the Task(s) in the passedConstraint

Should include:

Actual Behavior

After #61 we will have a simple Pipeline in which either the DAG is flat or all Tasks execute simultaneously; i.e. the passedConstraints are ignored.

Steps to Reproduce the Problem

  1. Deploy the Pipeline CRD + related CRDs to your cluster (note: should be docs on how to do this post kubebuilder)
  2. Create a Resource that brings in the hello-world docker image repo, and declares an image we will be building (gcr.io/my-registry must be changed to a registry that the specified serviceAccounts will have permission to push to):
apiVersion: pipeline.knative.dev/v1beta1
kind: Resource
metadata:
  name: resources
  namespace: default
spec:
  resources:
    - name: hello-world-resource
      type: github
      params:
      - name: url
        value: https://github.com/docker-library/hello-world
      - name: revision
        value: master
    - name: hello-world-image
      type: image
      params:
        - name: url
          value: gcr.io/my-registry
  1. Create a two Tasks, the first will build an image (using Kaniko, see Implement working kaniko build Task #62), the second will run the image:
apiVersion: pipeline.knative.dev/v1beta1
kind: Task
metadata:
  name: build-push
  namespace: default
spec:
    inputs:
        resources:
        - name: workspace
          type: github
        params:
            - name: pathToDockerFile
              value: string
    outputs:
        resources:
        - name: imageToBuild
          type: image
    buildSpec:
      steps:
         - name: build-and-push
           image: gcr.io/kaniko-project/executor
      args:
         - --dockerfile=${DOCKERFILE}
         - --destination=${IMAGE}
apiVersion: pipeline.knative.dev/v1beta1
kind: Task
metadata:
  name: run
  namespace: default
spec:
    inputs:
        resources:
        - name: imageToRun
          type: image
    buildSpec:
      steps:
         - name: run-image
           image: "${imageToRun.url}/${imageToRun.name}"
  1. Create a Pipeline which wires the two Tasks together using passedConstraints:
apiVersion: pipeline.knative.dev/v1beta1
kind: Pipeline
metadata:
  name: hello-world-pipeline
  namespace: default
spec:
    tasks:
        - name: build-hello-world
          taskRef:
              name: build-push
          inputSourceBindings:
          - key: workspace
            resourceRef:
                 name: hello-world-resource
          outputSourceBindings:
          - key: imageToBuild
             resourceRef:
                 name: hello-world-image
        - name: run-hello-world
          taskRef:
              name: run
          inputSourceBindings:
          - name: imageToRun
             resourceRef:
                 name: hello-world-image
             passedConstraints: ['build-hello-world']
  1. Create a PipelineRun which invokes the Tasks (you'll have to create a PipelineParams as well with your serviceAccount, in this case called hello-world-params:
apiVersion: pipeline.knative.dev/v1beta1
kind: PipelineRun
metadata:
  name: pipeline-run-1
  namespace: default
spec:
    pipelineRef:
        name: hello-world-pipeline
    resourceRef:
        name: hello-world-resource
    pipelineParamsRef:
        name: hello-world-params
    triggerRef:
        type: manual
  1. Creating the PipelineRun should have created TaskRuns with all of the values filled in, so with kubectl get taskRuns you should be able to see conditions updated appropriately

  2. Looking at the Task logs (if this has been implemented, see Implement working "hello world" Task + TaskRun #59) or at the logs for the running pod if not, you should be able to see that the hello world image was run and output the expected value from the hello world image

  3. You should be able to see hello-world image built and pushed to your registry.

Additional Info

@bobcatfish bobcatfish added this to the Mid October Demo milestone Sep 22, 2018
@aaron-prindle aaron-prindle removed this from the Mid October Demo milestone Oct 10, 2018
bobcatfish referenced this issue in bobcatfish/pipeline Oct 11, 2018
PipelineRun status will be based on the condition of the TaskRuns which
it has created, for #61. If any TaskRuns have failed, the PipelineRun
has failed. If all are successful, it is successful. If any are in
progress, it is in progress.

This is assuming a linear Pipeline, we will have to tweak this a bit
when we implement the graph (for #65)
@tejal29 tejal29 self-assigned this Oct 11, 2018
bobcatfish referenced this issue in bobcatfish/pipeline Oct 11, 2018
PipelineRun status will be based on the condition of the TaskRuns which
it has created, for #61. If any TaskRuns have failed, the PipelineRun
has failed. If all are successful, it is successful. If any are in
progress, it is in progress.

This is assuming a linear Pipeline, we will have to tweak this a bit
when we implement the graph (for #65)
@bobcatfish bobcatfish added the meaty-juicy-coding-work This task is mostly about implementation!!! And docs and tests of course but that's a given label Oct 12, 2018
bobcatfish referenced this issue in bobcatfish/pipeline Oct 12, 2018
PipelineRun status will be based on the condition of the TaskRuns which
it has created, for #61. If any TaskRuns have failed, the PipelineRun
has failed. If all are successful, it is successful. If any are in
progress, it is in progress.

This is assuming a linear Pipeline, we will have to tweak this a bit
when we implement the graph (for #65)
knative-prow-robot pushed a commit that referenced this issue Oct 13, 2018
PipelineRun status will be based on the condition of the TaskRuns which
it has created, for #61. If any TaskRuns have failed, the PipelineRun
has failed. If all are successful, it is successful. If any are in
progress, it is in progress.

This is assuming a linear Pipeline, we will have to tweak this a bit
when we implement the graph (for #65)
@bobcatfish
Copy link
Collaborator Author

Create a Pipeline which wires the two Tasks together using passedConstraints:

lol the passedConstraints - literally the most important part of this task - are missing from the example! 😅

@bobcatfish
Copy link
Collaborator Author

I think that between #168 (remove default linear execution) and #148 (link outputs and inputs) this issue will be taken care of, resolving as duplicate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
meaty-juicy-coding-work This task is mostly about implementation!!! And docs and tests of course but that's a given
Projects
None yet
Development

No branches or pull requests

3 participants