Array to use in matrix

[tcarac]

Can it be possible to return an array as an input to use in a matrix ?

  build:
    strategy:
      fail-fast: true
      matrix: 
        package: {{steps.filter.outputs}} //This is not working cause is not an array but a map ?
    if: ${{ matrix.package == 'true' }}
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: dorny/paths-filter@v2
        id: filter
        with:
          base: 'main'
          filters: |
            shared:
              - 'packages/shared/**'
            template:
              - 'packages/template/**'

Im getting a cannot unmarshal !!map into []interface {}

[dorny]

I think steps.<step-ID>.outputs will be always a map. Each step can set some output variables and all output variables are then available in that .outputs map. In this specific case it will be like this: { shared: true|false, template: true|false }.

From your example I'm not sure what exactly you are trying to do.
Could you please describe it a little more?

[tcarac]

Sure, im trying to run different steps in a matrix.
Matrix variables needs to be an array to re run the same steps with different parameters.
So in my ideal this would be

 build:
    strategy:
      fail-fast: true
      matrix: 
        package: ['shared', 'app1'] // Here
    if: ${{ matrix.package == 'true' }}
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: dorny/paths-filter@v2
        id: filter
        with:
          base: 'main'
          filters: |
            shared:
              - 'packages/shared/**'
            template:
              - 'packages/template/**'

If filter returned an array instead of a map this could be reused for each path that changed in my monorepo.

[dorny]

Ok I think I understand now. You have one job definition that you would like to run as a matrix for every package where some file was modified. Quite interesting use-case.

I have two solutions, quite cumbersome, but they work with current version of this action. I have also proposal for better solution but I will add it as a separate comment later.

1. Run build action for every package and do the change detection inside:

jobs:
  build:
    strategy:
      fail-fast: true
      matrix:
        package: ['shared', 'template' ] # run job for every package (not very efficient)
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: dorny/paths-filter@v2 # dynamically construct path filter based on current package
      id: filter
      with:
        filters: |
          changed:
            - 'packages/${{matrix.package}}/**'

    - if: steps.filter.outputs.changed == 'true' # execute this step only if there was a change
      name: 'Do something'
      run: echo ${{ matrix.package }}

2. Use separate job and some jq magic to get JSON array with names of changed packages

jobs:
  changes:
    runs-on: ubuntu-latest
    outputs:
      packages: ${{ steps.changes.outputs.list }}
    steps:
    - uses: actions/checkout@v2 # for PR not necessary
    - uses: dorny/paths-filter@v2
      id: filter
      with:
        filters: |
          shared:
            - 'packages/shared/**'
          template:
            - 'packages/template/**'
    # Store JSON map with filter results in a file
    - run: echo '${{ toJson(steps.filter.outputs) }}' > changes.json
    
    # Use jq to transform changes.json to array of changed packages and set it as list output variable
    - id: changes
      run: echo "::set-output name=list::$( jq '[ to_entries | .[] | select(.value == "true") | .key ]' changes.json -c )"

  build:
    needs: changes
    strategy:
      fail-fast: true
      matrix:        
        package: ${{ fromJson(needs.changes.outputs.packages) }}
    runs-on: ubuntu-latest
    steps:
      - name: 'Do something'
        run: echo ${{ matrix.package }}

Note:
You can't use the if condition on a build job. It is evaluated for whole job, not individual "instances" of matrix. Therefore the matrix variable is not available in the if expression.

[dorny]

To make it easier I could simply extend this action with changes output variable which would contain names of all matched filters as JSON array. Effectively it would have same value as you would get with jq in the second solution I posted before.

I'm just considering now if I will add it just as a non-breaking change or if I should completely replace the .<filtername> == 'true' syntax. I could then use the <filtername> output variable instead of <filtername_files> to list all matched files.

Please give me some feedback - which version looks better for you?
1.

    - uses: dorny/paths-filter@v2 # current version
      id: filter
      with:
        filters: |
          shared:
            - 'packages/shared/**'
    - if: steps.filter.outputs.shared == 'true'
      run: echo "shared package was changed"

    - uses: dorny/paths-filter@v3 # - this version does not exist yet!
      id: filter
      with:
        filters: |
          shared:
            - 'packages/shared/**'
    - if: contains(fromJson(steps.filter.outputs.changes, 'shared'))
      run: echo "shared package was changed"

[tcarac]

I guess using separate job and some jq magic to get JSON array with names of changed packages does the trick for me.

[akeemjenkins]

I love this feature! This is very useful in managing terraform apply and plan where each directory maps to an AWS account and you only want to run terraform on changed directories. It also allows for writing the terraform job once with automatic job spawning. Thank you!