Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

source/custom: expression based label rules #639

Merged

Conversation

marquiz
Copy link
Contributor

@marquiz marquiz commented Nov 2, 2021

Implement a framework for more flexible rule configuration and matching,
mimicking the MatchExpressions pattern from K8s nodeselector.

The basic building block is MatchExpression which contains an operator
and a list of values. The operator specifies that "function" that is
applied when evaluating a given input agains the list of values.
Available operators are:

  • MatchIn
  • MatchNotIn
  • MatchInRegexp
  • MatchExists
  • MatchDoesNotExist
  • MatchGt
  • MatchLt
  • MatchIsTrue
  • MatchIsFalse

Another building block of the framework is MatchExpressionSet which is a
map of string-MatchExpression pairs. It is a helper for specifying
multiple expressions that can be matched against a set of set of
features.

This patch converts all existing custom rules to utilize the new
expression-based framework.

The PR also contains a separate patch that implement GtLt, an operator for checking that an input (integer) is between two values. The purpose of keeping this patch separate is to demonstrate the addition of new operators

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Nov 2, 2021
@marquiz
Copy link
Contributor Author

marquiz commented Nov 2, 2021

Depends on #604
/hold

@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. labels Nov 2, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: marquiz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 2, 2021
@marquiz
Copy link
Contributor Author

marquiz commented Nov 10, 2021

e2e-tests were failing but fixed, now

@ArangoGutierrez
Copy link
Contributor

e2e-test

Ran 2 of 230 Specs in 26.560 seconds
SUCCESS! -- 2 Passed | 0 Failed | 0 Pending | 228 Skipped
--- PASS: TestE2E (26.58s)
PASS
ok      sigs.k8s.io/node-feature-discovery/test/e2e     26.615s

test

go test ./cmd/... ./pkg/... ./source/...                                                                               
?       sigs.k8s.io/node-feature-discovery/cmd/nfd-master       [no test files]                                        
ok      sigs.k8s.io/node-feature-discovery/cmd/nfd-topology-updater     (cached)                                       
ok      sigs.k8s.io/node-feature-discovery/cmd/nfd-worker       0.009s
?       sigs.k8s.io/node-feature-discovery/pkg/api/feature      [no test files]
?       sigs.k8s.io/node-feature-discovery/pkg/apihelper        [no test files]
?       sigs.k8s.io/node-feature-discovery/pkg/cpuid    [no test files]
ok      sigs.k8s.io/node-feature-discovery/pkg/kubeconf (cached)
?       sigs.k8s.io/node-feature-discovery/pkg/labeler  [no test files]
?       sigs.k8s.io/node-feature-discovery/pkg/nfd-client       [no test files]
ok      sigs.k8s.io/node-feature-discovery/pkg/nfd-client/topology-updater      (cached)
ok      sigs.k8s.io/node-feature-discovery/pkg/nfd-client/worker        3.088s
ok      sigs.k8s.io/node-feature-discovery/pkg/nfd-master       (cached)
?       sigs.k8s.io/node-feature-discovery/pkg/podres   [no test files]
ok      sigs.k8s.io/node-feature-discovery/pkg/resourcemonitor  (cached)
?       sigs.k8s.io/node-feature-discovery/pkg/topologypolicy   [no test files]
?       sigs.k8s.io/node-feature-discovery/pkg/topologyupdater  [no test files]
ok      sigs.k8s.io/node-feature-discovery/pkg/utils    (cached)
?       sigs.k8s.io/node-feature-discovery/pkg/version  [no test files]
ok      sigs.k8s.io/node-feature-discovery/source       0.004s
ok      sigs.k8s.io/node-feature-discovery/source/cpu   (cached)
?       sigs.k8s.io/node-feature-discovery/source/custom        [no test files]
ok      sigs.k8s.io/node-feature-discovery/source/custom/expression     0.078s
?       sigs.k8s.io/node-feature-discovery/source/custom/rules  [no test files]
?       sigs.k8s.io/node-feature-discovery/source/fake  [no test files]
?       sigs.k8s.io/node-feature-discovery/source/iommu [no test files]
ok      sigs.k8s.io/node-feature-discovery/source/kernel        (cached)
ok      sigs.k8s.io/node-feature-discovery/source/local (cached)
?       sigs.k8s.io/node-feature-discovery/source/memory        [no test files]
?       sigs.k8s.io/node-feature-discovery/source/network       [no test files]
ok      sigs.k8s.io/node-feature-discovery/source/pci   (cached)
?       sigs.k8s.io/node-feature-discovery/source/storage       [no test files]
ok      sigs.k8s.io/node-feature-discovery/source/system        (cached)
ok      sigs.k8s.io/node-feature-discovery/source/usb   (cached)

test image =quay.io/eduardoarango/node-feature-discovery:639

@ArangoGutierrez
Copy link
Contributor

/assign

@marquiz
Copy link
Contributor Author

marquiz commented Nov 11, 2021

/unhold

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 11, 2021
Implement a framework for more flexible rule configuration and matching,
mimicking the MatchExpressions pattern from K8s nodeselector.

The basic building block is MatchExpression which contains an operator
and a list of values. The operator specifies that "function" that is
applied when evaluating a given input agains the list of values.
Available operators are:

- MatchIn
- MatchNotIn
- MatchInRegexp
- MatchExists
- MatchDoesNotExist
- MatchGt
- MatchLt
- MatchIsTrue
- MatchIsFalse

Another building block of the framework is MatchExpressionSet which is a
map of string-MatchExpression pairs. It is a helper for specifying
multiple expressions that can be matched against a set of set of
features.

This patch converts all existing custom rules to utilize the new
expression-based framework.
A new operator for checking that an input (integer) is between two
values.
@marquiz
Copy link
Contributor Author

marquiz commented Nov 11, 2021

/assign @zvonkok

@zvonkok
Copy link
Contributor

zvonkok commented Nov 12, 2021

Tested and works.

                    feature.node.kubernetes.io/cpu-cpuid.ADX=true
                    feature.node.kubernetes.io/cpu-cpuid.AESNI=true
                    feature.node.kubernetes.io/cpu-cpuid.AVX=true
                    feature.node.kubernetes.io/cpu-cpuid.AVX2=true
                    feature.node.kubernetes.io/cpu-cpuid.AVX512BW=true
                    feature.node.kubernetes.io/cpu-cpuid.AVX512CD=true
                    feature.node.kubernetes.io/cpu-cpuid.AVX512DQ=true
                    feature.node.kubernetes.io/cpu-cpuid.AVX512F=true
                    feature.node.kubernetes.io/cpu-cpuid.AVX512VL=true
                    feature.node.kubernetes.io/cpu-cpuid.AVX512VNNI=true
                    feature.node.kubernetes.io/cpu-cpuid.FMA3=true
                    feature.node.kubernetes.io/cpu-cpuid.HYPERVISOR=true
                    feature.node.kubernetes.io/cpu-cpuid.MPX=true
                    feature.node.kubernetes.io/cpu-hardware_multithreading=true
                    feature.node.kubernetes.io/kernel-config.NO_HZ=true
                    feature.node.kubernetes.io/kernel-config.NO_HZ_FULL=true
                    feature.node.kubernetes.io/kernel-version.full=4.18.0-305.el8.x86_64
                    feature.node.kubernetes.io/kernel-version.major=4
                    feature.node.kubernetes.io/kernel-version.minor=18
                    feature.node.kubernetes.io/kernel-version.revision=0
                    feature.node.kubernetes.io/pci-0300_1d0f.present=true
                    feature.node.kubernetes.io/pci-0302_10de.present=true
                    feature.node.kubernetes.io/storage-nonrotationaldisk=true
                    feature.node.kubernetes.io/system-os_release.ID=ubuntu
                    feature.node.kubernetes.io/system-os_release.VERSION_ID=20.04
                    feature.node.kubernetes.io/system-os_release.VERSION_ID.major=20
                    feature.node.kubernetes.io/system-os_release.VERSION_ID.minor=04

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 12, 2021
@k8s-ci-robot k8s-ci-robot merged commit cfc1c82 into kubernetes-sigs:master Nov 12, 2021
@marquiz marquiz deleted the devel/matchexpression-rules branch November 12, 2021 14:41
@marquiz marquiz mentioned this pull request Dec 22, 2021
22 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants