OPA v0.34.0 Release

[peteroneilljr]

This release includes a number of enhancements and fixes. In particular, this
release adds a new keyword for membership and iteration (in) and a specialized
built-in function (print) for debugging.

The in operator

This release adds a new in operator that provides syntactic sugar for
references that perform membership tests or iteration on collections (i.e.,
arrays, sets, and objects.) The following table shows common patterns for arrays
with the old and new syntax:

Pattern	Existing Syntax	New Syntax
Check if 7 exists in array	7 == arr[_]	7 in arr
Check if 7 does not exist in array	n/a (requires helper rule)	not 7 in arr
Iterate over the elements of array	x := arr[_]	some x in arr

For more information on the in operator see Membership and iteration: in
in the docs.

The print function

This release adds a new print function for debugging purposes. The print
function can be used to output any value inside of the policy. The print
function has special handling for undefined values so that execution does not
stop if any of the operands are undefined. Instead, a special marker is emitted
in the output. For example:

package example

default allow = false

allow {
  print("the subject's username is:", input.subject.username)
  input.subject.username == "admin"
}

Given the policy above, we can see the output of the print function via STDERR when using opa eval:

echo '{"subject": {"username": "admin"}}' | opa eval -d policy.rego -I -f pretty 'data.example.allow'

Output:

the subject's username is: admin
true

If the username, subject, or entire input document was undefined, the print function will still execute:

echo '{}' | opa eval -d policy.rego -I -f pretty 'data.example.allow'

Output:

the subject's username is: <undefined>
false

The print function is integrated into the opa subcommands, REPL, server, VS
Code extension, and the playground. Library users must opt-in to print
statements. For more information see the Debugging section in the docs.

Enhancements

• SDK: Allow map of plugins to be passed to SDK (#3826) authored by @edpaget
• opa test: Change exit status when tests are skipped (#3773) authored by @kirk-patton
• Bundles: Improve loading performance (#3860) authored by @0xAP
• opa fmt: Keep new lines in between function arguments (#3836) reported by @anbrsap
• opa inspect: Add experimental subcommand for bundle inspection (#3754)

Fixes

• Bundles/API: When deleting a policy, the check determining if it's bundle-owned was using the path prefix, which would yield false positives under certain circumstances.
  It now checks the path properly, piece-by-piece. (#3863 authored by @edpaget

• CLI: Using --set with null value again translates to empty object (#3846)

• Rego: Forbid dynamic recursion with hidden (system.*) document (#3876

• Rego: Raise conflict errors in functions when output not captured (#3912)

  This change has the potential to break policies that previously evaluated successfully!
  See Backwards Compatibility notes below for details.

• Experimental disk storage: React to "txn too big" errors (#3879), reported and authored by @floriangasc

Documentation

• Kubernetes and Istio: Update tutorials for recent Kubernetes versions (#3910) authored by @olamiko
• Deployment: Add section about Capabilities (#3769)
• Built-in functions: Add warning to http.send and extension docs about side-effects in other systems (#3922) (#3893)
• Docker Authorization: The tutorial now uses a Bundles API server.
• SDK: An example of SDK use is provided.

Miscellaneous

• Runtime: Refactor logger usage -- see below for Backwards Compatibility notes.
• Wasm: fix an issue with undefined, plain input references (#3891)
• test/e2e: Extend TestRuntime to avoid global fixture
• types: Fix Arity function to return zero when type is known (#3932)
• Wasm/builder: bump LLVM to 13.0.0, latest versions of wabt and binaryen (#3908)
• Wasm: deal with importing memory in the compiler (#3763)

Backwards Compatibility

• Function return values need to be well-defined: for a single input x, the function's
  output f(x) can only be one value. When evaluating policies, this condition had not
  been ensured for function calls that don't make use of their values, like

  package p
  r {
      f(1)
  }
  f(_) = true
  f(_) = false

  Before, data.p.r evaluated to true. Now, it will (correctly) return an error:

  eval_conflict_error: functions must not produce multiple outputs for same inputs
  

  In more realistic settings, this can be encountered when true/false return values
  are captured and returned where they don't need to be:

  package p
  r {
      f("any", "baz")
  }
  f(path, _) = r {
      r := path == "any"
  }
  f(path, x) = r {
      r := glob.match(path, ["/"], x)
  }

  In this example, any function input containing "any" would make the function yield
  two different results:

  1. The first function body returns true, matching the "any" argument.
  2. The second function body returns the result of the glob.match call -- false.

  The fix here would be to not capture the return value in the function bodies:

  f(path, _) {
      path == "any"
  }
  f(path, x) {
      glob.match(path, ["/"], x)
  }

• The github.com/open-policy-agent/opa/runtime#NewLoggingHandler function now
  requires a logger instance. Requiring the logger avoids the need for the
  logging handler to depend on the global logrus logger (which is useful for
  test purposes.) This change is unlikely to affect users.