Partial evaluation optimization for production

[iamatwork]

Hello, I was wondering, if the optimization feature should be used for production setups?

At first it failed, because the built bundle's REGO-syntax was broken (v.0.34). Not optimized bundle worked fine, but with -O2 it failed to load in OPA. Generated files contained function calls without closing bracket and missing parameters.

Now I'm not sure, if it is a good idea to use it at all.

Now my implementation is mostly done, and I tried again (v0.35). The bundle loads, but doesn't produce the same result as the not optimized one. I can't share my code, but this simplified example:

certificate_user = cn_value {
	input.requestContext.securityProtocol == "SSL"

	subject := input.requestContext.principal.name
	regex.match("^doesntmatter,CN=[a-z0-9.]+,morestuffhere$", subject)
	cn_part := split(subject, ",")[1]
	regex.match("^CN=[a-z0-9]+$", cn_part)
	cn_value := trim_prefix(cn_part, "CN=")
}

allow {
  some_rule_that_returns_the_accessed_topics_name = data.users[certificate_user].topics[_]
}

allow {
	input.action.operation = "DESCRIBE"
	input.action.resourcePattern.resourceType = "TOPIC"
	input.action.resourcePattern.patternType = "LITERAL"
	regex.match(
		input.action.resourcePattern.name,
		"^[a-zA-Z0-9_-]{3,249}$",
	)
	input.requestContext.securityProtocol = "SSL"

	regex.match(
		input.requestContext.principal.name,
		"^X=\\d+,CN=[a-z0-9.]+,morestuffhere$"
	)
	split(input.requestContext.principal.name, ",", __local29__1428)
	regex.match("^CN=[a-z0-9]+$", __local29__1428[1])
	trim_prefix(__local29__1428[1], "CN=", "someusername")

	"accessed_topic_name" = input.action.resourcePattern.name
}

Such a block is generated for each user and their topics.

This looks like assignments from function calls, get rewritten to have the assigned variable as additional parameter (See regex.match)? I did not find anything about that in the documentation.

But I'm confused by the trim_prefix having a string as 3rd parameter? Is that evaluated as equality comparison? Because all requests are rejected with optimization. Original code was cn_value := trim_prefix(cn_part, "CN=")

[srenatus]

Hi there! Thanks for the detailed question.

if the optimization feature should be used for production setups?

It's not experimental or work in progress, and any results of optimization leading to different output are bugs. Under the hood, it's using partial evaluation, which is also a bit of an ancient feature in OPA.

All that said, there are certainly coverage blinds spots in our testing process when it comes to optimized bundles, which we should fix. (Contributions would be welcome as always, I think extending TestRego to also have a run with all our yaml tests that optimizes first and then asserts the result.)

This looks like assignments from function calls, get rewritten to have the assigned variable as additional parameter (See regex.match)? I did not find anything about that in the documentation.

This is correct, any n-ary function can be used as f(a1, ..., an, res) and it means the same as res = f(a1, ..., an).

That change alone should not cause your optimized policies to fail, see this example.

You could try extracing the optimized policy code, and run it against your input with tracing enabled to see what exactly goes wrong, maybe? 🔍