Code

perf: Implement compiler sharding#202

Code

Merged

willbeason merged 75 commits intoopen-policy-agent:masteropen-policy-agent/frameworks:masterfrom willbeason:mega-1willbeason/frameworks:mega-1Copy head branch name to clipboardMar 15, 2022

+3,756-5,311

Commits

Commits on Mar 2, 2022

• Use constant target name in Driver

  Show description for 6bfbf8c
  

  Will Beason

  committed
  6bfbf8c

• Move externs to Driver

  Show description for a7cf67b
  

  Will Beason

  committed
  a7cf67b

• Use matcherKey type

  Show description for 3d25a5c
  

  Will Beason

  committed
  3d25a5c

• Refactor constraint matcher

  Show description for 0b05bcb
  

  Will Beason

  committed
  0b05bcb

• Move autoreject logic to Go

  Show description for 35c7c44
  

  Will Beason

  committed
  35c7c44

• Add reminders for autorejection responses

  Show description for c9e12ae
  

  Will Beason

  committed
  c9e12ae

• Add constraints to constraintMatcher

  Show description for f3060d7
  

  Will Beason

  committed
  f3060d7

• Move audit to constraint-per-query

  Show description for c77331e
  

  Will Beason

  committed
  c77331e

• Move to one query-per-constraint for Review

  Show description for a79e8d7
  

  Will Beason

  committed
  a79e8d7

• Switch Query functions out

  Show description for 75ca02f
  

  Will Beason

  committed
  75ca02f

• Remove Constraints from Driver

  Show description for bf7b250
  

  Will Beason

  committed
  bf7b250

• Remove need for inits

  Show description for 6626728
  

  Will Beason

  committed
  6626728

• Remove target-specific pathing from Driver

  Show description for 7b8a0bd
  

  Will Beason

  committed
  7b8a0bd

• Add per-template per-target compilers

  Show description for 81e8ec4
  

  Will Beason

  committed
  81e8ec4

• Wire up sharded compilers

  Show description for aee67be
  

  Will Beason

  committed
  aee67be

• Shard the compiler

  Show description for 460fe47
  

  Will Beason

  committed
  460fe47

• Start fixing local tests

  Show description for 420db85
  

  Will Beason

  committed
  420db85

• Make path driver-independent

  Show description for 3b57ee8
  

  Will Beason

  committed
  3b57ee8

• Make keys for storage

  Show description for 9788f28
  

  Will Beason

  committed
  9788f28

• Remove Audit

  Show description for 562e808
  

  Will Beason

  committed
  562e808

• Remove object cache from Client

  Show description for 7cc2f72
  

  Will Beason

  committed
  7cc2f72

• Re-add AddConstraint/RemoveConstraint

  Show description for a696001
  

  Will Beason

  committed
  a696001

• Write Constraints to Driver

  Show description for 80a8399
  

  Will Beason

  committed
  80a8399

• Reference Constraint by Key

  Show description for dddca68
  

  Will Beason

  committed
  dddca68

• Switch query

  Show description for 8e59274
  

  Will Beason

  committed
  8e59274

• Fill in review outside of Rego

  Show description for 2ce4f66
  

  Will Beason

  committed
  2ce4f66

• Remove unnecessary json marshalling from result

  Show description for 5dd39bf
  

  Will Beason

  committed
  5dd39bf

• Prepare for multi-constraint queries

  Show description for 4585802
  

  Will Beason

  committed
  4585802

• Make Query multi-constraint

  Show description for f5c6114
  

  Will Beason

  committed
  f5c6114

• Write review object to data.tmp

  Show description for b3bc325
  

  Will Beason

  committed
  b3bc325

• Execute one query per Template

  Show description for 8ce51ea
  

  Will Beason

  committed
  8ce51ea

• Don't use separate object for Constraint

  Show description for 6386625
  

  Will Beason

  committed
  6386625

• Make constraint map once

  Show description for 7fae109
  

  Will Beason

  committed
  7fae109

• Split compilation logic out of Driver

  Show description for 6f26404
  

  Will Beason

  committed
  6f26404

• Improve documentation

  Show description for cf108bd
  

  Will Beason

  committed
  cf108bd

• Move rego code to Driver

  Show description for 435f89f
  

  Will Beason

  committed
  435f89f

• Improve documentation

  Show description for df42dbb
  

  Will Beason

  committed
  df42dbb

• Pre-parse input for queries

  Show description for 5d87c06
  

  Will Beason

  committed
  5d87c06

• Continue improving documentation

  Show description for 307c3a3
  

  Will Beason

  committed
  307c3a3

• Simplify Client handling of Template artifacts

  Show description for f5ecbd4
  

  Will Beason

  committed
  f5ecbd4

• Continue simplifying Client

  Show description for 4e2b896
  

  Will Beason

  committed
  4e2b896

• Move per-template logic to own file

  Show description for 848ef16
  

  Will Beason

  committed
  848ef16

• Add special-casing for autoreject

  Show description for 07b4f42
  

  Will Beason

  committed
  07b4f42

• Add test cases for autorejection

  Show description for 7fe2491
  

  Will Beason

  committed
  7fe2491

• Refine Client's usage of mutex

  Show description for c7370ca
  

  Will Beason

  committed
  c7370ca

• Continue working on synchronization

  Show description for 95c4a7f
  

  Will Beason

  committed
  95c4a7f

• Simplify locking.

  Show description for aab498d
  

  Will Beason

  committed
  aab498d

• Continue adding tests

  Show description for 3e5e7ad
  

  Will Beason

  committed
  3e5e7ad

• Re-add AddConstraint tests for Driver

  Show description for 94b7512
  

  Will Beason

  committed
  94b7512

• Remove unnecessary changes

  Show description for dc98379
  

  Will Beason

  committed
  dc98379

• golangci-lint run --fix

  Show description for 05dd767
  

  Will Beason

  committed
  05dd767

• Correct locking comments

  Show description for 2c82d83
  

  Will Beason

  committed
  2c82d83

Commits on Mar 3, 2022

• Respond to reviewer comments

  Show description for e261275
  

  Will Beason

  committed
  e261275

• Don't return error for deleting Constraint if Template deleted

  Show description for 75b7483
  

  Will Beason

  committed
  75b7483

Commits on Mar 4, 2022

• Prevent concurrent writes for Client

  Show description for 9088586
  

  Will Beason

  committed
  9088586

• Add clarification about DeepCopying Constraints

  Show description for 1f3375d
  

  Will Beason

  committed
  1f3375d

Commits on Mar 7, 2022

• Start resolving referential constraints problem

  Show description for 4ea5d9a
  

  Will Beason

  committed
  4ea5d9a

Commits on Mar 8, 2022

• Add e2e tests for referential Constraints

  Show description for b782596
  

  Will Beason

  committed
  b782596

• Fix per-target referential Constraints

  Show description for 5589236
  

  Will Beason

  committed
  5589236

• Remove unnecessary lock

  Show description for 3557527
  

  Will Beason

  committed
  3557527

• Remove unnecessary print statements

  Show description for 6ca6570
  

  Will Beason

  committed
  6ca6570

• Reverse inventory and target in path

  Show description for e3121f1
  

  Will Beason

  committed
  e3121f1

• Stuff

  Show description for e5edbcd
  

  Will Beason

  committed
  e5edbcd

Commits on Mar 9, 2022

• Use "external" instead of "inventory" for data

  Show description for 85a2824
  

  Will Beason

  committed
  85a2824

• Remove unnecessary print statements

  Show description for d1daf1a
  

  Will Beason

  committed
  d1daf1a

• Improve Review performance by 5-10%

  Show description for 315aebd
  

  Will Beason

  committed
  315aebd

• Remove map traversal for inventory

  Show description for beb9ef0
  

  Will Beason

  committed
  beb9ef0

• Use mutexes to guard storage maps

  Show description for 83152af
  

  Will Beason

  committed
  83152af

• Add docs

  Show description for e7ccd2f
  

  Will Beason

  committed
  e7ccd2f

Commits on Mar 10, 2022

• Don't check for directory in storage

  Show description for e9f68f1
  

  Will Beason

  committed
  e9f68f1

• Fix locking in Driver

  Show description for 3eea979
  

  Will Beason

  committed
  3eea979

Commits on Mar 14, 2022

• Move storage logic to own class

  Show description for 5084c7b
  

  Will Beason

  committed
  5084c7b

• Make locking in Driver robust

  Show description for 1db6737
  

  Will Beason

  committed
  1db6737

• Resolve reviewer comment

  Show description for 78c2b1d
  

  Will Beason

  committed
  78c2b1d

• Resolve reviewer comments

  Show description for 2d1a214
  

  Will Beason

  committed
  2d1a214