Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Package invariants #810

Open
wants to merge 71 commits into
base: master
Choose a base branch
from
Open

Package invariants #810

wants to merge 71 commits into from

Conversation

jcp19
Copy link
Contributor

@jcp19 jcp19 commented Dec 17, 2024

Changelog:

  • we introduce the notion of package invariants to describe the contents of global state (we distinguish between non-duplicable and duplicable invariants - more details to follow soon). We introduce the statement openDupPkgInv to open the duplicable invariants of the current package if initialization is guaranteed to be finished, and we introduce all necessary proof obligations for the init functions and main function.
  • we introduce a notion of a friendPkg, i.e., a package to which we may send resources after initialization is finished. These resources are made available as a precondition to the initializer of the selected package (no other package may access these resources). Due to some challenges with resolving paths to imported packages in imported packages, this feature is still heavily configuration dependent and may break easily, and thus, it is still experimental (I opened an issue to keep track of its evolution - Stabilize support for friend clauses #830)
  • we introduce the notion of functions that may be called during initialization (marked with mayInit) to make sure we never open invariants that might not hold yet - more details on this soon
  • we deprecate initEnsures clauses to allow for modularly checking a package. We also deprecate the --lazyImports flag, as it is no longer necessary: If the package under verification does not declare any package invariants nor initRequires clauses, then nothing is checked for that package. Because our methodology is now modular, we do not have to re-check initialization of all imported packages.
  • we make the Desugarer sequential again to fix the regression reported in Regression same_package/import-fail01/main fails non-deterministically #762

Minor TODOs:

  • protect the use of friend clauses under a feature flag --enableFriendClauses
  • add more tests

@jcp19 jcp19 mentioned this pull request Dec 17, 2024
@jcp19 jcp19 changed the title Package invariants [WIP] Package invariants Dec 17, 2024
@jcp19 jcp19 linked an issue Jan 6, 2025 that may be closed by this pull request
@jcp19 jcp19 linked an issue Jan 19, 2025 that may be closed by this pull request
@jcp19 jcp19 marked this pull request as ready for review January 30, 2025 18:19
@jcp19 jcp19 requested a review from ArquintL January 31, 2025 10:04
Copy link
Member

@ArquintL ArquintL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the first batch of comments and I still need to look at the desugarer

Comment on lines 3460 to 3465
// Check that the initialization code satisfies the contract of every file in the package.
val globalDecls = sortedGlobalVariableDecls(mainPkg)
val fileInitTranslations: Vector[in.Function] = mainPkg.programs.zip(globalDecls).map {
case (p, sortedDecls) => checkProgramInitContract(p)(sortedDecls)
}
fileInitTranslations.foreach(AdditionalMembers.addMember)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this.

  • why are the global decls need to be sorted? Or does this return the global decls per file such that they can get zipped on the next line?
  • what exactly is checked? That the per-file global vars are sufficient to satisfy the (generated) per-file init precondition and all postconditions of init functions of the package imply the package invariant?

Copy link
Contributor Author

@jcp19 jcp19 Feb 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • in go, globals are initialized according to a dependency order (variables initialize in lexicographic order, when the variables used in the rhs of the declaration are already initialize; cyclic dependencies are disallowed). Right now, we check that variables are declared in dependency order and we need to get them in the order they execute so that the order of effectful operations to match that of the go compiler
  • regarding the second question, yes (in addition, we may assume the invariants of other imported packages, as long as they have been reestablished by the end of the initialization)

Copy link
Member

@ArquintL ArquintL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

final batch of comments

* it is ok to check the initialization of a package by separately checking the initialization of each of
* its programs.
* @param p
* @return
*/
def checkProgramInitContract(p: PProgram)(sortedGlobVarDecls: Vector[in.GlobalVarDecl]): in.Function = {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is sortedGlobVarDecls a parameter? Couldn't we invoke sortedGlobalVariableDecls on p (instead of PPackage) in here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sortedGlobalVariableDecls returns a vector of sorted declarations (containing one element per file in the package). In this function, we pass the list of sorted files for the corresponding file (i.e., a PProgram)

Comment on lines 3669 to 3670
val pkgInvariantsImportedPackages: Vector[in.Assertion] =
initSpecs.getNonDupPkgInvariants().filter{ case (k, _) => k != currPkg }.values.flatten.toVector
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this. Doesn't initSpecs collect all package invariants? Why is it sufficient to filter out the invariants of this package? It seems that pkgInvariantsImportedPackages will contain the invariants of transitively imported packages. Is this desirable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, this collects all package invariants, even of those packages that are transitively imported. This does not impact soundness, but I agree that it is suboptimal. I changed this to filter only by packages that are directly imported

* @return
*/
def generateMainFuncProofObligation(mainPkg: PPackage, initSpecs: PackageInitSpecCollector): Option[in.Function] = {
val mainFuncOpt = mainPkg.declarations.collectFirst {
case f: PFunctionDecl if f.id.name == Constants.MAIN_FUNC_NAME => f
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this also that into account that the package name is "main"? Go playground allows to write functions called "main" in non-main packages

Comment on lines +5172 to +5187
private var nonDupPkgInvs: Map[PPackage, Vector[in.Assertion]] = Map.empty
private var dupPkgInvs: Map[PPackage, Vector[in.Assertion]] = Map.empty

// Register the invariants of pkg. Each invariant is accompanied by a bool that
// is true iff the invariants are meant to be duplicable
def registerPkgInvariants(pkg: PPackage, dInvs: Vector[(in.Assertion, Boolean)]) = {
assert(!nonDupPkgInvs.contains(pkg) && !dupPkgInvs.contains(pkg))
val (dups, nonDups) = dInvs.partitionMap { inv =>
if (inv._2) Left(inv._1) else Right(inv._1)
}
dupPkgInvs += pkg -> dups
nonDupPkgInvs += pkg -> nonDups
}

def getNonDupPkgInvariants(): Map[PPackage, Vector[in.Assertion]] = nonDupPkgInvs
def getDupPkgInvariants(): Map[PPackage, Vector[in.Assertion]] = dupPkgInvs
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it be sufficient to simply store PPackages and compute (& if necessary memoize) nonDupPkgInvs and dupPkgInvs? It seems this would result in a much simpler interface and also make clients easier to read by encapsulating more functionality in PackageInitSpecCollector

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this question is very related to the first one (if we move registerPkgInitData to the PackageInitSpecCollector class, we end up effectively doing this). Once again, the reason I didn't do this is that I wanted PackageInitSpecCollector to act only as a store of simple metadata, instead of having more complicated desugaring logic here.

@jcp19 jcp19 requested a review from ArquintL March 1, 2025 19:31
@jcp19
Copy link
Contributor Author

jcp19 commented Mar 5, 2025

@ArquintL I have addressed your comments, please take a look at this when you find the time

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants