Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fuzz and historically-guided tests #43

Open
leighmcculloch opened this issue May 11, 2022 · 2 comments
Open

Add fuzz and historically-guided tests #43

leighmcculloch opened this issue May 11, 2022 · 2 comments

Comments

@leighmcculloch
Copy link
Member

Requirements:

  • Test both valid and invalid types.
  • Cover major types used by a large number of applications like Transaction, TransactionResult, TransactionMeta, as well as objs critical to core (that is probably all of them), but at least LedgerEntry and related types and meta types.
  • Reasonably realistic entries.
  • Test against a set of invariants expected to be true.
@leighmcculloch
Copy link
Member Author

Ideas:

  • Typical code-coverage guided fuzz testing to identify:
    • Crashes.
    • Invariants.
  • Use proptest to hand craft some tests of major types such as Transaction.
  • Look at randxdr which is used to fuzz test the Go Stellar XDR impl, and possibly:
    • Adopt the same pattern here from scratch.
    • Or, add a CLI build to randxdr that outputs random xdr to stdout, which can be piped into tests here.
  • So something similar to Horizon's verify-range testing, where ledgers and data are pulled from Stellar archives and run through this lib and a set of invariants are checked. This would be different to what Horizon does but possibly:
    • Pick random transactions from archives.
    • Pick random ledgers and try parsing the entire ledger.

@leighmcculloch leighmcculloch changed the title Add fuzz testing Add fuzz and historically-guided tests May 11, 2022
@leighmcculloch
Copy link
Member Author

leighmcculloch commented May 11, 2022

@MonsieurNicolas @graydon Which quarter-end do we need this lib to be stable / well tested? Q4?

@leighmcculloch leighmcculloch changed the title Add fuzz and historically-guided tests Tests: Add fuzz and historically-guided tests Jul 6, 2022
@leighmcculloch leighmcculloch changed the title Tests: Add fuzz and historically-guided tests Add fuzz and historically-guided tests Jul 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants
@leighmcculloch and others