Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add statestore test helpers and unit tests #19227

Merged
merged 2 commits into from
Jun 17, 2020

Conversation

urso
Copy link

@urso urso commented Jun 16, 2020

  • Enhancement

What does this PR do?

This change introduces the libbeat/statestore/storetest package and unit
tests to the statestore frontend itself.
The storetest package provides helpers for writing tests. For example
does it emulate a key value store in memory by storing all k/v-pairs
in a map[string]interface{}, that can optionally provided by users.
The internal storecompliance test-suite is used to validate the
storetest package to be fully compatible with the statestore
requirements.

The addition of the statestore package is split up into multiple
changeset to ease review. The final version of the package can be found
here.

Once finalized, the libbeat/statestore package contains:

  • The statestore frontend and interface for use within Beats
  • Interfaces for the store backend
  • A common set of tests store backends need to support
  • a storetest package for testing new features that require a store. The
    testing helpers use map[string]interface{} that can be initialized or
    queried after the test run for validation purposes.
  • The default memlog backend + tests

This change introduces the second last item to libbeat: test helpers and
additional unit tests.

Test coverage of statetest package:

$ gocov test | gocov report
ok  	github.com/elastic/beats/v7/libbeat/statestore/storetest	0.004s	coverage: 90.0% of statements

github.com/elastic/beats/v7/libbeat/statestore/storetest/storetest.go	 MemoryStore.Access	 100.00% (9/9)
github.com/elastic/beats/v7/libbeat/statestore/storetest/storetest.go	 MapStore.Close		 100.00% (4/4)
github.com/elastic/beats/v7/libbeat/statestore/storetest/storetest.go	 MapStore.IsClosed	 100.00% (3/3)
github.com/elastic/beats/v7/libbeat/statestore/storetest/storetest.go	 MapStore.Reopen	 100.00% (3/3)
github.com/elastic/beats/v7/libbeat/statestore/storetest/storetest.go	 MemoryStore.init	 100.00% (2/2)
github.com/elastic/beats/v7/libbeat/statestore/storetest/storetest.go	 MapStore.init		 100.00% (2/2)
github.com/elastic/beats/v7/libbeat/statestore/storetest/storetest.go	 MemoryStore.Close	 100.00% (1/1)
github.com/elastic/beats/v7/libbeat/statestore/storetest/storetest.go	 NewMemoryStoreBackend	 100.00% (1/1)
github.com/elastic/beats/v7/libbeat/statestore/storetest/storetest.go	 CreateValueDecoder	 100.00% (1/1)
github.com/elastic/beats/v7/libbeat/statestore/storetest/storetest.go	 valueUnpacker.Decode	 100.00% (1/1)
github.com/elastic/beats/v7/libbeat/statestore/storetest/storetest.go	 MapStore.Each		 90.00% (9/10)
github.com/elastic/beats/v7/libbeat/statestore/storetest/storetest.go	 MapStore.Has		 85.71% (6/7)
github.com/elastic/beats/v7/libbeat/statestore/storetest/storetest.go	 MapStore.Remove	 85.71% (6/7)
github.com/elastic/beats/v7/libbeat/statestore/storetest/storetest.go	 MapStore.Set		 80.00% (8/10)
github.com/elastic/beats/v7/libbeat/statestore/storetest/storetest.go	 MapStore.Get		 77.78% (7/9)
github.com/elastic/beats/v7/libbeat/statestore/storetest		 ---------------------	 90.00% (63/70)

Test coverage of statestore package:

$ gocov test | gocov report
ok  	github.com/elastic/beats/v7/libbeat/statestore	0.006s	coverage: 82.0% of statements

github.com/elastic/beats/v7/libbeat/statestore/store.go		 Store.Has			 100.00% (8/8)
github.com/elastic/beats/v7/libbeat/statestore/store.go		 Store.Get			 100.00% (8/8)
github.com/elastic/beats/v7/libbeat/statestore/store.go		 Store.Remove			 100.00% (7/7)
github.com/elastic/beats/v7/libbeat/statestore/store.go		 Store.Set			 100.00% (7/7)
github.com/elastic/beats/v7/libbeat/statestore/store.go		 Store.Close			 100.00% (6/6)
github.com/elastic/beats/v7/libbeat/statestore/store.go		 Store.Each			 100.00% (4/4)
github.com/elastic/beats/v7/libbeat/statestore/store.go		 sharedStore.Release		 100.00% (3/3)
github.com/elastic/beats/v7/libbeat/statestore/registry.go	 Registry.Close			 100.00% (2/2)
github.com/elastic/beats/v7/libbeat/statestore/store.go		 newStore			 100.00% (2/2)
github.com/elastic/beats/v7/libbeat/statestore/store.go		 sharedStore.Retain		 100.00% (1/1)
github.com/elastic/beats/v7/libbeat/statestore/store.go		 newSharedStore			 100.00% (1/1)
github.com/elastic/beats/v7/libbeat/statestore/registry.go	 NewRegistry			 100.00% (1/1)
github.com/elastic/beats/v7/libbeat/statestore/registry.go	 Registry.Get			 91.67% (11/12)
github.com/elastic/beats/v7/libbeat/statestore/store.go		 sharedStore.tryUnregister	 83.33% (5/6)
github.com/elastic/beats/v7/libbeat/statestore/registry.go	 Registry.unregisterStore	 80.00% (4/5)
github.com/elastic/beats/v7/libbeat/statestore/error.go		 IsClosed			 75.00% (3/4)
github.com/elastic/beats/v7/libbeat/statestore/error.go		 ErrorAccess.Error		 0.00% (0/3)
github.com/elastic/beats/v7/libbeat/statestore/error.go		 ErrorOperation.Error		 0.00% (0/1)
github.com/elastic/beats/v7/libbeat/statestore/error.go		 ErrorOperation.Operation	 0.00% (0/1)
github.com/elastic/beats/v7/libbeat/statestore/error.go		 ErrorOperation.Store		 0.00% (0/1)
github.com/elastic/beats/v7/libbeat/statestore/error.go		 ErrorClosed.Error		 0.00% (0/1)
github.com/elastic/beats/v7/libbeat/statestore/error.go		 ErrorClosed.Operation		 0.00% (0/1)
github.com/elastic/beats/v7/libbeat/statestore/error.go		 ErrorClosed.Store		 0.00% (0/1)
github.com/elastic/beats/v7/libbeat/statestore/error.go		 ErrorOperation.Unwrap		 0.00% (0/1)
github.com/elastic/beats/v7/libbeat/statestore/error.go		 ErrorAccess.Unwrap		 0.00% (0/1)
github.com/elastic/beats/v7/libbeat/statestore/error.go		 ErrorAccess.Store		 0.00% (0/1)
github.com/elastic/beats/v7/libbeat/statestore			 -------------------------	 82.02% (73/89)

Why is it important?

The statestore introduces a simple key-value store to Beats. The statestore will be used to replace the registry in filebeat in the future.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
    - [ ] I have made corresponding changes to the documentation
    - [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
    - [ ] I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

  • [ ]

Related issues

This change introduces the libbeat/statestore/storetest package and unit
tests to the statestore frontend itself.
The storetest package provides helpers for writing tests. For example
does it emulate a key value store in memory by storing all k/v-pairs
in a map[string]interface{}, that can optionally provided by users.
The internal storecompliance test-suite is used to validate the
storetest package to be fully compatible with the statestore
requirements.

The addition of the statestore package is split up into multiple
changeset to ease review. The final version of the package can be found
[here](https://github.com/urso/beats/tree/fb-input-v2-combined/libbeat/statestore).

Once finalized, the libbeat/statestore package contains:
- The statestore frontend and interface for use within Beats
- Interfaces for the store backend
- A common set of tests store backends need to support
- a storetest package for testing new features that require a store. The
  testing helpers use map[string]interface{} that can be initialized or
  queried after the test run for validation purposes.
- The default memlog backend + tests

This change introduces the second last item to libbeat: test helpers and
additional unit tests.
@urso urso added review Filebeat Filebeat libbeat needs_backport PR is waiting to be backported to other branches. Project:Filebeat-Input-v2 Team:Services (Deprecated) Label for the former Integrations-Services team v7.9.0 labels Jun 16, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-services (Team:Services)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jun 16, 2020
@elasticmachine
Copy link
Collaborator

elasticmachine commented Jun 16, 2020

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #19227 updated]

  • Start Time: 2020-06-16T22:01:13.702+0000

  • Duration: 83 min 54 sec

Test stats 🧪

Test Results
Failed 0
Passed 9441
Skipped 1574
Total 11015

Steps errors

Expand to view the steps failures

  • Name: Report to Codecov
    • Description: curl -sSLo codecov https://codecov.io/bash for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat do FILE="${i}/build/coverage/full.cov" if [ -f "${FILE}" ]; then bash codecov -f "${FILE}" fi done

    • Duration: 2 min 22 sec

    • Start Time: 2020-06-16T22:38:45.753+0000

    • log

@urso urso merged commit 372c3ae into elastic:master Jun 17, 2020
@urso urso deleted the statestore-unit-tests branch June 17, 2020 20:37
@urso urso removed the needs_backport PR is waiting to be backported to other branches. label Jul 6, 2020
urso pushed a commit to urso/beats that referenced this pull request Jul 7, 2020
Add statestore test helpers and unit tests

This change introduces the libbeat/statestore/storetest package and unit
tests to the statestore frontend itself.
The storetest package provides helpers for writing tests. For example
does it emulate a key value store in memory by storing all k/v-pairs
in a map[string]interface{}, that can optionally provided by users.
The internal storecompliance test-suite is used to validate the
storetest package to be fully compatible with the statestore
requirements.

The addition of the statestore package is split up into multiple
changeset to ease review. The final version of the package can be found
[here](https://github.com/urso/beats/tree/fb-input-v2-combined/libbeat/statestore).

Once finalized, the libbeat/statestore package contains:
- The statestore frontend and interface for use within Beats
- Interfaces for the store backend
- A common set of tests store backends need to support
- a storetest package for testing new features that require a store. The
  testing helpers use map[string]interface{} that can be initialized or
  queried after the test run for validation purposes.
- The default memlog backend + tests

This change introduces the second last item to libbeat: test helpers and
additional unit tests.

(cherry picked from commit 372c3ae)
urso pushed a commit that referenced this pull request Jul 7, 2020
…#19689)

Add statestore test helpers and unit tests

This change introduces the libbeat/statestore/storetest package and unit
tests to the statestore frontend itself.
The storetest package provides helpers for writing tests. For example
does it emulate a key value store in memory by storing all k/v-pairs
in a map[string]interface{}, that can optionally provided by users.
The internal storecompliance test-suite is used to validate the
storetest package to be fully compatible with the statestore
requirements.

The addition of the statestore package is split up into multiple
changeset to ease review. The final version of the package can be found
[here](https://github.com/urso/beats/tree/fb-input-v2-combined/libbeat/statestore).

Once finalized, the libbeat/statestore package contains:
- The statestore frontend and interface for use within Beats
- Interfaces for the store backend
- A common set of tests store backends need to support
- a storetest package for testing new features that require a store. The
  testing helpers use map[string]interface{} that can be initialized or
  queried after the test run for validation purposes.
- The default memlog backend + tests

This change introduces the second last item to libbeat: test helpers and
additional unit tests.

(cherry picked from commit 372c3ae)
melchiormoulin pushed a commit to melchiormoulin/beats that referenced this pull request Oct 14, 2020
Add statestore test helpers and unit tests

This change introduces the libbeat/statestore/storetest package and unit
tests to the statestore frontend itself.
The storetest package provides helpers for writing tests. For example
does it emulate a key value store in memory by storing all k/v-pairs
in a map[string]interface{}, that can optionally provided by users.
The internal storecompliance test-suite is used to validate the
storetest package to be fully compatible with the statestore
requirements.

The addition of the statestore package is split up into multiple
changeset to ease review. The final version of the package can be found
[here](https://github.com/urso/beats/tree/fb-input-v2-combined/libbeat/statestore).

Once finalized, the libbeat/statestore package contains:
- The statestore frontend and interface for use within Beats
- Interfaces for the store backend
- A common set of tests store backends need to support
- a storetest package for testing new features that require a store. The
  testing helpers use map[string]interface{} that can be initialized or
  queried after the test run for validation purposes.
- The default memlog backend + tests

This change introduces the second last item to libbeat: test helpers and
additional unit tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Filebeat Filebeat libbeat Project:Filebeat-Input-v2 review skip-test-plan Team:Services (Deprecated) Label for the former Integrations-Services team v7.9.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants