Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(test): instructions to run regression test suites locally #1234

Merged
merged 2 commits into from
May 26, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docker/tester/root/usr/bin/usage
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ How to use.

How to build.

* cd docker/builder && DOCKER_BUILDKIT=1 docker build -t falcosecurity/falco-tester .
* cd docker/tester && DOCKER_BUILDKIT=1 docker build -t falcosecurity/falco-tester .

Environment.

Expand Down
35 changes: 34 additions & 1 deletion test/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,39 @@
# Falco Regression tests
# Falco regression tests

This folder contains the Regression tests suite for Falco.

You can find instructions on how to run this test suite on the Falco website [here](https://falco.org/docs/source/#run-regression-tests).

## Test suites

- [falco_tests](./falco_tests.yaml)
- [falco_traces](./falco_traces.yaml)
- [falco_tests_package](./falco_tests_package.yaml)
- [falco_k8s_audit_tests](./falco_k8s_audit_tests.yaml)
- [falco_tests_psp](./falco_tests_psp.yaml)

## Running locally

Using `virtualenv` the steps to locally run a specific test suite are the following ones (from this directory):

```console
virtualenv venv
source venv/bin/activate
pip install -r requirements.txt
BUILD_DIR="../build" avocado run --mux-yaml falco_tests.yaml --job-results-dir /tmp/job-results -- falco_test.py
deactivate
```

The name of the specific test suite to run is `falco_tests.yaml` in this case. Change it to run others test suites.

In case you want to only execute a specific test case, use the `--mux-filter-only` parameter as follows:

```console
BUILD_DIR="../build" avocado run --mux-yaml falco_tests.yaml --job-results-dir /tmp/job-results --mux-filter-only /run/trace_files/program_output -- falco_test.py
```

To obtain the path of all the available variants, execute:

```console
avocado variants --mux-yaml falco_test.yaml
```
8 changes: 4 additions & 4 deletions test/confs/program_output.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright (C) 2019 The Falco Authors.
# Copyright (C) 2020 The Falco Authors.
#
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand All @@ -21,14 +21,14 @@ rules_file: /etc/falco_rules.yaml
# Whether to output events in json or text
json_output: false

# Send information logs to stderr and/or syslog Note these are *not* security
# notification logs! These are just Falco lifecycle (and possibly error) logs.
# Send information logs to stderr and/or syslog
# Note these are *not* security notification logs!
# These are just Falco lifecycle (and possibly error) logs.
log_stderr: false
log_syslog: false

# Where security notifications should go.
# Multiple outputs can be enabled.

syslog_output:
enabled: false

Expand Down
11 changes: 11 additions & 0 deletions test/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
avocado-framework==69.0
avocado-framework-plugin-varianter-yaml-to-mux==69.0
certifi==2020.4.5.1
chardet==3.0.4
idna==2.9
pbr==5.4.5
PyYAML==5.3.1
requests==2.23.0
six==1.14.0
stevedore==1.32.0
urllib3==1.25.9
2 changes: 1 addition & 1 deletion userspace/falco/falco.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1203,7 +1203,7 @@ int falco_init(int argc, char **argv)
webserver.start();
}

// grpc server
// gRPC server
if(config.m_grpc_enabled)
{
// TODO(fntlnz,leodido): when we want to spawn multiple threads we need to have a queue per thread, or implement
Expand Down