Skip to content

Latest commit

 

History

History
1773 lines (1152 loc) · 55.5 KB

CHANGELOG.md

File metadata and controls

1773 lines (1152 loc) · 55.5 KB
Raw

Changelog

Unreleased

1.44.0 - 2024-08-25

### Fixed

  • We have to iterate over all existing distributions in the current environment to hopefully locate the package name of the distribution we are discovering

Changed

  • Bump dependencies

1.43.2 - 2024-05-07

Fixed

  • Remove Z suffix when datetime is timezone aware in discovery payload

1.43.1 - 2024-05-07

Fixed

  • We cannot change all extensions signatures for discovering capabilities so we remove arguments after the collection. Less efficient but more transparent
  • Switched from utcnow() to now(timezone.utc) as utcnow has been deprecated

1.43.0 - 2024-05-07

Changed

  • Switching to importlib.metadata builtin function to reverse lookup the package name (import name) from the distribution name during discovery. This should be more reliable
  • Switched log level to debug during discovery to keep output less noisy
  • Allow to ignore arguments during discovery for a lighter output
  • Update dependencies

1.42.1 - 2024-03-11

Changed

  • Tolerate colorama down to 0.4.4 to satisfy awscli

1.42.0 - 2024-02-20

Changed

  • Moved to PDM aways from setuptools as a more modern and robust approach to manage dependencies
  • Moved to ruff away from flake8 for linter
  • Switched to a Pypi trusted publisher
  • Upgraded pdm and checkout actions to clean out nodejs deprecation warnings
  • Removed logzero dependency
  • Dropped Python 3.7 support as it has been EOL since June 2023
  • Read version from scm and package via importlib_metadata

Fixed

  • Fixed loading of multiple controls from a global control file.

1.41.0 - 2023-11-16

Changed

  • Interrupting a blocking rollback led to not having any rollbacks being available to the journal. Now all rollbacks up to the one that gets interrupted (inclusive) should be un the journal

1.40.0 - 2023-10-28

Changed:

  • When working with jsonpath tolerances and substitutions like ${myvalue} chaostoolkit now gives the user an error message with all substituted values when the expect block does not match the output of the probe.

  • Tolerance field can now be substitued with variables even when the tolerance remains a native type. So this is now supported:

    {
      "configuration": {
          "expected": 3
      },
      "steady-state-hypothesis": {
          "title": "",
          "probes": [
              {
                  "name": "check-stuff",
                  "type": "probe",
                  "tolerance": "${expected}",
                  "provider": {
                      "type": "python",
                      "module": "statistics",
                      "func": "mean",
                      "arguments": {
                          "data": [1, 3, 4, 4]
                      }
                  }
              }
          ]
      }
  }

1.39.0 - 2023-09-12

Changed

  • Dropped chardet and cchardet (optionals dependencies) in favour to the faster and friendly-licenced charset-normalizer which will be installed by default
  • Removed simplejson support entirely as this library doesn't seem to be much updated anymore. Sticking with stdlib json and maybe one day we'll byte the bullet and migrate to orjson
  • Fix different behaviors in checking status from steady-state in Python 3.7

1.38.0 - 2023-08-02

Changed

  • The http activity provider can now substitute its timeout field so it can be read from the configuration
  • The expect field of a jsonpath tolerance can now be substituted with configuration values 267
  • Substitution of list of lists now returns a list of lists instead of flattening

1.37.0 - 2023-07-14

Added

  • A new "skip" strategy to explicitely dismiss the hypothesis entirely
  • A log message when an activity is in dry mode
  • Bump dependencies

1.36.3 - 2023-07-13

Changed

  • When trying to substitute a single value in an argument but that value isn't actually a variable, but instead a string to pass as-is to the function of the activity, we failed with a KeyError because we were looking into the configuration/secrets for a key that was not meant to be there. Now, we properly handle this case and leave the string as it should be. This case may easily happen for process activities where you make a reference to an environment variable when the process is executed. For instance:

    "provider": {
      "type": "process",
      "path": "cat",
      "arguments": "$MY_FILE"
  },

    Here "$MY_FILE" is not a pattern for chaos toolkit to replace with a value, it's a literal string to be passed to the cat command.

1.36.2 - 2023-07-11

Changed

  • Allow for missing environment variable when default property is set, even to a None value

1.36.1 - 2023-07-10

Fixed

  • Converting using JSON is a bypass when value is an empty string or None

1.36.0 - 2023-06-29

Added

  • When declaring an environment variable in a configuration block, you can ask Chaos Toolkit to convert the value read (which is always a string) into an appropriate type: int, float, str, bytes, bool and json

    {
  "my_var": {
      "type": "env",
      "key": "SOME_VAR",
      "env_var_type": "int"
  }
}

1.35.1 - 2023-06-19

Fixed

  • When a var file has no extension, it couldn't be properly parsed because we searched for an empty string into a string sequence, and that was truthey #263

1.35.0 - 2023-06-13

Changed

  • Set the status of the experiment to completed when we deviated but tolerances did not fail. Only set the status to failed if one of the tolerance actually failed to execute. #258
  • The order of applying controls. Before, global controls were applied after experiment controls. This has been changed so global controls get applied first. This was never specified so let me know if you get errors.

1.34.1 - 2023-03-04

Changed

  • Removed restriction on the importlib-metadata package

1.34.0 - 2023-03-04

Changed

  • Use builtin importlib-metadata package for Python 3.8+

1.33.1 - 2023-01-29

Changed

  • Position log correctly in the load_global_controls function

1.33.0 - 2023-01-27

Added

  • Support for extra control files loaded as global controls. A control file contains an object where the keys are control names and their values are control definitions

1.32.1 - 2023-01-25

Changed

  • Switched from a queue to a simple list for tracking method runs internally as the list is thread-safe for insertions anyway

1.32.0 - 2023-01-25

Changed

  • Internal refactoring so that, when a termination signal is received, the journal still contains the trace of runs that whappened so far. This is essential or we were losing information. As this is internal only, this should not have any side effect but, if you see strange behaviors from your runs, please ping us.

1.31.0 - 2023-01-20

Added

  • New start_activity and activity_completed callbackx in the handler registry to wrap the activities when they run.
  • Stop logging from the default handler callbacks

1.30.0 - 2023-01-14

Added

  • A new running callback in the handler registry as the started callback happens too soon in the control lifecycle.

1.29.0 - 2022-09-28

Added

  • A new CHAOSTOOLKIT_LOADER_AUTH_BEARER_TOKEN environment variable used when loading an experiment from http with an authorization bearer token. It takes precedence over the same information in the settings file if it is defined there too.

1.28.0 - 2022-06-15

Changed

  • When env variables are provided, first take their values before loading the configuration. This allows to have different scenarios stored outside the experiment itself. #252
  • Allow to substitute pauses values with configuration or secret values

1.27.3 - 2022-03-31

Changed

  • Ensure experiment level controls are only played once

1.27.2 - 2022-03-23

Changed

  • Ensure experiment level controls are applied to activity level controls when activity declares a control too #251

1.27.1 - 2022-02-25

Changed

  • Ensure dynamic configuration can reference itself #250

1.27.0 - 2022-02-17

Added

  • chaoslib.canonical_json that returns a fairly canoniocal view of the experiment
  • chaoslib.experiment_hash which builds a hash of the experiment's canonical json view
  • When no steady-state hypothesis strategy was provided, we use the default one

1.26.0 - 2022-02-16

Changed

  • Pass the event registry to the configure_control control if requested through event_registry argument. This allows to add your own event handler to the main registry. Mostly for quite advanced cases only.

1.25.0 - 2022-01-05

Changed

  • Requires Python 3.7 as a baseline as Python 3.6 is now EOL

Added

  • Added Python 3.10 support

1.24.0 - 2021-12-20

Added

  • configuration.py now have the ability to have load a dynamic configurations. The dynamic config is a regular activity (probe) in the configuration section. If there's a use-case for setting a configuration dynamically right before the experiment is starting. It executes the probe, and then the return value of this probe will be the config you wish to set.

1.23.1 - 2021-12-02

Added

  • Configuration now supports providing a type to convert environment variables to a provided type. Addressing this issue

    To specifically convert an environment variable loaded into configuration to a type, provide the env_var_type key in the same block as the "env" declaration. Supported values are: str, int, float, bytes.

1.23.0 - 2021-11-05

Fixed

  • notifications.py now uses PayloadEncoder to serialise datetime objects and Exception objects for when using notify_with_http.

Changed

  • Updated tests/test_notification.py to provide more explicit coverage of notify, notify_via_plugin, and notify_with_http
  • Pinned version of importlib-metadata for Python < 3.8 was too strict. Relaxed a bit.

Added

  • Added callee, freezegun, and responses as dev requirements to make testing notification.py easier
  • Added chaoslib.PayloadEncoder to correctly encode objects when serialising them to JSON, currently supporting:
    • date/datetimes
    • UUIDs
    • Decimals
    • Exceptions

1.22.1 - 2021-10-04

Changed

  • Only display dry flag value when new flag Dry was provided. Prevents breaking environments where the cli was not updated yet

1.22.0 - 2021-10-04

Changed

  • Switched from pycodestyle/pylama to black, flake8, isort
  • Update CI builds to build, lint, and test
  • Allow nesting of secrets
  • Secrets loaded from a vault can be overridden by variables
  • In secrets testcases, change the os.environ to be a patch, not an assignment within the testcase.
  • Add .github/workflows/close_stale_issues.yaml to mark issues as Stale after 365 days of inactivity and to close them after 7 days of being marked Stale.
  • Run pyupgrade --py36-plus across whole project
  • Updated .github/workflows/release.yaml to check that both CHANGELOG.md and chaoslib/__init__.py get updated in line with a new version
  • Updated Makefile to specify python3 instead of python

Added

  • Added .github/workflows/check_pr.yaml which checks if a PR has modified the CHANGELOG.md and if it changed/added tests

  • Added dry run options:

    • --dry=activities runs all activities as dry run. (same as the old flag)
    • --dry=actions runs all actions as dry run.
    • --dry=probes runs all probes as dry run.
    • --dry=pause runs all activities with no pausing.

1.21.0 - 2021-08-23

Changed

  • In run.py, changed method names containing continous to continuous

  • In run.py, changed variable continous_hypo_event to continuous_hypo_event

  • In run.py, changed Strategy.CONTINOUS to Strategy.CONTINUOUS

  • In types.py, changed Strategy.CONTINOUS to Strategy.CONTINUOUS (BREAKING COMPATABILITY)

  • In types.py, changed expected value from continously to continuously

  • In types.py, changed expected paramater from continous_hypothesis_frequency to continuous_hypothesis_frequency

  • In test_run.py, changed method names containing continous to continuous and method names containing immediatly to immediately

  • In test_run.py, changed parameter continous_hypothesis_frequency to continuous_hypothesis_frequency and variable strategy from Strategy.CONTINOUS to Strategy.CONTINUOUS

  • In test_run.py, changed method names containing continous to continuous

  • In run_handlers.py, changed method names containing continous to continuous

  • Changed other minor typos

1.20.0 - 2021-08-17

Added

  • Added a new control function, called validate_control that takes a control declaration and allows the control implementation to validate itself. This is useful when you want to ensure your control is well-formed before starting to run the experiment. The control validates itself. It's an optional control hook, like others. See #7 for context.

Changed

  • Fix activity name in the logs shown when the Python provider links to an unknown function #223

Removed

  • Remove the problematic log line from exit.py

While using the safeguards addon, when some safeguard probe fails, sometimes it doesn't stop the experiment and instead continues to run. In some cases it gets completely stuck.

Problem is that the chaostoolkit uses Signals in order to interrupt threads. Sometimes the interrupt exception gets caught by that logger and it just swallows it. Therefore the experiment continues to run. #210

1.19.0 - 2021-02-16

Changed

  • Library now requires Python 3.6 #208

1.18.3 - 2021-02-16

Changed

  • Fix typo in setup.cfg from install_require to install_requires

1.18.2 - 2021-02-16

Changed

  • Fix release workflow by updating to latest setuptools

1.18.1 - 2021-02-16

Changed

  • Fix release package

1.18.0 - 2021-02-16

Changed

  • Moved from setup.py to declarative setup.cfg #204
  • Moved to pylama from pycodestyle #205

1.17.0 - 2021-02-15

Added

  • Activities can be requested not to be executed on a per activity basis, not just as all or none. This can be used by a control to dynamically make a decision on what to run.
  • Test that demonstrate how to wrap controls into decorators #197

Changed

  • Moved from TravisCI to GitHub Workflows #201

1.16.0 - 2020-12-02

Changed

  • Allow substitution to keep its original type when done in isolation. #180 #195. So let's assume we have the following configuration:

    "configuration": {
    "value": 8
}

    We can now use it as an argument of an action:

    "provider": {
    "arguments": {
        "quantity": "${value}"
    }
}

    Now, the quantity argument will indeed be an integer, when it was converted to string until now. This works with any support Python objects, whether they are scalars or nested.

    If, you have something like this however

    "provider": {
    "arguments": {
        "quantity": "I want ${value} apples"
    }
}

    This will always remain a string.

1.15.1 - 2020-11-02

Changed

  • Fixed a regression where the extra vars were not passed onto configuration and secrets for overriding substitution #192

1.15.0 - 2020-09-11

Added

  • Raise the chaoslib.exceptions.InterruptExecution on SIGTERM
  • New exception chaoslib.exceptions.ExperimentExitedException that can only be injected into blocking background activities when we received the SIGUSR2 signal
  • We now inject chaoslib.exceptions.ExperimentExitedException into blocking background activities when we received the SIGUSR2 signal. This is relying on https://docs.python.org/3/c-api/init.html#c.PyThreadState_SetAsyncExc There is much we can to interrupt blocking calls and the limit is now reached because we have no control over any call that is out of the Python VM (just calling time.sleep() will get you in that situation). This is a constraint we have to live in and this extension authors must keep this in mind when they create their blocking calls. #185

1.14.1 - 2020-09-10

Added

  • SIGTERM signal is now listened for. This triggers the interruption mechanism of the experiment.

1.14.0 - 2020-09-09

Added

  • Add two functions to programmatically exit the experiment as soon as feasible by the Python VM #185:

    • chaoslib.exit.exit_gracefully: terminates but abides to the rollback strategy
    • chaoslib.exit.exit_ungracefully: terminates but bypasses rollbacks entirely and does not wait for background actions/probes still running This should mostly be useful to have a harsh way to interrupt an execution and is therefore an advanced concept with undesirable side effects (though Chaos Toolkit tries to do as right as it can).

    CAVEAT: Only works on Unix/Linux systems implementing SIGUSR1/SIGUSR2 signals

Changed

  • Fix chaostoolkit#192 to make sure a failing hypothesis prevents the method to be executed.

1.13.1 - 2020-09-07

Changed

  • Ensure the method is always executed, even when no steady-state was provided in the experiment.

1.13.0 - 2020-09-07

Added

  • Steady-state hypothesos runtime strategies can be now set to determine if the the hypothesis is executed before/after the method as usual (default behavior) or continuously throughout the method too. You can also make it applied before or after only, or even only during the method. #191

Changed

  • Always pass all secrets to control hookpoints #187
  • Massive refactor of the run_experiment function so that it raises now events of where it is during the execution so that external plugins to the Chaos Toolkit can react and impact the run. This goes beyond mere controls and is advanced usage. Still, this is a public interface. #178 In theory, this is an internal change only and the run_experiment function has not changed its API. However, this may have deep impact if you already depended on its internals.

1.12.0 - 2020-08-17

Added

  • Added ways to override configuration and secrets from var files or passed to the chaostoolkit cli chaostoolkit#175.

Changed

  • Always ensure the status in the journal is one of the specified value, with a default to "completed".

1.11.1 - 2020-07-29

Changed

  • Controls can now update/change configuration and secrets on the fly as per the specification #181

1.11.0 - 2020-07-06

Added

  • Added runtime strategies for rollback as per chaostoolkit#176. Until now, they were never played an activity would fail during the hypothesis or if the execution was interrupted from a control. With the strategies, you can now decide that they are always applied, never or only when the experiment deviated. This is a flag passed to the settings as follows:

    runtime:
  rollbacks:
    strategy: "always|never|default|deviated"

    The "default" strategy remains backward compatible.

1.10.0 - 2020-06-19

Added

  • Added function to lookup a value in the settings from a dotted path key chaostoolkit#65

1.9.0 - 2020-04-29

Added

  • Only apply rollbacks if experiment has progressed past the initial steady state hypothesis #168
  • Allow to not verify certificates when connecting to a HTTPS endpoint using #163 self-signed certificate.

Added

  • Refactored run of the experiment so that we can have multiple run strategies such as not running the steady-state hypothesis before or after the method but also one strategy so that the steady-state is continuously applied during the method, with the option to bail the experiment as soon as it deviates. #149

Changed

  • Potentially breaking. The following functions have moved from chaoslib.experiment to chaoslib.run: initialize_run_journal, apply_activities, apply_rollbacks.

Changed

  • Fix error on empty string variables call #165

1.8.1 - 2020-02-20

Added

  • Build and test on Python 3.8

Changed

  • Fix importlib_metadata call #162

1.8.0 - 2020-02-20

Added

  • Optional default value for environment variable in configuration
  • Warn the user for an action process returning a non-zero exit code
  • Support for process path relative to homedir ~
  • Indicate path in validation when path is not found nor executable #159

Changed

  • Changed the method's one-step minimum requirement. An experiment with an empty method (without any activities) is now valid.

1.7.1 - 2019-09-27

Added

  • Catch InterruptExecution during rollbacks so that the experiment terminates gracefully #132. The remaining rollbacks are not applied.
  • Catch SIGINT and SystemExit during rollbacks so that the experiment terminates gracefully #133. The remaining rollbacks are not applied.

1.7.0 - 2019-09-21

Changed

  • Fix to ensure a dry run ignores pauses
  • Allow substitution from configuration and secrets into regex and jsonpath tolerances

1.6.0 - 2019-09-03

Added

  • Two new control hook points before and after the experiment loading. These hooks allow a control to perform certain operations before the experiment is actually applied.
  • Experiments can be loaded from HTTP server which returns them using a text/plain content-type #130. This allows notably to run directly experiments hosted on GitHub.

Changed

  • Add the max_retries parameter to the HTTP provider
  • Changed nested tolerance Python probes to return True to the experiment in case the nested probe returns an object or True, and to return False in case of None or False. Before, True was returned every time except in case of an ActivityFailed exception. [#128][128].
  • Global Python controls are now imported from a specific function called load_global_controls(settings: Settings). The reason from loading them separately is so that we can have them declared, and registered, as soon as we can. Before, they were loaded when the experiments started to be executed. Now, we can also run controls when we read the experiment itself, which wasn't possible before.
  • Read package version by readsing __version__ from the chaoslib/__init__.py file directly without importing it, to avoid dependency issues of uninstalled third-party packages.

1.5.0 - 2019-07-01

Changed

  • Fix expect validation when the value is a scalar rather than a sequence.
  • Controls inheritance was getting too greedy because we were extending the same list over and over again.
  • Prevent control initializations' failure to bubble up
  • Pass arguments to control initialization function when declared
  • Interrupt nicely the experiment from control at the activity level

1.4.0 - 2019-05-16

Changed

  • [POTENTIALLY BREAKING] Changed the JSONPath implementation from jsonpath-ng to jsonpath2 as the former does not fully, or correctly, implement the JSON Path specification. The latter is based on the ANTLR grammar and implements filters appropriately. The breaking aspect is due to the fact that your path may not work anylonger as they may be rejected by the new implementation which is more correct. #119.

    Please install jsonpath2:

    $ pip install jsonpath2

1.3.1 - 2019-05-10

Changed

  • Fix to pass control's state when probe fails during hypothesis #118

1.3.0 - 2019-05-04

Changed

  • Fix to ensure a control's configuration parameter is populated when it the control is being configured #114

  • Load and apply global controls, those declared in the settings, from the run_experiment function rather than out of band #116

    This means that global controls will not be applied before the experiment is validated but only when it's executed.

    It also means the global controls will be provided with the experiment, its configuration and secrets in case global controls must access these.

1.2.0 - 2019-04-17

Changed

  • Create settings directory when it doesn't already exist #97
  • Pass configuration to tolerance checks so that probe tolerance can access it #98
  • Fix using a probe as a tolerance validator #98
  • Make settings globally available during the run #99
  • Load controls from settings too now #99
  • Do not build on MacOSX platform #110
  • Run stable Python 3.7 build

1.1.2 - 2019-03-22

Changed

  • Use a YAML safe loader for settings #92

1.1.1 - 2019-03-22

Changed

  • Fix a failure when no top-level controls were provided but an activity control was declared #88
  • Warn when control Python package is not installed but do not fail experiment during validation #90. Controls should remain optional.

1.1.0 - 2019-03-11

Added

  • a new tolerance type called range to support scenarios such as:

    value type is:

    {
    "status": 200,
    "headers": {"content-type": "text/plain"},
    "body": "1234.8"
}

    tolerance is as follows:

    {
    "type": "range",
    "range": [1000.0, 2000.0]
    }
}

    lower and upper bounds may be integers or floats.

1.0.0 - 2019-02-21

Changed

  • Delint to clean things up before 1.0
  • Do not pin too strictly or this causes havoc when updating chaoslib

1.0.0rc3 - 2019-01-29

Changed

  • Fix differences of API between Vault KV secret v1 and v2 #80
  • Catch Vault AppRole client error #81
  • Support now Service Account Vault authentication to access Vault secrets #82

1.0.0rc2 - 2019-01-28

Changed

  • Ensure Python control can be found #63

  • Ensure activity is looked up before control is applied #64

  • Ensure controls are applied only once before/after #65

  • Declare the name property in the catch block of the python control validator #66

  • Use ImportError exception instead of ModuleNotFoundError which is not declared in Python 3.5 #67

  • Disallow empty JSON path #68

  • Pass the experiment to each control when requested via the experiment parameter of the Python function #69

  • Specify the Vault KV secret default version to be used via the "vault_kv_version" configuration property, defaulting to v2

  • Support now AppRole Vault authentication to access Vault secrets thanks to @AlexShemeshWix #74

  • [BREAKING] Vault secrets are now accessed via the path property #77 instead of the key property as before. A warning message will be displayed.

        {
        "k8s": {
            "mykey": {
                "type": "vault",
                "path": "foo/bar"
            }
        }
    }

    In that case, you get the whole secrets payload from Vault set to the mykey property. If you want a specific value from that payload, specify its key as follows:

    {
    "k8s": {
        "mykey": {
            "type": "vault",
            "path": "foo/bar",
            "key": "mypassword"
        }
    }
}

  • Switch to yaml.safe_load() vs the unsafe yaml.load() to load YAML experiments #78

1.0.0rc1 - 2018-11-30

Changed

  • Let's get ready to roll 1.0.0
  • Pinned dependency versions

0.22.2 - 2018-11-30

Changed

  • Remove NoReturn import as it is not available prior Python 3.6.5 #90

0.22.1 - 2018-11-29

Added

  • Bug fix release for missing exporting the chaoslib/control package
  • Exposing readme as markdown in https://pypi.org/

0.22.0 - 2018-11-29

Added

  • listing installed Chaos Toolkit extensions #64

  • log (at DEBUG level) which Python file holds an activity or control provider function #59

  • add controls to provide entry points into the execution flow to perform out of band tasks such as tracing, monitoring or run's control #84 Simply add a block such as:

    "controls": [
   {
        "name": "tracing",
        "provider": {
            "type": "python",
            "module": "chaostracing.control"
        }
    }
]

    At the experiment, steady-state and/or activity level. This would apply the tracing control before and after the element it is enclosed in.

    By default, a control defined at the experiment level will be applied before and after, if you want to limit to one or the other, use the scope property:

    "controls": [
   {
        "name": "tracing",
        "scope": "after",
        "provider": {
            "type": "python",
            "module": "chaostracing.control"
        }
    }
]

    By default, a control defined at the experiment level will be applied at all sub-levels. You can change that behavior like this:

    "controls": [
   {
        "name": "tracing",
        "automatic": false,
        "provider": {
            "type": "python",
            "module": "chaostracing.control"
        }
    }
]

0.21.0 - 2018-09-19

Added

  • add codecov integration
  • add a new "deviated" flag to the journal to signal when the experiment ran fully but the steady state deviated after the experimental method was executed
  • add a new RunFlowEvent.RunDeviated declaration which can be used to signal when the experiment deviated. This can be sent in addition to the other events. This would allow subscribers to be notified when an experiment failed because the system deviated, while not being notified for any other failures that aborted the experiment run altogether [#56][56]
  • attempt to detect the encoding of the stdout/stderr streams of the process activities. This is only the case when either the chardet or cchardet packages are installed. When decoding does fail, raise an ActivityFailed exception

Changed

  • wording of the messages when the experiment deviated [#56][56]

0.20.1 - 2018-08-24

Changed

  • Pass an Accept: application/json, application/x+yaml header when fetching the experiment over HTTP

0.20.0 - 2018-08-09

Changed

  • renamed FailedActivity to a more active style: ActivityFailed #17

Added

  • support for the extensions block. A sequence of extension objects which are free to contain any piece of data they need. Their only mandatory property is a name #60

  • support for loading experiments from a remote endpoint using the HTTP protocol is now supported #53. Note that we do not perform any validation that the endpoint is safe, it's up to the user to be careful here (for instance by downloading the experiment using curl and reviewing it). We also do not tolerate self-signed endpoints for now.

  • added an auths entry to the settings file so that it can be used when loading from a remote endpoint that requires credentials. For now, the credentials should be manually added but this may be addressed when #65 is tackled, below is an example of such an entry:

    auths:
  mydomain.com:
    type: basic
    value: XYZ
  otherdomain.com:
    type: bearer
    value: UIY
  localhost:8081:
    type: digest
    value: UIY

[0.19.0][] - 2018-07-05

Added

  • jsonpath tolerance can now take an ̀expect property to compare the returned results with a given value (which is not really well supported in jsonpath when not dealing with arrays)
  • Ability to save settings as well as load them.
  • Display a warning to the user when a HTTP call (using the HTTP provider) returned a status code above 399. This makes it easier to track, during the experimental method or rollbacks, when the endpoint responded something we may not expected. During the steady state hypothesis checks, this warning is not displayed as this may be expected #46

Changed

  • when the probe of the steady state hypothesis fails with a FailedActivity exception, we now say that the steady state hypothesis was not met. Before, it would simply bail the whole experiment very sadly

0.17.0 - 2018-04-27

Added

  • Experiments can now be loaded from YAML as well #54

0.16.0 - 2018-04-13

Changed

  • HTTP provider can pass request body as JSON when "Content-Type: application/json" is passed in the headers object of the provider

  • Process provider can also take its arguments as a string now so you can pass the command line arguments as-is. This is needed because some commands do not respect POSIX and this fails with Python subprocess

  • Output a dict from process probes, with keys 'status', 'stdout' and 'stderr'. This is a first step to better tolerance checks on process probes. #21

  • Tolerance is richer now as well. If you provide a dictionary, it won't be considered only as a new probe to run. Instead, it will look for the type key. It's a follow-up of #21:

    • if type is "probe", then it is considered a new probe to run as usual, and the status of the probe makes the status of the tolerance.
    • if type is "regex", then a pattern key must also be present with a valid Python regex. If a target key is also present, it must indicate where to find the content to search for in the tested value. If the tested value comes from a process probe, the target key must be either "stdout" or "stderr". If it is a HTTP probe, the target key must be "body". In all other cases, the tested value is checked as-is.
    • if type is "jsonpath", then a path key must also be present with a valid JSON Path. If a target key is also present, it must indicate where to find the content to search for in the tested value. If the tested value comes from a process probe, the target key must be either "stdout" or "stderr". If it is a HTTP probe, the target key must be "body". In all other cases, the tested value is checked as-is. Note that, when the tested value is a string (no matter where read from), we try to parse it as a JSON payload first. Finally, you can provide a ̀count key as well, there are exactly that number of matches.

0.15.1 - 2018-03-09

Changed

  • Log a message when loading the configuration
  • Raise InvalidExperiment when a configuration or secret references a key in the environment and that key does not exist (it may not be set however) #40. This bails the experiment at validation time so before it runs.

0.15.0 - 2018-02-20

Changed

  • Process activities can now take their arguments as a list rather than a dictionary. The rationale is that command line arguments sometimes need ordering and dictionaries only provided ordering (as in insertion ordering) only since Python 3.6. Assuming older versions of Python will eventually get it backported is not reasonable. Therefore, a list is more appropriate now and a warning message will be displayed when your experiment uses the dictionary approach. Notice that HTTP and Python activities will remain mapping only. #34
  • Added module to warn about deprecated features.
  • Added settings for the user-supplied configurations #35
  • Added event support #33

0.14.0 - 2018-02-06

Changed

  • Do not fail the discovery when an extension is missing the __all__ attribute #28
  • Include extension activity arguments type when discovering extension #19

0.13.1 - 2018-01-30

Changed

  • Steady state is optional, so don't expect it here

0.13.0 - 2018-01-28

Changed

  • HTTP provider must be able to connect to self-signed HTTPS endpoint #25
  • Steady state is now added to the generated json report

0.12.2 - 2018-01-24

Changed

  • missing hvac dependency should not break chaos with an exception #23

0.12.1 - 2018-01-20

Changed

  • yaml is not required in this package

0.12.0 - 2018-01-20

Changed

  • Various dependencies cleanups
  • Added the DCO notice for contributors

0.11.0 - 2018-01-17

Changed

  • Steady state hypothesis is not mandatory when exploring #18

0.10.0 - 2018-01-16

Added

  • New discovery of extension capabilities #16

Changed

  • Pinning ply dependency version to 3.4 to avoid random install failures #8

0.9.4 - 2018-01-10

Changed

  • Proper merge of secrets from the different supported loaders

0.9.3 - 2018-01-09

Changed

  • Do not forget to load inlined secrets

0.9.2 - 2018-01-05

Changed

  • HTTP probe header dict must be serialiable back to JSON

0.9.1 - 2018-01-05

Removed

  • HTTP activity will not check status code any longer #10
  • Process activity will not check return code any longer #10

0.9.0 - 2018-01-05

Added

  • Steady state is now run after the method as well #19
  • Allow tolerance to be based on the HTTP status code of the probe #20

Changed

  • Logging when an activity runs in the background
  • Pause after we capture the end time of an activity
  • HTTP Probe now returns a dict with status code, headers and body of the response

0.8.2 - 2017-12-18

Changed

  • Missing transitive dependency "ply"

0.8.1 - 2017-12-18

Changed

  • Packaging up the dev requirements for install from the tarball

0.8.0 - 2017-12-17

### Added

  • Substitution from configuration and secrets so values can be dynamically set
  • The journal contains now a global status of the run

Changed

  • Pausing activity message is logged before the activity running message for better clarity
  • Arguments are now substituted with their configuration and secrets values before being applied

0.7.0 - 2017-12-12

Added

  • Configuration schema support
  • Lots of logging added at the DEBUG level
  • [EXPERIMENTAL] Loading secrets from HashiCorp vault

Changed

  • [BREAKING] Secrets from environment variable now respects the API and do not get detected by parsing the secret's value
  • Better handling of the steady hypothesis not matching its expectations Now the experiment terminates cleanly

0.6.0 - 2017-12-06

Changed

  • [BREAKING]: The specification has changed in a non-compatible way. We are working towards a 1.0.0 version of the specification as the initial draft was merely to set the stage. The method has been simplified and there are no more steady and close probes, only probes. We also now define a steady state hypothesis that would fail the experiment should it not meet its expected state. Finally, we define a rollbacks entry so that experiments can revert changes they made

Added

  • Experiment captures now SIGINT and system exit signals and swallows them so the journal is returned and can be inspected
  • Activities can be cached and referenced from within experiment

0.5.1 - 2017-11-23

Changed

  • Do not shadow functions
  • Log full errors on debug level
  • Always pause even if activity failed

0.5.0 - 2017-11-19

Changed

  • Split the inner of run_experiment into various functions for better composability
  • Pause before/after activity

0.4.0 - 2017-10-18

Changed

  • Better logging

0.3.0 - 2017-10-18

Added

  • An expected_status for HTTP probe to consider HTTP errors as expected in some cases

0.2.0 - 2017-10-10

Added

  • A first draft to loading secrets from environment
  • Passing those secrets as a mapping to actions and probes expecting them
  • Added support for background activities
  • Dry mode support

### Removed

  • Unused layer module

0.1.1 - 2017-10-06

Changed

  • Package up extra files when installed from source

0.1.0 - 2017-10-06

Added

  • Initial release