Skip to content

Commit

Permalink
Move config documentation into docstrings
Browse files Browse the repository at this point in the history
  • Loading branch information
Ayushsunny committed May 10, 2021
1 parent 889b54d commit 51b22a6
Show file tree
Hide file tree
Showing 2 changed files with 17 additions and 31 deletions.
32 changes: 1 addition & 31 deletions docs/Configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,37 +41,7 @@ of configurations of `class<BrowserParams>`.
- [`save_content`](#save_content)


## Platform Configuration Options

- `data_directory`
- The directory into which screenshots and page dumps will be saved
- [Intended to be removed by #232](https://github.com/mozilla/OpenWPM/issues/232)
- `log_path` -> supported file extensions are `.log`
- The path to the file in which OpenWPM will log. The
directory given will be created if it does not exist.
- `failure_limit` -> has to be either of type `int` or `None`
- The number of command failures the platform will tolerate before raising a
`CommandExecutionError` exception. Otherwise the default is set to 2 x the
number of browsers plus 10. The failure counter is reset at the end of each
successfully completed command sequence.
- For non-blocking command sequences that cause the number of failures to
exceed `failure_limit` the `CommandExecutionError` is raised when
attempting to execute the next command sequence.
- `testing`
- A platform wide flag that can be used to only run certain functionality
while testing. For example, the Javascript instrumentation
[exposes its instrumentation function](https://github.com/mozilla/OpenWPM/blob/91751831647c37b769f0039d99d0a164384c76ae/automation/Extension/firefox/data/content.js#L447-L449)
on the page script global to allow test scripts to instrument objects
on-the-fly. Depending on where you would like to add test functionality,
you may need to propagate the flag.
- This is not something you should enable during normal crawls.
- `process_watchdog`
- It is part of default manager_params. It is set to false by default which can manually be set to true.
- It is used to create another thread that kills off `GeckoDriver` (or `Xvfb`) instances that haven't been spawned by OpenWPM. (GeckoDriver is used by Selenium to control Firefox and Xvfb a "virtual display" so we simulate having graphics when running on a server).
- `memory_watchdog`
- It is part of default manager_params. It is set to false by default which can manually be set to true.
- A watchdog that tries to ensure that no Firefox instance takes up too much memory. It is set to false by default
- It is mostly useful for long running cloud crawls
<!--- ## Platform Configuration Options -->

## Browser Configuration Options

Expand Down
16 changes: 16 additions & 0 deletions openwpm/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -120,15 +120,31 @@ class ManagerParams(DataClassJsonMixin):
default=Path.home() / "openwpm",
metadata=DCJConfig(encoder=path_to_str, decoder=str_to_path),
)
"""The directory into which screenshots and page dumps will be saved"""
log_path: Path = field(
default=Path.home() / "openwpm" / "openwpm.log",
metadata=DCJConfig(encoder=path_to_str, decoder=str_to_path),
)
"""The path to the file in which OpenWPM will log. The
directory given will be created if it does not exist."""
testing: bool = False
"""A platform wide flag that can be used to only run certain functionality
while testing. For example, the Javascript instrumentation"""
memory_watchdog: bool = False
"""A watchdog that tries to ensure that no Firefox instance takes up too much memory.
It is mostly useful for long running cloud crawls"""
process_watchdog: bool = False
"""- It is used to create another thread that kills off `GeckoDriver` (or `Xvfb`) instances that haven't been spawned by OpenWPM. (GeckoDriver is used by
Selenium to control Firefox and Xvfb a "virtual display" so we simulate having graphics when running on a server)."""
num_browsers: int = 1
_failure_limit: Optional[int] = None
"""- The number of command failures the platform will tolerate before raising a
`CommandExecutionError` exception. Otherwise the default is set to 2 x the
number of browsers plus 10. The failure counter is reset at the end of each
successfully completed command sequence.
- For non-blocking command sequences that cause the number of failures to
exceed `failure_limit` the `CommandExecutionError` is raised when
attempting to execute the next command sequence."""

@property
def failure_limit(self) -> int:
Expand Down

0 comments on commit 51b22a6

Please sign in to comment.