Skip to content

Commit

Permalink
Log parsing example updates (#28)
Browse files Browse the repository at this point in the history
* use os.cpu_count() instead of psutil for cybert cli

* fix run script path in cybert example readme

* add vocab file for log parsing pipeline example

* remove log parsing vocab file for now

* updated log parsing readme to use sid-models hash and vocab
  • Loading branch information
efajardo-nv authored Apr 25, 2022
1 parent c352335 commit 0e289ef
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 5 deletions.
6 changes: 3 additions & 3 deletions examples/log_parsing/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,12 +53,12 @@ docker run --gpus=1 --rm -p8000:8000 -p8001:8001 -p8002:8002 -v $PWD:/models nvc
Run the following in your Morpheus container to start the log parsing pipeline:

```
python ./examples/log-parsing/run.py \
python ./examples/log_parsing/run.py \
--num_threads 1 \
--input_file ./models/datasets/validation-data/log-parsing-validation-data-input.csv \
--output_file ./log-parsing-output.jsonlines \
--model_vocab_hash_file=./data/bert-base-cased-hash.txt \
--model_vocab_file=./data/bert-base-cased-vocab.txt \
--model_vocab_hash_file=./models/training-tuning-scripts/sid-models/resources/bert-base-cased-hash.txt \
--model_vocab_file=./models/training-tuning-scripts/sid-models/resources/bert-base-cased-vocab.txt \
--model_seq_length=256 \
--model_name log-parsing-onnx \
--model_config_file=./models/log-parsing-models/log-parsing-config-20220418.json \
Expand Down
4 changes: 2 additions & 2 deletions examples/log_parsing/run.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
# limitations under the License.

import click
import psutil
import os
from inference import LogParsingInferenceStage
from postprocessing import LogParsingPostProcessingStage
from preprocessing import PreprocessLogParsingStage
Expand All @@ -32,7 +32,7 @@
@click.command()
@click.option(
"--num_threads",
default=psutil.cpu_count(),
default=os.cpu_count(),
type=click.IntRange(min=1),
help="Number of internal pipeline threads to use",
)
Expand Down

0 comments on commit 0e289ef

Please sign in to comment.