Reading report details from config.yaml

[KunalJadhav5]

Added enums for reports to efficiently utilize specific details needed for generating report.

Updated Config

daemon:
  port: 8000
  host: localhost
logging:
  level: info
reports:
  format: pdf
  renderer: weasyprint
  outputDir:  ~/.pebblo

After Updating enum values we are using details for generating report.

Default Enums value for Renderer, format, Report File Name while running pebblo command

(venv) PS C:\Users\KunalJadhav\Desktop\Kunal_Desktop\kunal-report-config\pebblo> pebblo                                                                                                             
Microsoft Visual C++ Redistributable is not installed, this may lead to the DLL load failure.
                 It can be downloaded at https://aka.ms/vs/16/release/vc_redist.x64.exe
C:\Users\KunalJadhav\Desktop\Kunal_Desktop\kunal-report-config\venv\Lib\site-packages\transformers\utils\generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.regist
er_pytree_node instead.
  _torch_pytree._register_pytree_node(
C:\Users\KunalJadhav\Desktop\Kunal_Desktop\kunal-report-config\venv\Lib\site-packages\transformers\utils\generic.py:309: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.regist
er_pytree_node instead.
  _torch_pytree._register_pytree_node(
Some weights of the model checkpoint at daxa-ai/pebblo-classifier were not used when initializing DistilBertForSequenceClassification: ['classifier.lora_B.default.weight', 'classifier.lora_A.default.weight', 'pre_classifier.lora
_B.default.weight', 'pre_classifier.lora_A.default.weight']
- This IS expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a B
ertForPreTraining model).
- This IS NOT expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequence
Classification model).
C:\Users\KunalJadhav\Desktop\Kunal_Desktop\kunal-report-config\venv\Lib\site-packages\transformers\pipelines\text_classification.py:105: UserWarning: `return_all_scores` is now deprecated,  if want a similar functionality use `t
op_k=None` instead of `return_all_scores=True` or `top_k=1` instead of `return_all_scores=False`.
  warnings.warn(
----Enum value for Renderer weasyprint----
----Enum value for Format pdf----
----Enum value for Report file name pebblo_report.pdf----
Some weights of the model checkpoint at daxa-ai/pebblo-classifier were not used when initializing DistilBertForSequenceClassification: ['classifier.lora_B.default.weight', 'classifier.lora_A.default.weight', 'pre_classifier.lora
_B.default.weight', 'pre_classifier.lora_A.default.weight']
- This IS expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a B
ertForPreTraining model).
- This IS NOT expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequence
Classification model).
INFO:     Started server process [11328]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://localhost:8000 (Press CTRL+C to quit)

Config value for Enums value for Renderer, format, Report File Name while running pebblo --config command

Config.yaml

daemon:
  port: 8000
  host: localhost
logging:
  level: info
reports:
  format: configValue
  renderer: configValue
  outputDir:  configValue

(venv) PS C:\Users\KunalJadhav\Desktop\Kunal_Desktop\kunal-report-config\pebblo> pebblo --config C:\Users\KunalJadhav\Desktop\Kunal_Desktop\kunal-report-config\pebblo\pebblo\app\config\config.yaml
Microsoft Visual C++ Redistributable is not installed, this may lead to the DLL load failure.
                 It can be downloaded at https://aka.ms/vs/16/release/vc_redist.x64.exe
C:\Users\KunalJadhav\Desktop\Kunal_Desktop\kunal-report-config\venv\Lib\site-packages\transformers\utils\generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.regist
er_pytree_node instead.
  _torch_pytree._register_pytree_node(
C:\Users\KunalJadhav\Desktop\Kunal_Desktop\kunal-report-config\venv\Lib\site-packages\transformers\utils\generic.py:309: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.regist
er_pytree_node instead.
  _torch_pytree._register_pytree_node(
Some weights of the model checkpoint at daxa-ai/pebblo-classifier were not used when initializing DistilBertForSequenceClassification: ['pre_classifier.lora_A.default.weight', 'pre_classifier.lora_B.default.weight', 'classifier.
lora_B.default.weight', 'classifier.lora_A.default.weight']
- This IS expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a B
ertForPreTraining model).
- This IS NOT expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequence
Classification model).
C:\Users\KunalJadhav\Desktop\Kunal_Desktop\kunal-report-config\venv\Lib\site-packages\transformers\pipelines\text_classification.py:105: UserWarning: `return_all_scores` is now deprecated,  if want a similar functionality use `t
op_k=None` instead of `return_all_scores=True` or `top_k=1` instead of `return_all_scores=False`.
  warnings.warn(
----Enum value for Renderer configValue----
----Enum value for Format configValue----
----Enum value for Report file name pebblo_report.configValue----
Some weights of the model checkpoint at daxa-ai/pebblo-classifier were not used when initializing DistilBertForSequenceClassification: ['pre_classifier.lora_A.default.weight', 'pre_classifier.lora_B.default.weight', 'classifier.
lora_B.default.weight', 'classifier.lora_A.default.weight']
- This IS expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a B
ertForPreTraining model).
- This IS NOT expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequence
Classification model).
INFO:     Started server process [23436]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://localhost:8000 (Press CTRL+C to quit)

Runing App using config.yaml details

Added OutputDir as ~/.temp

daemon:
  port: 8000
  host: localhost
logging:
  level: info
reports:
  format: pdf
  renderer: weasyprint
  outputDir: ~/.temp

Output dir with ./temp got created

(venv) daxa-kunal@kunal-ubu-test:~/.temp$ ls
acme-corp-rag-1
(venv) daxa-kunal@kunal-ubu-test:~/.temp$

(venv) daxa-kunal@kunal-ubu-test:~/kunal_pebblo/pebblo$ pebblo --config /home/daxa-kunal/kunal_pebblo/pebblo/pebblo/app/config/config.yaml
Config values : {'daemon': {'host': 'localhost', 'port': 8000}, 'reports': {'format': 'pdf', 'renderer': 'weasyprint', 'outputDir': '~/.temp'}, 'logging': {'level': 'info'}}
Downloading models...
Topic Classifier Initializing.
Topic Classifier Initialized...
Entity Classifier Initializing.
Entity Classifier Initialized...
Pebblo server Starting.
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:13<00:00,  1.37s/it]
INFO:     Started server process [32008]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://localhost:8000 (Press CTRL+C to quit)
INFO:     App Discover Request Processed Successfully
INFO:     127.0.0.1:48254 - "POST /v1/app/discover HTTP/1.1" 200 OK
INFO:     PDF report generated at : /home/daxa-kunal/.temp/acme-corp-rag-1/pebblo_report.pdf
INFO:     Loader Doc request Request processed successfully.
INFO:     127.0.0.1:48260 - "POST /v1/loader/doc HTTP/1.1" 200 OK

Running APP using default values

(venv) daxa-kunal@kunal-ubu-test:~/kunal_pebblo/pebblo$ pebblo
Config values : {'daemon': {'host': 'localhost', 'port': 8000}, 'reports': {'format': 'pdf', 'renderer': 'weasyprint', 'outputDir': '~/.pebblo'}, 'logging': {'level': 'info'}}
Downloading models...
Topic Classifier Initializing.
Topic Classifier Initialized...
Entity Classifier Initializing.
 50%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌                                                                                                                     | 5/10 [00:04<00:04,  1.22it/s]Collecting en-core-web-lg==3.7.1
  Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_lg-3.7.1/en_core_web_lg-3.7.1-py3-none-any.whl (587.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 587.7/587.7 MB 1.4 MB/s eta 0:00:00
Requirement already satisfied: spacy<3.8.0,>=3.7.2 in ./venv/lib/python3.11/site-packages (from en-core-web-lg==3.7.1) (3.7.2)
Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in ./venv/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (3.0.12)
Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in ./venv/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (1.0.5)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in ./venv/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (1.0.10)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in ./venv/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (2.0.8)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in ./venv/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (3.0.9)
Requirement already satisfied: thinc<8.3.0,>=8.1.8 in ./venv/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (8.2.3)
Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in ./venv/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (1.1.2)
Requirement already satisfied: srsly<3.0.0,>=2.4.3 in ./venv/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (2.4.8)
Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in ./venv/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (2.0.10)
Requirement already satisfied: weasel<0.4.0,>=0.1.0 in ./venv/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (0.3.4)
Requirement already satisfied: typer<0.10.0,>=0.3.0 in ./venv/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (0.9.0)
Requirement already satisfied: smart-open<7.0.0,>=5.2.1 in ./venv/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (6.4.0)
Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in ./venv/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (4.66.2)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in ./venv/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (2.31.0)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4 in ./venv/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (1.10.8)
Requirement already satisfied: jinja2 in ./venv/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (3.1.3)
Requirement already satisfied: setuptools in ./venv/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (65.5.0)
Requirement already satisfied: packaging>=20.0 in ./venv/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (23.2)
Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in ./venv/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (3.3.0)
Requirement already satisfied: numpy>=1.19.0 in ./venv/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (1.26.4)
Requirement already satisfied: typing-extensions>=4.2.0 in ./venv/lib/python3.11/site-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (4.9.0)
Requirement already satisfied: charset-normalizer<4,>=2 in ./venv/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in ./venv/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (3.6)
Requirement already satisfied: urllib3<3,>=1.21.1 in ./venv/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (2.2.0)
Requirement already satisfied: certifi>=2017.4.17 in ./venv/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (2024.2.2)
Requirement already satisfied: blis<0.8.0,>=0.7.8 in ./venv/lib/python3.11/site-packages (from thinc<8.3.0,>=8.1.8->spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (0.7.11)
Requirement already satisfied: confection<1.0.0,>=0.0.1 in ./venv/lib/python3.11/site-packages (from thinc<8.3.0,>=8.1.8->spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (0.1.4)
Requirement already satisfied: click<9.0.0,>=7.1.1 in ./venv/lib/python3.11/site-packages (from typer<0.10.0,>=0.3.0->spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (8.1.7)
Requirement already satisfied: cloudpathlib<0.17.0,>=0.7.0 in ./venv/lib/python3.11/site-packages (from weasel<0.4.0,>=0.1.0->spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (0.16.0)
Requirement already satisfied: MarkupSafe>=2.0 in ./venv/lib/python3.11/site-packages (from jinja2->spacy<3.8.0,>=3.7.2->en-core-web-lg==3.7.1) (2.1.5)
Installing collected packages: en-core-web-lg
Successfully installed en-core-web-lg-3.7.1

[notice] A new release of pip is available: 23.2.1 -> 24.0
[notice] To update, run: pip install --upgrade pip
Entity Classifier Initialized...
Pebblo server Starting.
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:29<00:00,  2.96s/it]
INFO:     Started server process [31336]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://localhost:8000 (Press CTRL+C to quit)
INFO:     App Discover Request Processed Successfully
INFO:     127.0.0.1:56462 - "POST /v1/app/discover HTTP/1.1" 200 OK
INFO:     PDF report generated at : /home/daxa-kunal/.pebblo/acme-corp-rag-1/pebblo_report.pdf
INFO:     Loader Doc request Request processed successfully.
INFO:     127.0.0.1:56474 - "POST /v1/loader/doc HTTP/1.1" 200 OK

[shreyas-damle]

LGTM.