Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[infra] Adicionar mensagem de Sucesso ao table.create e table.publish #1059

Closed
3 tasks done
fernandascovino opened this issue Jan 14, 2022 · 0 comments · Fixed by #1085
Closed
3 tasks done

[infra] Adicionar mensagem de Sucesso ao table.create e table.publish #1059

fernandascovino opened this issue Jan 14, 2022 · 0 comments · Fixed by #1085
Assignees
@isadorabugarin
Labels
enhancement New feature or request

Comments

@fernandascovino
Copy link
Contributor

fernandascovino commented Jan 14, 2022 •
edited by isadorabugarin
Loading

Sua solicitação de recurso está relacionada a um problema? Por favor descreva.
Não sei se a tabela foi criada/publicada depois de rodar table.create/publish, hoje indicamos somente se foi realizado o upload do dado no storage.

Descreva a solução que você gostaria
Algo do tipo: Success! Table [created/published] at: [url]

Como resolver

Opções disponíveis:

  • verbose bool: True

  • billing_project_id str: None

  • project_config_path str: None

  • Criar dataclass no arquivo de constants.py chamado config com as opções de configuração

  • Escrever no table.create a mensagem de erro condicionada ao config.verbose

  • Explorar loguru para retorno dos erros
@fernandascovino fernandascovino added the enhancement New feature or request label Jan 14, 2022
@JoaoCarabetta JoaoCarabetta linked a pull request Jan 24, 2022 that will close this issue
[infra] Feedback messages in upload methods [issue #1059] #1085
Merged
10 tasks
@JoaoCarabetta JoaoCarabetta mentioned this issue Jan 31, 2022
Merged
10 tasks
lucascr91 pushed a commit that referenced this issue Mar 14, 2022
@isadorabugarin @guissalustiano
[infra] Feedback messages in upload methods [issue #1059] (#1085)
05f9621 
* Creating dataclass config

* Success messages - create and update (table.py) using loguru

* feat: improve log level control

* refa: move logger config to Base.__init__

* Improving log level control

* Adjusting log level control function in base.py

* Fixing repeated 'DELETE' messages everytime Table is replaced.

* Importing 'dataclass' from 'dataclasses' to make config work.

* Fixing repeated 'UPDATE' messages inside other functions.

* Defining a new script message format.

* Definng standard log messages for 'dataset.py' functions

* Definng standard log messages for 'storage.py' functions

* Definng standard log messages for 'table.py' functions

* Definng standard log messages for 'metadata.py' functions

* Adds standard configuration to billing_project_id in download.py

* Configuring billing_project_id in download.py

* Configuring config_path in base.py

Co-authored-by: Guilherme Salustiano <guissalustiano@gmail.com>
Co-authored-by: Isadora Bugarin <isadorabugarin@gmail.com>
lucascr91 added a commit that referenced this issue Mar 14, 2022
@d116626 @rdahis
@lucasnascm @gustavoairestiago @lucascr91 @isadorabugarin @gabriel-milan @guissalustiano
[infra] Python 1.6.2 (#1109)
a9b5807 
* feat(infra): create version 1.6.2

* feat(infra): create version 1.6.2

* feat(infra): create version 1.6.2

* [infra] python-v1.6.2 (#1089)

* [infra] fix dataset_config.yaml folder path (#1067)

* feat(infra) merge master

* [infra] conform Metadata to new metadata changes (#1093)

* [dados-bot] br_ms_vacinacao_covid19 (2022-01-23) (#1086)

Co-authored-by: terminal_name <github_email>

* [dados] br_bd_diretorios_brasil.etnia_indigena (#1087)

* Sobe diretorio etnia_indigena

* Update table_config.yaml

* Update table_config.yaml

* feat: conform Metadata's schema to new one

* fix: conform yaml generation to new schema

* fix: delete test_dataset folder

Co-authored-by: Lucas Moreira <65978482+lucasnascm@users.noreply.github.com>
Co-authored-by: Gustavo Aires Tiago <36206956+gustavoairestiago@users.noreply.github.com>

Co-authored-by: Ricardo Dahis <6617207+rdahis@users.noreply.github.com>
Co-authored-by: Lucas Moreira <65978482+lucasnascm@users.noreply.github.com>
Co-authored-by: Gustavo Aires Tiago <36206956+gustavoairestiago@users.noreply.github.com>

* feat(infra): 1.6.2a3 version

* feat(infra): 1.6.2a3 version

* fix(ingra): edit partitions and update_locally

* feat(infra): update_columns new fields and accepts local files

* [infra] option to make dataset public (#1020)

* feat(infra): option to make dataset public

* feat(infra): fix None data

* fix(infra): roll back

* fix(infra): fix retry in storage upload

* fix(infra): add option to dataset data location

* feat(infra): make staging dataset not public

* feat(infra): make staging dataset not public

* fix(infra): change bd version in actions

* fix(infra): add toml to install in ci

* fix(infra): remove a forget print

* fix(infra): fix location location

* fix(infra): fix dataset description

* feat(infra): bump-version

* feat(infra): temporal coverage as list in update_columns

* feat(infra): add new parameters to cli

* feat(infra): fix cli options

* [infra] change download functions to consume CKAN endpoints #1129  (#1130)

* [infra] add function to wrap bd_dataset_search endpoint

* Update download.py

* [infra] modify list_datasets function to consume CKAN endpoint

* [infra] fix list_dataset function to include limit and remove order_by

* [infra] change function list_dataset_tables to use CKAN endpoint

* [infra] apply PEP8 to list_dataset_tables and respective tests

* add get_dataset_description, get_table_description, get_table_columns

* [infra] fix dataset_config.yaml folder path (#1067)

* feat(infra) merge master

* fix files organization to match master

* remove download.py

* remove test_download

* Delete test_download.py

* remove test files

* remove test_download.py

* remove test_download.py

* remove test_download.py

* remove test_download.py

* add tests metadata

* remove test_download.py

* remove unused imports

* [infra] add _safe_fetch and get_table_size functions

Co-authored-by: lucascr91 <lucas.ecomg@gmail.com>

* fix(infra): add a empty list to not a partition

* [infra] Adiciona suporte a Avro e Parquet (#1145)

* adiciona suporte a Avro e Parquet para upload

* Adds test for source formats

* [infra] update tests for avro, parquet, and csv upload

Co-authored-by: Gabriel Gazola Milan <gabriel.gazola@poli.ufrj.br>
Co-authored-by: Isadora Bugarin  <isadorabugarin@gmail.com >
Co-authored-by: lucascr91 <lucas.ecomg@gmail.com>

* [infra] Feedback messages in upload methods [issue #1059] (#1085)

* Creating dataclass config

* Success messages - create and update (table.py) using loguru

* feat: improve log level control

* refa: move logger config to Base.__init__

* Improving log level control

* Adjusting log level control function in base.py

* Fixing repeated 'DELETE' messages everytime Table is replaced.

* Importing 'dataclass' from 'dataclasses' to make config work.

* Fixing repeated 'UPDATE' messages inside other functions.

* Defining a new script message format.

* Definng standard log messages for 'dataset.py' functions

* Definng standard log messages for 'storage.py' functions

* Definng standard log messages for 'table.py' functions

* Definng standard log messages for 'metadata.py' functions

* Adds standard configuration to billing_project_id in download.py

* Configuring billing_project_id in download.py

* Configuring config_path in base.py

Co-authored-by: Guilherme Salustiano <guissalustiano@gmail.com>
Co-authored-by: Isadora Bugarin <isadorabugarin@gmail.com>

* update toml

Co-authored-by: Ricardo Dahis <6617207+rdahis@users.noreply.github.com>
Co-authored-by: Lucas Moreira <65978482+lucasnascm@users.noreply.github.com>
Co-authored-by: Gustavo Aires Tiago <36206956+gustavoairestiago@users.noreply.github.com>
Co-authored-by: lucascr91 <lucas.ecomg@gmail.com>
Co-authored-by: Isadora Bugarin <57679195+isadorabugarin@users.noreply.github.com>
Co-authored-by: Gabriel Gazola Milan <gabriel.gazola@poli.ufrj.br>
Co-authored-by: Isadora Bugarin  <isadorabugarin@gmail.com >
Co-authored-by: Guilherme Salustiano <guissalustiano@gmail.com>
Co-authored-by: Isadora Bugarin <isadorabugarin@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@isadorabugarin isadorabugarin

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[infra] Feedback messages in upload methods [issue #1059] isadorabugarin/mais
5 participants
@d116626 @JoaoCarabetta @fernandascovino @lucascr91 @isadorabugarin