Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[dados] metadata #1340

Merged
merged 164 commits into from
Mar 15, 2023
Merged

[dados] metadata #1340

merged 164 commits into from
Mar 15, 2023

Conversation

rdahis
Copy link
Member

@rdahis rdahis commented Oct 3, 2022 •
edited by laura-l-amaral
Loading

  • Terminar de preencher metadados

@rdahis rdahis added the data Data to add on BigQuery label Oct 3, 2022
@rdahis rdahis self-assigned this Oct 3, 2022
@github-actions

This comment was marked as off-topic.

Sign in to view
1 similar comment
@github-actions
Copy link

github-actions bot commented Oct 3, 2022

===> 8 warning(s) found:

  • warning [W3101] Missing timeout argument for method 'requests.get' can cause your program to hang indefinitely (python-package/basedosdados/download/metadata.py:18:19)
  • warning [W3101] Missing timeout argument for method 'requests.get' can cause your program to hang indefinitely (python-package/basedosdados/upload/metadata.py:71:24)
  • warning [W3101] Missing timeout argument for method 'requests.get' can cause your program to hang indefinitely (python-package/basedosdados/upload/metadata.py:105:19)
  • warning [W3101] Missing timeout argument for method 'requests.get' can cause your program to hang indefinitely (python-package/basedosdados/upload/metadata.py:208:15)
  • warning [W3101] Missing timeout argument for method 'requests.get' can cause your program to hang indefinitely (python-package/basedosdados/upload/metadata.py:217:19)
  • warning [W3101] Missing timeout argument for method 'requests.get' can cause your program to hang indefinitely (python-package/basedosdados/upload/metadata.py:220:15)
  • warning [W3101] Missing timeout argument for method 'requests.get' can cause your program to hang indefinitely (python-package/basedosdados/upload/metadata.py:239:25)
  • warning [W3101] Missing timeout argument for method 'requests.get' can cause your program to hang indefinitely (python-package/basedosdados/upload/table.py:239:40)
    ===> 2 convention(s) found:
  • convention [C3001] Lambda expression assigned to a variable. Define a function using the "def" keyword instead. (python-package/basedosdados/upload/dataset.py:43:22)
  • convention [C1803] 'not_found_columns != []' can be simplified to 'not_found_columns' as an empty sequence is falsey (python-package/basedosdados/upload/table.py:329:11)

@github-actions

This comment was marked as off-topic.

Sign in to view
1 similar comment
@github-actions
Copy link

github-actions bot commented Oct 4, 2022

===> 8 warning(s) found:

  • warning [W3101] Missing timeout argument for method 'requests.get' can cause your program to hang indefinitely (python-package/basedosdados/download/metadata.py:18:19)
  • warning [W3101] Missing timeout argument for method 'requests.get' can cause your program to hang indefinitely (python-package/basedosdados/upload/metadata.py:71:24)
  • warning [W3101] Missing timeout argument for method 'requests.get' can cause your program to hang indefinitely (python-package/basedosdados/upload/metadata.py:105:19)
  • warning [W3101] Missing timeout argument for method 'requests.get' can cause your program to hang indefinitely (python-package/basedosdados/upload/metadata.py:208:15)
  • warning [W3101] Missing timeout argument for method 'requests.get' can cause your program to hang indefinitely (python-package/basedosdados/upload/metadata.py:217:19)
  • warning [W3101] Missing timeout argument for method 'requests.get' can cause your program to hang indefinitely (python-package/basedosdados/upload/metadata.py:220:15)
  • warning [W3101] Missing timeout argument for method 'requests.get' can cause your program to hang indefinitely (python-package/basedosdados/upload/metadata.py:239:25)
  • warning [W3101] Missing timeout argument for method 'requests.get' can cause your program to hang indefinitely (python-package/basedosdados/upload/table.py:239:40)
    ===> 2 convention(s) found:
  • convention [C3001] Lambda expression assigned to a variable. Define a function using the "def" keyword instead. (python-package/basedosdados/upload/dataset.py:43:22)
  • convention [C1803] 'not_found_columns != []' can be simplified to 'not_found_columns' as an empty sequence is falsey (python-package/basedosdados/upload/table.py:329:11)

@github-actions

This comment was marked as off-topic.

Sign in to view
@github-actions

This comment was marked as off-topic.

Sign in to view
@rdahis
Copy link
Member Author

rdahis commented Oct 4, 2022

@lucascr91 @d116626 tem como desabilitar esses comentários do bot? não me ajuda em nada (mas deveria...) e só polui o PR.

@lucascr91
Copy link
Collaborator

A maior parte dos comentários são do pylint. Vou editar a Action para ver se tem como pegar apenas o pylint dos arquivos que foram editados.

@lucascr91 lucascr91 mentioned this pull request Oct 4, 2022
Closed
@github-actions

This comment was marked as off-topic.

Sign in to view
@github-actions

This comment was marked as off-topic.

Sign in to view
@github-actions

This comment was marked as off-topic.

Sign in to view
@github-actions

This comment was marked as off-topic.

Sign in to view
laura-l-amaral
laura-l-amaral reviewed Mar 10, 2023
View reviewed changes
bases/br_bd_metadados/columns/table_config.yaml Outdated
observation_level:
- entity: other
columns:
- table_id
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aqui é a coluna não a tabela

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

O nome da coluna é table_id, meu bem.
Screenshot_1

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aaa certo, mas mesmo assim não é o table_id que identifica unicamente a linha, precisamos de mais uma variavel, acho que pde ficar assim:

observation_level:
- entity: other
columns:
- table_id
- entity: other
columns:
- name

laura-l-amaral
laura-l-amaral reviewed Mar 10, 2023
View reviewed changes
bases/br_bd_metadados/external_links/table_config.yaml Outdated
is_partition: false
- name: id
bigquery_type: string
description: ID
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resource ID

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Por que resource ID? É a tabela de links externos.

laura-l-amaral
laura-l-amaral reviewed Mar 10, 2023
View reviewed changes
Copy link
Contributor

@laura-l-amaral laura-l-amaral left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ficou ótimo querido.

Coloquei alguns comentários nas primeiras tabelas, mas acho que precisa ver em todas elas que elas ficaram super parecidas né?

Inclusive não sei pq o metadata_modified ta vindo preenchido nos últimos tempos, mesmo quando nao temos tabelas

Se tiver qualquer dúvida pode me chamar ;)

tricktx
tricktx reviewed Mar 13, 2023
View reviewed changes
bases/br_bd_metadados/resources/table_config.yaml Outdated
table_id: resources

# Título da tabela.
title: Recursos na BD+, incluindo tabelas, pedidos LAI e links externos
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@laura-l-amaral eu não sou tão bom escrevendo, então só inclui "incluindo, tabelas, pedidos LAI e links externos", caso não esteja bom, me avisa que altero na hora.

laura-l-amaral
laura-l-amaral approved these changes Mar 14, 2023
View reviewed changes
Copy link
Contributor

@laura-l-amaral laura-l-amaral left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Acho que tá ótimo querido, amanha a gente marca um horario pra publicar em staging e fechar esse PR!

@laura-l-amaral laura-l-amaral merged commit 0ca8873 into master Mar 15, 2023
@laura-l-amaral laura-l-amaral deleted the metadata branch March 15, 2023 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tricktx tricktx tricktx left review comments

@laura-l-amaral laura-l-amaral laura-l-amaral approved these changes

@gustavoalcantara gustavoalcantara Awaiting requested review from gustavoalcantara

@lucasnascm lucasnascm Awaiting requested review from lucasnascm

@IngridSRabelo IngridSRabelo Awaiting requested review from IngridSRabelo

@arthurfg arthurfg Awaiting requested review from arthurfg

@folhesgabriel folhesgabriel Awaiting requested review from folhesgabriel

@gabrielle-carv gabrielle-carv Awaiting requested review from gabrielle-carv

Assignees

@rdahis rdahis

@tricktx tricktx

Labels
data Data to add on BigQuery
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@rdahis @lucascr91 @laura-l-amaral @tricktx