Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix remaining D401 checks #37256

Closed
Taragolis opened this issue Feb 8, 2024 · 4 comments · Fixed by #37434
Closed

Fix remaining D401 checks #37256

Taragolis opened this issue Feb 8, 2024 · 4 comments · Fixed by #37434
Labels
good first issue kind:meta High-level information important to the community

Comments

@Taragolis
Copy link
Contributor

Taragolis commented Feb 8, 2024 •
edited
Loading

Body

Part of #10742. Recently we enable D401 pydocstyle rule, however we still have some modules which do not follow D401 rule yet

Easy way to find all linting problem into the module

  1. Remove corresponding line to particular module into the pyproject.toml in [tool.ruff.lint.per-file-ignores]
  2. Run ruff via pre-commit hook
❯ pre-commit run ruff --all-files
Run 'ruff' for extremely fast Python linting.............................Failed
- hook id: ruff
- exit code: 1

airflow/www/auth.py:90:5: D401 First line of docstring should be in imperative mood: "This decorator is used to check permissions on views."
airflow/www/auth.py:348:5: D401 First line of docstring should be in imperative mood: "Decorator that checks current user's permissions to access the website."
Found 2 errors.

Airflow Core

  • airflow/auth/managers/utils/fab.py
  • airflow/decorators/bash.py
  • airflow/executors/debug_executor.py
  • airflow/models/baseoperator.py
  • airflow/models/dagrun.py
  • airflow/models/taskinstance.py
  • airflow/operators/python.py
  • airflow/plugins_manager.py
  • airflow/providers_manager.py
  • airflow/serialization/serde.py
  • airflow/utils/log/task_context_logger.py
  • airflow/utils/sqlalchemy.py
  • airflow/www/auth.py
  • airflow/www/blueprints.py
  • airflow/www/views.py
  • airflow/utils/file.py

Provider airbyte

  • airflow/providers/airbyte/hooks/airbyte.py
  • airflow/providers/airbyte/operators/airbyte.py
  • airflow/providers/airbyte/sensors/airbyte.py
  • airflow/providers/airbyte/triggers/airbyte.py

Provider amazon

  • airflow/providers/amazon/aws/auth_manager/cli/avp_commands.py
  • airflow/providers/amazon/aws/auth_manager/views/auth.py
  • airflow/providers/amazon/aws/executors/ecs/boto_schema.py
  • airflow/providers/amazon/aws/executors/ecs/ecs_executor.py
  • airflow/providers/amazon/aws/executors/ecs/utils.py
  • airflow/providers/amazon/aws/hooks/redshift_sql.py
  • airflow/providers/amazon/aws/hooks/logs.py
  • airflow/providers/amazon/aws/triggers/redshift_cluster.py
  • airflow/providers/amazon/aws/triggers/redshift_data.py
  • airflow/providers/amazon/aws/triggers/sagemaker.py

Provider apache.hive

  • airflow/providers/apache/hive/hooks/hive.py

Provider apache.spark

  • airflow/providers/apache/spark/hooks/spark_connect.py
  • airflow/providers/apache/spark/hooks/spark_sql.py
  • airflow/providers/apache/spark/hooks/spark_submit.py

Provider atlassian.jira

  • airflow/providers/atlassian/jira/hooks/jira.py

Provider cncf.kubernetes

  • airflow/providers/cncf/kubernetes/callbacks.py
  • airflow/providers/cncf/kubernetes/operators/custom_object_launcher.py
  • airflow/providers/cncf/kubernetes/operators/spark_kubernetes.py
  • airflow/providers/cncf/kubernetes/resource_convert/configmap.py
  • airflow/providers/cncf/kubernetes/resource_convert/env_variable.py
  • airflow/providers/cncf/kubernetes/resource_convert/secret.py

Provider common.io

  • airflow/providers/common/io/xcom/backend.py

Provider databricks

  • airflow/providers/databricks/hooks/databricks.py
  • airflow/providers/databricks/operators/databricks.py

Provider discord

  • airflow/providers/discord/hooks/discord_webhook.py

Provider fab

  • airflow/providers/fab/auth_manager/decorators/auth.py
  • airflow/providers/fab/auth_manager/fab_auth_manager.py
  • airflow/providers/fab/auth_manager/security_manager/override.py

Provider google

  • airflow/providers/google/cloud/hooks/automl.py
  • airflow/providers/google/cloud/hooks/bigquery.py
  • airflow/providers/google/cloud/hooks/bigquery_dts.py
  • airflow/providers/google/cloud/hooks/bigtable.py
  • airflow/providers/google/cloud/hooks/cloud_batch.py
  • airflow/providers/google/cloud/hooks/cloud_build.py
  • airflow/providers/google/cloud/hooks/cloud_composer.py
  • airflow/providers/google/cloud/hooks/cloud_memorystore.py
  • airflow/providers/google/cloud/hooks/cloud_run.py
  • airflow/providers/google/cloud/hooks/cloud_sql.py
  • airflow/providers/google/cloud/hooks/cloud_storage_transfer_service.py
  • airflow/providers/google/cloud/hooks/compute.py
  • airflow/providers/google/cloud/hooks/datacatalog.py
  • airflow/providers/google/cloud/hooks/dataflow.py
  • airflow/providers/google/cloud/hooks/dataform.py
  • airflow/providers/google/cloud/hooks/datafusion.py
  • airflow/providers/google/cloud/hooks/datapipeline.py
  • airflow/providers/google/cloud/hooks/dataplex.py
  • airflow/providers/google/cloud/hooks/dataprep.py
  • airflow/providers/google/cloud/hooks/dataproc_metastore.py
  • airflow/providers/google/cloud/hooks/datastore.py
  • airflow/providers/google/cloud/hooks/dlp.py
  • airflow/providers/google/cloud/hooks/gcs.py
  • airflow/providers/google/cloud/hooks/gdm.py
  • airflow/providers/google/cloud/hooks/kms.py
  • airflow/providers/google/cloud/hooks/kubernetes_engine.py
  • airflow/providers/google/cloud/hooks/life_sciences.py
  • airflow/providers/google/cloud/hooks/looker.py
  • airflow/providers/google/cloud/hooks/mlengine.py
  • airflow/providers/google/cloud/hooks/natural_language.py
  • airflow/providers/google/cloud/hooks/os_login.py
  • airflow/providers/google/cloud/hooks/pubsub.py
  • airflow/providers/google/cloud/hooks/secret_manager.py
  • airflow/providers/google/cloud/hooks/spanner.py
  • airflow/providers/google/cloud/hooks/speech_to_text.py
  • airflow/providers/google/cloud/hooks/stackdriver.py
  • airflow/providers/google/cloud/hooks/tasks.py
  • airflow/providers/google/cloud/hooks/text_to_speech.py
  • airflow/providers/google/cloud/hooks/translate.py
  • airflow/providers/google/cloud/hooks/vertex_ai/auto_ml.py
  • airflow/providers/google/cloud/hooks/vertex_ai/batch_prediction_job.py
  • airflow/providers/google/cloud/hooks/vertex_ai/custom_job.py
  • airflow/providers/google/cloud/hooks/vertex_ai/dataset.py
  • airflow/providers/google/cloud/hooks/vertex_ai/endpoint_service.py
  • airflow/providers/google/cloud/hooks/vertex_ai/hyperparameter_tuning_job.py
  • airflow/providers/google/cloud/hooks/vertex_ai/model_service.py
  • airflow/providers/google/cloud/hooks/vertex_ai/pipeline_job.py
  • airflow/providers/google/cloud/hooks/video_intelligence.py
  • airflow/providers/google/cloud/hooks/vision.py
  • airflow/providers/google/cloud/hooks/workflows.py
  • airflow/providers/google/cloud/links/vertex_ai.py
  • airflow/providers/google/cloud/log/stackdriver_task_handler.py
  • airflow/providers/google/cloud/operators/bigquery.py
  • airflow/providers/google/cloud/operators/bigquery_dts.py
  • airflow/providers/google/cloud/operators/cloud_base.py
  • airflow/providers/google/cloud/operators/cloud_build.py
  • airflow/providers/google/cloud/operators/cloud_sql.py
  • airflow/providers/google/cloud/operators/cloud_storage_transfer_service.py
  • airflow/providers/google/cloud/operators/dataflow.py
  • airflow/providers/google/cloud/operators/datafusion.py
  • airflow/providers/google/cloud/operators/dataplex.py
  • airflow/providers/google/cloud/operators/dataproc.py
  • airflow/providers/google/cloud/operators/functions.py
  • airflow/providers/google/cloud/operators/gcs.py
  • airflow/providers/google/cloud/operators/kubernetes_engine.py
  • airflow/providers/google/cloud/operators/mlengine.py
  • airflow/providers/google/cloud/operators/pubsub.py
  • airflow/providers/google/cloud/operators/spanner.py
  • airflow/providers/google/cloud/operators/vertex_ai/auto_ml.py
  • airflow/providers/google/cloud/operators/vertex_ai/batch_prediction_job.py
  • airflow/providers/google/cloud/operators/vertex_ai/custom_job.py
  • airflow/providers/google/cloud/operators/vertex_ai/hyperparameter_tuning_job.py
  • airflow/providers/google/cloud/operators/vertex_ai/pipeline_job.py
  • airflow/providers/google/cloud/sensors/bigquery.py
  • airflow/providers/google/cloud/sensors/cloud_composer.py
  • airflow/providers/google/cloud/sensors/gcs.py
  • airflow/providers/google/cloud/sensors/pubsub.py
  • airflow/providers/google/cloud/transfers/bigquery_to_gcs.py
  • airflow/providers/google/cloud/transfers/bigquery_to_sql.py
  • airflow/providers/google/cloud/transfers/cassandra_to_gcs.py
  • airflow/providers/google/cloud/transfers/gcs_to_bigquery.py
  • airflow/providers/google/cloud/transfers/gcs_to_gcs.py
  • airflow/providers/google/cloud/transfers/gcs_to_sftp.py
  • airflow/providers/google/cloud/transfers/local_to_gcs.py
  • airflow/providers/google/cloud/transfers/mssql_to_gcs.py
  • airflow/providers/google/cloud/transfers/mysql_to_gcs.py
  • airflow/providers/google/cloud/transfers/oracle_to_gcs.py
  • airflow/providers/google/cloud/transfers/postgres_to_gcs.py
  • airflow/providers/google/cloud/transfers/presto_to_gcs.py
  • airflow/providers/google/cloud/transfers/s3_to_gcs.py
  • airflow/providers/google/cloud/transfers/sftp_to_gcs.py
  • airflow/providers/google/cloud/transfers/sql_to_gcs.py
  • airflow/providers/google/cloud/transfers/trino_to_gcs.py
  • airflow/providers/google/cloud/triggers/bigquery.py
  • airflow/providers/google/cloud/triggers/bigquery_dts.py
  • airflow/providers/google/cloud/triggers/cloud_batch.py
  • airflow/providers/google/cloud/triggers/cloud_build.py
  • airflow/providers/google/cloud/triggers/cloud_run.py
  • airflow/providers/google/cloud/triggers/cloud_storage_transfer_service.py
  • airflow/providers/google/cloud/triggers/dataflow.py
  • airflow/providers/google/cloud/triggers/datafusion.py
  • airflow/providers/google/cloud/triggers/dataplex.py
  • airflow/providers/google/cloud/triggers/dataproc.py
  • airflow/providers/google/cloud/triggers/gcs.py
  • airflow/providers/google/cloud/triggers/kubernetes_engine.py
  • airflow/providers/google/cloud/triggers/mlengine.py
  • airflow/providers/google/cloud/triggers/pubsub.py
  • airflow/providers/google/cloud/utils/bigquery.py
  • airflow/providers/google/cloud/utils/credentials_provider.py
  • airflow/providers/google/cloud/utils/dataform.py
  • airflow/providers/google/cloud/utils/field_validator.py
  • airflow/providers/google/cloud/utils/helpers.py
  • airflow/providers/google/cloud/utils/mlengine_operator_utils.py
  • airflow/providers/google/cloud/utils/mlengine_prediction_summary.py
  • airflow/providers/google/common/auth_backend/google_openid.py
  • airflow/providers/google/common/hooks/base_google.py
  • airflow/providers/google/common/hooks/discovery_api.py
  • airflow/providers/google/common/utils/id_token_credentials.py
  • airflow/providers/google/firebase/hooks/firestore.py
  • airflow/providers/google/leveldb/hooks/leveldb.py
  • airflow/providers/google/marketing_platform/hooks/analytics.py
  • airflow/providers/google/marketing_platform/hooks/campaign_manager.py
  • airflow/providers/google/marketing_platform/hooks/display_video.py
  • airflow/providers/google/marketing_platform/hooks/search_ads.py
  • airflow/providers/google/suite/hooks/calendar.py
  • airflow/providers/google/suite/hooks/drive.py
  • airflow/providers/google/suite/hooks/sheets.py

Provider grpc

  • airflow/providers/grpc/hooks/grpc.py

Provider hashicorp

  • airflow/providers/hashicorp/_internal_client/vault_client.py
  • airflow/providers/hashicorp/hooks/vault.py

Provider http

  • airflow/providers/http/operators/http.py
  • airflow/providers/http/triggers/http.py

Provider imap

  • airflow/providers/imap/hooks/imap.py

Provider jenkins

  • airflow/providers/jenkins/hooks/jenkins.py

Provider microsoft.azure

  • airflow/providers/microsoft/azure/hooks/adx.py
  • airflow/providers/microsoft/azure/hooks/asb.py
  • airflow/providers/microsoft/azure/hooks/base_azure.py
  • airflow/providers/microsoft/azure/hooks/batch.py
  • airflow/providers/microsoft/azure/hooks/container_instance.py
  • airflow/providers/microsoft/azure/hooks/container_registry.py
  • airflow/providers/microsoft/azure/hooks/container_volume.py
  • airflow/providers/microsoft/azure/hooks/cosmos.py
  • airflow/providers/microsoft/azure/hooks/data_factory.py
  • airflow/providers/microsoft/azure/hooks/data_lake.py
  • airflow/providers/microsoft/azure/hooks/fileshare.py
  • airflow/providers/microsoft/azure/hooks/synapse.py
  • airflow/providers/microsoft/azure/hooks/wasb.py
  • airflow/providers/microsoft/azure/log/wasb_task_handler.py
  • airflow/providers/microsoft/azure/operators/adx.py
  • airflow/providers/microsoft/azure/operators/asb.py
  • airflow/providers/microsoft/azure/operators/data_factory.py
  • airflow/providers/microsoft/azure/operators/synapse.py
  • airflow/providers/microsoft/azure/sensors/data_factory.py
  • airflow/providers/microsoft/azure/sensors/wasb.py
  • airflow/providers/microsoft/azure/transfers/sftp_to_wasb.py
  • airflow/providers/microsoft/azure/triggers/data_factory.py
  • airflow/providers/microsoft/azure/triggers/wasb.py

Provider microsoft.mssql

  • airflow/providers/microsoft/mssql/hooks/mssql.py

Provider microsoft.psrp

  • airflow/providers/microsoft/psrp/hooks/psrp.py

Provider mongo

  • airflow/providers/mongo/hooks/mongo.py

Provider mysql

  • airflow/providers/mysql/hooks/mysql.py
  • airflow/providers/mysql/transfers/s3_to_mysql.py

Provider neo4j

  • airflow/providers/neo4j/hooks/neo4j.py

Provider odbc

  • airflow/providers/odbc/hooks/odbc.py

Provider openfaas

  • airflow/providers/openfaas/hooks/openfaas.py

Provider openlineage

  • airflow/providers/openlineage/plugins/adapter.py
  • airflow/providers/openlineage/sqlparser.py
  • airflow/providers/openlineage/utils/sql.py

Provider opensearch

  • airflow/providers/opensearch/hooks/opensearch.py
  • airflow/providers/opensearch/operators/opensearch.py

Provider opsgenie

  • airflow/providers/opsgenie/hooks/opsgenie.py

Provider oracle

  • airflow/providers/oracle/hooks/oracle.py

Provider pagerduty

  • airflow/providers/pagerduty/hooks/pagerduty.py
  • airflow/providers/pagerduty/hooks/pagerduty_events.py

Provider papermill

  • airflow/providers/papermill/hooks/kernel.py

Provider pinecone

  • airflow/providers/pinecone/hooks/pinecone.py

Provider postgres

  • airflow/providers/postgres/hooks/postgres.py

Provider presto

  • airflow/providers/presto/hooks/presto.py

Provider qdrant

  • airflow/providers/qdrant/hooks/qdrant.py

Provider redis

  • airflow/providers/redis/hooks/redis.py

Provider salesforce

  • airflow/providers/salesforce/hooks/salesforce.py
  • airflow/providers/salesforce/operators/bulk.py
  • airflow/providers/salesforce/operators/salesforce_apex_rest.py

Provider samba

  • airflow/providers/samba/hooks/samba.py
  • airflow/providers/samba/transfers/gcs_to_samba.py

Provider segment

  • airflow/providers/segment/hooks/segment.py

Provider sftp

  • airflow/providers/sftp/decorators/sensors/sftp.py
  • airflow/providers/sftp/hooks/sftp.py
  • airflow/providers/sftp/operators/sftp.py
  • airflow/providers/sftp/sensors/sftp.py
  • airflow/providers/sftp/triggers/sftp.py

Provider slack

  • airflow/providers/slack/hooks/slack.py
  • airflow/providers/slack/hooks/slack_webhook.py
  • airflow/providers/slack/operators/slack.py

Provider smtp

  • airflow/providers/smtp/hooks/smtp.py

Provider snowflake

  • airflow/providers/snowflake/hooks/snowflake.py
  • airflow/providers/snowflake/hooks/snowflake_sql_api.py
  • airflow/providers/snowflake/operators/snowflake.py
  • airflow/providers/snowflake/transfers/copy_into_snowflake.py
  • airflow/providers/snowflake/triggers/snowflake_trigger.py
  • airflow/providers/snowflake/utils/sql_api_generate_jwt.py

Provider sqlite

  • airflow/providers/sqlite/hooks/sqlite.py

Provider ssh

  • airflow/providers/ssh/hooks/ssh.py

Provider tableau

  • airflow/providers/tableau/hooks/tableau.py
  • airflow/providers/tableau/operators/tableau.py

Provider tabular

  • airflow/providers/tabular/hooks/tabular.py

Provider telegram

  • airflow/providers/telegram/hooks/telegram.py
  • airflow/providers/telegram/operators/telegram.py

Provider trino

  • airflow/providers/trino/hooks/trino.py

Provider weaviate

  • airflow/providers/weaviate/hooks/weaviate.py

Provider yandex

  • airflow/providers/yandex/hooks/yandex.py

Committer

  • I acknowledge that I am a maintainer/committer of the Apache Airflow project.
@Taragolis Taragolis added good first issue kind:meta High-level information important to the community labels Feb 8, 2024
This was referenced Feb 8, 2024
Closed
Merged
@guptarohit
Copy link
Contributor

I can help on these, if not all modules are picked or in progress.

@rawwar rawwar mentioned this issue Feb 9, 2024
Merged
@Taragolis
Copy link
Contributor Author

@guptarohit feel free to pick up any providers or even core

@rawwar rawwar mentioned this issue Feb 9, 2024
Merged
16 tasks
This was referenced Feb 9, 2024
Closed
Merged
@rawwar
Copy link
Collaborator

rawwar commented Feb 9, 2024 •
edited
Loading

@Taragolis , What to do when the docstring doesn't actually explain anything about the method?

Example:

airflow/airflow/providers/google/cloud/hooks/bigquery.py

Line 2350 in 7835fd2

def close(self) -> None:

Can i add noqa for docstrings like this?

EDIT:

slack message: https://apache-airflow.slack.com/archives/CCPRP7943/p1707489704051959?thread_ts=1707489399.332569&cid=CCPRP7943

possible rewrite : “Do nothing. Not needed for BigQueryConnection” ?

@rawwar rawwar mentioned this issue Feb 9, 2024
Closed
20 tasks
@vincbeck vincbeck mentioned this issue Feb 9, 2024
Merged
@rawwar rawwar mentioned this issue Feb 9, 2024
Merged
51 tasks
This was referenced Feb 10, 2024
Merged
Merged
@okirialbert
Copy link
Contributor

Hey, I'd like to work on any pending modules.

@guptarohit guptarohit mentioned this issue Feb 10, 2024
Merged
8 tasks
@rawwar rawwar mentioned this issue Feb 10, 2024
Merged
42 tasks
@jmelot jmelot mentioned this issue Feb 10, 2024
Merged
@DjVinnii DjVinnii mentioned this issue Feb 11, 2024
Merged
@rawwar rawwar mentioned this issue Feb 12, 2024
Merged
35 tasks
@Satoshi-Sh Satoshi-Sh mentioned this issue Feb 12, 2024
Merged
4 tasks
@rawwar rawwar mentioned this issue Feb 15, 2024
Merged
@Taragolis Taragolis mentioned this issue Feb 19, 2024
Closed
77 tasks
@Taragolis Taragolis mentioned this issue Apr 4, 2024
Merged
@eladkal eladkal mentioned this issue Apr 16, 2024
Closed
16 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
good first issue kind:meta High-level information important to the community
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix remaining D401 checks rawwar/airflow
4 participants
@Taragolis @guptarohit @rawwar @okirialbert