Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ETQ tech: je veux publier les fichiers opendata selon les bonnes pratiques de data.gouv #11100

Draft
wants to merge 18 commits into
base: main
Choose a base branch
from

Conversation

Benoit-MINT
Copy link
Contributor

@Benoit-MINT Benoit-MINT commented Nov 28, 2024

Dans le dataset "utilisation de DS" : https://www.data.gouv.fr/fr/datasets/utilisation-du-service-demarches-simplifiees/
Aujourd'hui, on ajoute chaque mois une nouvelle resource par type de données :

  • Nombre d'instructeurs connectés par mois
  • Nombre de comptes usagers créés avec France Connect par mois
  • Nombre de démarches supprimées par mois
  • Nombre de démarches closes par mois
  • Nombre de démarches publiées par mois
  • Nombre de comptes créés par mois
  • Nombre de dossiers créés par mois
  • Nombre d'instructeurs créés par mois
  • Nombre d'administrateurs créés par mois
    Ceci ne suit pas les bonnes pratiques de data.gouv pour faciliter la réutilisation.
    Il est préférable de maintenir à jour, chaque mois, une même resource par type de données. Cela revient à ajouter de nouvelles lignes aux mêmes fichiers.

PR en draft, cas d'usage sur "Nombre de comptes créés par mois"

Copy link

codecov bot commented Nov 28, 2024

Codecov Report

Attention: Patch coverage is 79.48718% with 32 lines in your changes missing coverage. Please review.

Project coverage is 77.00%. Comparing base (08c963a) to head (4a3cf07).
Report is 319 commits behind head on main.

Files with missing lines Patch % Lines
app/lib/api_datagouv/api.rb 36.36% 14 Missing ⚠️
app/jobs/cron/datagouv/account_by_month_job.rb 53.84% 6 Missing ⚠️
app/jobs/cron/datagouv/procedure_by_month_job.rb 53.84% 6 Missing ⚠️
...user_connected_with_france_connect_by_month_job.rb 53.84% 6 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (08c963a) and HEAD (4a3cf07). Click for more details.

HEAD has 10 uploads less than BASE
Flag BASE (08c963a) HEAD (4a3cf07)
16 6
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #11100      +/-   ##
==========================================
- Coverage   84.35%   77.00%   -7.36%     
==========================================
  Files        1176     1186      +10     
  Lines       25942    26703     +761     
  Branches     4896     4848      -48     
==========================================
- Hits        21884    20562    -1322     
- Misses       4058     6141    +2083     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Benoit-MINT Benoit-MINT force-pushed the opendata-new-publication-dataset branch from fa02bd0 to 72004a0 Compare December 4, 2024 08:26
@Benoit-MINT Benoit-MINT force-pushed the opendata-new-publication-dataset branch 6 times, most recently from a232d00 to aae1858 Compare December 9, 2024 15:22
@Benoit-MINT Benoit-MINT force-pushed the opendata-new-publication-dataset branch from 1c96411 to 035a21b Compare December 9, 2024 16:15
@LeSim LeSim force-pushed the opendata-new-publication-dataset branch from dfab2f4 to 40bb9a0 Compare December 13, 2024 09:53
@Benoit-MINT Benoit-MINT force-pushed the opendata-new-publication-dataset branch from 40bb9a0 to e235152 Compare December 13, 2024 17:59

url = JSON.parse(response.body)["url"]

response = Typhoeus.get(url)

Check failure

Code scanning / CodeQL

Server-side request forgery Critical

The URL of this request depends on a
user-provided value
.

Copilot Autofix AI 22 days ago

To fix the problem, we need to ensure that the URL extracted from the response.body is validated before being used in the subsequent HTTP request. One way to do this is to check that the URL belongs to a trusted domain. We can use the URI module to parse the URL and verify its host.

  • Add a method to validate the URL.
  • Use this method to check the URL before making the HTTP request on line 28.
Suggested changeset 1
app/lib/api_datagouv/api.rb

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/app/lib/api_datagouv/api.rb b/app/lib/api_datagouv/api.rb
--- a/app/lib/api_datagouv/api.rb
+++ b/app/lib/api_datagouv/api.rb
@@ -1,3 +1,3 @@
 # frozen_string_literal: true
-
+require 'uri'
 class APIDatagouv::API
@@ -26,2 +26,3 @@
       url = JSON.parse(response.body)["url"]
+      validate_url(url)
 
@@ -94,2 +95,10 @@
   end
+  private
+
+  def validate_url(url)
+    uri = URI.parse(url)
+    raise "Invalid URL" unless uri.host == "www.data.gouv.fr"
+  rescue URI::InvalidURIError
+    raise "Invalid URL"
+  end
 end
EOF
@@ -1,3 +1,3 @@
# frozen_string_literal: true

require 'uri'
class APIDatagouv::API
@@ -26,2 +26,3 @@
url = JSON.parse(response.body)["url"]
validate_url(url)

@@ -94,2 +95,10 @@
end
private

def validate_url(url)
uri = URI.parse(url)
raise "Invalid URL" unless uri.host == "www.data.gouv.fr"
rescue URI::InvalidURIError
raise "Invalid URL"
end
end
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
@Benoit-MINT Benoit-MINT force-pushed the opendata-new-publication-dataset branch from e235152 to 87c86c7 Compare December 13, 2024 18:19
@Benoit-MINT Benoit-MINT force-pushed the opendata-new-publication-dataset branch 3 times, most recently from 6d6cf8e to 137797a Compare December 23, 2024 15:16
@Benoit-MINT Benoit-MINT force-pushed the opendata-new-publication-dataset branch from 137797a to 4a3cf07 Compare December 23, 2024 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant