Skip to content

Commit

Permalink
Document field parsing typing via DocTests
Browse files Browse the repository at this point in the history
  • Loading branch information
thbar committed Nov 29, 2024
1 parent f1a9bcf commit 975c661
Showing 1 changed file with 45 additions and 0 deletions.
45 changes: 45 additions & 0 deletions apps/transport/lib/irve/data_frame.ex
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,51 @@ defmodule Transport.IRVE.DataFrame do

@doc """
Parse an in-memory binary of CSV content into a typed `Explorer.DataFrame` for IRVE use.
Current behaviour is that the embedded static IRVE schema enforces the field type, for fields
that are known.
For instance, a `string` field will be considered as a string:
iex> Transport.IRVE.DataFrame.dataframe_from_csv_body!("id_pdc_itinerance\\nABC123")
#Explorer.DataFrame<
Polars[1 x 1]
id_pdc_itinerance string ["ABC123"]
>
Even if it contains something that would be considered a float:
iex> Transport.IRVE.DataFrame.dataframe_from_csv_body!("id_pdc_itinerance\\n22.0")
#Explorer.DataFrame<
Polars[1 x 1]
id_pdc_itinerance string ["22.0"]
>
An `integer` field will be mapped to a integer variant:
iex> Transport.IRVE.DataFrame.dataframe_from_csv_body!("nbre_pdc\\n123")
#Explorer.DataFrame<
Polars[1 x 1]
nbre_pdc s64 [123]
>
A `boolean` field in the schema, similary, will correctly result into a `boolean` `DataFrame` field:
iex> Transport.IRVE.DataFrame.dataframe_from_csv_body!("reservation\\nfalse")
#Explorer.DataFrame<
Polars[1 x 1]
reservation boolean [false]
>
And dates are also handled correctly:
iex> Transport.IRVE.DataFrame.dataframe_from_csv_body!("date_mise_en_service\\n2024-10-02")
#Explorer.DataFrame<
Polars[1 x 1]
date_mise_en_service date [2024-10-02]
>
TODO: document behaviour on extra fields.
"""
def dataframe_from_csv_body!(body, schema \\ Transport.IRVE.StaticIRVESchema.schema_content()) do
dtypes =
Expand Down

0 comments on commit 975c661

Please sign in to comment.