CHANGELOG.md

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Unreleased

Added

• Add confidentiality property to WebSource #518
• Add missing properties methods in extensions points #781
• Add support of catalogs
• Add check sources command #515

Changed

• Update ILO website URL #780

Fixed

• Fix hidden IO exception when using data stream
• Fix sources file env variable in CLI #779

3.0.0-beta.13 - 2024-09-06

This is the thirteenth beta release of sdmx-dl.
sdmx-dl follows semantic versioning.

This release improves the modularity of the API and continues to clean up the configuration.
The experimental PxWebDriver has also been improved but is not yet ready for daily use.
The OECD, INEGI and ILO endpoints have been updated.

Disclaimer: sdmx-dl is still in development and is available for test only. Do not use in production!

Added

• Move FileFormat and Persistence to API
• Add registry API to load custom web sources
• Add support of databases in PxWebDriver
• Add parsing of language in PxWebDriver
• Add websites in PxWebDriver
• Improve desktop UI
• Add webservice transcoding in gRPC

Changed

• Force use of screaming-snake-case pattern on driver ID
• Rename source-file property from sdmxdl.sources to sdmxdl.registry.sourcesFile
• Rename user-agent property from http.agent to sdmxdl.driver.userAgent
• Rename dump-folder property from sdmxdl.ri.web.dump.folder to sdmxdl.driver.dumpFolder
• Replace curl-backend property sdmxdl.networking.curlBackend with url-backend property sdmxdl.networking.urlBackend
• Replace list drivers command with list plugins command

Fixed

• Update OECD source #543
• Update INEGI source #639
• Update ILO source #690

3.0.0-beta.12 - 2023-10-16

This is the twelfth beta release of sdmx-dl.
sdmx-dl follows semantic versioning.

This release brings a more flexible configuration system: cache, network and language can be specified by source if needed and environment variables are used as default values.
The API has been refactored to make it both simpler and more flexible.
The ECB endpoint has also been updated.

Disclaimer: sdmx-dl is still in development and is available for test only. Do not use in production!

Added

• Add support of partial ISO date/time in TimeInterval
• Allow caching & networking configuration by environment variables #516
• Allow caching & networking configuration per source #493

Fixed

• Fix file locking in cache

Changed

• Refactor cache API #500
• Refactor network API #503
• Remove dialect API #505
• Refactor listeners API #506
• Handle languages per connection instead of per manager #507
• Simplify naming and structure #508
• Change pattern of drivers properties
• Improve support of ISO-8601 duration
• Update ECB endpoint #495
• Refactor file format API #502
• Move curl backend to an external project
• Move PxWebDriver to its own module
• Move dialects drivers to their own module
• Use environment variables as default values
• Migrate gRPC module to Quarkus framework

3.0.0-beta.11 - 2023-06-02

This is the eleventh beta release of sdmx-dl.
sdmx-dl follows semantic versioning.

This release adds new sources from Eurostat and the European Commission. It also improves performance of several drivers alongside the usual bug fixes.

Disclaimer: sdmx-dl is still in development and is available for test only. Do not use in production!

Added

• Add feature descriptor DATA_QUERY_ALL_KEYWORD
• Add sources from Directorates General of the European Commission #414
• Add source from Eurostat International trade in goods #415

Fixed

• Fix URL squashing in curl backend #417
• Fix cache when two sources share the same host #413

Changed

• Improve request for available key codes
• Improve source from Economic and Social Commission for Asia and the Pacific #418
• Use protobuf as default cache format
• Improve Eurostat driver #416
• Add support of DATA_QUERY_ALL_KEYWORD feature in RI
• Replace error by empty stream when no data is available

Removed

• Remove feature descriptor DATA_QUERY_KEY

3.0.0-beta.10 - 2023-02-13

This is a bugfix release for the deployment of v3.0.0-beta.9.
See v3.0.0-beta.9 for complete changelog.

Fixed

• Fix missing javadoc resource

3.0.0-beta.9 - 2023-02-13

This is the ninth beta release of sdmx-dl.
sdmx-dl follows semantic versioning.

This release brings the API closer to the SDMX standard. It fixes an important problem with Eurostat. It also adds two proof of concept for future use.

Disclaimer: sdmx-dl is still in development and is available for test only. Do not use in production!

Added

• Add support of time intervals in observations #394
• Add support of redirections in curl backend #363
• Add desktop application proof of concept #401
• Add gRPC service proof of concept #402

Changed

• Sort observations chronologically #396
• Enforce non-null observations fields #396
• Invert fields in DataDetail #396
• Set default values to DataQuery #396
• Rename field Dimension#label as Dimension#name #395
• Rename field Attribute#label as Attribute#name #395
• Rename field DataStructure#label as DataStructure#name #395
• Rename field SdmxWebSource#name as SdmxWebSource#id #395
• Rename field SdmxWebSource#description as SdmxWebSource#name #395
• Set field Dataflow#description as optional #395
• Set field Obs#getPeriod as TimeInterval #394
• Improve parsing of time formats #394
• Rename module sdmx-dl-format-util as smdx-dl-format-base
• Rename module sdmx-dl-provider-util as smdx-dl-provider-base

Fixed

• Fix language consistency in names and descriptions #397
• Fix Eurostat endpoint #346

3.0.0-beta.8 - 2022-11-22

This is the eighth beta release of sdmx-dl.
sdmx-dl follows semantic versioning.

This release brings support of description in data flows. It also updates a few sources alongside the usual bugfixes.

Disclaimer: sdmx-dl is still in development and is available for test only. Do not use in production!

Added

• Add field Dataflow#description #287
• Add source from UN International Children's Emergency Fund #95

Changed

• Improve code coherence by replacing Dataflow#of(...) and DataQuery#of(...) with builders
• Rename field Dataflow#label as Dataflow#name
• Move SdmxCubeUtil from provider util to API
• Modify ISTAT endpoint #339
• Modify list/flows command headers to follow Dataflow changes
• Enforce https in doc URL

Fixed

• Fix dependency inheritance in BOM

3.0.0-beta.7 - 2022-08-10

This is the seventh beta release of sdmx-dl.
sdmx-dl follows semantic versioning.

This release reshapes CLI commands, simplifies the network configuration and improves startup time. It introduces a new dedicated documentation to replace the wiki. It finalizes the support of SDMX time formats and attribute relationship in the API. A new source is also added alongside the usual bugfixes.

Disclaimer: sdmx-dl is still in development and is available for test only. Do not use in production!

Added

• Add URL connection factory in Network
• Add SdmxWebSource#monitorWebsite field
• Add SdmxWebSource#getDescription(LanguagePriorityList) method
• Add support of attribute relationship #81
• Add source from National Statistical Office of Thailand #262
• Add support of time range in time formats
• Add support of POST requests in HTTP backend
• Add driver for International Monetary Fund source #156
• Add list/availability command
• Add MonitorWebsite column in list/sources command
• Add Languages column in list/sources command #266

Changed

• Improve error reporting on content-type in HTTP response header
• Use option instead of property to enable curl backend
• Remove cache folder option
• Replace position column with index column in list/concepts command
• Split command list/concepts into list/dimensions and list/attributes
• Improve startup time when using custom sources
• Remove sdmx-dl-provider-connectors dependency
• Remove custom name of shaded binary
• Migrate documentation to GitHub Pages #268

Fixed

• Fix uptime parsing on multi-thread environment
• Fix parsing of HTTP response header in curl backend
• Fix NPE on missing HTTP response message
• Fix DataflowRef validation
• Fix Statistics Canada revisions #252
• Fix missing charsets in native image
• Fix registration of system SSL in native image
• Fix usage of dependency-reduced pom

3.0.0-beta.6 - 2022-04-21

This is the sixth beta release of sdmx-dl.
sdmx-dl follows semantic versioning.

This release focuses on API refactoring to allow future improvements. It introduces a mechanism that validates the input parameters to give a better feedback in case of error. A few sources are also added.

Disclaimer: sdmx-dl is still in development and is available for test only. Do not use in production!

Added

• Add parameters validity check #138
• Add multi-language descriptions in SdmxWebSource #203
• Add source from El Salvador Labour Market Information System #202
• Add source from Statistics Luxembourg #245
• Add Maven BOM

Changed

• Refactor data queries #218
• Refactor features discovery #89
• Simplify class hierarchy #222
• Simplify class naming scheme #220
• Refactor dialects handling #227
• Simplify module names
• Refactor format and provider utilities
• Enforce https on ISTAT source
• Change FileDriver to be optional

Fixed

• Fix missing series header in compact data #172
• Fix input validation in Statistics Canada driver #171
• Fix curl certificate revocation checks on missing/offline check lists
• Fix Java package requirement in Homebrew distribution #206
• Fix use of reflection in native image
• Fix missing resources in native image

3.0.0-beta.5 - 2021-11-23

This is the fifth beta release of sdmx-dl.
sdmx-dl follows semantic versioning.

Note that sdmx-dl is still in heavy development and might change a lot between versions, so you shouldn't use it in production.

This release adds the support of most SDMX time formats. The execution time of CLI is reduced greatly by using a smarter cache and by avoiding to load unnecessary resources. A few sources are also added, including a random data generator that can be used for tests/demos.

Added

• Add support of codelist in API #158
• Add native driver for INSEE #157
• Add native driver for Eurostat #155
• Add second layer of cache in CLI
• Add availability check on driver SPI #162
• Add optional RNG driver #163
• Add cache-folder-path option in CLI
• Add source from Statistics Canada #165
• Add support of non-calendar reporting periods

Changed

• Refactor network API to avoid unnecessary resource loading
• Invert default activation of auto-proxy option in CLI
• Simplify name of auto-proxy option in CLI
• Replace multiple parameters with DataRef in API
• Change SdmxWebSource#endpoint type to URI
• Change SdmxWebSource#monitor type to URI
• Change cache default folder to java.io.tmpdir/sdmx-dl/VERSION
• Refactor parsing of SDMX time formats

Fixed

• Improve error reporting in CLI
• Restore setup-completion command
• Fix setup commands names
• Fix datetime parsing of undefined freq
• Fix dimension ordering when used in key
• Fix external references in flows
• Migrate ABS source to the new server #168

3.0.0-beta.4 - 2021-10-21

This is the fourth beta release of sdmx-dl.
sdmx-dl follows semantic versioning.

Note that sdmx-dl is still in heavy development and might change a lot between versions, so you shouldn't use it in production.

This release simplifies the CLI options and fixes web monitors by using a new self-hosted provider.
The CLI binary is now also available in a JBang catalog and in a snapshot repository .

Added

• Add a link to documentation in CLI #137
• Add support of average response time in monitors #144
• Add Maven deployment of CLI
• Add distribution to JBang #141

Changed

• Rename command check.properties as check.config #136
• Migrate web monitors to Upptime #142
• Refactor cache to handle web monitors
• Simplify CSV command options #88

Fixed

• Fix missing content in subsequent calls to fetch-meta command #154
• Fix detection of console encoding

3.0.0-beta.3 - 2021-09-13

This is the third beta release of sdmx-dl.
sdmx-dl follows semantic versioning.

Note that sdmx-dl is still in heavy development and might change a lot between versions, so you shouldn't use it in production.

This release adds new sources and improves feedback on request parameters such as Key and Flow.
The CLI binary is now available in some popular package managers.
Finally, sdmx-dl advertises itself on web queries with the user-agent sdmx-dl/3.0.0-beta.3.

Added

• Add support of multiple media-types in HTTP request header (RI)
• Add support of media-type charset in HTTP response header (RI)
• Add response event #onSuccess(MediaType) (RI)
• Add default user-agent sdmx-dl/version (RI) #117
• Add source from Bundesbank #104
• Add source from Economic and Social Commission for Asia and the Pacific #99
• Add source from National Statistical Institute of Cambodia #98
• Add source from SDMX Global Registry #97
• Add source from Pacific Data Hub #96
• Add source from UK Data Service #93
• Add distribution to Scoop (Windows) #83
• Add distribution to Homebrew (macOS & Linux) #84

Changed

• Enforce https on ABS source #108
• Improve CLI version option #79
• Refactor SdmxWebAuthenticator as an SPI
• Improve feedback on missing data flow #123

Fixed

• Fix parsing of blank labels
• Update ILO source with new endpoint #107
• Fix key parsing when time dimension is not last in data structure #110
• Fix key validity check on input #118
• Fix parsing of media types in SDMX21 driver

3.0.0-beta.2 - 2021-05-03

This is the second beta release of sdmx-dl.
sdmx-dl follows semantic versioning.

Note that sdmx-dl is still in heavy development and might change a lot between versions, so you shouldn't use it in production.

This release adds new sources, the support of attributes and modify the CLI commands.
These command modifications are quite extended and concern command names, overall structure and output.

Added

• Add FileDriver to allow demo data
• Add initial support of native image
• Add file reading feedback
• Improve file performance in worst-case scenarios
• Add source from Bank for International Settlements (BIS)
• Add source from Norges Bank (NB)
• Add web service endpoint monitoring
• Add reading of attributes
• Add library deployment to Maven Central
• Add GPG signature of binaries
• Add basic documentation

Changed

• Modify CLI command name, structure and output

3.0.0-beta.1 - 2020-09-25

This is the first beta release of sdmx-dl.
sdmx-dl follows semantic versioning.

Note that sdmx-dl is still in heavy development and might change a lot between versions, so you shouldn't use it in production.

Added

• Initial release