Switch parameters to YAML

.gitignore

@@ -1,6 +1,7 @@
 .project
 .spyderproject
 .pydevproject
+.vscode
 .settings/
 build/
 dist/
@@ -12,4 +13,4 @@ dist/
 /.coverage
 /tags
 .tags*
-.noseids
+.noseids

CHANGELOG.md

@@ -1,5 +1,199 @@
 # Changelog
 
+# 17.0.0 - [#552](https://github.com/openfisca/openfisca-core/pull/552)
+
+#### Breaking changes
+
+##### Parameter files
+
+* Load the legislation parameters from a directory `parameters` containing YAML files, instead of XML files
+
+  Before:
+  ```XML
+  <NODE code="root">
+    <NODE code="impot">
+      <CODE code="taux" description="" format="percent">
+        <END deb="2016-01-01"/>   
+        <VALUE deb="2015-01-01" valeur="0.32" />
+        <VALUE deb="1998-01-01" valeur="0.3" />
+      </CODE>
+    </NODE>
+  </NODE>
+  ```
+
+  Now:
+  ```yaml
+  impot:
+    taux:
+      description: Taux d'impôt sur les salaires
+      unit: /1
+      values:
+        2016-01-01:
+          value: null
+        2015-01-01:
+          value: 0.32
+        1991-01-01:
+          value: 0.3
+  ```
+
+  - The XML attributes `format` and `type` are replaced by the YAML attribute `unit`, which is a free text field.
+
+##### reforms
+
+* Rename `Reform.modify_legislation_json()` -> `Reform.modify_parameters()`
+* Remove `reforms.compose_reforms()`.
+* Remove `reforms.update_legislation()`.
+  - To modify an existing parameter, use `ValuesHistory.update()` instead.
+  - You can navigate the parameters using `.` (e.g. `parameters.taxes.tax_on_salaries.public_sector.rate`)
+  - Before:
+    ```python
+    new_legislation_parameters = update_legislation(
+        legislation_json = original_legislation_parameters,
+        path = ('children', 'impot_revenu', 'children', 'bareme', 'brackets', 1, 'threshold'),
+        period = reform_period,
+        value = 6011,
+        )
+    ```
+  - Now:
+    ```python
+    parameters.impot_revenu.bareme[1].threshold.update(period = reform_period, value = 6011)
+    ```
+
+* Change the syntax to dynamically create new parameters
+  - Before :
+    ```python
+    reform_legislation_subtree = {
+        "@type": "Node",
+        "description": "PLF 2016 sur revenus 2015",
+        "children": {
+            "decote_seuil_celib": {
+                "@type": "Parameter",
+                "description": "Seuil de la décôte pour un célibataire",
+                "format": "integer",
+                "unit": "currency",
+                "values": [
+                    {'start': u'2016-01-01', },
+                    {'start': u'2015-01-01', 'value': round(1135 * (1 + inflation))},
+                    ],
+                },
+            "decote_seuil_couple": {
+                "@type": "Parameter",
+                "description": "Seuil de la décôte pour un couple",
+                "format": "integer",
+                "unit": "currency",
+                "values": [
+                    {'start': u'2065-01-01', },
+                    {'start': u'2015-01-01', 'value': round(1870 * (1 + inflation))},
+                    ],
+                },
+            },
+        }
+    reference_legislation_copy['children']['plf2016_conterfactual'] = reform_legislation_subtree
+    ```
+
+  - Now:
+    ```python
+    from openfisca_core.parameters import ParameterNode
+
+    inflation = .001
+    reform_parameters_subtree = ParameterNode('plf2016_conterfactual', data = {
+        'decote_seuil_celib': {
+          'values': {
+            "2015-01-01": {'value': round(1135 * (1 + inflation))},
+            "2016-01-01": {'value': None}
+            }
+          },
+        'decote_seuil_couple': {
+          'values': {
+            "2015-01-01": {'value': round(1870 * (1 + inflation))},
+            "2065-01-01": {'value': None}
+            }
+          },
+        })
+    reference_parameters.add_child('plf2016_conterfactual', reform_parameters_subtree)
+    ```
+
+  - Note that this way of creating parameters is only recommanded when using dynamically computed values (for instance `round(1135 * (1 + inflation))` in the previous example). If the values are static, the new parameters can be directly built from YAML (See New features section).
+
+##### TaxBenefitSystem
+
+* Remove parameter `legislation_json` in constructor
+* Remove methods/attributes:
+  - `compact_legislation_by_instant_cache`
+  - `get_baseline_compact_legislation`
+  - `compute_legislation`
+  - `get_legislation`
+    + We can now directly use the `parameters` attribute.
+* Rename
+  - `preprocess_legislation` -> `preprocess_parameters`
+  - `add_legislation_params` -> `load_parameters`
+  - `get_compact_legislation` -> `get_parameters_at_instant`
+    + The signature of the method has changed. Check the [documentation](http://openfisca.readthedocs.io/en/latest/tax-benefit-system.html#openfisca_core.taxbenefitsystems.TaxBenefitSystem.load_parameters).
+
+##### Simulation
+
+* Remove methods/attributes:
+  - `compact_legislation_by_instant_cache`
+  - `get_baseline_compact_legislation`
+* Rename
+    `baseline_compact_legislation_by_instant_cache` -> `baseline_parameters_at_instant_cache`
+    `legislation_at` -> `parameters_at`
+
+#### New features
+
+* In reforms, new parameters can be added from a YAML file.
+  - The function `parameters.load_parameter_file()` loads a YAML file.
+  - The function `ParameterNode.add_child()` adds a new child to an existing legislation node.
+  - Example:
+    ```python
+    import os
+    from openfisca_core.parameters import load_parameter_file
+
+    dir_path = os.path.dirname(__file__)
+
+    def reform_modify_parameters(parameters):
+        file_path = os.path.join(dir_path, 'plf2016.yaml')
+        reform_parameters_subtree = load_parameter_file(name = 'plf2016', file_path=file_path)
+        parameters.add_child('plf2016', reform_parameters_subtree)
+        return parameters
+
+    ...
+    ```
+
+* In module model_api, add classes that are needed to build reforms:
+  - `load_parameter_file`
+  - `ParameterNodeNode`
+  - `Scale`
+  - `Bracket`
+  - `Parameter`
+  - `ValuesHistory`
+  - `periods.period`.
+
+
+#### Technical changes
+
+* Refactor the internal representation and the interface of legislation parameters
+  - The parameters of a legislation are wraped into the classes `Node`, `Parameter`, `Scale`, `Bracket`, `ValuesHistory`, `ValueAtInstant` instead of bare python dict.
+  - The parameters of a legislation at a given instant are wraped into the classes `NodeAtInstant`, `ValueAtInstant` and tax scales instead of bare python objects.
+  - The file `parameters.py` and the classes defined inside are responsible both for loading and accessing the parameters. Before the loading was implemented in `legislationsxml.py` and the other processings were implemented in `legislations.py`
+  - The validation of the XML files was performed against a XML schema defined in `legislation.xsd`. Now the YAML files are loaded with the library `yaml` and then validated in basic Python.
+
+* The word "legislation" is replaced by the word "parameters" in several internal variables and internal methods. It Reduced the ambiguity between the legislation as a tax and benefit system and the legislation as the parameters.
+  - `TaxBenefitSystem.compact_legislation_by_instant_cache` -> `TaxBenefitSystem._parameters_at_instant_cache`
+  - `TaxBenefitSystem.get_baseline_compact_legislation()` -> `TaxBenefitSystem._get_baseline_parameters_at_instant()`
+  - `Simulation.compact_legislation_by_instant_cache` -> `Simulation._parameters_at_instant_cache`
+  - `Simulation.get_compact_legislation()` -> `Simulation._get_parameters_at_instant()`
+  - `Simulation.get_baseline_compact_legislation()` -> `Simulation._get_baseline_parameters_at_instant()`
+
+* The optionnal parameter `traced_simulation` is removed in function `TaxBenefitSystem.get_compact_legislation()` (now `TaxBenefitSystem.get_parameters_at_instant()`). This parameter had no effect.
+
+* The optional parameter `with_source_file_infos` is removed in functions `TaxBenefitSystem.compute_legislation()` (now `TaxBenefitSystem._compute_parameters()`) and `TaxBenefitSystem.get_legislation()`. This parameter had no effect.
+
+* In the API preview, update the internal transformation of the parameters.
+
+* In the directory `script`, add a subdirectory `migrations`.
+
+
 ## 16.3.0
 
 - Support `reference` attributes on all parameter XML nodes.

MANIFEST.in

@@ -1,3 +1,2 @@
 recursive-include openfisca_core/scripts *
-include openfisca_core/legislation.xsd
 include openfisca_web_api_preview/openAPI.yml

doc/source/conf.py

@@ -13,3 +13,4 @@
 html_context = {
     'css_files': ['_static/css/custom.css']
     }
+autoclass_content = 'both'

doc/source/index.rst

@@ -7,6 +7,8 @@ Modules:
    :maxdepth: 1
 
    tax-benefit-system
+   parameters
+   reforms
    test_runner
 
 Scripts:
@@ -16,19 +18,6 @@ Scripts:
 
    openfisca-run-test
 
-Web-API routes:
-
-.. toctree::
-   :maxdepth: 1
-
-   parameter
-   parameters
-   variable
-   variables
-
-
-
-
 Indices and tables
 ==================
 

doc/source/parameters.rst

@@ -1,46 +1,6 @@
-==============
-``/parameters``
-==============
+==========
+Parameters
+==========
 
-.. http:get:: /parameters
-
-    Get the parameters contained in the tax and benefit system.
-
-    The response is formatted as an object whose keys are the parameters id and values objects with a single property, description.
-
-    :resheader Country-Package: The name of the country package served by the API
-    :resheader Country-Package-Version: The version of the country package served by the API
-    :statuscode 200: The list of all parameters is sent back in the response body.
-
-    **Example request**:
-
-    .. sourcecode:: http
-
-        GET /parameters HTTP/1.1
-
-    **Example response**:
-
-    .. sourcecode:: http
-
-        HTTP/1.1 200 OK
-        Country-Package: openfisca-france
-        Country-Package-Version: 18.2.1
-        Content-Type: application/json
-
-        {
-          "benefits.basic_income": {
-            "description": "Amount of the basic income"
-          },
-          "benefits.housing_allowance": {
-            "description": "Housing allowance amount (as a fraction of the rent)"
-          },
-          "general.age_of_majority": {
-            "description": "Age of majority"
-          },
-          "taxes.income_tax_rate": {
-            "description": "Income tax rate"
-          },
-          "taxes.social_security_contribution": {
-            "description": "Social security contribution tax scale"
-          }
-        }
+.. automodule:: openfisca_core.parameters
+    :members: