feat(*): anonymization & archive functionality (#2012)

Entity Details views in the context (three-dot) menu now have actions to "anonymize" and "archive" in addition to "delete".
This functionality includes extensive confirmation dialogs and info boxes to explain the implications and show the state of an entity to the user. Inactive (i.e. archived) records always hidden by default and a toggle to include them is always available below entity lists.

Anonymization logic can be configured for each entity using the EntityField schema, specifying whether a property shall be retained even after anonymization. By default values are removed if not configured otherwise. For more details see the Developer Documentation.

MIGRATION RECOMMENDED:
existing list filters using "isActive" should be removed, as there now is a default filter built in

closes #1674, closes #899
Co-authored-by: Simon <simon@aam-digital.com>

---------
This functionality has been developed for the project “codo”.
codo is developed under the projects “Landungsbrücken – Patenschaften in Hamburg stärken” and “openTransfer Patenschaften”. It is funded through the program “Menschen stärken Menschen” by the German Federal Ministry of Family Affairs, Senior Citizens, Women and Youth.
More information at https://github.com/codo-mentoring

“Landungsbrücken – Patenschaften in Hamburg stärken” is a project of BürgerStiftung Hamburg in cooperation with the Mentor.Ring Hamburg. With a mix of networking opportunities, capacity building and financial support the project strengthens Hamburg’s scene of mentoring projects since its founding in 2016.

The “Stiftung Bürgermut” foundation since 2007 supports the digital and real exchange of experiences and connections of active citizens. Within the federal program “Menschen stärken Menschen” the foundation as part of its program “openTransfer Patenschaften” offers support services for connecting, spreading and upskilling mentoring organisations across Germany.

Diese Funktion wurde entwickelt für das Projekt codo.
codo wird entwickelt im Rahmen der Projekte Landungsbrücken – Patenschaften in Hamburg stärken und openTransfer Patenschaften. Er ist gefördert durch das Bundesprogramm Menschen stärken Menschen des Bundesministeriums für Familie, Senioren, Frauen und Jugend.
Mehr Informationen unter https://github.com/codo-mentoring

“Landungsbrücken – Patenschaften in Hamburg stärken” ist ein Projekt der BürgerStiftung Hamburg in Kooperation mit dem Mentor.Ring Hamburg. Mit einer Mischung aus Vernetzungsangeboten, Qualifizierungsmaßnahmen und finanzieller Förderung stärkt das Projekt die Hamburger Szene der Patenschaftsprojekte seit der Gründung im Jahr 2016.

Die Stiftung Bürgermut fördert seit 2007 den digitalen und realen Erfahrungsaustausch und die Vernetzung von engagierten Bürger:innen. Innerhalb des Bundesprogramms „Menschen stärken Menschen” bietet die Stiftung im Rahmen ihres Programms openTransfer Patenschaften Unterstützungsleistungen zur Vernetzung, Verbreitung und Qualifizierung von Patenschafts- und Mentoringorganisationen bundesweit.

Co-authored-by: codo-mentoring <117934638+codo-mentoring@users.noreply.github.com>

doc/compodoc_sources/concepts/entity-schema-system.md

@@ -1,8 +1,14 @@
-# Entity Schema 
+# Entities & Entity Schema 
 -----
-The Entity Schema defines details of the properties of an entity type.
-(An "entity" is an object of a certain type that users work with and save to the database, like "Child" or "School")
+For us, an "Entity" is an object in the database (and a representation of something in the user's real world, e.g. a "Child" or "School").
+Entities are at the core of the Aam Digital platform and the primary way to customize the system is to adapt and add new entity types.
+
+The Entity Schema defines the data structure as well as how it is displayed in the UI.
+Entity instances also have some generic functionality inherited from the `Entity` base class.
 
+
+## Entity Schema
+The Entity Schema defines details of the properties of an entity type.
 We define an entity type and its schema in code through a plain TypeScript class and some custom annotations.
 Read more on the background and practical considerations in [How to create a new Entity Type](../how-to-guides/create-a-new-entity-type.html).
 
@@ -45,9 +51,45 @@ providing these through Angular dependency injection using `multi: true`.
 
 Also see: [How to create a new Datatype](../how-to-guides/create-a-new-datatype.html).
 
-## Schema options
+### Schema options
 
 The schema definitions contains information regarding the schema transformation as well as how a property can be displayed.
 The [EntitySchemaField](../../interfaces/EntitySchemaField.html) interface shows all configuration options.
 If the `editComponent` and the `viewComponent` are not set, the default components of this property's datatype will be used.
 The `description` field allows adding further explanation which will be displayed as a tooltip.
+
+
+## Generic Entity functionalities
+
+### Metadata (created, updated)
+Each record automatically holds basic data of timestamp and user who created and last updated the record.
+(see `Entity` class)
+
+### Archive / Anonymize
+Any entity can be archived (i.e. marked as inactive and hidden from UI by default) or anonymized (i.e. discarding most data and keeping a few selected properties for statistical reports).
+This is often preferable to deleting a record completely. Deleting data also affects statistical reports, even for previous time periods.
+By anonymizing records, all personal identifiable data can be removed and the remaining stub record can be stored indefinitely, as it is not subject to data protection regulations like GDPR anymore.
+
+Anonymization is configured as part of the entity schema.
+Data of fields that are not explicitly marked to be retained during anonymization is always deleted (anonymization by default).
+
+To keep some data even after the user "anonymized" a record, configure the `anonymize` property of the `@DatabaseField` decorator:
+- `anonymize: "retain"` will keep this field unchanged and prevent it from being deleted
+- `anonymize: "retain-anonymized"` will trigger a special "partial" deletion that depends on the dataType (e.g. date types will be changed to 1st July of the given year, thereby largely removing details but keeping data to calculate a rough age)
+
+#### Data Protection & GDPR regarding anonymization / pseudonomyzation
+The "anonymize" function is implemented specifically for data protection rules requiring to delete personal data.
+According to the EU's "General Data Protection Regulation" (GDPR) "anonymous" data does not fall under its regulations:
+
+- GDPR is not applicable to anonymous data: "The principles of data protection should therefore not apply to [...] personal data rendered anonymous in such a manner that the data subject is not or no longer identifiable." [<sup>GDPR Recital 26</sup>](https://gdpr-info.eu/recitals/no-26/)
+  - "To determine whether a natural person is identifiable, account should be taken of all the means reasonably likely to be used, such as singling out, either by the controller or by another person to identify the natural person directly or indirectly."
+  - "To ascertain whether means are reasonably likely to be used to identify the natural person, account should be taken of all objective factors, such as the costs of and the amount of time required for identification, taking into consideration the available technology at the time of the processing and technological developments."
+- "Pseudonymisation enables the personal data to become unidentifiable unless more information is available whereas anonymization allows the processing of personal data to irreversibly prevent re-identification." [<sup>source</sup>](https://www.privacycompany.eu/blogpost-en/what-are-the-differences-between-anonymisation-and-pseudonymisation)
+- _also see this [good overview of anonymization misunderstandings and considerations](https://edps.europa.eu/system/files/2021-04/21-04-27_aepd-edps_anonymisation_en_5.pdf)_
+
+In the case of records being retained "anonymized" in Aam Digital, we provide a context that makes re-identification even harder:
+- only authorized users of the system can access even the anonymized record (where only a few properties have been retained). Unless the organisation actively shares the data, it remains as securely protected as the personal data managed in Aam Digital.
+- those authorized users with access to the anonymized records (and therefor a theoretical chance to attempt re-identification) are team members of an organization. They have been screened to be responsible persons and are usually legally bound to keep information confidential.
+- by default only a few, explicitly selected properties in anonymized records are retained (data minimization by default). As such, both re-identification likelihood and the impact in case of re-identification are reduced as far as possible.
+
+--> If our anonymization process is configured thoughfully on a case by case basis to only retain a few data fields that are not easy indirect identifiers, it seems reasonably unlikely that the person can be identified after the anonymization process. Therefore, GDPR should not apply to these records and it is legitimate to retain these for statistical reporting.

src/app/child-dev-project/attendance/model/recurring-activity.ts

@@ -55,6 +55,7 @@ export class RecurringActivity extends Entity {
     validators: {
       required: true,
     },
+    anonymize: "retain",
   })
   title: string = "";
 
@@ -67,6 +68,7 @@ export class RecurringActivity extends Entity {
     label: $localize`:Label for the interaction type of a recurring activity:Type`,
     dataType: "configurable-enum",
     innerDataType: INTERACTION_TYPE_CONFIG_ID,
+    anonymize: "retain",
   })
   type: InteractionType;
 
@@ -75,6 +77,7 @@ export class RecurringActivity extends Entity {
     label: $localize`:Label for the participants of a recurring activity:Participants`,
     dataType: "entity-array",
     additional: Child.ENTITY_TYPE,
+    anonymize: "retain",
   })
   participants: string[] = [];
 
@@ -83,6 +86,7 @@ export class RecurringActivity extends Entity {
     label: $localize`:Label for the linked schools of a recurring activity:Groups`,
     dataType: "entity-array",
     additional: School.ENTITY_TYPE,
+    anonymize: "retain",
   })
   linkedGroups: string[] = [];
 
@@ -91,6 +95,7 @@ export class RecurringActivity extends Entity {
     label: $localize`:Label for excluded participants of a recurring activity:Excluded Participants`,
     dataType: "entity-array",
     additional: Child.ENTITY_TYPE,
+    anonymize: "retain",
   })
   excludedParticipants: string[] = [];
 
@@ -99,6 +104,7 @@ export class RecurringActivity extends Entity {
     label: $localize`:Label for the assigned user(s) of a recurring activity:Assigned user(s)`,
     dataType: "entity-array",
     additional: User.ENTITY_TYPE,
+    anonymize: "retain",
   })
   assignedTo: string[] = [];
 

src/app/child-dev-project/children/health-checkup/model/health-check.ts

@@ -30,8 +30,15 @@ export class HealthCheck extends Entity {
     return Object.assign(new HealthCheck(), contents);
   }
 
-  @DatabaseField() child: string;
-  @DatabaseField({ label: $localize`:Label for date of a health check:Date` })
+  @DatabaseField({
+    anonymize: "retain",
+  })
+  child: string;
+
+  @DatabaseField({
+    label: $localize`:Label for date of a health check:Date`,
+    anonymize: "retain-anonymized",
+  })
   date: Date;
 
   /** height measurement in cm **/

src/app/child-dev-project/children/model/child.spec.ts

@@ -57,5 +57,13 @@ describe("Child", () => {
     const testEntity3 = new Child();
     testEntity3["status"] = "Dropout";
     expect(testEntity3.isActive).withContext("Dropout").toBeFalse();
+
+    // always give "inactive" precedence over other logic (as it is trigger in UI)
+    const testEntityPrec = new Child();
+    testEntityPrec["inactive"] = false;
+    testEntityPrec["status"] = "Dropout";
+    expect(testEntityPrec.isActive)
+      .withContext("inactive taking precedence")
+      .toBeTrue();
   });
 });

src/app/child-dev-project/children/model/child.ts

@@ -54,31 +54,36 @@ export class Child extends Entity {
     label: $localize`:Label for the project number of a child:Project Number`,
     labelShort: $localize`:Short label for the project number:PN`,
     searchable: true,
+    anonymize: "retain",
   })
   projectNumber: string;
 
   @DatabaseField({
     label: $localize`:Label for the date of birth of a child:Date of birth`,
     labelShort: $localize`:Short label for the date of birth:DoB`,
+    anonymize: "retain-anonymized",
   })
   dateOfBirth: DateWithAge;
 
   @DatabaseField({
     dataType: "configurable-enum",
     label: $localize`:Label for the gender of a child:Gender`,
     innerDataType: "genders",
+    anonymize: "retain",
   })
   gender: ConfigurableEnumValue;
 
   @DatabaseField({
     dataType: "configurable-enum",
     innerDataType: "center",
     label: $localize`:Label for the center of a child:Center`,
+    anonymize: "retain",
   })
   center: Center;
 
   @DatabaseField({
     label: $localize`:Label for the admission date of a child:Admission`,
+    anonymize: "retain-anonymized",
   })
   admissionDate: Date;
 
@@ -89,11 +94,13 @@ export class Child extends Entity {
 
   @DatabaseField({
     label: $localize`:Label for the dropout date of a child:Dropout Date`,
+    anonymize: "retain-anonymized",
   })
   dropoutDate: Date;
 
   @DatabaseField({
     label: $localize`:Label for the type of dropout of a child:Dropout Type`,
+    anonymize: "retain",
   })
   dropoutType: string;
 
@@ -120,6 +127,11 @@ export class Child extends Entity {
   phone: string;
 
   get isActive(): boolean {
+    if (this.inactive !== undefined) {
+      // explicit property set through UI has to take precedence
+      return super.isActive;
+    }
+
     return (
       this.status !== "Dropout" &&
       !this["dropoutDate"] &&

src/app/child-dev-project/children/model/childSchoolRelation.ts

@@ -16,6 +16,7 @@ export class ChildSchoolRelation extends TimePeriod {
     validators: {
       required: true,
     },
+    anonymize: "retain",
   })
   childId: string;
 
@@ -26,6 +27,7 @@ export class ChildSchoolRelation extends TimePeriod {
     validators: {
       required: true,
     },
+    anonymize: "retain",
   })
   schoolId: string;
 

src/app/child-dev-project/notes/model/note.ts

@@ -77,6 +77,7 @@ export class Note extends Entity {
     dataType: "entity-array",
     additional: Child.ENTITY_TYPE,
     editComponent: "EditAttendance",
+    anonymize: "retain",
   })
   children: string[] = [];
 
@@ -85,42 +86,55 @@ export class Note extends Entity {
    *
    * No direct access to change this property. Use the `.getAttendance()` method to have safe access.
    */
-  @DatabaseField({ innerDataType: "schema-embed", additional: EventAttendance })
+  @DatabaseField({
+    innerDataType: "schema-embed",
+    additional: EventAttendance,
+    anonymize: "retain",
+  })
   private childrenAttendance: Map<string, EventAttendance> = new Map();
 
   @DatabaseField({
     label: $localize`:Label for the date of a note:Date`,
     dataType: "date-only",
     defaultValue: PLACEHOLDERS.NOW,
+    anonymize: "retain",
   })
   date: Date;
+
   @DatabaseField({ label: $localize`:Label for the subject of a note:Subject` })
   subject: string;
+
   @DatabaseField({
     label: $localize`:Label for the actual notes of a note:Notes`,
     editComponent: "EditLongText",
   })
   text: string;
+
   /** IDs of users that authored this note */
   @DatabaseField({
     label: $localize`:Label for the social worker(s) who created the note:SW`,
     dataType: "entity-array",
     additional: User.ENTITY_TYPE,
     defaultValue: PLACEHOLDERS.CURRENT_USER,
+    anonymize: "retain",
   })
   authors: string[] = [];
 
   @DatabaseField({
     label: $localize`:Label for the category of a note:Category`,
     dataType: "configurable-enum",
     innerDataType: INTERACTION_TYPE_CONFIG_ID,
+    anonymize: "retain",
   })
   category: InteractionType;
 
   /**
    * id referencing a different entity (e.g. a recurring activity) this note is related to
    */
-  @DatabaseField() relatesTo: string;
+  @DatabaseField({
+    anonymize: "retain",
+  })
+  relatesTo: string;
 
   /**
    * other records (e.g. a recurring activity, group membership, ...) to which this note is related in some way,
@@ -134,6 +148,7 @@ export class Note extends Entity {
     editComponent: "EditEntityArray",
     // by default no additional relatedEntities can be linked apart from children and schools, overwrite this in config to display (e.g. additional: "ChildSchoolRelation")
     additional: undefined,
+    anonymize: "retain",
   })
   relatedEntities: string[] = [];
 
@@ -144,13 +159,15 @@ export class Note extends Entity {
     label: $localize`:label for the linked schools:Groups`,
     dataType: "entity-array",
     additional: School.ENTITY_TYPE,
+    anonymize: "retain",
   })
   schools: string[] = [];
 
   @DatabaseField({
     label: $localize`:Status of a note:Status`,
     dataType: "configurable-enum",
     innerDataType: "warning-levels",
+    anonymize: "retain",
   })
   warningLevel: Ordering.EnumValue;