diff --git a/doc/compodoc_sources/concepts/entities.md b/doc/compodoc_sources/concepts/entities.md new file mode 100644 index 0000000000..0a26de0dae --- /dev/null +++ b/doc/compodoc_sources/concepts/entities.md @@ -0,0 +1,10 @@ +# Entities & Entity Schema +----- +For us, an "Entity" is an object in the database (and a representation of something in the user's real world, e.g. a "Child" or "School"). +Entities are at the core of the Aam Digital platform and the primary way to customize the system is to adapt and add new entity types. + +The Entity Schema defines the data structure as well as how it is displayed in the UI. +Entity instances also have some generic functionality inherited from the `Entity` base class. + +------ +_see the sub-pages here for details of the various concepts related to the Entity system_ diff --git a/doc/compodoc_sources/concepts/entity-anonymization.md b/doc/compodoc_sources/concepts/entity-anonymization.md new file mode 100644 index 0000000000..421b176ea0 --- /dev/null +++ b/doc/compodoc_sources/concepts/entity-anonymization.md @@ -0,0 +1,39 @@ +# Archive / Anonymize Entities +----- +Any entity can be archived (i.e. marked as inactive and hidden from UI by default) or anonymized (i.e. discarding most data and keeping a few selected properties for statistical reports). +This is often preferable to deleting a record completely. Deleting data also affects statistical reports, even for previous time periods. +By anonymizing records, all personal identifiable data can be removed and the remaining stub record can be stored indefinitely, as it is not subject to data protection regulations like GDPR anymore. + +Anonymization is configured as part of the entity schema. +Data of fields that are not explicitly marked to be retained during anonymization is always deleted (anonymization by default). + +To keep some data even after the user "anonymized" a record, configure the `anonymize` property of the `@DatabaseField` decorator: +- `anonymize: "retain"` will keep this field unchanged and prevent it from being deleted +- `anonymize: "retain-anonymized"` will trigger a special "partial" deletion that depends on the dataType (e.g. date types will be changed to 1st July of the given year, thereby largely removing details but keeping data to calculate a rough age) + + +## Cascading anonymization / deletion +Relationships between entities are automatically handled when the user anonymizes or deletes an entity. +Any related entities that reference the anonymized/deleted entity are checked +and - depending on their configured role - may be updated or anonymized as well. + +The logic follows the scenarios shown below: +![](../../images/cascading-delete.png) + + +## Data Protection & GDPR regarding anonymization / pseudonomyzation +The "anonymize" function is implemented specifically for data protection rules requiring to delete personal data. +According to the EU's "General Data Protection Regulation" (GDPR) "anonymous" data does not fall under its regulations: + +- GDPR is not applicable to anonymous data: "The principles of data protection should therefore not apply to [...] personal data rendered anonymous in such a manner that the data subject is not or no longer identifiable." [GDPR Recital 26](https://gdpr-info.eu/recitals/no-26/) + - "To determine whether a natural person is identifiable, account should be taken of all the means reasonably likely to be used, such as singling out, either by the controller or by another person to identify the natural person directly or indirectly." + - "To ascertain whether means are reasonably likely to be used to identify the natural person, account should be taken of all objective factors, such as the costs of and the amount of time required for identification, taking into consideration the available technology at the time of the processing and technological developments." +- "Pseudonymisation enables the personal data to become unidentifiable unless more information is available whereas anonymization allows the processing of personal data to irreversibly prevent re-identification." [source](https://www.privacycompany.eu/blogpost-en/what-are-the-differences-between-anonymisation-and-pseudonymisation) +- _also see this [good overview of anonymization misunderstandings and considerations](https://edps.europa.eu/system/files/2021-04/21-04-27_aepd-edps_anonymisation_en_5.pdf)_ + +In the case of records being retained "anonymized" in Aam Digital, we provide a context that makes re-identification even harder: +- only authorized users of the system can access even the anonymized record (where only a few properties have been retained). Unless the organisation actively shares the data, it remains as securely protected as the personal data managed in Aam Digital. +- those authorized users with access to the anonymized records (and therefor a theoretical chance to attempt re-identification) are team members of an organization. They have been screened to be responsible persons and are usually legally bound to keep information confidential. +- by default only a few, explicitly selected properties in anonymized records are retained (data minimization by default). As such, both re-identification likelihood and the impact in case of re-identification are reduced as far as possible. + +--> If our anonymization process is configured thoughfully on a case by case basis to only retain a few data fields that are not easy indirect identifiers, it seems reasonably unlikely that the person can be identified after the anonymization process. Therefore, GDPR should not apply to these records and it is legitimate to retain these for statistical reporting. diff --git a/doc/compodoc_sources/concepts/entity-schema-system.md b/doc/compodoc_sources/concepts/entity-schema-system.md index 3de8a37d8f..9d55ab7aad 100644 --- a/doc/compodoc_sources/concepts/entity-schema-system.md +++ b/doc/compodoc_sources/concepts/entity-schema-system.md @@ -1,13 +1,5 @@ -# Entities & Entity Schema +# Entity Schema ----- -For us, an "Entity" is an object in the database (and a representation of something in the user's real world, e.g. a "Child" or "School"). -Entities are at the core of the Aam Digital platform and the primary way to customize the system is to adapt and add new entity types. - -The Entity Schema defines the data structure as well as how it is displayed in the UI. -Entity instances also have some generic functionality inherited from the `Entity` base class. - - -## Entity Schema The Entity Schema defines details of the properties of an entity type. We define an entity type and its schema in code through a plain TypeScript class and some custom annotations. Read more on the background and practical considerations in [How to create a new Entity Type](../how-to-guides/create-a-new-entity-type.html). @@ -64,32 +56,3 @@ The `description` field allows adding further explanation which will be displaye ### Metadata (created, updated) Each record automatically holds basic data of timestamp and user who created and last updated the record. (see `Entity` class) - -### Archive / Anonymize -Any entity can be archived (i.e. marked as inactive and hidden from UI by default) or anonymized (i.e. discarding most data and keeping a few selected properties for statistical reports). -This is often preferable to deleting a record completely. Deleting data also affects statistical reports, even for previous time periods. -By anonymizing records, all personal identifiable data can be removed and the remaining stub record can be stored indefinitely, as it is not subject to data protection regulations like GDPR anymore. - -Anonymization is configured as part of the entity schema. -Data of fields that are not explicitly marked to be retained during anonymization is always deleted (anonymization by default). - -To keep some data even after the user "anonymized" a record, configure the `anonymize` property of the `@DatabaseField` decorator: -- `anonymize: "retain"` will keep this field unchanged and prevent it from being deleted -- `anonymize: "retain-anonymized"` will trigger a special "partial" deletion that depends on the dataType (e.g. date types will be changed to 1st July of the given year, thereby largely removing details but keeping data to calculate a rough age) - -#### Data Protection & GDPR regarding anonymization / pseudonomyzation -The "anonymize" function is implemented specifically for data protection rules requiring to delete personal data. -According to the EU's "General Data Protection Regulation" (GDPR) "anonymous" data does not fall under its regulations: - -- GDPR is not applicable to anonymous data: "The principles of data protection should therefore not apply to [...] personal data rendered anonymous in such a manner that the data subject is not or no longer identifiable." [GDPR Recital 26](https://gdpr-info.eu/recitals/no-26/) - - "To determine whether a natural person is identifiable, account should be taken of all the means reasonably likely to be used, such as singling out, either by the controller or by another person to identify the natural person directly or indirectly." - - "To ascertain whether means are reasonably likely to be used to identify the natural person, account should be taken of all objective factors, such as the costs of and the amount of time required for identification, taking into consideration the available technology at the time of the processing and technological developments." -- "Pseudonymisation enables the personal data to become unidentifiable unless more information is available whereas anonymization allows the processing of personal data to irreversibly prevent re-identification." [source](https://www.privacycompany.eu/blogpost-en/what-are-the-differences-between-anonymisation-and-pseudonymisation) -- _also see this [good overview of anonymization misunderstandings and considerations](https://edps.europa.eu/system/files/2021-04/21-04-27_aepd-edps_anonymisation_en_5.pdf)_ - -In the case of records being retained "anonymized" in Aam Digital, we provide a context that makes re-identification even harder: -- only authorized users of the system can access even the anonymized record (where only a few properties have been retained). Unless the organisation actively shares the data, it remains as securely protected as the personal data managed in Aam Digital. -- those authorized users with access to the anonymized records (and therefor a theoretical chance to attempt re-identification) are team members of an organization. They have been screened to be responsible persons and are usually legally bound to keep information confidential. -- by default only a few, explicitly selected properties in anonymized records are retained (data minimization by default). As such, both re-identification likelihood and the impact in case of re-identification are reduced as far as possible. - ---> If our anonymization process is configured thoughfully on a case by case basis to only retain a few data fields that are not easy indirect identifiers, it seems reasonably unlikely that the person can be identified after the anonymization process. Therefore, GDPR should not apply to these records and it is legitimate to retain these for statistical reporting. diff --git a/doc/compodoc_sources/summary.json b/doc/compodoc_sources/summary.json index d0fa044c62..244192ddaa 100644 --- a/doc/compodoc_sources/summary.json +++ b/doc/compodoc_sources/summary.json @@ -38,8 +38,18 @@ "file": "concepts/extendability.md" }, { - "title": "Entity Schema", - "file": "concepts/entity-schema-system.md" + "title": "Entity System", + "file": "concepts/entities.md", + "children": [ + { + "title": "Entity Schema", + "file": "concepts/entity-schema-system.md" + }, + { + "title": "Archiving, Anonymizing and Deleting Entities", + "file": "concepts/entity-anonymization.md" + } + ] }, { "title": "Configuration", diff --git a/doc/images/cascading-delete.png b/doc/images/cascading-delete.png new file mode 100644 index 0000000000..178781d676 Binary files /dev/null and b/doc/images/cascading-delete.png differ diff --git a/src/app/child-dev-project/attendance/add-day-attendance/roll-call/roll-call.component.html b/src/app/child-dev-project/attendance/add-day-attendance/roll-call/roll-call.component.html index 8a9617e58d..ecdd75d394 100644 --- a/src/app/child-dev-project/attendance/add-day-attendance/roll-call/roll-call.component.html +++ b/src/app/child-dev-project/attendance/add-day-attendance/roll-call/roll-call.component.html @@ -5,7 +5,7 @@ [value]="(currentIndex / children.length) * 100" > -
+
{{ currentIndex + 1 }} / {{ children.length }}
-
+
+ +
+ +
) { return Object.assign(new HealthCheck(), contents); } @DatabaseField({ + dataType: "entity", + additional: Child.ENTITY_TYPE, + entityReferenceRole: "composite", anonymize: "retain", }) child: string; diff --git a/src/app/child-dev-project/children/model/child.ts b/src/app/child-dev-project/children/model/child.ts index 5a955b252a..4cd743f56c 100644 --- a/src/app/child-dev-project/children/model/child.ts +++ b/src/app/child-dev-project/children/model/child.ts @@ -31,6 +31,7 @@ export class Child extends Entity { static label = $localize`:label for entity:Participant`; static labelPlural = $localize`:label (plural) for entity:Participants`; static color = "#1565C0"; + static override hasPII = true; static create(name: string): Child { const instance = new Child(); diff --git a/src/app/child-dev-project/children/model/childSchoolRelation.ts b/src/app/child-dev-project/children/model/childSchoolRelation.ts index 0152865bb3..4362b98af5 100644 --- a/src/app/child-dev-project/children/model/childSchoolRelation.ts +++ b/src/app/child-dev-project/children/model/childSchoolRelation.ts @@ -9,10 +9,13 @@ import { TimePeriod } from "../../../core/entity-details/related-time-period-ent */ @DatabaseEntity("ChildSchoolRelation") export class ChildSchoolRelation extends TimePeriod { + static override hasPII = true; + @DatabaseField({ label: $localize`:Label for the child of a relation:Child`, dataType: "entity", additional: Child.ENTITY_TYPE, + entityReferenceRole: "composite", validators: { required: true, }, @@ -24,6 +27,7 @@ export class ChildSchoolRelation extends TimePeriod { label: $localize`:Label for the school of a relation:School`, dataType: "entity", additional: School.ENTITY_TYPE, + entityReferenceRole: "aggregate", validators: { required: true, }, @@ -38,6 +42,7 @@ export class ChildSchoolRelation extends TimePeriod { dataType: "date-only", label: $localize`:Label for the start date of a relation:Start date`, description: $localize`:Description of the start date of a relation:The date a child joins a school`, + anonymize: "retain", }) start: Date; @@ -45,6 +50,7 @@ export class ChildSchoolRelation extends TimePeriod { dataType: "date-only", label: $localize`:Label for the end date of a relation:End date`, description: $localize`:Description of the end date of a relation:The date of a child leaving the school`, + anonymize: "retain", }) end: Date; diff --git a/src/app/child-dev-project/notes/model/note.ts b/src/app/child-dev-project/notes/model/note.ts index 2cae607385..378815bb0d 100644 --- a/src/app/child-dev-project/notes/model/note.ts +++ b/src/app/child-dev-project/notes/model/note.ts @@ -39,6 +39,7 @@ export class Note extends Entity { static toStringAttributes = ["subject"]; static label = $localize`:label for entity:Note`; static labelPlural = $localize`:label (plural) for entity:Notes`; + static override hasPII = true; static create( date: Date, @@ -76,6 +77,7 @@ export class Note extends Entity { label: $localize`:Label for the children of a note:Children`, dataType: "entity-array", additional: Child.ENTITY_TYPE, + entityReferenceRole: "composite", editComponent: "EditAttendance", anonymize: "retain", }) @@ -165,6 +167,7 @@ export class Note extends Entity { label: $localize`:label for the linked schools:Groups`, dataType: "entity-array", additional: School.ENTITY_TYPE, + entityReferenceRole: "composite", anonymize: "retain", }) schools: string[] = []; diff --git a/src/app/core/basic-datatypes/date/display-date/display-date.component.html b/src/app/core/basic-datatypes/date/display-date/display-date.component.html new file mode 100644 index 0000000000..78fd5b6411 --- /dev/null +++ b/src/app/core/basic-datatypes/date/display-date/display-date.component.html @@ -0,0 +1,14 @@ + + {{ value | date: config }} + + + + {{ value | date: "YYYY" }} + + diff --git a/src/app/core/basic-datatypes/date/display-date/display-date.component.ts b/src/app/core/basic-datatypes/date/display-date/display-date.component.ts index 76001a89f0..7db5675d5e 100644 --- a/src/app/core/basic-datatypes/date/display-date/display-date.component.ts +++ b/src/app/core/basic-datatypes/date/display-date/display-date.component.ts @@ -1,7 +1,9 @@ -import { Component } from "@angular/core"; +import { Component, Input } from "@angular/core"; import { ViewDirective } from "../../../entity/default-datatype/view.directive"; import { DynamicComponent } from "../../../config/dynamic-components/dynamic-component.decorator"; -import { DatePipe } from "@angular/common"; +import { DatePipe, NgIf } from "@angular/common"; +import { FontAwesomeModule } from "@fortawesome/angular-fontawesome"; +import { MatTooltipModule } from "@angular/material/tooltip"; /** * This component displays a date attribute using the shortDate format. @@ -12,8 +14,13 @@ import { DatePipe } from "@angular/common"; @DynamicComponent("DisplayDate") @Component({ selector: "app-display-date", - template: `{{ value | date: config }}`, + templateUrl: "./display-date.component.html", standalone: true, - imports: [DatePipe], + imports: [DatePipe, NgIf, FontAwesomeModule, MatTooltipModule], }) -export class DisplayDateComponent extends ViewDirective {} +export class DisplayDateComponent extends ViewDirective { + @Input() displayAsAnonymized: boolean; + + /** formatting string for date pipe */ + @Input() config: string; +} diff --git a/src/app/core/basic-datatypes/date/edit-date/edit-date.component.html b/src/app/core/basic-datatypes/date/edit-date/edit-date.component.html index caf141aebe..31f53b3378 100644 --- a/src/app/core/basic-datatypes/date/edit-date/edit-date.component.html +++ b/src/app/core/basic-datatypes/date/edit-date/edit-date.component.html @@ -5,6 +5,15 @@ [formControlName]="formControlName" [matDatepicker]="datepickerComp" /> + + + { testDatatype(new EntityDatatype(null, null), "1", "1", "User"); @@ -50,7 +50,7 @@ describe("Schema data type: entity", () => { const entityMapper = mockEntityMapper([referencedEntity]); spyOn(entityMapper, "save"); - const mockRemoveService: jasmine.SpyObj = + const mockRemoveService: jasmine.SpyObj = jasmine.createSpyObj("EntityRemoveService", ["anonymize"]); const dataType = new EntityDatatype(entityMapper, mockRemoveService); diff --git a/src/app/core/basic-datatypes/entity/entity.datatype.ts b/src/app/core/basic-datatypes/entity/entity.datatype.ts index 33070ffc5b..eb1281c586 100644 --- a/src/app/core/basic-datatypes/entity/entity.datatype.ts +++ b/src/app/core/basic-datatypes/entity/entity.datatype.ts @@ -20,7 +20,7 @@ import { StringDatatype } from "../string/string.datatype"; import { EntitySchemaField } from "../../entity/schema/entity-schema-field"; import { EntityMapperService } from "../../entity/entity-mapper/entity-mapper.service"; import { ColumnMapping } from "../../import/column-mapping"; -import { EntityRemoveService } from "../../entity/entity-remove.service"; +import { EntityActionsService } from "../../entity/entity-actions/entity-actions.service"; /** * Datatype for the EntitySchemaService to handle a single reference to another entity @@ -40,7 +40,7 @@ export class EntityDatatype extends StringDatatype { constructor( private entityMapper: EntityMapperService, - private removeService: EntityRemoveService, + private removeService: EntityActionsService, ) { super(); } diff --git a/src/app/core/basic-datatypes/month/display-month/display-month.component.ts b/src/app/core/basic-datatypes/month/display-month/display-month.component.ts index ecdf6591e5..8eddf4dd3a 100644 --- a/src/app/core/basic-datatypes/month/display-month/display-month.component.ts +++ b/src/app/core/basic-datatypes/month/display-month/display-month.component.ts @@ -1,11 +1,16 @@ import { Component } from "@angular/core"; -import { DatePipe } from "@angular/common"; +import { DatePipe, NgIf } from "@angular/common"; import { ViewDirective } from "../../../entity/default-datatype/view.directive"; +import { DisplayDateComponent } from "../../date/display-date/display-date.component"; @Component({ selector: "app-display-month", standalone: true, - template: `{{ value | date: "YYYY-MM" }}`, - imports: [DatePipe], + template: ``, + imports: [DatePipe, NgIf, DisplayDateComponent], }) export class DisplayMonthComponent extends ViewDirective {} diff --git a/src/app/core/basic-datatypes/month/edit-month/edit-month.component.html b/src/app/core/basic-datatypes/month/edit-month/edit-month.component.html index 56239701c7..ab01510f80 100644 --- a/src/app/core/basic-datatypes/month/edit-month/edit-month.component.html +++ b/src/app/core/basic-datatypes/month/edit-month/edit-month.component.html @@ -5,6 +5,15 @@ [formControl]="formControl" [matDatepicker]="datepickerComp" /> + + + + {{ data.message }} + + +
+ +
diff --git a/src/app/core/common-components/confirmation-dialog/progress-dialog/progress-dialog.component.spec.ts b/src/app/core/common-components/confirmation-dialog/progress-dialog/progress-dialog.component.spec.ts new file mode 100644 index 0000000000..8926eec000 --- /dev/null +++ b/src/app/core/common-components/confirmation-dialog/progress-dialog/progress-dialog.component.spec.ts @@ -0,0 +1,32 @@ +import { ComponentFixture, TestBed, waitForAsync } from "@angular/core/testing"; + +import { ProgressDialogComponent } from "./progress-dialog.component"; +import { MAT_DIALOG_DATA, MatDialogRef } from "@angular/material/dialog"; + +describe("ProgressDialogComponent", () => { + let component: ProgressDialogComponent; + let fixture: ComponentFixture; + + beforeEach(waitForAsync(() => { + TestBed.configureTestingModule({ + imports: [ProgressDialogComponent], + providers: [ + { provide: MatDialogRef, useValue: {} }, + { + provide: MAT_DIALOG_DATA, + useValue: { message: "test title" }, + }, + ], + }).compileComponents(); + })); + + beforeEach(() => { + fixture = TestBed.createComponent(ProgressDialogComponent); + component = fixture.componentInstance; + fixture.detectChanges(); + }); + + it("should create", () => { + expect(component).toBeTruthy(); + }); +}); diff --git a/src/app/core/common-components/confirmation-dialog/progress-dialog/progress-dialog.component.ts b/src/app/core/common-components/confirmation-dialog/progress-dialog/progress-dialog.component.ts new file mode 100644 index 0000000000..3c213d838c --- /dev/null +++ b/src/app/core/common-components/confirmation-dialog/progress-dialog/progress-dialog.component.ts @@ -0,0 +1,16 @@ +import { Component, Inject } from "@angular/core"; +import { MAT_DIALOG_DATA, MatDialogModule } from "@angular/material/dialog"; +import { MatProgressBarModule } from "@angular/material/progress-bar"; + +/** + * A simple progress indicator dialog + * used via the {@link ConfirmationDialogService}. + */ +@Component({ + templateUrl: "./progress-dialog.component.html", + imports: [MatProgressBarModule, MatDialogModule], + standalone: true, +}) +export class ProgressDialogComponent { + constructor(@Inject(MAT_DIALOG_DATA) public data: { message: string }) {} +} diff --git a/src/app/core/common-components/entity-subrecord/entity-subrecord/entity-subrecord.component.ts b/src/app/core/common-components/entity-subrecord/entity-subrecord/entity-subrecord.component.ts index 2a096c5f8f..31f2fb0ee8 100644 --- a/src/app/core/common-components/entity-subrecord/entity-subrecord/entity-subrecord.component.ts +++ b/src/app/core/common-components/entity-subrecord/entity-subrecord/entity-subrecord.component.ts @@ -24,7 +24,7 @@ import { } from "../../entity-form/entity-form.service"; import { LoggingService } from "../../../logging/logging.service"; import { AnalyticsService } from "../../../analytics/analytics.service"; -import { EntityRemoveService } from "../../../entity/entity-remove.service"; +import { EntityActionsService } from "../../../entity/entity-actions/entity-actions.service"; import { EntityMapperService } from "../../../entity/entity-mapper/entity-mapper.service"; import { tableSort } from "./table-sort"; import { @@ -180,7 +180,7 @@ export class EntitySubrecordComponent implements OnChanges { private router: Router, private analyticsService: AnalyticsService, private loggingService: LoggingService, - public entityRemoveService: EntityRemoveService, + public entityRemoveService: EntityActionsService, private entityMapper: EntityMapperService, private filterService: FilterService, ) { diff --git a/src/app/core/database/pouch-database.spec.ts b/src/app/core/database/pouch-database.spec.ts index 49f9774b61..7d455c9049 100644 --- a/src/app/core/database/pouch-database.spec.ts +++ b/src/app/core/database/pouch-database.spec.ts @@ -259,7 +259,11 @@ describe("PouchDatabase tests", () => { }, ]; - const results = await database.putAll(dataWithConflicts); + await expectAsync(database.putAll(dataWithConflicts)).toBeRejectedWith([ + conflictError, + jasmine.objectContaining({ id: "4", ok: true }), + jasmine.objectContaining({ id: "5", ok: true }), + ]); expect(resolveConflictSpy.calls.allArgs()).toEqual([ [ { @@ -271,11 +275,6 @@ describe("PouchDatabase tests", () => { jasmine.objectContaining({ status: 409 }), ], ]); - expect(results).toEqual([ - conflictError, - jasmine.objectContaining({ id: "4", ok: true }), - jasmine.objectContaining({ id: "5", ok: true }), - ]); }); it("should correctly determine if database is empty", async () => { diff --git a/src/app/core/database/pouch-database.ts b/src/app/core/database/pouch-database.ts index 8a3ae8e649..90fc67fdd8 100644 --- a/src/app/core/database/pouch-database.ts +++ b/src/app/core/database/pouch-database.ts @@ -211,7 +211,8 @@ export class PouchDatabase extends Database { * Save an array of documents to the database * @param objects the documents to be saved * @param forceOverwrite whether conflicting versions should be overwritten - * @returns array holding `{ ok: true, ... }` or `{ error: true, ... }` depending on whether the document could be saved + * @returns array with the result for each object to be saved, if any item fails to be saved, this returns a rejected Promise. + * The save can partially fail and return a mix of success and error states in the array (e.g. `[{ ok: true, ... }, { error: true, ... }]`) */ async putAll(objects: any[], forceOverwrite = false): Promise { if (forceOverwrite) { @@ -227,11 +228,17 @@ export class PouchDatabase extends Database { if (result.status === 409) { results[i] = await this.resolveConflict( objects.find((obj) => obj._id === result.id), - false, + forceOverwrite, result, - ).catch((e) => e); + ).catch((e) => { + return new DatabaseException(e); + }); } } + + if (results.some((r) => r instanceof Error)) { + return Promise.reject(results); + } return results; } diff --git a/src/app/core/entity-details/entity-actions-menu/entity-actions-menu.component.spec.ts b/src/app/core/entity-details/entity-actions-menu/entity-actions-menu.component.spec.ts index 7422bfd47c..9854810c04 100644 --- a/src/app/core/entity-details/entity-actions-menu/entity-actions-menu.component.spec.ts +++ b/src/app/core/entity-details/entity-actions-menu/entity-actions-menu.component.spec.ts @@ -6,21 +6,21 @@ import { } from "@angular/core/testing"; import { EntityActionsMenuComponent } from "./entity-actions-menu.component"; -import { EntityRemoveService } from "../../entity/entity-remove.service"; +import { EntityActionsService } from "../../entity/entity-actions/entity-actions.service"; import { MockedTestingModule } from "../../../utils/mocked-testing.module"; describe("EntityActionsMenuComponent", () => { let component: EntityActionsMenuComponent; let fixture: ComponentFixture; - let mockEntityRemoveService: jasmine.SpyObj; + let mockEntityRemoveService: jasmine.SpyObj; beforeEach(() => { mockEntityRemoveService = jasmine.createSpyObj(["delete"]); TestBed.configureTestingModule({ imports: [EntityActionsMenuComponent, MockedTestingModule], providers: [ - { provide: EntityRemoveService, useValue: mockEntityRemoveService }, + { provide: EntityActionsService, useValue: mockEntityRemoveService }, ], }); fixture = TestBed.createComponent(EntityActionsMenuComponent); diff --git a/src/app/core/entity-details/entity-actions-menu/entity-actions-menu.component.ts b/src/app/core/entity-details/entity-actions-menu/entity-actions-menu.component.ts index 54101bd2fe..625821d4af 100644 --- a/src/app/core/entity-details/entity-actions-menu/entity-actions-menu.component.ts +++ b/src/app/core/entity-details/entity-actions-menu/entity-actions-menu.component.ts @@ -6,7 +6,7 @@ import { Output, SimpleChanges, } from "@angular/core"; -import { EntityRemoveService } from "../../entity/entity-remove.service"; +import { EntityActionsService } from "../../entity/entity-actions/entity-actions.service"; import { Entity } from "../../entity/model/entity"; import { NgForOf, NgIf } from "@angular/common"; import { MatButtonModule } from "@angular/material/button"; @@ -87,7 +87,7 @@ export class EntityActionsMenuComponent implements OnChanges { }, ]; - constructor(private entityRemoveService: EntityRemoveService) {} + constructor(private entityRemoveService: EntityActionsService) {} ngOnChanges(changes: SimpleChanges): void { if (changes.entity) { @@ -97,13 +97,16 @@ export class EntityActionsMenuComponent implements OnChanges { private filterAvailableActions() { this.actions = this.defaultActions.filter((action) => { - if (this.entity?.anonymized) { - return action.action !== "anonymize" && action.action !== "archive"; + switch (action.action) { + case "archive": + return this.entity?.isActive && !this.entity?.anonymized; + case "anonymize": + return ( + !this.entity?.anonymized && this.entity?.getConstructor().hasPII + ); + default: + return true; } - if (!this.entity?.isActive) { - return action.action !== "archive"; - } - return true; }); } diff --git a/src/app/core/entity-details/entity-archived-info/entity-archived-info.component.spec.ts b/src/app/core/entity-details/entity-archived-info/entity-archived-info.component.spec.ts index d9b557ead8..9ac50cea43 100644 --- a/src/app/core/entity-details/entity-archived-info/entity-archived-info.component.spec.ts +++ b/src/app/core/entity-details/entity-archived-info/entity-archived-info.component.spec.ts @@ -1,7 +1,7 @@ import { ComponentFixture, TestBed } from "@angular/core/testing"; import { EntityArchivedInfoComponent } from "./entity-archived-info.component"; -import { EntityRemoveService } from "../../entity/entity-remove.service"; +import { EntityActionsService } from "../../entity/entity-actions/entity-actions.service"; describe("EntityArchivedInfoComponent", () => { let component: EntityArchivedInfoComponent; @@ -10,7 +10,7 @@ describe("EntityArchivedInfoComponent", () => { beforeEach(() => { TestBed.configureTestingModule({ imports: [EntityArchivedInfoComponent], - providers: [{ provide: EntityRemoveService, useValue: null }], + providers: [{ provide: EntityActionsService, useValue: null }], }); fixture = TestBed.createComponent(EntityArchivedInfoComponent); component = fixture.componentInstance; diff --git a/src/app/core/entity-details/entity-archived-info/entity-archived-info.component.ts b/src/app/core/entity-details/entity-archived-info/entity-archived-info.component.ts index ce357afc3e..0724b65363 100644 --- a/src/app/core/entity-details/entity-archived-info/entity-archived-info.component.ts +++ b/src/app/core/entity-details/entity-archived-info/entity-archived-info.component.ts @@ -4,7 +4,7 @@ import { MatCardModule } from "@angular/material/card"; import { MatButtonModule } from "@angular/material/button"; import { FontAwesomeModule } from "@fortawesome/angular-fontawesome"; import { Entity } from "../../entity/model/entity"; -import { EntityRemoveService } from "../../entity/entity-remove.service"; +import { EntityActionsService } from "../../entity/entity-actions/entity-actions.service"; /** * Informs users that the entity is inactive (or anonymized) and provides options to change the status. @@ -19,5 +19,5 @@ import { EntityRemoveService } from "../../entity/entity-remove.service"; export class EntityArchivedInfoComponent { @Input() entity: Entity; - constructor(public entityRemoveService: EntityRemoveService) {} + constructor(public entityRemoveService: EntityActionsService) {} } diff --git a/src/app/core/entity-details/entity-details/entity-details.component.spec.ts b/src/app/core/entity-details/entity-details/entity-details.component.spec.ts index e76003429d..f324442a7b 100644 --- a/src/app/core/entity-details/entity-details/entity-details.component.spec.ts +++ b/src/app/core/entity-details/entity-details/entity-details.component.spec.ts @@ -11,7 +11,7 @@ import { EntityDetailsConfig, PanelConfig } from "../EntityDetailsConfig"; import { Child } from "../../../child-dev-project/children/model/child"; import { ChildrenService } from "../../../child-dev-project/children/children.service"; import { MockedTestingModule } from "../../../utils/mocked-testing.module"; -import { EntityRemoveService } from "../../entity/entity-remove.service"; +import { EntityActionsService } from "../../entity/entity-actions/entity-actions.service"; import { EntityAbility } from "../../permissions/ability/entity-ability"; import { EntityMapperService } from "../../entity/entity-mapper/entity-mapper.service"; import { SimpleChange } from "@angular/core"; @@ -44,7 +44,7 @@ describe("EntityDetailsComponent", () => { }; let mockChildrenService: jasmine.SpyObj; - let mockEntityRemoveService: jasmine.SpyObj; + let mockEntityRemoveService: jasmine.SpyObj; let mockAbility: jasmine.SpyObj; beforeEach(waitForAsync(() => { @@ -61,7 +61,7 @@ describe("EntityDetailsComponent", () => { imports: [EntityDetailsComponent, MockedTestingModule.withState()], providers: [ { provide: ChildrenService, useValue: mockChildrenService }, - { provide: EntityRemoveService, useValue: mockEntityRemoveService }, + { provide: EntityActionsService, useValue: mockEntityRemoveService }, { provide: EntityAbility, useValue: mockAbility }, ], }).compileComponents(); diff --git a/src/app/core/entity-details/entity-details/entity-details.component.ts b/src/app/core/entity-details/entity-details/entity-details.component.ts index ff3dbb492b..bc7f1fe164 100644 --- a/src/app/core/entity-details/entity-details/entity-details.component.ts +++ b/src/app/core/entity-details/entity-details/entity-details.component.ts @@ -28,6 +28,9 @@ import { LoggingService } from "../../logging/logging.service"; import { UnsavedChangesService } from "../form/unsaved-changes.service"; import { EntityActionsMenuComponent } from "../entity-actions-menu/entity-actions-menu.component"; import { EntityArchivedInfoComponent } from "../entity-archived-info/entity-archived-info.component"; +import { filter } from "rxjs/operators"; +import { UntilDestroy, untilDestroyed } from "@ngneat/until-destroy"; +import { Subscription } from "rxjs"; /** * This component can be used to display an entity in more detail. @@ -36,6 +39,7 @@ import { EntityArchivedInfoComponent } from "../entity-archived-info/entity-arch * The subcomponents will be provided with the Entity object and the creating new status, as well as its static config. */ @RouteTarget("EntityDetails") +@UntilDestroy() @Component({ selector: "app-entity-details", templateUrl: "./entity-details.component.html", @@ -62,6 +66,7 @@ import { EntityArchivedInfoComponent } from "../entity-archived-info/entity-arch export class EntityDetailsComponent implements EntityDetailsConfig, OnChanges { creatingNew = false; isLoading = true; + private changesSubscription: Subscription; /** @deprecated use "entityType" instead, this remains for config backwards compatibility */ @Input() set entity(v: string) { @@ -91,12 +96,25 @@ export class EntityDetailsComponent implements EntityDetailsConfig, OnChanges { } if (changes.id) { this.loadEntity(this.id); + this.subscribeToEntityChanges(); // `initPanels()` is already called inside `loadEntity()` } else if (changes.panels) { this.initPanels(); } } + private subscribeToEntityChanges() { + this.changesSubscription?.unsubscribe(); + this.changesSubscription = this.entityMapperService + .receiveUpdates(this.entityConstructor) + .pipe( + filter(({ entity }) => entity.getId() === this.id), + filter(({ type }) => type !== "remove"), + untilDestroyed(this), + ) + .subscribe(({ entity }) => (this.record = entity)); + } + private async loadEntity(id: string) { if (id === "new") { if (this.ability.cannot("create", this.entityConstructor)) { diff --git a/src/app/core/entity-details/related-entities/related-entities.component.html b/src/app/core/entity-details/related-entities/related-entities.component.html index 7ca6a6ebfc..897ca94e57 100644 --- a/src/app/core/entity-details/related-entities/related-entities.component.html +++ b/src/app/core/entity-details/related-entities/related-entities.component.html @@ -3,5 +3,6 @@ [filter]="filter" [columns]="columns" [newRecordFactory]="createNewRecordFactory()" + [showInactive]="showInactive" [isLoading]="isLoading" > diff --git a/src/app/core/entity-details/related-entities/related-entities.component.ts b/src/app/core/entity-details/related-entities/related-entities.component.ts index 7e5a4eb93e..ddefb12814 100644 --- a/src/app/core/entity-details/related-entities/related-entities.component.ts +++ b/src/app/core/entity-details/related-entities/related-entities.component.ts @@ -44,6 +44,8 @@ export class RelatedEntitiesComponent implements OnInit { @Input() filter?: DataFilter; + @Input() showInactive: boolean; + data: E[] = []; isLoading = false; private isArray = false; @@ -77,6 +79,10 @@ export class RelatedEntitiesComponent implements OnInit { : this.entity.getId(), }; + if (this.showInactive === undefined) { + this.showInactive = this.entity.anonymized; + } + this.isLoading = false; } diff --git a/src/app/core/entity-details/related-time-period-entities/related-time-period-entities.component.html b/src/app/core/entity-details/related-time-period-entities/related-time-period-entities.component.html index e095a8c8d1..52a97c8865 100644 --- a/src/app/core/entity-details/related-time-period-entities/related-time-period-entities.component.html +++ b/src/app/core/entity-details/related-time-period-entities/related-time-period-entities.component.html @@ -1,4 +1,4 @@ - + Currently there is no active entry. To add a new entry, click on the implements OnInit { */ additional?: any; + /** indicating that the value is not in its original state, so that components can explain this to the user */ + isPartiallyAnonymized: boolean; + ngOnInit() { if (!this.formFieldConfig?.forTable) { this.label = this.formFieldConfig?.label ?? this.propertySchema?.label; @@ -68,4 +71,11 @@ export abstract class EditComponent implements OnInit { // This type casts are needed as the normal types throw errors in the templates this.parent = this.formControl.parent as FormGroup; } + + ngOnChanges() { + this.isPartiallyAnonymized = + this.entity?.anonymized && + this.entity?.getSchema()?.get(this.formFieldConfig?.id)?.anonymize === + "retain-anonymized"; + } } diff --git a/src/app/core/entity/default-datatype/view.directive.ts b/src/app/core/entity/default-datatype/view.directive.ts index ffded6f6cc..cb155918e3 100644 --- a/src/app/core/entity/default-datatype/view.directive.ts +++ b/src/app/core/entity/default-datatype/view.directive.ts @@ -1,11 +1,20 @@ import { Entity } from "../model/entity"; -import { Directive, Input } from "@angular/core"; +import { Directive, Input, OnChanges } from "@angular/core"; @Directive() -export abstract class ViewDirective { +export abstract class ViewDirective implements OnChanges { @Input() entity: Entity; @Input() id: string; @Input() tooltip: string; @Input() value: T; @Input() config: C; + + /** indicating that the value is not in its original state, so that components can explain this to the user */ + isPartiallyAnonymized: boolean; + + ngOnChanges() { + this.isPartiallyAnonymized = + this.entity?.anonymized && + this.entity?.getSchema()?.get(this.id)?.anonymize === "retain-anonymized"; + } } diff --git a/src/app/core/entity/entity-actions/cascading-entity-action.spec.ts b/src/app/core/entity/entity-actions/cascading-entity-action.spec.ts new file mode 100644 index 0000000000..f72ccf7fef --- /dev/null +++ b/src/app/core/entity/entity-actions/cascading-entity-action.spec.ts @@ -0,0 +1,197 @@ +import { DatabaseEntity } from "../database-entity.decorator"; +import { Entity } from "../model/entity"; +import { DatabaseField } from "../database-field.decorator"; +import { + comparableEntityData, + expectEntitiesToMatch, +} from "../../../utils/expect-entity-data.spec"; +import { MockEntityMapperService } from "../entity-mapper/mock-entity-mapper-service"; + +/* + Deleting/Anonymizing referenced & related entities + also see doc/compodoc_sources/concepts/entity-anonymization.md + + we distinguish different roles / relations between entities: + ♢ "aggregate" (has-a): both entities have meaning independently + ♦ "composite" (is-part-of): the entity holding the reference is only meaningful in the context of the referenced + */ +@DatabaseEntity("EntityWithAnonRelations") +export class EntityWithAnonRelations extends Entity { + static override hasPII = true; + + @DatabaseField() name: string; + + @DatabaseField({ + dataType: "entity-array", + additional: "EntityWithAnonRelations", + anonymize: "retain", + entityReferenceRole: "aggregate", + }) + refAggregate: string[]; + + @DatabaseField({ + dataType: "entity-array", + additional: "EntityWithAnonRelations", + anonymize: "retain", + entityReferenceRole: "composite", + }) + refComposite: string[]; + + static create(name: string, properties?: Partial) { + return Object.assign(new EntityWithAnonRelations(), { + name: name, + ...properties, + }); + } +} + +export function expectAllUnchangedExcept( + changedEntities: EntityWithAnonRelations[], + entityMapper: MockEntityMapperService, +) { + const isExpectedUnchanged = (entity: EntityWithAnonRelations) => { + !changedEntities.some((c) => entity.getId() === c.getId()); + }; + + const actualEntitiesAfter = + entityMapper.getAllData() as EntityWithAnonRelations[]; + + expectEntitiesToMatch( + actualEntitiesAfter.filter(isExpectedUnchanged), + allEntities.filter(isExpectedUnchanged), + true, + ); +} + +export function expectDeleted( + deletedEntities: Entity[], + entityMapper: MockEntityMapperService, +) { + const actualEntitiesAfter = entityMapper.getAllData(); + + for (const deletedEntity of deletedEntities) { + expect(actualEntitiesAfter).not.toContain(deletedEntity); + } +} + +export function expectUpdated( + updatedEntities: EntityWithAnonRelations[], + entityMapper: MockEntityMapperService, +) { + const actualEntitiesAfter = entityMapper.getAllData(); + + for (const updatedEntity of updatedEntities) { + const actualEntity = actualEntitiesAfter.find( + (e) => e.getId() === updatedEntity.getId(), + ); + expect(comparableEntityData(actualEntity)).toEqual( + comparableEntityData(updatedEntity), + ); + } +} + +const WithoutRelations = EntityWithAnonRelations.create( + "entity without relations", +); + +const ReferencedAsComposite = EntityWithAnonRelations.create( + "entity referenced as composite", +); +const ReferencingSingleComposite = EntityWithAnonRelations.create( + "entity having a composite reference", + { + refComposite: [ReferencedAsComposite.getId()], + }, +); + +const ReferencedAsOneOfMultipleComposites1 = EntityWithAnonRelations.create( + "entity referenced as one composite (1)", +); +const ReferencedAsOneOfMultipleComposites2 = EntityWithAnonRelations.create( + "entity referenced as one composite (2)", +); +const ReferencingTwoComposites = EntityWithAnonRelations.create( + "entity referencing two entities as composites", + { + refComposite: [ + ReferencedAsOneOfMultipleComposites1.getId(), + ReferencedAsOneOfMultipleComposites2.getId(), + ], + }, +); + +const ReferencingCompositeAndAggregate_refComposite = + EntityWithAnonRelations.create( + "referenced as composite from composite+aggregate referencing entity", + ); +const ReferencingCompositeAndAggregate_refAggregate = + EntityWithAnonRelations.create( + "referenced as aggregate from composite+aggregate referencing entity", + ); +const ReferencingCompositeAndAggregate = EntityWithAnonRelations.create( + "having both a composite and a aggregate reference", + { + refComposite: [ReferencingCompositeAndAggregate_refComposite.getId()], + refAggregate: [ReferencingCompositeAndAggregate_refAggregate.getId()], + }, +); + +const ReferencingAggregate_ref = EntityWithAnonRelations.create( + "entity referenced as aggregate", +); +const ReferencingAggregate = EntityWithAnonRelations.create( + "entity having an aggregate reference", + { + refAggregate: [ReferencingAggregate_ref.getId()], + }, +); + +const ReferencingTwoAggregates_ref1 = EntityWithAnonRelations.create( + "entity referenced as one aggregate (1)", +); +const ReferencingTwoAggregates_ref2 = EntityWithAnonRelations.create( + "entity referenced as one aggregate (2)", +); +const ReferencingTwoAggregates = EntityWithAnonRelations.create( + "entity referencing two entities as aggregates", + { + refAggregate: [ + ReferencingTwoAggregates_ref1.getId(), + ReferencingTwoAggregates_ref2.getId(), + ], + }, +); + +export const ENTITIES = { + WithoutRelations, + ReferencedAsComposite, + ReferencingSingleComposite, + ReferencedAsOneOfMultipleComposites1, + ReferencedAsOneOfMultipleComposites2, + ReferencingTwoComposites, + ReferencingCompositeAndAggregate_refComposite, + ReferencingCompositeAndAggregate_refAggregate, + ReferencingCompositeAndAggregate, + ReferencingAggregate_ref, + ReferencingAggregate, + ReferencingTwoAggregates_ref1, + ReferencingTwoAggregates_ref2, + ReferencingTwoAggregates, +}; + +export const allEntities: EntityWithAnonRelations[] = [ + ENTITIES.WithoutRelations, + ENTITIES.ReferencedAsComposite, + ENTITIES.ReferencingSingleComposite, + ENTITIES.ReferencedAsOneOfMultipleComposites1, + ENTITIES.ReferencedAsOneOfMultipleComposites2, + ENTITIES.ReferencingTwoComposites, + ENTITIES.ReferencingCompositeAndAggregate_refComposite, + ENTITIES.ReferencingCompositeAndAggregate_refAggregate, + ENTITIES.ReferencingCompositeAndAggregate, + ENTITIES.ReferencingAggregate_ref, + ENTITIES.ReferencingAggregate, + ENTITIES.ReferencingTwoAggregates_ref1, + ENTITIES.ReferencingTwoAggregates_ref2, + ENTITIES.ReferencingTwoAggregates, +]; diff --git a/src/app/core/entity/entity-actions/cascading-entity-action.ts b/src/app/core/entity/entity-actions/cascading-entity-action.ts new file mode 100644 index 0000000000..177094c56d --- /dev/null +++ b/src/app/core/entity/entity-actions/cascading-entity-action.ts @@ -0,0 +1,114 @@ +import { Entity } from "../model/entity"; +import { asArray } from "../../../utils/utils"; +import { EntitySchemaService } from "../schema/entity-schema.service"; +import { EntityMapperService } from "../entity-mapper/entity-mapper.service"; + +export class CascadingActionResult { + /** + * entities that have been updated in the process, in their original state + * (can be used for undo action) + */ + originalEntitiesBeforeChange: Entity[]; + + /** + * entities that may still contain PII related to the primary entity that could not be automatically removed + * (may need manual review by the user) + */ + potentiallyRetainingPII: Entity[]; + + constructor(changedEntities?: Entity[], potentiallyRetainingPII?: Entity[]) { + this.originalEntitiesBeforeChange = changedEntities ?? []; + this.potentiallyRetainingPII = potentiallyRetainingPII ?? []; + } + + mergeResults(otherResult: CascadingActionResult) { + this.originalEntitiesBeforeChange = [ + ...this.originalEntitiesBeforeChange, + ...otherResult.originalEntitiesBeforeChange.filter( + (e) => + !this.originalEntitiesBeforeChange.some( + (x) => x.getId() === e.getId(), + ), + ), + ]; + this.potentiallyRetainingPII = [ + ...this.potentiallyRetainingPII, + ...otherResult.potentiallyRetainingPII.filter( + (e) => + !this.potentiallyRetainingPII.some((x) => x.getId() === e.getId()), + ), + ]; + + return this; + } +} + +/** + * extend this class to implement services that perform actions on an entity + * that require recursive actions to related entities as well. + */ +export abstract class CascadingEntityAction { + protected constructor( + protected entityMapper: EntityMapperService, + protected schemaService: EntitySchemaService, + ) {} + + /** + * Recursively call the given actions on all related entities that contain a reference to the given entity. + * + * Returns an array of all affected related entities (excluding the given entity) in their state before the action + * to support an undo action. + * + * @param entity + * @param compositeAction + * @param aggregateAction + * @private + */ + protected async cascadeActionToRelatedEntities( + entity: Entity, + compositeAction: ( + relatedEntity: Entity, + refField?: string, + entity?: Entity, + ) => Promise, + aggregateAction: ( + relatedEntity: Entity, + refField?: string, + entity?: Entity, + ) => Promise, + ): Promise { + const cascadeActionResult = new CascadingActionResult(); + + const entityTypesWithReferences = + this.schemaService.getEntityTypesReferencingType(entity.getType()); + + for (const refType of entityTypesWithReferences) { + const entities = await this.entityMapper.loadType(refType.entityType); + + for (const refField of refType.referencingProperties) { + const affectedEntities = entities.filter( + (e) => + asArray(e[refField]).includes(entity.getId()) || + asArray(e[refField]).includes(entity.getId(true)), + ); + + for (const e of affectedEntities) { + if ( + refType.entityType.schema.get(refField).entityReferenceRole === + "composite" && + asArray(e[refField]).length === 1 + ) { + // is only composite + const result = await compositeAction(e); + cascadeActionResult.mergeResults(result); + } else { + const result = await aggregateAction(e, refField, entity); + cascadeActionResult.mergeResults(result); + } + } + } + } + + return cascadeActionResult; + } +} diff --git a/src/app/core/entity/entity-actions/entity-actions.service.spec.ts b/src/app/core/entity/entity-actions/entity-actions.service.spec.ts new file mode 100644 index 0000000000..9fa08052a8 --- /dev/null +++ b/src/app/core/entity/entity-actions/entity-actions.service.spec.ts @@ -0,0 +1,141 @@ +import { fakeAsync, TestBed, tick } from "@angular/core/testing"; +import { EntityActionsService } from "./entity-actions.service"; +import { EntityMapperService } from "../entity-mapper/entity-mapper.service"; +import { + MatSnackBar, + MatSnackBarRef, + TextOnlySnackBar, +} from "@angular/material/snack-bar"; +import { ConfirmationDialogService } from "../../common-components/confirmation-dialog/confirmation-dialog.service"; +import { Entity } from "../model/entity"; +import { NEVER, of, Subject } from "rxjs"; +import { Router } from "@angular/router"; +import { CoreTestingModule } from "../../../utils/core-testing.module"; +import { EntityDeleteService } from "./entity-delete.service"; +import { EntityAnonymizeService } from "./entity-anonymize.service"; +import { CascadingActionResult } from "./cascading-entity-action"; + +describe("EntityActionsService", () => { + let service: EntityActionsService; + let mockedEntityMapper: jasmine.SpyObj; + let snackBarSpy: jasmine.SpyObj; + let mockSnackBarRef: jasmine.SpyObj>; + let mockConfirmationDialog: jasmine.SpyObj; + let mockRouter; + let mockedEntityDeleteService: jasmine.SpyObj; + + let primaryEntity: Entity; + + beforeEach(() => { + primaryEntity = new Entity(); + + mockedEntityDeleteService = jasmine.createSpyObj(["deleteEntity"]); + mockedEntityDeleteService.deleteEntity.and.resolveTo( + new CascadingActionResult([primaryEntity]), + ); + mockedEntityMapper = jasmine.createSpyObj(["save", "saveAll"]); + + snackBarSpy = jasmine.createSpyObj(["open"]); + mockSnackBarRef = jasmine.createSpyObj(["onAction", "afterDismissed"]); + mockSnackBarRef.onAction.and.returnValue(of()); + snackBarSpy.open.and.returnValue(mockSnackBarRef); + + mockConfirmationDialog = jasmine.createSpyObj([ + "getConfirmation", + "showProgressDialog", + ]); + mockConfirmationDialog.getConfirmation.and.resolveTo(true); + mockConfirmationDialog.showProgressDialog.and.returnValue( + jasmine.createSpyObj(["close"]), + ); + + TestBed.configureTestingModule({ + imports: [CoreTestingModule], + providers: [ + EntityActionsService, + { provide: EntityDeleteService, useValue: mockedEntityDeleteService }, + { provide: EntityAnonymizeService, useValue: null }, + { provide: EntityMapperService, useValue: mockedEntityMapper }, + { provide: MatSnackBar, useValue: snackBarSpy }, + Router, + { + provide: ConfirmationDialogService, + useValue: mockConfirmationDialog, + }, + ], + }); + mockRouter = TestBed.inject(Router); + spyOn(mockRouter, "navigate"); + + service = TestBed.inject(EntityActionsService); + }); + + it("should return false when user cancels confirmation", async () => { + mockConfirmationDialog.getConfirmation.and.resolveTo(false); + + const result = await service.delete(new Entity()); + + expect(result).toBe(false); + expect(snackBarSpy.open).not.toHaveBeenCalled(); + expect(mockedEntityDeleteService.deleteEntity).not.toHaveBeenCalled(); + }); + + it("should delete entity, show snackbar confirmation and navigate back", async () => { + // onAction is never called + mockSnackBarRef.onAction.and.returnValues(NEVER); + mockSnackBarRef.afterDismissed.and.returnValue(of(undefined)); + + const result = await service.delete(new Entity(), true); + + expect(result).toBe(true); + expect(snackBarSpy.open).toHaveBeenCalled(); + expect(mockedEntityDeleteService.deleteEntity).toHaveBeenCalled(); + expect(mockRouter.navigate).toHaveBeenCalled(); + }); + + it("should re-save all affected entities and navigate back to entity on undo", fakeAsync(() => { + const anotherAffectedEntity = new Entity(); + mockedEntityDeleteService.deleteEntity.and.resolveTo( + new CascadingActionResult([primaryEntity, anotherAffectedEntity]), + ); + + // Mock a snackbar where 'undo' is pressed + const onSnackbarAction = new Subject(); + mockSnackBarRef.onAction.and.returnValue(onSnackbarAction.asObservable()); + + mockedEntityMapper.save.and.resolveTo(); + + service.delete(primaryEntity, true); + tick(); + + mockRouter.navigate.calls.reset(); + onSnackbarAction.next(); + onSnackbarAction.complete(); + tick(); + + expect(mockedEntityDeleteService.deleteEntity).toHaveBeenCalled(); + expect(mockedEntityMapper.saveAll).toHaveBeenCalledWith( + [primaryEntity, anotherAffectedEntity], + true, + ); + expect(mockRouter.navigate).toHaveBeenCalled(); + })); + + it("should archive and save entity", async () => { + await service.archive(primaryEntity); + + expect(primaryEntity.isActive).toBeFalse(); + expect(mockedEntityMapper.save).toHaveBeenCalledWith(primaryEntity); + }); + + it("should archiveUndo and save entity", async () => { + await service.archive(primaryEntity); + expect(primaryEntity.isActive).toBeFalse(); + mockedEntityMapper.save.calls.reset(); + + await service.undoArchive(primaryEntity); + + expect(primaryEntity.isActive).toBeTrue(); + expect(mockedEntityMapper.save).toHaveBeenCalledWith(primaryEntity); + }); +}); diff --git a/src/app/core/entity/entity-remove.service.ts b/src/app/core/entity/entity-actions/entity-actions.service.ts similarity index 57% rename from src/app/core/entity/entity-remove.service.ts rename to src/app/core/entity/entity-actions/entity-actions.service.ts index 52dfc6a32e..ad44a37fcf 100644 --- a/src/app/core/entity/entity-remove.service.ts +++ b/src/app/core/entity/entity-actions/entity-actions.service.ts @@ -1,45 +1,35 @@ import { Injectable } from "@angular/core"; -import { ConfirmationDialogService } from "../common-components/confirmation-dialog/confirmation-dialog.service"; -import { EntityMapperService } from "./entity-mapper/entity-mapper.service"; +import { EntityMapperService } from "../entity-mapper/entity-mapper.service"; +import { Entity } from "../model/entity"; +import { ConfirmationDialogService } from "../../common-components/confirmation-dialog/confirmation-dialog.service"; import { MatSnackBar } from "@angular/material/snack-bar"; -import { Entity } from "./model/entity"; -import { getUrlWithoutParams } from "../../utils/utils"; import { Router } from "@angular/router"; -import { EntitySchemaService } from "./schema/entity-schema.service"; -import { FileDatatype } from "../../features/file/file.datatype"; -import { FileService } from "../../features/file/file.service"; -import { firstValueFrom } from "rxjs"; +import { getUrlWithoutParams } from "../../../utils/utils"; +import { EntityDeleteService } from "./entity-delete.service"; +import { EntityAnonymizeService } from "./entity-anonymize.service"; +import { OkButton } from "../../common-components/confirmation-dialog/confirmation-dialog/confirmation-dialog.component"; /** - * Additional options that can be (partly) specified - * for the several titles - */ -export interface RemoveEntityTextOptions { - dialogTitle?: string; - dialogText?: string; - deletedEntityInformation?: string; -} - -/** - * A service that can triggers a user flow to safely remove an entity, + * A service that can triggers a user flow for entity actions (e.g. to safely remove or anonymize an entity), * including a confirmation dialog. */ @Injectable({ providedIn: "root", }) -export class EntityRemoveService { +export class EntityActionsService { constructor( private confirmationDialog: ConfirmationDialogService, - private entityMapper: EntityMapperService, private snackBar: MatSnackBar, private router: Router, - private schemaService: EntitySchemaService, - private fileService: FileService, + private entityMapper: EntityMapperService, + private entityDelete: EntityDeleteService, + private entityAnonymize: EntityAnonymizeService, ) {} private showSnackbarConfirmation( entity: Entity, action: string, + previousEntitiesForUndo: Entity[], navigateBackToUrl?: string, ) { const snackBarTitle = $localize`:Entity action confirmation message:${ @@ -56,7 +46,12 @@ export class EntityRemoveService { // Undo Action snackBarRef.onAction().subscribe(async () => { - await this.entityMapper.save(entity, true); + const undoProgressRef = this.confirmationDialog.showProgressDialog( + $localize`:Undo entity action progress dialog: Reverting changes ...`, + ); + await this.entityMapper.saveAll(previousEntitiesForUndo, true); + undoProgressRef.close(); + if (navigateBackToUrl) { await this.router.navigate([navigateBackToUrl]); } @@ -70,9 +65,6 @@ export class EntityRemoveService { * This also triggers a toast message, enabling the user to undo the action. * * @param entity The entity to remove - * @param textOptions Options that you can specify to override the default options. - * You can only specify some of the options, the options that you don't specify will then be - * the default ones. * @param navigate whether upon delete the app will navigate back */ async delete( @@ -93,8 +85,22 @@ export class EntityRemoveService { return false; } - const originalEntity = entity.copy(); - await this.entityMapper.remove(entity); + const progressDialogRef = this.confirmationDialog.showProgressDialog( + $localize`:Entity action progress dialog:Processing ...`, + ); + const result = await this.entityDelete.deleteEntity(entity); + progressDialogRef.close(); + + if (result.potentiallyRetainingPII.length > 0) { + await this.confirmationDialog.getConfirmation( + $localize`:post-delete related PII warning title:Related records may still contain personal data`, + $localize`:post-delete related PII warning dialog:Some related records (e.g. notes) may still contain personal data in their text. We have automatically deleted all records that are linked to ONLY this ${ + entity.getConstructor().label + }. + However, there are some records that are linked to multiple records. We have not deleted these, so that you will not lose relevant data. Please review them manually to ensure all sensitive information is removed, if required (e.g. by looking through the linked notes and editing a note's text).`, + OkButton, + ); + } let currentUrl: string; if (navigate) { @@ -104,8 +110,9 @@ export class EntityRemoveService { } this.showSnackbarConfirmation( - originalEntity, + result.originalEntitiesBeforeChange[0], $localize`:Entity action confirmation message verb:Deleted`, + result.originalEntitiesBeforeChange, currentUrl, ); return true; @@ -134,48 +141,29 @@ export class EntityRemoveService { return false; } - const originalEntity = entity.copy(); - await this.anonymizeEntity(entity); - await this.entityMapper.save(entity); - - this.showSnackbarConfirmation( - originalEntity, - $localize`:Entity action confirmation message verb:Anonymized`, + const progressDialogRef = this.confirmationDialog.showProgressDialog( + $localize`:Entity action progress dialog:Processing ...`, ); - return true; - } - - private async anonymizeEntity(entity: Entity) { - for (const [key, schema] of entity.getSchema().entries()) { - if (entity[key] === undefined) { - continue; - } + const result = await this.entityAnonymize.anonymizeEntity(entity); + progressDialogRef.close(); - switch (schema.anonymize) { - case "retain": - break; - case "retain-anonymized": - await this.anonymizeProperty(entity, key); - break; - default: - await this.removeProperty(entity, key); - } + if (result.potentiallyRetainingPII.length > 0) { + await this.confirmationDialog.getConfirmation( + $localize`:post-anonymize related PII warning title:Related records may still contain personal data`, + $localize`:post-anonymize related PII warning dialog:Some related records (e.g. notes) may still contain personal data in their text. We have automatically anonymized all records that are linked to ONLY this ${ + entity.getConstructor().label + }. + However, there are some records that are linked to multiple records. We have not anonymized these, so that you will not lose relevant data. Please review them manually to ensure all sensitive information is removed (e.g. by looking through the linked notes and editing a note's text).`, + OkButton, + ); } - entity.anonymized = true; - entity.inactive = true; - } - - private async anonymizeProperty(entity: Entity, key: string) { - const dataType = this.schemaService.getDatatypeOrDefault( - entity.getSchema().get(key).dataType, - ); - - entity[key] = await dataType.anonymize( - entity[key], - entity.getSchema().get(key), - entity, + this.showSnackbarConfirmation( + result.originalEntitiesBeforeChange[0], + $localize`:Entity action confirmation message verb:Anonymized`, + result.originalEntitiesBeforeChange, ); + return true; } /** @@ -190,6 +178,7 @@ export class EntityRemoveService { this.showSnackbarConfirmation( originalEntity, $localize`:Entity action confirmation message verb:Archived`, + [originalEntity], ); return true; } @@ -205,18 +194,8 @@ export class EntityRemoveService { this.showSnackbarConfirmation( originalEntity, $localize`:Entity action confirmation message verb:Reactivated`, + [originalEntity], ); return true; } - - private async removeProperty(entity: Entity, key: string) { - if ( - entity.getSchema().get(key).dataType === FileDatatype.dataType && - entity[key] - ) { - await firstValueFrom(this.fileService.removeFile(entity, key)); - } - - delete entity[key]; - } } diff --git a/src/app/core/entity/entity-actions/entity-anonymize.service.spec.ts b/src/app/core/entity/entity-actions/entity-anonymize.service.spec.ts new file mode 100644 index 0000000000..1befc74733 --- /dev/null +++ b/src/app/core/entity/entity-actions/entity-anonymize.service.spec.ts @@ -0,0 +1,315 @@ +import { TestBed } from "@angular/core/testing"; +import { EntityMapperService } from "../entity-mapper/entity-mapper.service"; +import { Entity } from "../model/entity"; +import { of } from "rxjs"; +import { DatabaseEntity } from "../database-entity.decorator"; +import { DatabaseField } from "../database-field.decorator"; +import { + mockEntityMapper, + MockEntityMapperService, +} from "../entity-mapper/mock-entity-mapper-service"; +import { + comparableEntityData, + expectEntitiesToMatch, +} from "../../../utils/expect-entity-data.spec"; +import { UpdateMetadata } from "../model/update-metadata"; +import { FileService } from "../../../features/file/file.service"; +import { CoreTestingModule } from "../../../utils/core-testing.module"; +import { DefaultDatatype } from "../default-datatype/default.datatype"; +import { FileDatatype } from "../../../features/file/file.datatype"; +import moment from "moment"; +import { + allEntities, + ENTITIES, + EntityWithAnonRelations, + expectAllUnchangedExcept, +} from "./cascading-entity-action.spec"; +import { EntityAnonymizeService } from "./entity-anonymize.service"; + +describe("EntityAnonymizeService", () => { + let service: EntityAnonymizeService; + let entityMapper: MockEntityMapperService; + let mockFileService: jasmine.SpyObj; + + beforeEach(() => { + entityMapper = mockEntityMapper(allEntities.map((e) => e.copy())); + + mockFileService = jasmine.createSpyObj(["removeFile"]); + mockFileService.removeFile.and.returnValue(of(null)); + + TestBed.configureTestingModule({ + imports: [CoreTestingModule], + providers: [ + EntityAnonymizeService, + { provide: EntityMapperService, useValue: entityMapper }, + { provide: FileService, useValue: mockFileService }, + { provide: DefaultDatatype, useClass: FileDatatype, multi: true }, + ], + }); + + service = TestBed.inject(EntityAnonymizeService); + }); + + /* + * ANONYMIZATION + */ + @DatabaseEntity("AnonymizableEntity") + class AnonymizableEntity extends Entity { + static override hasPII = true; + + @DatabaseField() defaultField: string; + + @DatabaseField({ anonymize: "retain" }) + retainedField: string; + + @DatabaseField({ + anonymize: "retain-anonymized", + dataType: "array", + innerDataType: "date-only", + }) + retainAnonymizedDates: Date[]; + + @DatabaseField({ dataType: "file" }) file: string; + + @DatabaseField({ anonymize: "retain-anonymized", dataType: "entity-array" }) + referencesToRetainAnonymized: string[]; + + static create(properties: Partial) { + return Object.assign(new AnonymizableEntity(), properties); + } + + static expectAnonymized( + entityId: string, + expectedEntity: AnonymizableEntity, + checkAllBaseProperties = false, + ) { + const actualResult = entityMapper + .get(expectedEntity.getType(), entityId) + .copy(); + + if (!checkAllBaseProperties) { + delete actualResult.inactive; + delete actualResult.anonymized; + } + + expect(comparableEntityData(actualResult, true)).toEqual( + comparableEntityData(expectedEntity, true), + ); + } + } + + it("should anonymize and only keep properties marked to be retained", async () => { + const entity = new AnonymizableEntity(); + entity.defaultField = "test"; + entity.retainedField = "test"; + + await service.anonymizeEntity(entity); + + AnonymizableEntity.expectAnonymized( + entity.getId(), + AnonymizableEntity.create({ retainedField: "test" }), + ); + }); + + it("should anonymize and keep empty record without any fields", async () => { + const entity = new AnonymizableEntity(); + entity.defaultField = "test"; + + await service.anonymizeEntity(entity); + + AnonymizableEntity.expectAnonymized( + entity.getId(), + AnonymizableEntity.create({}), + ); + }); + + it("should anonymize and retain created and updated", async () => { + const entityProperties = { + created: new UpdateMetadata("CREATOR", new Date("2020-01-01")), + updated: new UpdateMetadata("UPDATER", new Date("2020-01-02")), + }; + const entity = AnonymizableEntity.create({ + defaultField: "test", + ...entityProperties, + }); + + await service.anonymizeEntity(entity); + + AnonymizableEntity.expectAnonymized( + entity.getId(), + AnonymizableEntity.create({ + inactive: true, + anonymized: true, + ...entityProperties, + }), + true, + ); + }); + + it("should mark anonymized entities as inactive", async () => { + const entity = new AnonymizableEntity(); + entity.defaultField = "test"; + + await service.anonymizeEntity(entity); + + AnonymizableEntity.expectAnonymized( + entity.getId(), + AnonymizableEntity.create({ inactive: true, anonymized: true }), + true, + ); + }); + + it("should anonymize array values recursively and use datatype implementation for 'retain-anonymized", async () => { + const entity = new AnonymizableEntity(); + entity.retainAnonymizedDates = [ + moment("2023-09-25").toDate(), + moment("2023-10-04").toDate(), + ]; + + await service.anonymizeEntity(entity); + + AnonymizableEntity.expectAnonymized( + entity.getId(), + AnonymizableEntity.create({ + retainAnonymizedDates: [ + moment("2023-07-01").toDate(), + moment("2023-07-01").toDate(), + ], + }), + ); + }); + + it("should anonymize file values, actively deleting file attachments", async () => { + const entity = new AnonymizableEntity(); + entity.file = "test-file.txt"; + + await service.anonymizeEntity(entity); + + AnonymizableEntity.expectAnonymized( + entity.getId(), + AnonymizableEntity.create({}), + ); + expect(mockFileService.removeFile).toHaveBeenCalled(); + }); + + it("should not anonymize fields if Entity type is set to not have PII", async () => { + AnonymizableEntity.hasPII = false; + const entity = new AnonymizableEntity(); + // make sure the original entity is available initially (we expect it to remain unchanged) + entityMapper.add(entity); + entity.defaultField = "test"; + + await service.anonymizeEntity(entity); + + AnonymizableEntity.expectAnonymized( + entity.getId(), + AnonymizableEntity.create({ defaultField: "test" }), + true, + ); + + // reset actual state + AnonymizableEntity.hasPII = true; + }); + + /* + CASCADING ANONYMIZATION + */ + function expectAnonymized( + expectedToGetAnonymized: EntityWithAnonRelations[], + entityMapper: MockEntityMapperService, + ) { + const actualEntitiesAfter = entityMapper.getAllData(); + + for (const anonEntity of expectedToGetAnonymized) { + const actualEntity = actualEntitiesAfter.find( + (e) => e.getId() === anonEntity.getId(), + ); + + const expectedAnonymizedEntity = new EntityWithAnonRelations( + anonEntity.getId(), + ); + // copy over properties that are marked as `anonymize: "retain"` + expectedAnonymizedEntity.refAggregate = anonEntity.refAggregate; + expectedAnonymizedEntity.refComposite = anonEntity.refComposite; + expectedAnonymizedEntity.inactive = true; + expectedAnonymizedEntity.anonymized = true; + + expect(comparableEntityData(actualEntity)).toEqual( + comparableEntityData(expectedAnonymizedEntity), + ); + } + + expectAllUnchangedExcept(expectedToGetAnonymized, entityMapper); + } + + it("should not cascade anonymize the related entity if the entity holding the reference is anonymized", async () => { + // for direct references (e.g. x.referencesToRetainAnonymized --> recursively calls anonymize on referenced entities) + // see EntityDatatype & EntityArrayDatatype for unit tests + + await service.anonymizeEntity(ENTITIES.ReferencingSingleComposite); + + expectAnonymized([ENTITIES.ReferencingSingleComposite], entityMapper); + }); + + it("should cascade anonymize the 'composite'-type entity that references the entity user acts on", async () => { + await service.anonymizeEntity(ENTITIES.ReferencedAsComposite); + + expectAnonymized( + [ENTITIES.ReferencedAsComposite, ENTITIES.ReferencingSingleComposite], + entityMapper, + ); + }); + + it("should not cascade anonymize the 'composite'-type entity that still references additional other entities but ask user", async () => { + const result = await service.anonymizeEntity( + ENTITIES.ReferencedAsOneOfMultipleComposites1, + ); + + expectAnonymized( + [ENTITIES.ReferencedAsOneOfMultipleComposites1], + entityMapper, + ); + // warn user that there may be personal details in referencing entity which have not been deleted + expectEntitiesToMatch(result.potentiallyRetainingPII, [ + ENTITIES.ReferencingTwoComposites, + ]); + }); + + it("should cascade anonymize the 'composite'-type entity that references the entity user acts on even when another property holds other id (e.g. ChildSchoolRelation)", async () => { + await service.anonymizeEntity( + ENTITIES.ReferencingCompositeAndAggregate_refComposite, + ); + + expectAnonymized( + [ + ENTITIES.ReferencingCompositeAndAggregate_refComposite, + ENTITIES.ReferencingCompositeAndAggregate, + ], + entityMapper, + ); + }); + + it("should not cascade anonymize the 'aggregate'-type entity that only references the entity user acts on but ask user", async () => { + const result = await service.anonymizeEntity( + ENTITIES.ReferencingAggregate_ref, + ); + + expectAnonymized([ENTITIES.ReferencingAggregate_ref], entityMapper); + // warn user that there may be personal details in referencing entity which have not been deleted + expectEntitiesToMatch(result.potentiallyRetainingPII, [ + ENTITIES.ReferencingAggregate, + ]); + }); + + it("should not cascade anonymize the 'aggregate'-type entity that still references additional other entities but ask user", async () => { + const result = await service.anonymizeEntity( + ENTITIES.ReferencingTwoAggregates_ref1, + ); + + expectAnonymized([ENTITIES.ReferencingTwoAggregates_ref1], entityMapper); + // warn user that there may be personal details in referencing entity which have not been deleted + expectEntitiesToMatch(result.potentiallyRetainingPII, [ + ENTITIES.ReferencingTwoAggregates, + ]); + }); +}); diff --git a/src/app/core/entity/entity-actions/entity-anonymize.service.ts b/src/app/core/entity/entity-actions/entity-anonymize.service.ts new file mode 100644 index 0000000000..731e5757e6 --- /dev/null +++ b/src/app/core/entity/entity-actions/entity-anonymize.service.ts @@ -0,0 +1,101 @@ +import { Injectable } from "@angular/core"; +import { EntityMapperService } from "../entity-mapper/entity-mapper.service"; +import { EntitySchemaService } from "../schema/entity-schema.service"; +import { + CascadingActionResult, + CascadingEntityAction, +} from "./cascading-entity-action"; +import { firstValueFrom } from "rxjs"; +import { FileDatatype } from "../../../features/file/file.datatype"; +import { FileService } from "../../../features/file/file.service"; +import { Entity } from "../model/entity"; + +/** + * Anonymize an entity including handling references with related entities. + * This service is usually used in combination with the `EntityActionsService`, which provides user confirmation processes around this. + */ +@Injectable({ + providedIn: "root", +}) +export class EntityAnonymizeService extends CascadingEntityAction { + constructor( + protected entityMapper: EntityMapperService, + protected schemaService: EntitySchemaService, + private fileService: FileService, + ) { + super(entityMapper, schemaService); + } + + /** + * The actual anonymize action without user interactions. + * @param entity + * @private + */ + async anonymizeEntity(entity: Entity): Promise { + if (!entity.getConstructor().hasPII) { + // entity types that are generally without PII by default retain all fields + // this should only be called through a cascade action anyway + return new CascadingActionResult(); + } + + const originalEntity = entity.copy(); + + for (const [key, schema] of entity.getSchema().entries()) { + if (entity[key] === undefined) { + continue; + } + + switch (schema.anonymize) { + case "retain": + break; + case "retain-anonymized": + await this.anonymizeProperty(entity, key); + break; + default: + await this.removeProperty(entity, key); + } + } + + entity.anonymized = true; + entity.inactive = true; + + await this.entityMapper.save(entity); + + const cascadeResult = await this.cascadeActionToRelatedEntities( + entity, + (e) => this.anonymizeEntity(e), + (e) => this.keepEntityUnchanged(e), + ); + + return new CascadingActionResult([originalEntity]).mergeResults( + cascadeResult, + ); + } + + private async anonymizeProperty(entity: Entity, key: string) { + const dataType = this.schemaService.getDatatypeOrDefault( + entity.getSchema().get(key).dataType, + ); + + entity[key] = await dataType.anonymize( + entity[key], + entity.getSchema().get(key), + entity, + ); + } + + private async removeProperty(entity: Entity, key: string) { + if ( + entity.getSchema().get(key).dataType === FileDatatype.dataType && + entity[key] + ) { + await firstValueFrom(this.fileService.removeFile(entity, key)); + } + + delete entity[key]; + } + + private async keepEntityUnchanged(e: Entity): Promise { + return new CascadingActionResult([], e.getConstructor().hasPII ? [e] : []); + } +} diff --git a/src/app/core/entity/entity-actions/entity-delete.service.spec.ts b/src/app/core/entity/entity-actions/entity-delete.service.spec.ts new file mode 100644 index 0000000000..182f37cf43 --- /dev/null +++ b/src/app/core/entity/entity-actions/entity-delete.service.spec.ts @@ -0,0 +1,205 @@ +import { TestBed } from "@angular/core/testing"; +import { CoreTestingModule } from "../../../utils/core-testing.module"; +import { EntityDeleteService } from "./entity-delete.service"; +import { + mockEntityMapper, + MockEntityMapperService, +} from "../entity-mapper/mock-entity-mapper-service"; +import { EntityMapperService } from "../entity-mapper/entity-mapper.service"; +import { + allEntities, + ENTITIES, + EntityWithAnonRelations, + expectAllUnchangedExcept, + expectDeleted, + expectUpdated, +} from "./cascading-entity-action.spec"; +import { expectEntitiesToMatch } from "../../../utils/expect-entity-data.spec"; +import { Note } from "../../../child-dev-project/notes/model/note"; +import { Child } from "../../../child-dev-project/children/model/child"; + +describe("EntityDeleteService", () => { + let service: EntityDeleteService; + let entityMapper: MockEntityMapperService; + + beforeEach(() => { + entityMapper = mockEntityMapper(allEntities.map((e) => e.copy())); + + TestBed.configureTestingModule({ + imports: [CoreTestingModule], + providers: [ + EntityDeleteService, + { provide: EntityMapperService, useValue: entityMapper }, + ], + }); + + service = TestBed.inject(EntityDeleteService); + }); + + function removeReference( + entity: EntityWithAnonRelations, + property: "refAggregate" | "refComposite", + referencedEntity: EntityWithAnonRelations, + ) { + const result = entity.copy(); + result[property] = result[property].filter( + (id) => + id !== referencedEntity.getId() && id !== referencedEntity.getId(true), + ); + return result; + } + + it("should not cascade delete the related entity if the entity holding the reference is deleted", async () => { + // for direct references (e.g. x.referencesToRetainAnonymized --> recursively calls anonymize on referenced entities) + // see EntityDatatype & EntityArrayDatatype for unit tests + + await service.deleteEntity(ENTITIES.ReferencingSingleComposite); + + expectDeleted([ENTITIES.ReferencingSingleComposite], entityMapper); + expectAllUnchangedExcept( + [ENTITIES.ReferencingSingleComposite], + entityMapper, + ); + }); + + it("should cascade delete the 'composite'-type entity that references the entity user acts on", async () => { + await service.deleteEntity(ENTITIES.ReferencedAsComposite); + + expectDeleted( + [ENTITIES.ReferencedAsComposite, ENTITIES.ReferencingSingleComposite], + entityMapper, + ); + expectAllUnchangedExcept( + [ENTITIES.ReferencedAsComposite, ENTITIES.ReferencingSingleComposite], + entityMapper, + ); + }); + + it("should not cascade delete the 'composite'-type entity that still references additional other entities but remove id", async () => { + const result = await service.deleteEntity( + ENTITIES.ReferencedAsOneOfMultipleComposites1, + ); + + const expectedUpdatedRelEntity = removeReference( + ENTITIES.ReferencingTwoComposites, + "refComposite", + ENTITIES.ReferencedAsOneOfMultipleComposites1, + ); + expectDeleted( + [ENTITIES.ReferencedAsOneOfMultipleComposites1], + entityMapper, + ); + expectUpdated([expectedUpdatedRelEntity], entityMapper); + expectAllUnchangedExcept( + [ + ENTITIES.ReferencedAsOneOfMultipleComposites1, + ENTITIES.ReferencingTwoComposites, + ], + entityMapper, + ); + // warn user that there may be personal details in referencing entity which have not been deleted + expectEntitiesToMatch(result.potentiallyRetainingPII, [ + expectedUpdatedRelEntity, + ]); + }); + + it("should cascade delete the 'composite'-type entity that references the entity user acts on even when another property holds other id (e.g. ChildSchoolRelation)", async () => { + await service.deleteEntity( + ENTITIES.ReferencingCompositeAndAggregate_refComposite, + ); + + expectDeleted( + [ + ENTITIES.ReferencingCompositeAndAggregate_refComposite, + ENTITIES.ReferencingCompositeAndAggregate, + ], + entityMapper, + ); + expectAllUnchangedExcept( + [ + ENTITIES.ReferencingCompositeAndAggregate_refComposite, + ENTITIES.ReferencingCompositeAndAggregate, + ], + entityMapper, + ); + }); + + it("should not cascade delete the 'aggregate'-type entity that only references the entity user acts on but remove id", async () => { + const result = await service.deleteEntity( + ENTITIES.ReferencingAggregate_ref, + ); + + const expectedUpdatedRelEntity = removeReference( + ENTITIES.ReferencingAggregate, + "refAggregate", + ENTITIES.ReferencingAggregate_ref, + ); + expectDeleted([ENTITIES.ReferencingAggregate_ref], entityMapper); + expectUpdated([expectedUpdatedRelEntity], entityMapper); + expectAllUnchangedExcept( + [ENTITIES.ReferencingAggregate_ref, ENTITIES.ReferencingAggregate], + entityMapper, + ); + // warn user that there may be personal details in referencing entity which have not been deleted + expectEntitiesToMatch(result.potentiallyRetainingPII, [ + expectedUpdatedRelEntity, + ]); + }); + + it("should not cascade delete the 'aggregate'-type entity that still references additional other entities but remove id", async () => { + await service.deleteEntity(ENTITIES.ReferencingTwoAggregates_ref1); + + expectDeleted([ENTITIES.ReferencingTwoAggregates_ref1], entityMapper); + expectUpdated( + [ + removeReference( + ENTITIES.ReferencingTwoAggregates, + "refAggregate", + ENTITIES.ReferencingTwoAggregates_ref1, + ), + ], + entityMapper, + ); + expectAllUnchangedExcept( + [ + ENTITIES.ReferencingTwoAggregates_ref1, + ENTITIES.ReferencingTwoAggregates, + ], + entityMapper, + ); + }); + + it("should remove multiple ref ids from related note", async () => { + const schemaField = Note.schema.get("relatedEntities"); + const originalSchemaAdditional = schemaField.additional; + schemaField.additional = [Child.ENTITY_TYPE]; + + const primary = new Child(); + const note = new Note(); + note.subject = "test"; + note.children = [primary.getId(), "some-other"]; + note.relatedEntities = [primary.getId(true)]; + const originalNote = note.copy(); + await entityMapper.save(primary); + await entityMapper.save(note); + + const result = await service.deleteEntity(primary); + + const actualNote = entityMapper.get( + Note.ENTITY_TYPE, + note.getId(true), + ) as Note; + + expect(actualNote.relatedEntities).toEqual([]); + expect(actualNote.children).toEqual(["some-other"]); + + expect(result.originalEntitiesBeforeChange.length).toBe(2); + expectEntitiesToMatch(result.originalEntitiesBeforeChange, [ + primary, + originalNote, + ]); + + // restore original schema + schemaField.additional = originalSchemaAdditional; + }); +}); diff --git a/src/app/core/entity/entity-actions/entity-delete.service.ts b/src/app/core/entity/entity-actions/entity-delete.service.ts new file mode 100644 index 0000000000..6f28fb8280 --- /dev/null +++ b/src/app/core/entity/entity-actions/entity-delete.service.ts @@ -0,0 +1,89 @@ +import { Injectable } from "@angular/core"; +import { EntityMapperService } from "../entity-mapper/entity-mapper.service"; +import { Entity } from "../model/entity"; +import { EntitySchemaService } from "../schema/entity-schema.service"; +import { + CascadingActionResult, + CascadingEntityAction, +} from "./cascading-entity-action"; + +/** + * Safely delete an entity including handling references with related entities. + * This service is usually used in combination with the `EntityActionsService`, which provides user confirmation processes around this. + */ +@Injectable({ + providedIn: "root", +}) +export class EntityDeleteService extends CascadingEntityAction { + constructor( + protected entityMapper: EntityMapperService, + protected schemaService: EntitySchemaService, + ) { + super(entityMapper, schemaService); + } + + /** + * The actual delete action without user interactions. + * + * Returns an array of all affected entities (including the given entity) in their state before the action + * to support an undo action. + * + * @param entity + * @private + */ + async deleteEntity(entity: Entity): Promise { + const cascadeResult = await this.cascadeActionToRelatedEntities( + entity, + (e) => this.deleteEntity(e), + (e, refField, entity) => + this.removeReferenceFromEntity(e, refField, entity), + ); + + const originalEntity = entity.copy(); + await this.entityMapper.remove(entity); + + return new CascadingActionResult([originalEntity]).mergeResults( + cascadeResult, + ); + } + + /** + * Change and save the entity, removing referenced ids of the given referenced entity. + * + * Returns an array of the affected entities (which here is only the given entity) in the state before the action + * to support an undo action. + * + * @param relatedEntityWithReference + * @param refField + * @param referencedEntity + * @private + */ + private async removeReferenceFromEntity( + relatedEntityWithReference: Entity, + refField: string, + referencedEntity: Entity, + ): Promise { + const originalEntity = relatedEntityWithReference.copy(); + + if (Array.isArray(relatedEntityWithReference[refField])) { + relatedEntityWithReference[refField] = relatedEntityWithReference[ + refField + ].filter( + (id) => + id !== referencedEntity.getId() && + id !== referencedEntity.getId(true), + ); + } else { + delete relatedEntityWithReference[refField]; + } + + await this.entityMapper.save(relatedEntityWithReference); + + return new CascadingActionResult( + [originalEntity], + relatedEntityWithReference.getConstructor().hasPII + ? [relatedEntityWithReference] + : [], + ); + } +} diff --git a/src/app/core/entity/entity-config.service.ts b/src/app/core/entity/entity-config.service.ts index 13df3bb5eb..b33bbd307d 100644 --- a/src/app/core/entity/entity-config.service.ts +++ b/src/app/core/entity/entity-config.service.ts @@ -76,6 +76,7 @@ export class EntityConfigService { ), ); } + // TODO: shall we just assign all properties that are present in the config object? entityType.toStringAttributes = entityConfig.toStringAttributes ?? entityType.toStringAttributes; entityType.label = entityConfig.label ?? entityType.label; @@ -83,6 +84,7 @@ export class EntityConfigService { entityType.icon = (entityConfig.icon as IconName) ?? entityType.icon; entityType.color = entityConfig.color ?? entityType.color; entityType.route = entityConfig.route ?? entityType.route; + entityType.hasPII = entityConfig.hasPII ?? entityType.hasPII; entityType._isCustomizedType = true; } @@ -156,4 +158,9 @@ export interface EntityConfig { * when a new entity is created, all properties from this class will also be available */ extends?: string; + + /** + * whether the type can contain personally identifiable information (PII) + */ + hasPII?: boolean; } diff --git a/src/app/core/entity/entity-mapper/entity-mapper.service.ts b/src/app/core/entity/entity-mapper/entity-mapper.service.ts index b7c800b417..29c0e54102 100644 --- a/src/app/core/entity/entity-mapper/entity-mapper.service.ts +++ b/src/app/core/entity/entity-mapper/entity-mapper.service.ts @@ -155,13 +155,18 @@ export class EntityMapperService { * This method should be chosen whenever a bigger number of entities needs to be * saved * @param entities The entities to save + * @param forceUpdate Optional flag whether any conflicting version in the database will be quietly overwritten. + * if a conflict occurs without the forceUpdate flag being set, the save will fail, rejecting the returned promise. */ - public async saveAll(entities: Entity[]): Promise { + public async saveAll( + entities: Entity[], + forceUpdate: boolean = false, + ): Promise { entities.forEach((e) => this.setEntityMetadata(e)); const rawData = entities.map((e) => this.entitySchemaService.transformEntityToDatabaseFormat(e), ); - const results = await this._db.putAll(rawData); + const results = await this._db.putAll(rawData, forceUpdate); results.forEach((res, idx) => { if (res.ok) { const entity = entities[idx]; diff --git a/src/app/core/entity/entity-remove.service.spec.ts b/src/app/core/entity/entity-remove.service.spec.ts deleted file mode 100644 index 0e9861f513..0000000000 --- a/src/app/core/entity/entity-remove.service.spec.ts +++ /dev/null @@ -1,300 +0,0 @@ -import { fakeAsync, TestBed, tick } from "@angular/core/testing"; -import { EntityRemoveService } from "./entity-remove.service"; -import { EntityMapperService } from "./entity-mapper/entity-mapper.service"; -import { - MatSnackBar, - MatSnackBarRef, - TextOnlySnackBar, -} from "@angular/material/snack-bar"; -import { ConfirmationDialogService } from "../common-components/confirmation-dialog/confirmation-dialog.service"; -import { Entity } from "./model/entity"; -import { NEVER, of, Subject } from "rxjs"; -import { Router } from "@angular/router"; -import { DatabaseEntity } from "./database-entity.decorator"; -import { DatabaseField } from "./database-field.decorator"; -import { mockEntityMapper } from "./entity-mapper/mock-entity-mapper-service"; -import { expectEntitiesToMatch } from "../../utils/expect-entity-data.spec"; -import { UpdateMetadata } from "./model/update-metadata"; -import { FileService } from "../../features/file/file.service"; -import { CoreTestingModule } from "../../utils/core-testing.module"; -import { DefaultDatatype } from "./default-datatype/default.datatype"; -import { FileDatatype } from "../../features/file/file.datatype"; -import moment from "moment"; - -describe("EntityRemoveService", () => { - let service: EntityRemoveService; - let mockedEntityMapper: jasmine.SpyObj; - let snackBarSpy: jasmine.SpyObj; - let mockSnackBarRef: jasmine.SpyObj>; - let mockConfirmationDialog: jasmine.SpyObj; - let mockFileService: jasmine.SpyObj; - let mockRouter; - - beforeEach(() => { - mockedEntityMapper = jasmine.createSpyObj(["remove", "save"]); - snackBarSpy = jasmine.createSpyObj(["open"]); - mockSnackBarRef = jasmine.createSpyObj(["onAction", "afterDismissed"]); - mockSnackBarRef.onAction.and.returnValue(of()); - mockConfirmationDialog = jasmine.createSpyObj(["getConfirmation"]); - mockConfirmationDialog.getConfirmation.and.resolveTo(true); - snackBarSpy.open.and.returnValue(mockSnackBarRef); - mockedEntityMapper.remove.and.resolveTo(); - mockFileService = jasmine.createSpyObj(["removeFile"]); - mockFileService.removeFile.and.returnValue(of(null)); - - TestBed.configureTestingModule({ - imports: [CoreTestingModule], - providers: [ - EntityRemoveService, - { provide: EntityMapperService, useValue: mockedEntityMapper }, - { provide: MatSnackBar, useValue: snackBarSpy }, - Router, - { - provide: ConfirmationDialogService, - useValue: mockConfirmationDialog, - }, - { provide: FileService, useValue: mockFileService }, - { provide: DefaultDatatype, useClass: FileDatatype, multi: true }, - ], - }); - mockRouter = TestBed.inject(Router); - spyOn(mockRouter, "navigate"); - - service = TestBed.inject(EntityRemoveService); - }); - - it("should return false when user cancels confirmation", async () => { - mockConfirmationDialog.getConfirmation.and.resolveTo(false); - - const result = await service.delete(new Entity()); - - expect(result).toBe(false); - expect(snackBarSpy.open).not.toHaveBeenCalled(); - expect(mockedEntityMapper.remove).not.toHaveBeenCalled(); - }); - - it("should delete entity, show snackbar confirmation and navigate back", async () => { - // onAction is never called - mockSnackBarRef.onAction.and.returnValues(NEVER); - mockSnackBarRef.afterDismissed.and.returnValue(of(undefined)); - - const result = await service.delete(new Entity(), true); - - expect(result).toBe(true); - expect(snackBarSpy.open).toHaveBeenCalled(); - expect(mockedEntityMapper.remove).toHaveBeenCalled(); - expect(mockRouter.navigate).toHaveBeenCalled(); - }); - - it("should re-save entity and navigate back to entity on undo", fakeAsync(() => { - const entity = new Entity(); - - // Mock a snackbar where 'undo' is immediately pressed - const onSnackbarAction = new Subject(); - mockSnackBarRef.onAction.and.returnValue(onSnackbarAction.asObservable()); - mockSnackBarRef.afterDismissed.and.returnValue(NEVER); - - mockedEntityMapper.save.and.resolveTo(); - - service.delete(entity, true); - tick(); - - mockRouter.navigate.calls.reset(); - onSnackbarAction.next(); - onSnackbarAction.complete(); - tick(); - - expect(mockedEntityMapper.remove).toHaveBeenCalled(); - expect(mockedEntityMapper.save).toHaveBeenCalledWith(entity, true); - expect(mockRouter.navigate).toHaveBeenCalled(); - })); - - it("should archive and save entity", async () => { - const entity = new Entity(); - - await service.archive(entity); - - expect(entity.isActive).toBeFalse(); - expect(mockedEntityMapper.save).toHaveBeenCalledWith(entity); - }); - - it("should archiveUndo and save entity", async () => { - const entity = new Entity(); - - await service.archive(entity); - expect(entity.isActive).toBeFalse(); - mockedEntityMapper.save.calls.reset(); - - await service.undoArchive(entity); - - expect(entity.isActive).toBeTrue(); - expect(mockedEntityMapper.save).toHaveBeenCalledWith(entity); - }); - - /* - * ANONYMIZATION - */ - @DatabaseEntity("AnonymizableEntity") - class AnonymizableEntity extends Entity { - @DatabaseField() defaultField: string; - - @DatabaseField({ anonymize: "retain" }) - retainedField: string; - - @DatabaseField({ - anonymize: "retain-anonymized", - dataType: "array", - innerDataType: "date-only", - }) - retainAnonymizedDates: Date[]; - - @DatabaseField({ dataType: "file" }) file: string; - - @DatabaseField({ anonymize: "retain-anonymized", dataType: "entity-array" }) - referencesToRetainAnonymized: string[]; - - static create(properties: Partial) { - return Object.assign(new AnonymizableEntity(), properties); - } - } - - async function testAnonymization( - entity: AnonymizableEntity, - entitiesBefore: any[], - expectedEntitiesAfter: any[], - checkAllBaseProperties: boolean = false, - ) { - const entityMapper = mockEntityMapper(entitiesBefore); - - // @ts-ignore - service.entityMapper = entityMapper; - - await service.anonymize(entity); - - const actualEntitiesAfter = entityMapper.getAllData(); - - if (!checkAllBaseProperties) { - actualEntitiesAfter.forEach((e) => { - delete e.inactive; - delete e.anonymized; - }); - } - - expectEntitiesToMatch(actualEntitiesAfter, expectedEntitiesAfter, true); - } - - it("should anonymize and only keep properties marked to be retained", async () => { - const entity = new AnonymizableEntity(); - entity.defaultField = "test"; - entity.retainedField = "test"; - - await testAnonymization( - entity, - [entity], - [AnonymizableEntity.create({ retainedField: "test" })], - ); - }); - - it("should anonymize and keep empty record without any fields", async () => { - const entity = new AnonymizableEntity(); - entity.defaultField = "test"; - - await testAnonymization(entity, [entity], [AnonymizableEntity.create({})]); - }); - - it("should anonymize and retain created and updated", async () => { - const entityProperties = { - created: new UpdateMetadata("CREATOR", new Date("2020-01-01")), - updated: new UpdateMetadata("UPDATER", new Date("2020-01-02")), - }; - const entity = AnonymizableEntity.create({ - defaultField: "test", - ...entityProperties, - }); - - await testAnonymization( - entity, - [entity], - [ - AnonymizableEntity.create({ - inactive: true, - anonymized: true, - ...entityProperties, - }), - ], - true, - ); - }); - - it("should mark anonymized entities as inactive", async () => { - const entity = new AnonymizableEntity(); - entity.defaultField = "test"; - - await testAnonymization( - entity, - [entity], - [AnonymizableEntity.create({ inactive: true, anonymized: true })], - true, - ); - }); - - it("should anonymize array values recursively and use datatype implementation for 'retain-anonymized", async () => { - const entity = new AnonymizableEntity(); - entity.retainAnonymizedDates = [ - moment("2023-09-25").toDate(), - moment("2023-10-04").toDate(), - ]; - - await testAnonymization( - entity, - [entity], - [ - AnonymizableEntity.create({ - retainAnonymizedDates: [ - moment("2023-07-01").toDate(), - moment("2023-07-01").toDate(), - ], - }), - ], - ); - }); - - it("should anonymize file values, actively deleting file attachments", async () => { - const entity = new AnonymizableEntity(); - entity.file = "test-file.txt"; - - await testAnonymization(entity, [entity], [AnonymizableEntity.create({})]); - expect(mockFileService.removeFile).toHaveBeenCalled(); - }); - - // - // Anonymizing referenced & related entities - // - - // for direct references (e.g. x.referencesToRetainAnonymized --> recursively calls anonymize on referenced entities) - // see EntityDatatype & EntityArrayDatatype for unit tests - - xit("should anonymize cascadingly entities that reference the entity being anonymized", async () => { - // TODO: cascading anonymization - see https://github.com/Aam-Digital/ndb-core/issues/220 - - const entity = new AnonymizableEntity(); - entity.retainedField = "entity being anonymized"; - - const ref1 = new AnonymizableEntity("ref-1"); - ref1.defaultField = "test-1"; - ref1.retainedField = "test-1"; - ref1.referencesToRetainAnonymized = [entity.getId()]; - - await testAnonymization( - entity, - [entity, ref1], - [ - entity, - AnonymizableEntity.create({ - retainedField: "test-1", - referencesToRetainAnonymized: [entity.getId()], - }), - ], - ); - }); -}); diff --git a/src/app/core/entity/model/entity.ts b/src/app/core/entity/model/entity.ts index 6e12ba4d1e..1bf954d818 100644 --- a/src/app/core/entity/model/entity.ts +++ b/src/app/core/entity/model/entity.ts @@ -166,6 +166,13 @@ export class Entity { return; } + /** + * whether this entity type can contain "personally identifiable information" (PII) + * and therefore should follow strict data protection requirements + * and offer a function to anonymize records. + */ + static hasPII: boolean = false; + /** * Internal database id. * This is usually combined from the ENTITY_TYPE as a prefix with the entityId field `EntityType:entityId` diff --git a/src/app/core/entity/schema/entity-schema-field.ts b/src/app/core/entity/schema/entity-schema-field.ts index 011b5816c2..c6c5f210b0 100644 --- a/src/app/core/entity/schema/entity-schema-field.ts +++ b/src/app/core/entity/schema/entity-schema-field.ts @@ -16,6 +16,7 @@ */ import { FormValidatorConfig } from "../../common-components/entity-form/dynamic-form-validators/form-validator-config"; +import { EntityReferenceRole } from "../../basic-datatypes/entity/entity-reference-role"; /** * Interface for additional configuration about a DatabaseField schema. @@ -66,6 +67,17 @@ export interface EntitySchemaField { */ additional?: any; + /** + * (Optional) If the dataType of this field references another entity, + * define the role of this relationship for the entity containing this field. + * + * i.e. how "important" is the entity this field is referencing? + * Does this the entity containing this field (not the referenced entity) still have meaning after the referenced entity has been deleted? + * + * see options of the `EntityReferenceRole` type + */ + entityReferenceRole?: EntityReferenceRole; + /** * (Optional) Define using which component this property should be displayed in lists and forms. * diff --git a/src/app/core/entity/schema/entity-schema.service.spec.ts b/src/app/core/entity/schema/entity-schema.service.spec.ts index 2b72caac62..05bede894e 100644 --- a/src/app/core/entity/schema/entity-schema.service.spec.ts +++ b/src/app/core/entity/schema/entity-schema.service.spec.ts @@ -26,6 +26,7 @@ import { EntitySchemaField } from "./entity-schema-field"; import { ConfigurableEnumDatatype } from "../../basic-datatypes/configurable-enum/configurable-enum-datatype/configurable-enum.datatype"; import { MockedTestingModule } from "../../../utils/mocked-testing.module"; import { EntityDatatype } from "../../basic-datatypes/entity/entity.datatype"; +import { DatabaseEntity, EntityRegistry } from "../database-entity.decorator"; describe("EntitySchemaService", () => { let service: EntitySchemaService; @@ -133,6 +134,52 @@ describe("EntitySchemaService", () => { EntityDatatype, ); }); + + it("should getEntityTypesReferencingType with all entity types having schema fields referencing the given type", () => { + @DatabaseEntity("ReferencingEntity") + class ReferencingEntity extends Entity { + @DatabaseField({ + dataType: "entity-array", + additional: "Child", + }) + refChildren: string[]; + + @DatabaseField({ + dataType: "entity", + additional: "Child", + }) + refChild: string; + + @DatabaseField({ + dataType: "entity", + additional: "School", + }) + refSchool: string; + + @DatabaseField({ + dataType: "entity-array", + additional: ["Child", "School"], + }) + multiTypeRef: string[]; + } + + const entities = new EntityRegistry(); + entities.addAll([ + [ReferencingEntity.ENTITY_TYPE, ReferencingEntity], + [Entity.ENTITY_TYPE, Entity], + ]); + const injector = TestBed.inject(Injector); + spyOn(injector, "get").and.returnValue(entities); + + const result = service.getEntityTypesReferencingType("Child"); + + expect(result).toEqual([ + { + entityType: ReferencingEntity, + referencingProperties: ["refChildren", "refChild", "multiTypeRef"], + }, + ]); + }); }); export function testDatatype( diff --git a/src/app/core/entity/schema/entity-schema.service.ts b/src/app/core/entity/schema/entity-schema.service.ts index ca89c52a94..4fe5075420 100644 --- a/src/app/core/entity/schema/entity-schema.service.ts +++ b/src/app/core/entity/schema/entity-schema.service.ts @@ -15,11 +15,13 @@ * along with ndb-core. If not, see . */ -import { Entity } from "../model/entity"; +import { Entity, EntityConstructor } from "../model/entity"; import { Injectable, Injector } from "@angular/core"; import { EntitySchema } from "./entity-schema"; import { EntitySchemaField } from "./entity-schema-field"; import { DefaultDatatype } from "../default-datatype/default.datatype"; +import { EntityRegistry } from "../database-entity.decorator"; +import { asArray } from "../../../utils/utils"; /** * Transform between entity instances and database objects @@ -236,4 +238,31 @@ export class EntitySchemaService { schemaField.dataType, ).transformToObjectFormat(value, schemaField, dataObject); } + + /** + * Get all entity types whose schema includes fields referencing the given type. + * + * e.g. given Child -> [Note, ChildSchoolRelation, ...] + * @param type + */ + getEntityTypesReferencingType(type: string): { + entityType: EntityConstructor; + referencingProperties: string[]; + }[] { + const referencingTypes = []; + for (const t of this.injector.get(EntityRegistry).values()) { + for (const [key, field] of t.schema.entries()) { + if (asArray(field.additional).includes(type)) { + let refType = referencingTypes.find((e) => e.entityType === t); + if (!refType) { + refType = { entityType: t, referencingProperties: [] }; + referencingTypes.push(refType); + } + + refType.referencingProperties.push(key); + } + } + } + return referencingTypes; + } } diff --git a/src/app/core/import/import.service.spec.ts b/src/app/core/import/import.service.spec.ts index 39efecfda9..34b196179a 100644 --- a/src/app/core/import/import.service.spec.ts +++ b/src/app/core/import/import.service.spec.ts @@ -9,7 +9,6 @@ import { expectEntitiesToBeInDatabase, expectEntitiesToMatch, } from "../../utils/expect-entity-data.spec"; -import { HealthCheck } from "../../child-dev-project/children/health-checkup/model/health-check"; import moment from "moment"; import { Child } from "../../child-dev-project/children/model/child"; import { RecurringActivity } from "../../child-dev-project/attendance/model/recurring-activity"; @@ -17,6 +16,8 @@ import { ChildSchoolRelation } from "../../child-dev-project/children/model/chil import { EntityArrayDatatype } from "../basic-datatypes/entity-array/entity-array.datatype"; import { mockEntityMapper } from "../entity/entity-mapper/mock-entity-mapper-service"; import { CoreTestingModule } from "../../utils/core-testing.module"; +import { EntityRegistry } from "../entity/database-entity.decorator"; +import { DatabaseField } from "../entity/database-field.decorator"; describe("ImportService", () => { let service: ImportService; @@ -55,49 +56,61 @@ describe("ImportService", () => { }); it("should transform raw data to mapped entities", async () => { - HealthCheck.schema.set("entityArray", { - dataType: EntityArrayDatatype.dataType, - additional: "Child", - }); + class ImportTestTarget extends Entity { + @DatabaseField() name: string; + @DatabaseField() counter: number; + @DatabaseField() date: Date; + @DatabaseField({ + dataType: EntityArrayDatatype.dataType, + additional: "Child", + }) + entityRefs: string[]; + } + spyOn(TestBed.inject(EntityRegistry), "get").and.callFake( + (entityType: string) => + entityType === "ImportTestTarget" ? ImportTestTarget : Child, + ); + const child = Child.create("Child Name"); await TestBed.inject(EntityMapperService).save(child); + const rawData: any[] = [ - { x: "John", y: "111" }, - { x: "Jane" }, - { x: "broken date", y: "foo" }, // date column; ("y") ignored - { x: "with broken mapping column", brokenMapping: "foo" }, // column mapped to non-existing property ignored - { x: "", onlyUnmappedColumn: "1" }, // only empty or unmapped columns => row skipped - { x: "with zero", y: "0" }, // 0 value mapped - { x: "custom mapping fn", z: "30.01.2023" }, - { x: "entity array", childName: child.name }, + { rawName: "John", rawCounter: "111" }, + { rawName: "Jane" }, + { rawName: "broken date", rawCounter: "foo" }, // number column; ("rawCounter") ignored + { rawName: "with broken mapping column", brokenMapping: "foo" }, // column mapped to non-existing property ignored + { rawName: "", onlyUnmappedColumn: "1" }, // only empty or unmapped columns => row skipped + { rawName: "with zero", rawCounter: "0" }, // 0 value mapped + { rawName: "custom mapping fn", rawDate: "30.01.2023" }, + { rawName: "entity array", rawRefName: child.name }, ]; const columnMapping: ColumnMapping[] = [ - { column: "x", propertyName: "child" }, - { column: "y", propertyName: "height" }, - { column: "z", propertyName: "date", additional: "DD.MM.YYYY" }, + { column: "rawName", propertyName: "name" }, + { column: "rawCounter", propertyName: "counter" }, + { column: "rawDate", propertyName: "date", additional: "DD.MM.YYYY" }, { column: "brokenMapping", propertyName: "brokenMapping" }, - { column: "childName", propertyName: "entityArray", additional: "name" }, + { column: "rawRefName", propertyName: "entityRefs", additional: "name" }, ]; const parsedEntities = await service.transformRawDataToEntities( rawData, - HealthCheck.ENTITY_TYPE, + "ImportTestTarget", columnMapping, ); let expectedEntities: any[] = [ - { child: "John", height: 111 }, - { child: "Jane" }, - { child: "broken date" }, - { child: "with broken mapping column" }, - { child: "with zero", height: 0 }, - { child: "custom mapping fn", date: moment("2023-01-30").toDate() }, - { child: "entity array", entityArray: [child.getId()] }, + { name: "John", counter: 111 }, + { name: "Jane" }, + { name: "broken date" }, + { name: "with broken mapping column" }, + { name: "with zero", counter: 0 }, + { name: "custom mapping fn", date: moment("2023-01-30").toDate() }, + { name: "entity array", entityRefs: [child.getId()] }, ]; expectEntitiesToMatch( parsedEntities, - expectedEntities.map((e) => Object.assign(new HealthCheck(), e)), + expectedEntities.map((e) => Object.assign(new ImportTestTarget(), e)), true, ); }); diff --git a/src/app/core/user/user.ts b/src/app/core/user/user.ts index 3fba9038d6..4c390db113 100644 --- a/src/app/core/user/user.ts +++ b/src/app/core/user/user.ts @@ -32,6 +32,7 @@ export class User extends Entity { static icon: IconName = "user"; static label = $localize`:label for entity:User`; static labelPlural = $localize`:label (plural) for entity:Users`; + static override hasPII = true; /** username used for login and identification */ @DatabaseField({ diff --git a/src/app/features/historical-data/model/historical-entity-data.ts b/src/app/features/historical-data/model/historical-entity-data.ts index e02667f7b9..6ed0d39c83 100644 --- a/src/app/features/historical-data/model/historical-entity-data.ts +++ b/src/app/features/historical-data/model/historical-entity-data.ts @@ -2,6 +2,7 @@ import { Entity } from "../../../core/entity/model/entity"; import { DatabaseEntity } from "../../../core/entity/database-entity.decorator"; import { DatabaseField } from "../../../core/entity/database-field.decorator"; import { PLACEHOLDERS } from "../../../core/entity/schema/entity-schema-field"; +import { Child } from "../../../child-dev-project/children/model/child"; /** * A general class that represents data that is collected for a entity over time. @@ -9,6 +10,8 @@ import { PLACEHOLDERS } from "../../../core/entity/schema/entity-schema-field"; */ @DatabaseEntity("HistoricalEntityData") export class HistoricalEntityData extends Entity { + static override hasPII = true; + @DatabaseField({ label: $localize`:Label for date of historical data:Date`, defaultValue: PLACEHOLDERS.NOW, @@ -17,6 +20,9 @@ export class HistoricalEntityData extends Entity { date: Date; @DatabaseField({ + dataType: "entity", + additional: Child.ENTITY_TYPE, + entityReferenceRole: "composite", anonymize: "retain", }) relatedEntity: string; diff --git a/src/app/features/todos/model/todo.ts b/src/app/features/todos/model/todo.ts index e53d8afb72..f99b13d458 100644 --- a/src/app/features/todos/model/todo.ts +++ b/src/app/features/todos/model/todo.ts @@ -32,6 +32,7 @@ export class Todo extends Entity { static label = $localize`:label for entity:Task`; static labelPlural = $localize`:label (plural) for entity:Tasks`; static toStringAttributes = ["subject"]; + static override hasPII = true; static create(properties: Partial): Todo { const instance = new Todo(); @@ -89,6 +90,7 @@ export class Todo extends Entity { School.ENTITY_TYPE, RecurringActivity.ENTITY_TYPE, ], + entityReferenceRole: "composite", showInDetailsView: true, anonymize: "retain", }) diff --git a/src/app/features/todos/todos-related-to-entity/todos-related-to-entity.component.html b/src/app/features/todos/todos-related-to-entity/todos-related-to-entity.component.html index d170afed77..7f11ce30ab 100644 --- a/src/app/features/todos/todos-related-to-entity/todos-related-to-entity.component.html +++ b/src/app/features/todos/todos-related-to-entity/todos-related-to-entity.component.html @@ -8,13 +8,3 @@ (rowClick)="showDetails($event)" [getBackgroundColor]="backgroundColorFn" > - -
- - Also show completed - -
diff --git a/src/app/features/todos/todos-related-to-entity/todos-related-to-entity.component.ts b/src/app/features/todos/todos-related-to-entity/todos-related-to-entity.component.ts index c20712ee65..b8f69d9846 100644 --- a/src/app/features/todos/todos-related-to-entity/todos-related-to-entity.component.ts +++ b/src/app/features/todos/todos-related-to-entity/todos-related-to-entity.component.ts @@ -66,7 +66,6 @@ export class TodosRelatedToEntityComponent implements OnInit { async ngOnInit() { this.entries = await this.loadDataFor(this.entity.getId(true)); - this.toggleInactive(); } private async loadDataFor(entityId: string): Promise { @@ -97,13 +96,4 @@ export class TodosRelatedToEntityComponent implements OnInit { showDetails(entity: Todo) { this.formDialog.openFormPopup(entity, this.columns, TodoDetailsComponent); } - - toggleInactive() { - // TODO: move the toggle into its own component to be used like a filter? this is almost copy & paste from ChildSchoolOverview - if (this.includeInactive) { - this.filter = {}; - } else { - this.filter = { isActive: true }; - } - } } diff --git a/src/app/utils/core-testing.module.ts b/src/app/utils/core-testing.module.ts index 6e0e1e326c..647ddaaa4f 100644 --- a/src/app/utils/core-testing.module.ts +++ b/src/app/utils/core-testing.module.ts @@ -7,8 +7,8 @@ import { import { EntityMapperService } from "../core/entity/entity-mapper/entity-mapper.service"; import { mockEntityMapper } from "../core/entity/entity-mapper/mock-entity-mapper-service"; import { ConfigurableEnumService } from "../core/basic-datatypes/configurable-enum/configurable-enum.service"; -import { EntityRemoveService } from "../core/entity/entity-remove.service"; import { ComponentRegistry } from "../dynamic-components"; +import { EntityActionsService } from "../core/entity/entity-actions/entity-actions.service"; /** * A basic module that can be imported in unit tests to provide default datatypes. @@ -24,10 +24,9 @@ import { ComponentRegistry } from "../dynamic-components"; useValue: new ConfigurableEnumService(mockEntityMapper(), null), }, { - provide: EntityRemoveService, + provide: EntityActionsService, useValue: jasmine.createSpyObj(["anonymize"]), }, - { provide: EntityRemoveService, useValue: null }, ComponentRegistry, ], }) diff --git a/src/app/utils/expect-entity-data.spec.ts b/src/app/utils/expect-entity-data.spec.ts index 515628a172..0faed45e0e 100644 --- a/src/app/utils/expect-entity-data.spec.ts +++ b/src/app/utils/expect-entity-data.spec.ts @@ -152,7 +152,10 @@ function printArrayDifferences(name: string, a1: Array, a2: Array) { * @param obj The object or array of objects to simplify * @param withoutId (Optional) set to true to remove _id as well */ -function comparableEntityData(obj: any | any[], withoutId: boolean = false) { +export function comparableEntityData( + obj: any | any[], + withoutId: boolean = false, +) { if (Array.isArray(obj)) { return obj.map((o) => comparableEntityData(o, withoutId)); } else {