Handle deleted entities / not_found errors

[TheSlimvReal]

issue summary

As we did not implement #220 from the beginning, we do have a number of invalid references and outdated docs in the database. These cause errors in our logs and sometimes might even show up in lists.
We should implement a way to clean this up and handle such exceptions more smoothly, without triggering errors in our monitoring.

Issues are, for example:

• if a Child was deleted, ChildSchoolRelation entities of it are not deleted and still floating around the db
• if a Child was deleted, its id is still referenced in some Note or RecurringActivity entities, which show up as errors in the logs

Possible approach:

• write an API endpoint (to be triggered manually by admin) to detect dead references and remove such ids from affected entities

---

original comment

When deleting an entity (e.g. a child) the ID of this child might be used in other entities (e.g. in ChildSchoolRelations or notes). This eventually will throw error messages when the according school or note is viewed.

In general, deleting should only happen, when a child is accidentally created or needs to be removed from the system completely. In this cases, also all the foreign keys should be removed as well (all the mentions of the ID in other entities).

In other cases, where the child does only not participate in the system anymore, the dropout system should be used in order to keep the information available in the system. This way, nothing will be deleted.

What are your thoughts about this topic? Currently we have a lot of error messages regarding deleted children which are very difficult to solve.

[sleidig]

Another idea we have discussed right now to avoid the "not_found" errors when an entity was deleted without cascading delete:
catch these cases in the app and display a "record deleted" in the UI without throwing the error in console and sentry.

[sleidig]

I am refocusing this issue on the topic of handling errors of existing broken references. Fixing the issue of generating these when deleting an entity is the topic of #220

[sleidig]

With the cascading delete functionality in ndb-core, detecting dead references could reuse the same code. Only challenge (if we want to do this in the backend as a migration): This code is not so easily ported to the backend because it relies on the config schema system.

[TheSlimvReal]

Should we close this issue as we now delete entities without leaving dead references? I think it is not too much effort to remove existing broken references once they pop up in sentry manually. Haven't seen them in a while.

[sleidig]

Fair enough. As this is not trivial to implement, let's close it here. Please flag cases from Sentry in our internal chat and let us clean these up on a mission towards zero Sentry errors!