Load schemas dynamically

[udgover]

This PR adds the concept of private schemas. Observables, entities and indicators are dynamically registered. Loading is implemented in core.schemas.__init__.py. It also automatically extends each type enums.

• Implement dynamic loader for observables
• Replace observables validators as field validators
• Implement dynamic loader for entities
• Implement dynamic loader for indicators

Types definition are based on filename.

[udgover]

generic_observable.py has been renamed generic.py and GenericObservable class has been renamed Generic. SsdeepHash class has been renamed Ssdeep for clarity as well.

core unit tests are currently broken because of add_text tests that relies on validators.

[tomchop]

This looks great! Happy to have this also for indicators and entities in this PR too. The field validators should go in another one IMHO.

Also, wdyt about merging all observables into 1 file (like indicators and entities)? having one line per file is weird... (also in another PR to keep changes clean)

[udgover]

This looks great! Happy to have this also for indicators and entities in this PR too.

Indicators and entities will be part of this PR as well.

The field validators should go in another one IMHO.

The problem with field validator is that's not possible to keep the current implementation as it's breaking add_text method. Since we are building the ObservableType enum dynamically from __init__.py which imports observable.py the TYPE_VALIDATOR_MAP cannot be created. I also think it makes sense to include validators as part of each object to validate field.

Also, wdyt about merging all observables into 1 file (like indicators and entities)? having one line per file is weird... (also in another PR to keep changes clean)

I do not have a strong opinion on this. However, I find it faster to know what is already defined just by browsing the folder rather than reading the code.

[tomchop]

Re: validation - Gotcha. Then let's bring the validation changes here

Let's keep the one-file-per-obs type in that case. We should still (at some point) also do the same for entities / indicators, just to be consistent.