Towards better inference: bits → nibbles #3808
Conversation
…coordination into set-default-risk-in-model
…coordination into set-default-risk-in-model
…uler from recreating already deleted oois trhough affirmations
…cheduler_from_reacreating_already_deleted_oois_through_affirmations' into feature/nibbles
This reverts commit 1b4aed6.
…rchitecture because at this point in time they always run, maximizing available information)
|
There was a problem hiding this comment.
Great work! I didn't check all the queries or dove too deep into some parts (such as run/infer and the whole going down the inference tree), but at least for the queries I am happy to see the integration tests.
There is for me one big caveat: the nibble runner, and in particular its stateful properties, are now in charge of managing enabled/disabled nibbles and selecting them after a query. This will not work in practice over application lifetime.
Some less urging comments are about the number of methods/endpoints and how flexible they should/could be.
Please have a look! I could definitely re-review this to go into more detail for other parts later on.
| self.method, | ||
| f"[{','.join(sorted([str(param) for param in self.parameters_references]))}]" | ||
| if self.parameters_references is not None | ||
| else "Null", |
There was a problem hiding this comment.
Perhaps "None" would be more consistent with other instances where fields in the natural key might be None
| self.__class__.__name__, | ||
| self.origin_type.value, | ||
| self.method, | ||
| f"[{','.join(sorted([str(param) for param in self.parameters_references]))}]" |
There was a problem hiding this comment.
Should this be the has of the parameters instead? Since there could be an arbitrary amount
| @@ -249,6 +262,30 @@ def deserialize(cls, data: dict[str, Any]) -> OOI: | |||
| stripped["user_id"] = user_id | |||
| return object_cls.model_validate(stripped) | |||
|
|
|||
| @classmethod | |||
| def objectify(cls, t: list | Any, obj: dict | list | set | Any) -> tuple | frozenset | Any: | |||
There was a problem hiding this comment.
Do you mean "hydrate"? This deserves a comment explaining what the method accepts and returns, as I doubt Any is the best we can do seeing this method only being called once? Also the parameter t and variables tt and o could be more explicit about what they might hold. For the main if/elif branches you could specify the scenario we are in.
|
|
||
| import structlog | ||
| from pydantic import BaseModel | ||
| from xxhash import xxh3_128_hexdigest as xxh3 # INFO: xxh3_64_hexdigest is faster but hash more collision probabilities |
| @@ -582,3 +582,73 @@ def migrate_origins( | |||
| session.add((OperationType.DELETE, origin.id, valid_time)) | |||
|
|
|||
| session.commit() # The save-delete order is important to avoid garbage collection of the results | |||
|
|
|||
|
|
|||
| @router.get("/nibbles/list", tags=["nibbles"]) | |||
There was a problem hiding this comment.
For consistency:
| @router.get("/nibbles/list", tags=["nibbles"]) | |
| @router.get("/nibbles", tags=["nibbles"]) |
| assert self.repository.objectify(int, set((3 * "9 ").split())) == frozenset([9]) | ||
| assert self.repository.objectify(int, {"1", "2", "5"}) == frozenset([1, 2, 5]) | ||
|
|
||
| assert self.repository.objectify(str, "potato") == "potato" |
There was a problem hiding this comment.
Given all these examples: should we rename to something like parse_recursively_as(data, parse_type: type)?
| "potato2": "king-edward", | ||
| } | ||
|
|
||
| assert self.repository.objectify([str, int], ["seven", "11"]) == tuple(["seven", 11]) |
There was a problem hiding this comment.
I think this gets a bit tricky. Some fields/query results might be strings that should remain strings even if they represent an integer? I.e. perhaps some of this parsing should happen outside the query funtion by users that know whether to expect a string of integer?
| @@ -13,6 +13,7 @@ class OriginType(Enum): | |||
| OBSERVATION = "observation" | |||
| INFERENCE = "inference" | |||
| AFFIRMATION = "affirmation" | |||
| NIBBLET = "nibblet" | |||
There was a problem hiding this comment.
After Nibble and Nibbler, perhaps we should reduce the introduced naming by making this:
| NIBBLET = "nibblet" | |
| NIBBLET = "nibble" |
And renaming the variables to nibble_origin
|
|
||
| STATIC_IP = ".".join((4 * "1 ").split()) | ||
|
|
||
|
|
There was a problem hiding this comment.
So these tests create objects in XTDB and then query them using nibble queries to assert the behavior? Then I'm very pleased to see this!
| if os.environ.get("CI") != "1": | ||
| pytest.skip("Needs XTDB multinode container.", allow_module_level=True) | ||
|
|
||
|
|
Changes
Bits → Nibbles
Issue link
N/A
Demo
T.B.D.
QA notes
Test all ported bits/new nibbles from front-end:
Code Checklist
.envchanges files if required and changed the.env-distaccordingly.Checklist for code reviewers:
Copy-paste the checklist from the docs/source/templates folder into your comment.
Checklist for QA:
Copy-paste the checklist from the docs/source/templates folder into your comment.