feat: add doccache #21
Conversation
cristianmtr
force-pushed
the
feat-doccahce
branch
3 times, most recently
from
8377d17 to
d4e9c1f
Compare
cristianmtr
force-pushed
the
feat-doccahce
branch
from
d4e9c1f to
95bc7ca
Compare
jinahub/indexers/cache/Dockerfile
Outdated
| # for testing the image | ||
| RUN pip install pytest && pytest -s -v tests | ||
|
|
||
| ENTRYPOINT ["jina", "pod", "--uses", "config.yml"] |
There was a problem hiding this comment.
I believe this should become ..."jina", "pea", ..., but it might anyhow change completely when JinaD 2.0 is finished. Opinions @deepankarm ?
There was a problem hiding this comment.
it should be jina pea I agree, but the sugar of using executor would make sense
There was a problem hiding this comment.
This needs to be decided in core & JinaD would adapt to it. With JinaD 2.0, we still support containerized pods & handle them by just modifying host and mapping ports wherever required.
jinahub/indexers/cache/Dockerfile
Outdated
| RUN pip install -r requirements.txt | ||
|
|
||
| # for testing the image | ||
| RUN pip install pytest && pytest -s -v tests |
There was a problem hiding this comment.
I think we want to establish the base from syntax.
jinahub/indexers/cache/__init__.py
Outdated
| ) | ||
|
|
||
| @requests(on='/index') | ||
| def index(self, docs: DocumentArray, **kwargs): |
There was a problem hiding this comment.
Name index_or_remove_from_request or something like that? Not sure anyhow.
jinahub/indexers/cache/__init__.py
Outdated
| def update(self, docs: DocumentArray, **kwargs): | ||
| """Update the documents in the cache with the new content | ||
|
|
||
| WARNING: If the Document doesn't exist in the cache |
There was a problem hiding this comment.
Feels like this sentence is not complete.
jinahub/indexers/cache/config.yml
Outdated
| @@ -0,0 +1,4 @@ | |||
| jtype: FileQueryIndexer | |||
jinahub/indexers/cache/manifest.yml
Outdated
| manifest_version: 1 | ||
| name: FileQueryIndexer | ||
| kind: indexer | ||
| description: A basic file based query indexer | ||
| author: Jina AI Dev-Team (dev-team@jina.ai) | ||
| url: https://jina.ai | ||
| vendor: Jina AI Limited | ||
| documentation: https://github.com/jina-ai/jina-hub | ||
| version: 0.0.1 | ||
| license: apache-2.0 | ||
| keywords: [database, FileQueryIndexer, indexer] | ||
| type: pod |
| @@ -0,0 +1,2 @@ | |||
| jina==2.0.0rc5.dev54 | |||
There was a problem hiding this comment.
I believe we are at RC6 already? or even beyond. not sure.
There was a problem hiding this comment.
removed, using master
|
@maximilianwerk recheck, please |
| @@ -12,4 +12,4 @@ RUN pip install -r requirements.txt | |||
| # for testing the image | |||
| RUN pip install pytest && pytest -s -v tests | |||
|
|
|||
| ENTRYPOINT ["jina", "pod", "--uses", "config.yml"] | |||
| ENTRYPOINT ["jina", "pea", "--uses", "config.yml"] | |||
There was a problem hiding this comment.
I think the FROM base is still missing, right @jakob1111996 ?
jinahub/indexers/cache/__init__.py
Outdated
| assert len(self.cache_handler.id_to_hash) == len( | ||
| self.cache_handler.hash_to_id | ||
| ), 'Cache state invalid. Length of mappings does not align. Try rebuilding the index' | ||
| def ids(self): |
There was a problem hiding this comment.
The naming seems to be confusing. Perhaps ids_count if size is no option
Part of jina-ai/serve#2479