Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

auto fetch gs-wiki articles #42

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

auto fetch gs-wiki articles #42

wants to merge 1 commit into from

Conversation

rti
Copy link
Owner

@rti rti commented Feb 26, 2024

No description provided.

Base automatically changed from integration to main February 27, 2024 15:12
@rti rti force-pushed the integration-gswiki branch from 35ad743 to 57ec054 Compare February 27, 2024 16:07
@rti
Copy link
Owner Author

rti commented Feb 27, 2024

@Silvan-WMDE this is really great. Thanks for your work on this.

Thinking about how to integrate this... I am currently working on a feature over here #41 that allows us to have the data fetching and embedding process decoupled from the application start up process as well as stores documents and embeddings in a postgres db (with vecto.rs extension). So we get rid of the json file caches.

I would really like to integrate your feature on top of the branch in #41. Is this ok for you?

The workflow would then be:

This way we also do not need to put article fetch lists into container builds.

What to you think?

@rti
Copy link
Owner Author

rti commented Feb 28, 2024

I the changes from this branch on top of #41

The fetching process is now decoupled from container start and is a manual process.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants