You can also include a gif with a full demo of the example
*ADD A TABLE OF CONTENTS HERE *
- Overview
- 🐍 Build the app with Python
- 🔮 Overview of the files in this example
- 🌀 Flow diagram
- 🔨 Next steps, building your own app
- 🐳 Deploy the prebuild application using Docker
- 🙍 Community
- 🦄 License
About this example: | |
---|---|
Learnings | Describe what the user will learn after running this example |
Used for indexing | What is the datatype of the indexing input |
Used for querying | What is the data type of the query input |
Dataset used | Link to the datasets |
Model used | Link to the model |
These instructions explain how to build the example yourself and deploy it with Python. If you want to skip the building steps and just run the example with Docker, check the Docker deployment instructions at the end of this README
Here outline in bullet points anything the user is expected to have before diving in.
For example:
- You have a working Python 3.8 environment.
- We recommend creating a new Python virtual environment to have a clean installation of Jina and prevent dependency conflicts.
- You have at least 2GB of free space on your hard drive.
Begin by cloning the repo, so you can get the required files and datasets. (If you already have the examples repository on your machine make sure to fetch the most recent version)
git clone https://github.com/jina-ai/examples
And enter the correct folder:
cd examples/example_to_use (replace as necessary)
In your terminal, you should now be located in you the enter example name folder. Let's install Jina and the other required Python libraries. For further information on installing Jina check out our documentation.
pip install -r requirements.txt
There are two different options here. You can either use the toy data we provide in this repo, which is quick to index but will give very poor results. Alternatively, you can download a larger dataset, which takes longer to index, but will have better results.
-
Toy dataset: Skip to step 3. No action is needed here.
-
Full dataset: In order to get the full dataset, follow the instructions below:
- Register for a free Kaggle account
- Set up your API token (see authentication section of their API docs)
- Run
pip install kaggle
- Run
sh get_data.sh
In this step, we will index our data.
Here describe the Index Flow. Be as specific as possible in describing how this Index Flow works and what is its input. You are encouraged to use code snippets, images, or whatever helps to clarify.
python app.py -t index (replace as necessary)
If you see the following output, it means your data has been correctly indexed.
Flow@5162[S]:flow is closed and all resources are released, current build level is 0
Next, we will deploy our query Flow.
Here describe the Query Flow. Be as specific as possible in describing how this Query Flow works and what is its input. You are encouraged to use code snippets, images, or whatever helps to clarify.
Run the query Flow in your terminal like this:
python app.py -t query (replace as necessary)
Here include a short description of the results and how to interpret them if needed.
This diagram provides a visual representation of the Flows in this example; Showing which executors are used in which order.
Here Show the Flow for this example.
Use this section to add extra information you think the user could benefit from. QueryLanguage, Faiss, Annoy for example.
Add a list with all folders/files in the example:
📂 flows/ |
Folder to store Flow configuration |
--- 📃 index.yml |
YAML file to configure indexing Flow |
--- 📃 query.yml |
YAML file to configure querying Flow |
📂 pods/ |
Folder to store Pod configuration |
--- 📃 encoder.yml |
YAML file to configure encoder Pod |
📂 workspace/ |
Folder to store indexed files (embeddings and documents). Automatically created after the first indexing |
To make it easier for you, we have built and published the Docker image for this example.
- You have Docker installed and working.
- You have at least 8GB of free space on your hard drive.
Running the following command will pull the Docker image and run it.
Replace below with the command to run the Docker image of this example
docker .
Did you like this example and are you interested in building your own? For a detailed tutorial on how to build your Jina app check out How to Build Your First Jina App guide in our documentation.
If you have any issues following this guide, you can always get support from our Slack community .
- Slack channel - a communication platform for developers to discuss Jina.
- LinkedIn - get to know Jina AI as a company and find job opportunities.
- - follow us and interact with us using hashtag
#JinaSearch
. - Company - know more about our company, we are fully committed to open-source!
Copyright (c) 2021 Jina AI Limited. All rights reserved.
Jina is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.