Merge pull request #157 from 20001LastOrder/refactoring

Add instructions to run the refactored project

docs/How_To/Slack Bot/1_slackbot_workspace.rst

@@ -38,66 +38,8 @@ Next, we will create a new Slack app and add it to the Slack Workspace we just c
 
 Run the Slackbot Locally
 ************************
-Next, we will run the slack app project locally. The slackapp project is part of this livebook's repository. If you haven't done so, clone the repository for this livebook at https://github.com/Aggregate-Intellect/sherpa. 
+Next, we will run the slack app project locally. The slackapp project is part of this livebook's repository. If you haven't done so, follow this guide: :doc:`Run and Develop Sherpa Slackbot <2_run_slackbot>` to set up the livebook repository.
 
-After you clone the repository, you can find the slackbot project under `apps/slackbot`. The `README` of the slackbot contains instruction to run the app with docker or a virtual environment, we will repeat the instruction with a local virual environment.
-
-1. Install `conda <https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html?>`__ or `miniconda <https://docs.conda.io/en/latest/miniconda.html>`__ if you don't have it already.
-2. Create a new conda environment with python 3.9. 
-    :: 
-
-        conda create -n slackbot python=3.9
-
-3. Activate the environment
-    :: 
-
-        conda activate slackbot
-
-4. Install the dependencies
-    :: 
-
-        pip install -r requirements.txt
-
-5. Configuration: Copy the contents of ``slackbot/.env-sample``
-   into your own ``slackbot/.env`` file.
-    :: 
-
-        cp .env-sample .env
-   * Then, modify your configuration as needed. You need to add the environment variables in ``.env`` following the format of ``.env_sample``. The following parts describe how to access the configuration values that you will add to ``.env``:
-
-
-  * You can find ``SLACK_SIGNING_SECRET`` and ``SLACK_VERIFICATION_TOKEN`` values in the `Basic Information` page of your Slack App.
-
-    .. image:: slackbot_imgs/slackbot.png
-        :width: 400
-  * The ``SLACK_OAUTH_TOKEN`` can be found in the `OAuth & Permissions` page of your Slack App.
-
-    .. image:: slackbot_imgs/slackbot2.png
-        :width: 400
-  * The ``OPENAI_API_KEY`` can be found in your OpenAI account page. If you you don't have one, create one at https://platform.openai.com/overview.
-
-  * Serper is a search engine that we use to search for relevant articles. You can find the API key 
-    in your Serper account page. If you don't have one, create one at https://serper.dev/. 
-    You don't need it to run the app, but it will enable the Internet search function of the SlackBot. If you don't want to have this functionality, you can leave the `SERPER_API_KEY` empty.
-
-  * You may also find there are optional *Pinecone* keys in the `.env` file. 
-    Pinecone is a cloud-based vector database, by default, the app runs with an in-memory vector 
-    database. However, if you want to build your own cloud vector database, you can learn more 
-    about Pinecone here: https://www.pinecone.io/ and on the :doc:`Pinecone setup <3_pinecone_setup>` page.
-
-6. We will need to have some files to load into the vector database and act as the long-term memory of the SlackBot. For the sake of this tutorial, copy the `README.md` file of this project to `apps/slackbot/file`. This file will be used as the long-term memory of the SlackBot.
-7. Now, run the app with the following command
-
-  ::
-
-      python bolt_app.py
-
-8. You should see something like below if everything is working properly. 
-
-      .. image:: slackbot_imgs/slackbot3.png
-        :width: 400
-
-Great! We are almost there to let you communicate with your SlackBot.
 
 Connect Slack Bot to the App
 ****************************

docs/How_To/Slack Bot/2_run_slackbot.rst

@@ -29,68 +29,88 @@ Preparation
 2. Create new app in slack workspace following the :doc:`setup
    tutorial <1_slackbot_workspace>`
 
-3. Configuration: Copy the contents of ``slackbot/.env-sample``
-   into your own ``slackbot/.env`` file, and then modify your configuration as needed.
+3. Configuration: Copy the contents of ``src/.env-sample``
+   into your own ``src/.env`` file, and then modify your configuration as needed.
    Remember not to share your private keys with anyone else.
 
-4. Add all the files which you want to build the Vector Db index to
-   the ``files`` folder. Currently, it works with ``PDFs`` and
-   ``Markdown`` files. (Ignore this step if you connect with your
-   Pinecone database)
+.. 4. Add all the files which you want to build the Vector Db index to
+..    the ``files`` folder. Currently, it works with ``PDFs`` and
+..    ``Markdown`` files. (Ignore this step if you connect with your
+..    Pinecone database)
 
 Usage
 -----
 
+Run with docker
+~~~~~~~~~~~~~~~
+
+1. Run the docker-compose the setup the sherpa-ai package and the local vector database prefilled with the production data.
+
+   .. code:: bash
+
+      cd src/
+      docker-compose up -d
+
+2. Expose the server to the internet using a tool like ngrok. This is not
+   required if your server is hosted on a public IP address.
+
+3. Set up the Slack app’s Event Subscriptions and provide the ngrok URL
+   as the Request URL.
+
+   -  **NOTE:** When adding the url to the Slack app, make sure to append
+      ``/slack/events`` at the end as this is the default path used by
+      Slack Bolt.
+
 Run with Virtual Environment
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-1. Install
+1. If you want to use the local vector database, you need to build the vector database with Docker. See :doc:`Setup Local VectorDB with Production Data <4_local_vectordb>` for how to do this.
+
+2. Install
    `conda <https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html>`__
    or `miniconda <https://docs.conda.io/en/latest/miniconda.html>`__
 
-2. Create a new conda environment:
+3. Create a new conda environment:
 
    .. code:: bash
 
       conda create -n slackbot python=3.9
 
-3. Activate the environment:
+4. Activate the environment:
 
    .. code:: bash
 
        conda activate slackbot
 
-4. Install the dependencies:
+5. Install the dependencies of the slackbote: 
+
+   * Install dependency with `Poetry <https://python-poetry.org/>`__ (recommended, this will give you the ability to edit the code in `sherpa_ai` without rebuild) 
 
-   .. code:: bash
+      .. code:: bash
 
-      pip install -r requirements.txt
+         cd app/slackapp
+         poetry install
+      
+   * Install dependency with `pip <https://pip.pypa.io/en/stable/>`__
 
-5. Run the server:
+      .. code:: bash
 
-   .. code:: bash
+         cd app/slackapp
+         pip install -e .
 
-       python bolt_app.py
+6. Run the server:
 
-Run with docker
-~~~~~~~~~~~~~~~
+   * Run with `Poetry <https://python-poetry.org/>`__
 
-1. Run the docker image:
+      .. code:: bash
 
-   .. code:: bash
+         poetry run sherpa_slack
 
-      docker build -it slackbot .
-      docker run -p 3000:3000 slackbot
+   * Run the script directly
 
-2. Expose the server to the internet using a tool like ngrok. This is not
-   required if your server is hosted on a public IP address.
+      .. code:: bash
 
-3. Set up the Slack app’s Event Subscriptions and provide the ngrok URL
-   as the Request URL.
-
-   -  **NOTE:** When adding the url to the Slack app, make sure to append
-      ``/slack/events`` at the end as this is the default path used by
-      Slack Bolt.
+         python apps/slackapp/slackapp/bolt_app.py
 
 Development
 -----------
@@ -99,12 +119,7 @@ Linting and formating
 ~~~~~~~~~~~~~~~~~~~~~
 
 This project uses ``flake8`` for linting, ``black`` and ``isort`` for
-formatting, and ``pytest`` for testing. To install the dependencies,
-run:
-
-.. code:: bash
-
-   pip install -r dev-requirements.txt
+formatting, and ``pytest`` for testing.
 
 To format the project, run:
 

src/.env-sample

@@ -17,7 +17,9 @@ SLACK_SIGNING_SECRET=             # from `Basic Information` page of your Slack
 SLACK_VERIFICATION_TOKEN=         # from `Basic Information` page of your Slack App
 
 # Vector databases, choose from Pinecone or Chroma, if none is configure, then an
-# in-memory version of Chroma is used.
+# in-memory version of Chroma is used. You don't need to configure the 
+# vector database if you are using docker-compose, as it is configured there.
+
 # Pinecone. Optional. Enables cloud-based storage of vector embeddings.
 # PINECONE_API_KEY=                 # The Pinecone vector database API key 
 # PINECONE_NAMESPACE="ReadTheDocs"  # Pinecone namespace to use for documents 

src/.gitignore

@@ -1,4 +1,5 @@
 __pycache__
 **/.env
 **/files/*
-**/testing*
+**/testing*
+**/dist/