suggestion: use a vector-db like pinecone to overcome the token limits? #1
Comments
|
The good news is that I haven't really found the context window to be a problem with GPT-4 and the diff based approach used by aider. But ya, I am familiar with that technique. I've used it with other LLM projects, and have been thinking about how it might work for aider. It's not obvious that it is the best approach, but I will certainly experiment with it more. I have previously done a fair bit of work experimenting with various approaches to work around the smaller gpt-3.5 context window. I shared some notes on HN at the time: |
|
FYI,I just released some improvements to how aider manages the context window https://github.com/paul-gauthier/aider/blob/main/docs/ctags.md |
|
Wow! That's an impressive solution and very "compressed". It amazes me, that ctags is totally sufficient. Great idea + great, that it works.! Contratulations! I like your blog-post about it, too! |
ghost
mentioned this issue
Closed
paul-gauthier
added a commit
that referenced
this issue
May 15, 2024
# Aider chat conversation: USER: will this pickup the github pages jekyll site that is in this repy? Like the _layouts _data _posts etc dirs? ASSISTANT: Yes, the current Dockerfile and scripts should pick up the GitHub Pages Jekyll site, including directories like `_layouts`, `_data`, `_posts`, etc., because the `COPY . /srv/jekyll` command in the Dockerfile copies the entire current directory (which should include those directories) into the Docker container. However, to ensure everything works correctly, let's make sure the Dockerfile and scripts are set up to handle the Jekyll site properly. Here are the steps: 1. Ensure the Dockerfile is correctly set up to copy the entire site and install dependencies. 2. Ensure the build and run scripts are correctly set up to build the Docker image and run the container. Let's review the current state of the files and make any necessary adjustments. scripts/Dockerfile ```python # Use the official Jekyll image from Docker Hub FROM jekyll/jekyll:latest # Set the working directory WORKDIR /srv/jekyll # Copy the current directory contents into the container at /srv/jekyll COPY . /srv/jekyll # Install any needed packages specified in Gemfile RUN bundle install # Expose port 4000 to the host EXPOSE 4000 # Run Jekyll server CMD ["jekyll", "serve", "--host", "0.0.0.0"] ``` scripts/build.sh ```python #!/bin/bash # Build the Docker image docker build -t my-jekyll-site -f scripts/Dockerfile . ``` scripts/run.sh ```python #!/bin/bash # Run the Docker container docker run --rm -v "$PWD:/srv/jekyll" -p 4000:4000 my-jekyll-site ``` These files should work as expected. The `COPY . /srv/jekyll` command in the Dockerfile will copy all the necessary Jekyll site files, including `_layouts`, `_data`, `_posts`, etc., into the Docker container. The `bundle install` command will install any dependencies specified in the `Gemfile`. To verify, you can run the following commands in your project directory: ```sh bash scripts/build.sh bash scripts/run.sh ``` This will build the Docker image and start a container that serves your Jekyll site on `http://localhost:4000`. If you encounter any issues, please let me know! USER: I ran this command: bash scripts/build.sh And got this output: #1 [internal] load build definition from Dockerfile #1 transferring dockerfile: 435B done #1 DONE 0.0s #2 [internal] load .dockerignore #2 transferring context: 101B 0.0s done #2 DONE 0.0s #3 [internal] load metadata for docker.io/jekyll/jekyll:latest #3 DONE 0.6s #4 [1/4] FROM docker.io/jekyll/jekyll:latest@sha256:400b8d1569f118bca8a3a09a25f32803b00a55d1ea241feaf5f904d66ca9c625 #4 DONE 0.0s #5 [internal] load build context #5 transferring context: 250.53kB 0.2s done #5 DONE 0.2s #6 [2/4] WORKDIR /srv/jekyll #6 CACHED #7 [3/4] COPY . /srv/jekyll #7 CACHED #8 [4/4] RUN bundle install #8 6.277 Could not locate Gemfile #8 ERROR: process "/bin/sh -c bundle install" did not complete successfully: exit code: 10 ------ > [4/4] RUN bundle install: #8 6.277 Could not locate Gemfile ------ Dockerfile:11 -------------------- 9 | 10 | # Install any needed packages specified in Gemfile 11 | >>> RUN bundle install 12 | 13 | # Expose port 4000 to the host -------------------- ERROR: failed to solve: process "/bin/sh -c bundle install" did not complete successfully: exit code: 10 ASSISTANT: The error indicates that the `Gemfile` is not being found in the context being copied into the Docker container. This could be because the `Gemfile` is not in the root directory of your project or it might be missing. To resolve this, we need to ensure that the `Gemfile` is present in the root directory of your project. If it is not, you should add it. If it is present, we need to make sure it is being copied correctly. Let's first verify that the `Gemfile` is in the root directory of your project. If it is not, please add it. If it is, we can proceed with the current Dockerfile and scripts. Assuming the `Gemfile` is present in the root directory, the Dockerfile should work as expected. If the `Gemfile` is in a different directory, please let me know its location so we can adjust the Dockerfile accordingly. If the `Gemfile` is missing, you can create a basic one with the following content: Gemfile ```ruby source 'https://rubygems.org' gem 'jekyll' ``` Once the `Gemfile` is in place, you can re-run the build script: ```sh bash scripts/build.sh ``` If you encounter any further issues, please let me know! USER: please make it ASSISTANT: Let's create the `Gemfile` in the root directory of your project. Gemfile ```ruby <<<<<<< SEARCH ======= source 'https://rubygems.org' gem 'jekyll' >>>>>>> REPLACE ``` This `Gemfile` specifies the source for Ruby gems and includes the `jekyll` gem, which is necessary for your Jekyll site. After adding this `Gemfile`, you can re-run the build script: ```sh bash scripts/build.sh ``` This should resolve the issue with the missing `Gemfile` and allow the Docker image to be built successfully. If you encounter any further issues, please let me know!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
No description provided.
The text was updated successfully, but these errors were encountered: