feat: USE_FULL_PROJECT_PROMPT + extracting code from ```

[goooseman]

Changes:

1. USE_FULL_PROJECT_PROMPT env var. If enabled for each file generation prompt we will include all the files generated before
2. Sometimes GPT-3.5 still returns some words around the content of the file. This change extracts code from ``` blocks.
3. OPENAI_DEFAULT_MODEL and OPENAI_DEFAULT_MAX_TOKENS env vars added

The change is tested with USE_FULL_PROJECT_PROMPT=True OPENAI_DEFAULT_MODEL=gpt-3.5-turbo-16k-0613 OPENAI_DEFAULT_MAX_TOKENS=9000 and USE_FULL_PROJECT_PROMPT makes a big change for small projects. My generated project is ~5000 tokens, so I'm safe to include full project files in the prompt.

[coditamar]

@CodiumAI-Agent please review

[CodiumAI-Agent]

PR Analysis

• 🎯 Main theme: Adding environment variables for configuration and improving code generation
• 🔍 Description and title: Yes
• 📌 Type of PR: Enhancement
• 🧪 Relevant tests added: No
• ✨ Minimal and focused: Yes, the PR is focused on improving the code generation process by adding environment variables for configuration and extracting code from blocks.
• 🔒 Security concerns: No, the PR does not introduce any obvious security concerns. The added environment variables are used for configuration and do not involve sensitive data.

PR Feedback

• 💡 General PR suggestions: The PR is generally well-structured and the changes are clear. However, it lacks tests to verify the new functionality. It would be beneficial to add tests that check the behavior when the new environment variables are set. Also, consider removing the print statement used for debugging.

• 🤖 Code suggestions:

  • relevant file: constants.py
    suggestion content: Consider using os.getenv instead of os.environ for getting environment variables. This will return None if the environment variable is not set, which can simplify the code. For example, USE_FULL_PROJECT_PROMPT = os.getenv('USE_FULL_PROJECT_PROMPT', 'False').lower() in ('true', '1') [important]

  • relevant file: constants.py
    suggestion content: Avoid using print statements for debugging in production code. If necessary, consider using a logger for this purpose. [medium]

  • relevant file: main.py
    suggestion content: Consider handling the case where the regex match fails in the get_code_from_string function. Currently, if the match fails, the entire input string is returned, which may not be the desired behavior. [important]

  • relevant file: main_no_modal.py
    suggestion content: Similar to the previous suggestion, consider handling the case where the regex match fails in the get_code_from_string function in this file as well. [important]

How to use

Tag me in a comment '@CodiumAI-Agent' to ask for a new review after you update the PR.
You can also tag me and ask any question, for example '@CodiumAI-Agent is the PR ready for merge?'