Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bitbucket pipeline #304

Merged
merged 18 commits into from
Sep 22, 2023
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
ed1816a
Merge branch 'main' of https://github.com/Codium-ai/pr-agent into fix…
sarbjitsinghgrewal Sep 8, 2023
ca8997b
Merge branch 'main' of https://github.com/Codium-ai/pr-agent into fix…
sarbjitsinghgrewal Sep 11, 2023
12bd9e8
add bitbucket pipeline
sarbjitsinghgrewal Sep 11, 2023
9bd5140
update access token
sarbjitsinghgrewal Sep 11, 2023
9bee305
add bitbucket access token
sarbjitsinghgrewal Sep 12, 2023
caee7cb
add bitbucket access token
sarbjitsinghgrewal Sep 12, 2023
67052aa
add bitbucket access token
sarbjitsinghgrewal Sep 12, 2023
4043dff
Merge branch 'main' of https://github.com/Codium-ai/pr-agent into fix…
sarbjitsinghgrewal Sep 14, 2023
978c56c
update pipeline
sarbjitsinghgrewal Sep 15, 2023
7b2c011
Merge pull request #3 from sarbjitsinghgrewal/fix_bitbucket_pipeline
sarbjitsinghgrewal Sep 15, 2023
e343ce8
bitbucket pipeline for adding reviews
sarbjitsinghgrewal Sep 15, 2023
43af4aa
remove token from config
sarbjitsinghgrewal Sep 15, 2023
a2ca43a
Merge branch 'main' of https://github.com/Codium-ai/pr-agent into fix…
sarbjitsinghgrewal Sep 18, 2023
f7abdc6
Merge branch 'main' of https://github.com/Codium-ai/pr-agent into fix…
sarbjitsinghgrewal Sep 20, 2023
5e608cc
Merge branch 'main' of https://github.com/Codium-ai/pr-agent into fix…
sarbjitsinghgrewal Sep 21, 2023
46ec3c0
implement suggestions in bitbucket pipeline
sarbjitsinghgrewal Sep 21, 2023
56cc804
Merge branch 'main' of https://github.com/Codium-ai/pr-agent into fix…
sarbjitsinghgrewal Sep 22, 2023
3e2e2d6
update install.md
sarbjitsinghgrewal Sep 22, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions Dockerfile.bitbucket_pipeline
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
FROM python:3.10 as base

WORKDIR /app
ADD pyproject.toml .
ADD requirements.txt .
RUN pip install . && rm pyproject.toml requirements.txt
ENV PYTHONPATH=/app
ADD pr_agent pr_agent
ADD bitbucket_pipeline/entrypoint.sh /
RUN chmod +x /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Consider using a multi-stage build to reduce the size of the final Docker image.

Suggested change
FROM python:3.10 as base
WORKDIR /app
ADD pyproject.toml .
ADD requirements.txt .
RUN pip install . && rm pyproject.toml requirements.txt
ENV PYTHONPATH=/app
ADD pr_agent pr_agent
ADD bitbucket_pipeline/entrypoint.sh /
RUN chmod +x /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
# First stage: build
FROM python:3.10 as builder
WORKDIR /app
ADD pyproject.toml .
ADD requirements.txt .
RUN pip install . && rm pyproject.toml requirements.txt
# Second stage: runtime
FROM python:3.10-slim as runtime
COPY --from=builder /app /app
ENV PYTHONPATH=/app
ADD pr_agent pr_agent
ADD bitbucket_pipeline/entrypoint.sh /
RUN chmod +x /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that won't work, no dependencies in "runtime" target. But I suggest moving this to the docker/Dockerfile as a new target, follow the other examples in this file @sarbjitsinghgrewal

58 changes: 58 additions & 0 deletions INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ There are several ways to use PR-Agent:
- [Method 6: Deploy as a Lambda Function](INSTALL.md#method-6---deploy-as-a-lambda-function)
- [Method 7: AWS CodeCommit](INSTALL.md#method-7---aws-codecommit-setup)
- [Method 8: Run a GitLab webhook server](INSTALL.md#method-8---run-a-gitlab-webhook-server)
- [Method 9: Run as a Bitbucket Pipeline](INSTALL.md#method-9-run-as-a-bitbucket-pipeline)
---

### Method 1: Use Docker image (no installation required)
Expand Down Expand Up @@ -370,4 +371,61 @@ In the "Trigger" section, check the ‘comments’ and ‘merge request events
6. Test your installation by opening a merge request or commenting or a merge request using one of CodiumAI's commands.



### Method 9: Run as a Bitbucket Pipeline


You can use our pre-build Bitbucket-Pipeline docker image to run as Bitbucket-Pipeline.

1. Add the following file in your repository bitbucket_pipelines.yml

pipelines:
pull-requests:
'**':
- step:
name: PR Agent Pipeline
caches:
- pip
image: python:3.8 # Use an appropriate Python image with pip
services:
- docker
script:
- git clone https://github.com/Codium-ai/pr-agent.git
- cd pr-agent
- pip install docker-compose
- docker-compose up --build

2. Add the following secret to your repository under Repository settings > Pipelines > Repository variables.
OPENAI_API_KEY: <your key>
BITBUCKET_BEARER_TOKEN: <your token>

3. To get BITBUCKET_BEARER_TOKEN follow these steps
So here is my step by step tutorial
i) Insert your workspace name instead of {workspace_name} and go to the following link in order to create an OAuth consumer.

https://bitbucket.org/{workspace_name}/workspace/settings/api

set callback URL to http://localhost:8976 (doesn't need to be a real server there)
select permissions: repository -> read

ii) use consumer's Key as a {client_id} and open the following URL in the browser

https://bitbucket.org/site/oauth2/authorize?client_id={client_id}&response_type=code

iii)
after you press "Grant access" in the browser it will redirect you to

http://localhost:8976?code=<CODE>

iv) use the code from the previous step and consumer's Key as a {client_id}, and consumer's Secret as {client_secret}

curl -X POST -u "{client_id}:{client_secret}" \
https://bitbucket.org/site/oauth2/access_token \
-d grant_type=authorization_code \
-d code={code} \


After completing this steps, you just to place this access token in the repository varibles.


=======
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,7 @@ There are several ways to use PR-Agent:
- [Method 6: Deploy as a Lambda Function](INSTALL.md#method-6---deploy-as-a-lambda-function)
- [Method 7: AWS CodeCommit](INSTALL.md#method-7---aws-codecommit-setup)
- [Method 8: Run a GitLab webhook server](INSTALL.md#method-8---run-a-gitlab-webhook-server)
- [Method 9: Run as a Bitbucket Pipeline](INSTALL.md#method-9-run-as-a-bitbucket-pipeline)

## How it works

Expand Down
2 changes: 2 additions & 0 deletions bitbucket_pipeline/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
python /app/pr_agent/servers/bitbucket_pipeline_runner.py
37 changes: 37 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
version: '3'
services:
myapp:
build:
context: .
dockerfile: Dockerfile.bitbucket_pipeline
ports:
- "8080:80"
environment:
- BITBUCKET_BRANCH=${BITBUCKET_BRANCH}
- BITBUCKET_BUILD_NUMBER=${BITBUCKET_BUILD_NUMBER}
- BITBUCKET_CLONE_DIR=${BITBUCKET_CLONE_DIR}
- BITBUCKET_COMMIT=${BITBUCKET_COMMIT}
- BITBUCKET_GIT_HTTP_ORIGIN=${BITBUCKET_GIT_HTTP_ORIGIN}
- BITBUCKET_GIT_SSH_ORIGIN=${BITBUCKET_GIT_SSH_ORIGIN}
- BITBUCKET_PIPELINE_UUID=${BITBUCKET_PIPELINE_UUID}
- BITBUCKET_PROJECT_KEY=${BITBUCKET_PROJECT_KEY}
- BITBUCKET_PROJECT_UUID=${BITBUCKET_PROJECT_UUID}
- BITBUCKET_PR_DESTINATION_BRANCH=${BITBUCKET_PR_DESTINATION_BRANCH}
- BITBUCKET_PR_DESTINATION_COMMIT=${BITBUCKET_PR_DESTINATION_COMMIT}
- BITBUCKET_PR_ID=${BITBUCKET_PR_ID}
- BITBUCKET_REPO_FULL_NAME=${BITBUCKET_REPO_FULL_NAME}
- BITBUCKET_REPO_IS_PRIVATE=${BITBUCKET_REPO_IS_PRIVATE}
- BITBUCKET_REPO_OWNER=${BITBUCKET_REPO_OWNER}
- BITBUCKET_REPO_OWNER_UUID=${BITBUCKET_REPO_OWNER_UUID}
- BITBUCKET_REPO_SLUG=${BITBUCKET_REPO_SLUG}
- BITBUCKET_REPO_UUID=${BITBUCKET_REPO_UUID}
- BITBUCKET_SSH_KEY_FILE=${BITBUCKET_SSH_KEY_FILE}
- BITBUCKET_STEP_RUN_NUMBER=${BITBUCKET_STEP_RUN_NUMBER}
- BITBUCKET_STEP_TRIGGERER_UUID=${BITBUCKET_STEP_TRIGGERER_UUID}
- BITBUCKET_STEP_UUID=${BITBUCKET_STEP_UUID}
- BITBUCKET_WORKSPACE=${BITBUCKET_WORKSPACE}
- CI=${CI}
- DOCKER_HOST=${DOCKER_HOST}
- PIPELINES_JWT_TOKEN=${PIPELINES_JWT_TOKEN}
- OPENAI_API_KEY=${OPENAI_API_KEY}
- BITBUCKET_BEARER_TOKEN=${BITBUCKET_BEARER_TOKEN}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Consider using version '3.8' or later for the Docker Compose file to take advantage of the latest features and options.

Suggested change
version: '3'
services:
myapp:
build:
context: .
dockerfile: Dockerfile.bitbucket_pipeline
ports:
- "8080:80"
environment:
- BITBUCKET_BRANCH=${BITBUCKET_BRANCH}
- BITBUCKET_BUILD_NUMBER=${BITBUCKET_BUILD_NUMBER}
- BITBUCKET_CLONE_DIR=${BITBUCKET_CLONE_DIR}
- BITBUCKET_COMMIT=${BITBUCKET_COMMIT}
- BITBUCKET_GIT_HTTP_ORIGIN=${BITBUCKET_GIT_HTTP_ORIGIN}
- BITBUCKET_GIT_SSH_ORIGIN=${BITBUCKET_GIT_SSH_ORIGIN}
- BITBUCKET_PIPELINE_UUID=${BITBUCKET_PIPELINE_UUID}
- BITBUCKET_PROJECT_KEY=${BITBUCKET_PROJECT_KEY}
- BITBUCKET_PROJECT_UUID=${BITBUCKET_PROJECT_UUID}
- BITBUCKET_PR_DESTINATION_BRANCH=${BITBUCKET_PR_DESTINATION_BRANCH}
- BITBUCKET_PR_DESTINATION_COMMIT=${BITBUCKET_PR_DESTINATION_COMMIT}
- BITBUCKET_PR_ID=${BITBUCKET_PR_ID}
- BITBUCKET_REPO_FULL_NAME=${BITBUCKET_REPO_FULL_NAME}
- BITBUCKET_REPO_IS_PRIVATE=${BITBUCKET_REPO_IS_PRIVATE}
- BITBUCKET_REPO_OWNER=${BITBUCKET_REPO_OWNER}
- BITBUCKET_REPO_OWNER_UUID=${BITBUCKET_REPO_OWNER_UUID}
- BITBUCKET_REPO_SLUG=${BITBUCKET_REPO_SLUG}
- BITBUCKET_REPO_UUID=${BITBUCKET_REPO_UUID}
- BITBUCKET_SSH_KEY_FILE=${BITBUCKET_SSH_KEY_FILE}
- BITBUCKET_STEP_RUN_NUMBER=${BITBUCKET_STEP_RUN_NUMBER}
- BITBUCKET_STEP_TRIGGERER_UUID=${BITBUCKET_STEP_TRIGGERER_UUID}
- BITBUCKET_STEP_UUID=${BITBUCKET_STEP_UUID}
- BITBUCKET_WORKSPACE=${BITBUCKET_WORKSPACE}
- CI=${CI}
- DOCKER_HOST=${DOCKER_HOST}
- PIPELINES_JWT_TOKEN=${PIPELINES_JWT_TOKEN}
- OPENAI_API_KEY=${OPENAI_API_KEY}
- BITBUCKET_BEARER_TOKEN=${BITBUCKET_BEARER_TOKEN}
version: '3.8'
services:
myapp:
build:
context: .
dockerfile: Dockerfile.bitbucket_pipeline
ports:
- "8080:80"
environment:
- BITBUCKET_BRANCH=${BITBUCKET_BRANCH}
- BITBUCKET_BUILD_NUMBER=${BITBUCKET_BUILD_NUMBER}
- BITBUCKET_CLONE_DIR=${BITBUCKET_CLONE_DIR}
- BITBUCKET_COMMIT=${BITBUCKET_COMMIT}
- BITBUCKET_GIT_HTTP_ORIGIN=${BITBUCKET_GIT_HTTP_ORIGIN}
- BITBUCKET_GIT_SSH_ORIGIN=${BITBUCKET_GIT_SSH_ORIGIN}
- BITBUCKET_PIPELINE_UUID=${BITBUCKET_PIPELINE_UUID}
- BITBUCKET_PROJECT_KEY=${BITBUCKET_PROJECT_KEY}
- BITBUCKET_PROJECT_UUID=${BITBUCKET_PROJECT_UUID}
- BITBUCKET_PR_DESTINATION_BRANCH=${BITBUCKET_PR_DESTINATION_BRANCH}
- BITBUCKET_PR_DESTINATION_COMMIT=${BITBUCKET_PR_DESTINATION_COMMIT}
- BITBUCKET_PR_ID=${BITBUCKET_PR_ID}
- BITBUCKET_REPO_FULL_NAME=${BITBUCKET_REPO_FULL_NAME}
- BITBUCKET_REPO_IS_PRIVATE=${BITBUCKET_REPO_IS_PRIVATE}
- BITBUCKET_REPO_OWNER=${BITBUCKET_REPO_OWNER}
- BITBUCKET_REPO_OWNER_UUID=${BITBUCKET_REPO_OWNER_UUID}
- BITBUCKET_REPO_SLUG=${BITBUCKET_REPO_SLUG}
- BITBUCKET_REPO_UUID=${BITBUCKET_REPO_UUID}
- BITBUCKET_SSH_KEY_FILE=${BITBUCKET_SSH_KEY_FILE}
- BITBUCKET_STEP_RUN_NUMBER=${BITBUCKET_STEP_RUN_NUMBER}
- BITBUCKET_STEP_TRIGGERER_UUID=${BITBUCKET_STEP_TRIGGERER_UUID}
- BITBUCKET_STEP_UUID=${BITBUCKET_STEP_UUID}
- BITBUCKET_WORKSPACE=${BITBUCKET_WORKSPACE}
- CI=${CI}
- DOCKER_HOST=${DOCKER_HOST}
- PIPELINES_JWT_TOKEN=${PIPELINES_JWT_TOKEN}
- OPENAI_API_KEY=${OPENAI_API_KEY}
- BITBUCKET_BEARER_TOKEN=${BITBUCKET_BEARER_TOKEN}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the purpose of docker compose here? Maybe you added it to the repo by mistake? It's not documented anywhere and I think it's not necessary. It's the responsibility of the user to host the docker image properly. We might need to add better documentation about it in the future

36 changes: 36 additions & 0 deletions pr_agent/servers/bitbucket_pipeline_runner.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
import os
from pr_agent.agent.pr_agent import PRAgent
from pr_agent.config_loader import get_settings
from pr_agent.tools.pr_reviewer import PRReviewer
import asyncio

async def run_action():
pull_request_id = os.environ.get("BITBUCKET_PR_ID", '')
slug = os.environ.get("BITBUCKET_REPO_SLUG", '')
workspace = os.environ.get("BITBUCKET_WORKSPACE", '')
bearer_token = os.environ.get('BITBUCKET_BEARER_TOKEN', None)
OPENAI_KEY = os.environ.get('OPENAI_API_KEY') or os.environ.get('OPENAI.KEY')
OPENAI_ORG = os.environ.get('OPENAI_ORG') or os.environ.get('OPENAI.ORG')

# Check if required environment variables are set
if not bearer_token:
print("BITBUCKET_BEARER_TOKEN not set")
return

if not OPENAI_KEY:
print("OPENAI_KEY not set")
return

# Set the environment variables in the settings
get_settings().set("BITBUCKET.BEARER_TOKEN", bearer_token)
get_settings().set("OPENAI.KEY", OPENAI_KEY)
if OPENAI_ORG:
get_settings().set("OPENAI.ORG", OPENAI_ORG)

if pull_request_id and slug and workspace:
pr_url = f"https://bitbucket.org/{workspace}/{slug}/pull-requests/{pull_request_id}"
await PRReviewer(pr_url).run()

if __name__ == "__main__":
asyncio.run(run_action())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Consider handling exceptions for the async function run_action to prevent the program from crashing in case of errors.

Suggested change
async def run_action():
pull_request_id = os.environ.get("BITBUCKET_PR_ID", '')
slug = os.environ.get("BITBUCKET_REPO_SLUG", '')
workspace = os.environ.get("BITBUCKET_WORKSPACE", '')
bearer_token = os.environ.get('BITBUCKET_BEARER_TOKEN', None)
OPENAI_KEY = os.environ.get('OPENAI_API_KEY') or os.environ.get('OPENAI.KEY')
OPENAI_ORG = os.environ.get('OPENAI_ORG') or os.environ.get('OPENAI.ORG')
# Check if required environment variables are set
if not bearer_token:
print("BITBUCKET_BEARER_TOKEN not set")
return
if not OPENAI_KEY:
print("OPENAI_KEY not set")
return
# Set the environment variables in the settings
get_settings().set("BITBUCKET.BEARER_TOKEN", bearer_token)
get_settings().set("OPENAI.KEY", OPENAI_KEY)
if OPENAI_ORG:
get_settings().set("OPENAI.ORG", OPENAI_ORG)
if pull_request_id and slug and workspace:
pr_url = f"https://bitbucket.org/{workspace}/{slug}/pull-requests/{pull_request_id}"
await PRReviewer(pr_url).run()
if __name__ == "__main__":
asyncio.run(run_action())
async def run_action():
try:
pull_request_id = os.environ.get("BITBUCKET_PR_ID", '')
slug = os.environ.get("BITBUCKET_REPO_SLUG", '')
workspace = os.environ.get("BITBUCKET_WORKSPACE", '')
bearer_token = os.environ.get('BITBUCKET_BEARER_TOKEN', None)
OPENAI_KEY = os.environ.get('OPENAI_API_KEY') or os.environ.get('OPENAI.KEY')
OPENAI_ORG = os.environ.get('OPENAI_ORG') or os.environ.get('OPENAI.ORG')
# Check if required environment variables are set
if not bearer_token:
print("BITBUCKET_BEARER_TOKEN not set")
return
if not OPENAI_KEY:
print("OPENAI_KEY not set")
return
# Set the environment variables in the settings
get_settings().set("BITBUCKET.BEARER_TOKEN", bearer_token)
get_settings().set("OPENAI.KEY", OPENAI_KEY)
if OPENAI_ORG:
get_settings().set("OPENAI.ORG", OPENAI_ORG)
if pull_request_id and slug and workspace:
pr_url = f"https://bitbucket.org/{workspace}/{slug}/pull-requests/{pull_request_id}"
await PRReviewer(pr_url).run()
except Exception as e:
print(f"An error occurred: {e}")
if __name__ == "__main__":
asyncio.run(run_action())

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also add support for pt comments?


18 changes: 17 additions & 1 deletion pr_agent/settings/configuration.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[config]
model="gpt-4"
fallback_models=["gpt-3.5-turbo-16k"]
git_provider="github"
git_provider="bitbucket"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
git_provider="bitbucket"
git_provider="github"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not change the default value here!

publish_output=true
publish_output_progress=true
verbosity_level=0 # 0,1,2
Expand Down Expand Up @@ -55,6 +55,7 @@ extra_instructions = ""
[github]
# The type of deployment to create. Valid values are 'app' or 'user'.
deployment_type = "user"
user_token = "" # A GitHub personal access token with 'repo' scope.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
user_token = "" # A GitHub personal access token with 'repo' scope.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be placed in the .secrets.toml file!

ratelimit_retries = 5

[github_app]
Expand Down Expand Up @@ -100,6 +101,21 @@ polling_interval_seconds = 30
# token to authenticate in the patch server
# patch_server_token = ""


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't belong here! Should be placed by every user in their .sercrets.toml file

[openai]
key = "" # Acquire through https://platform.openai.com
#org = "<ORGANIZATION>" # Optional, may be commented out.
# Uncomment the following for Azure OpenAI
#api_type = "azure"
#api_version = '2023-05-15' # Check Azure documentation for the current API version
#api_base = "" # The base URL for your Azure OpenAI resource. e.g. "https://<your resource name>.openai.azure.com"
#deployment_id = "" # The deployment name you chose when you deployed the engine
#fallback_deployments = [] # For each fallback model specified in configuration.toml in the [config] section, specify the appropriate deployment_id

[bitbucket]
# Bitbucket personal bearer token
bearer_token = ""

[litellm]
#use_client = false

Expand Down
Loading