Refactor/general deploy #58

wannacfuture · 2023-12-05T14:27:31Z

Some fixes for bot deploy

…iquibot into refactor/general

src/adapters/supabase/helpers/tables/logs.ts

src/handlers/shared/structured-metadata.ts

src/tests/commands-test.ts

wannacfuture · 2023-12-05T16:49:38Z

Documentation for App Deploy

1.Register your Github App

Go to https://github.com/settings/apps/new
Homepage URL: the URL to the GitHub repository for your app
Webhook URL: Use https://example.com/ for now, we'll come back in a minute to update this with the URL of your deployed app.
Webhook Secret: Generate a unique secret with (e.g. with openssl rand -base64 32) and save it because you'll need it in a minute to configure your Probot app.
Permissions: was not sure lol - so have set all of them Access: Read/Write
Subscribe to events: Issues, Issue comment, Label, Pull request, PR review, PR review comment, PR review thread, Push, Repository dispatch, Workflow dispatch, Workflow run
Create your App and install to your repository (the one you want to install)

2. Deploy the app

Go to netlify and select the github repository (ubiquibot) and select the branch you want to deploy
Set environment variables:
- APP_ID: the id of the app which is created just before (You can get it from app settings https://github.com/settings/apps/your_app_name)
- GITHUB_CLIENT_ID: also the one appear in app settings
- GITHUB_CLIENT_SECRET: you can generate the secret in app settings page
- GITHUB_TOKEN: your PAT token for the config repo which includes compute delegating
- PRIVATE_KEY: Scroll down in app settings page and generate private key and paste it
- SUPABASE_KEY: your SUPABASE key
- SUPABASE_URL: your SUPABASE url
- WEBHOOK_PROXY_URL: the proxy url in your '.env' file already
- WEBHOOK_SECRET: the env variable in your .env file (also set this value as webhook secret in webhook section in app settings page)
- X25519_PRIVATE_KEY: your X25519_PRIVATE_KEY
Then deploy your app
In app settings page, set your Webhook URL as deployed_url/.netlify/functions/webhooks
you can see the logs at app.netlify.com/sites/deployed_url/logs/functions/webhooks

0x4007 · 2023-12-06T12:14:34Z

const commitHash = process.env.COMMIT_REF;

Have you tried this? First suggestion that ChatGPT gave regarding Netlify.

0x4007 · 2023-12-06T12:37:26Z

GITHUB_TOKEN: your PAT token for the config repo which includes compute delegating

This doesn't make sense. There will only be one official deployment. It's obviously not going to have private repository access for everyone in the world. Why bother with this if we should be passing it in from the partner level?

You also didn't specify the permissions required. I enabled Contents: Read-only Metadata: Read-only under Repository permissions

wannacfuture · 2023-12-06T12:45:56Z

GITHUB_TOKEN: your PAT token for the config repo which includes compute delegating

Right, I just thought that we have only one backend for compute delegating and believed we need that PAT token for that backend repo.

What about just having only one backend?

If each user has their own backend repo, should we include that token to the config?

0x4007 · 2023-12-06T12:48:35Z

In app settings page, set your Webhook URL as deployed_url/.netlify/functions/webhooks
you can see the logs at app.netlify.com/sites/deployed_url/logs/functions/webhooks

I don't see where to do this.

Build failed. You did not specify the build settings etc.

9:45:36 PM: build-image version: 3ffff9df3d5419545acc1b673a54de348174406d (focal)
9:45:36 PM: buildbot version: 228d4ef7b03dfc60f183d3edd500451c6a4cc157
9:45:36 PM: Fetching cached dependencies
9:45:36 PM: Failed to fetch cache, continuing with build
9:45:36 PM: Starting to prepare the repo for build
9:45:36 PM: No cached dependencies found. Cloning fresh repo
9:45:36 PM: git clone --filter=blob:none https://github.com/pavlovcik/ubiquibot
9:45:37 PM: Preparing Git Reference refs/heads/refactor/general
9:45:38 PM: Custom functions path detected. Proceeding with the specified path: ".netlify/functions"
9:45:38 PM: Custom ignore command detected. Proceeding with the specified command: "is-core-module/package.json"
9:45:38 PM: Starting to install dependencies
9:45:38 PM: Python version set to 3.8
9:45:38 PM: Attempting Ruby version 2.7.2, read from environment
9:45:39 PM: Using Ruby version 2.7.2
9:45:40 PM: Started restoring cached go cache
9:45:40 PM: Finished restoring cached go cache
9:45:41 PM: go version go1.19.13 linux/amd64
9:45:41 PM: Using PHP version 8.0
9:45:42 PM: Attempting Node.js version "v20.3.0" from .nvmrc
9:45:42 PM: Downloading and installing node v20.3.0...
9:45:42 PM: Downloading https://nodejs.org/dist/v20.3.0/node-v20.3.0-linux-x64.tar.xz...
9:45:43 PM: Computing checksum with sha256sum
9:45:43 PM: Checksums matched!
9:45:46 PM: Now using node v20.3.0 (npm v9.6.7)
9:45:47 PM: Enabling Node.js Corepack
9:45:47 PM: Started restoring cached build plugins
9:45:47 PM: Finished restoring cached build plugins
9:45:47 PM: Started restoring cached corepack dependencies
9:45:47 PM: Finished restoring cached corepack dependencies
9:45:47 PM: Started restoring cached yarn cache
9:45:47 PM: Finished restoring cached yarn cache
9:45:47 PM: No yarn workspaces detected
9:45:47 PM: Started restoring cached node modules
9:45:47 PM: Finished restoring cached node modules
9:45:47 PM: Installing npm packages using Yarn version 1.22.19
9:45:48 PM: yarn install v1.22.19
9:45:48 PM: [1/5] Validating package.json...
9:45:48 PM: [2/5] Resolving packages...
9:45:48 PM: [3/5] Fetching packages...
9:46:01 PM: [4/5] Linking dependencies...
9:46:04 PM: [5/5] Building fresh packages...
9:46:05 PM: $ husky install
9:46:05 PM: husky - Git hooks installed
9:46:05 PM: Done in 17.06s.
9:46:05 PM: npm packages installed using Yarn
9:46:05 PM: Successfully installed dependencies
9:46:05 PM: Starting build script
9:46:06 PM: Detected 0 framework(s)
9:46:06 PM: Section completed: initializing
9:46:08 PM: 
9:46:08 PM: Netlify Build                                                 
9:46:08 PM: ────────────────────────────────────────────────────────────────
9:46:08 PM: 
9:46:08 PM: ❯ Version
9:46:08 PM:   @netlify/build 29.29.4
9:46:08 PM: 
9:46:08 PM: ❯ Flags
9:46:08 PM:   baseRelDir: true
9:46:08 PM:   buildId: 65706cdb9a475274621f7669
9:46:08 PM:   deployId: 65706cdb9a475274621f766b
9:46:08 PM: 
9:46:08 PM: ❯ Current directory
9:46:08 PM:   /opt/build/repo
9:46:08 PM: 
9:46:08 PM: ❯ Config file
9:46:08 PM:   /opt/build/repo/netlify.toml
9:46:08 PM: 
9:46:08 PM: ❯ Context
9:46:08 PM:   production
9:46:08 PM: 
9:46:08 PM: Build command from Netlify app                                
9:46:08 PM: ────────────────────────────────────────────────────────────────
9:46:08 PM: 
9:46:08 PM: $ yarn tsc
9:46:08 PM: yarn run v1.22.19
9:46:08 PM: $ /opt/build/repo/node_modules/.bin/tsc
9:46:12 PM: src/helpers/issue.ts(63,16): error TS6133: "listIssuesAndPullsForRepo" is declared but its value is never read.
9:46:12 PM: src/helpers/issue.ts(378,16): error TS6133: "getPullRequests" is declared but its value is never read.
9:46:12 PM: src/helpers/issue.ts(439,16): error TS6133: "getPullRequestReviews" is declared but its value is never read.
9:46:12 PM: src/helpers/parser.ts(4,1): error TS6133: "getPullByNumber" is declared but its value is never read.
9:46:12 PM: src/helpers/parser.ts(74,10): error TS6133: "isNewerPullRequest" is declared but its value is never read.
9:46:12 PM: src/tests/before-all-handler.ts(90,15): error TS2304: Cannot find name "orgConfig".
9:46:12 PM: src/tests/before-all-handler.ts(98,15): error TS2304: Cannot find name "repoConfig".
9:46:12 PM: src/types/payload.ts(366,6): error TS6196: "Organization" is declared but never used.
9:46:12 PM: src/utils/ajv.ts(1,15): error TS6133: "Schema" is declared but its value is never read.
9:46:12 PM: src/utils/ajv.ts(1,23): error TS6133: "ValidateFunction" is declared but its value is never read.
9:46:12 PM: error Command failed with exit code 2. (https://ntl.fyi/exit-code-2)
9:46:12 PM: info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
9:46:12 PM: 
9:46:12 PM: "build.command" failed                                        
9:46:12 PM: ────────────────────────────────────────────────────────────────
9:46:12 PM: 
9:46:12 PM:   Error message
9:46:12 PM:   Command failed with exit code 2: yarn tsc (https://ntl.fyi/exit-code-2)
9:46:12 PM: 
9:46:12 PM:   Error location
9:46:12 PM:   In Build command from Netlify app:
9:46:12 PM:   yarn tsc
9:46:12 PM: 
9:46:12 PM:   Resolved config
9:46:12 PM:   build:
9:46:12 PM:     command: yarn tsc
9:46:12 PM:     commandOrigin: ui
9:46:12 PM:     environment:
9:46:12 PM:       - UBIQUITY_BOUNTY_BOT_APP_ID
9:46:12 PM:       - private_key
9:46:12 PM:     ignore: is-core-module/package.json
9:46:12 PM:     publish: /opt/build/repo/lib
9:46:12 PM:     publishOrigin: ui
9:46:12 PM:   functionsDirectory: /opt/build/repo/.netlify/functions
9:46:13 PM: Failed during stage "building site": Build script returned non-zero exit code: 2
9:46:13 PM: Build failed due to a user error: Build script returned non-zero exit code: 2
9:46:13 PM: Failing build: Failed to build site
9:46:13 PM: Finished processing build request in 36.735s

0x4007 · 2023-12-06T12:52:27Z

What about just having only one backend?

If each user has their own backend repo, should we include that token to the config?

I need to do more research on this to answer conclusively but it makes a ton of sense to allow users to make their own plugins, which makes sense to load directly from their repositories. Without personal access tokens, there will not be a way to have private repositories/plugins.

When they add the UbiquiBot to their organization, they should allow access to their private ubiquibot-config repository.
Their ubiquibot-config.yml should have a personalAccessToken property that substitutes for this.

What are the benefits of having only one backend other than saving a little work time for this?

wannacfuture · 2023-12-06T12:55:56Z

In app settings page, set your Webhook URL as deployed_url/.netlify/functions/webhooks
you can see the logs at app.netlify.com/sites/deployed_url/logs/functions/webhooks

You can do that after you deployed.

And regarding build failure
Make sure you have selected the correct branch & make sure it is set like this in .netlify/functions/webhooks/webhook.ts
import app from "../../../src/main";

What are the benefits of having only one backend other than saving a little work time for this?

Maybe contributors just need to touch bot repo not configuring config repo maybe...? not so sure about that.
But yeah, we can add PAT to config

0x4007 · 2023-12-06T12:57:12Z

Current repository
github.com/pavlovcik/ubiquibot

Production branch
refactor/general

its
.netlify/functions/webhooks/webhooks.ts
not
.netlify/functions/webhooks/webhook.ts

wannacfuture · 2023-12-06T12:58:42Z

don't set any build command and directory

0x4007 · 2023-12-06T13:00:40Z

Successfully deployed.

wannacfuture · 2023-12-06T13:01:08Z

In app settings page, set your Webhook URL as deployed_url/.netlify/functions/webhooks
you can see the logs at app.netlify.com/sites/deployed_url/logs/functions/webhooks

wannacfuture · 2023-12-06T13:01:52Z

0x4007 · 2023-12-06T13:03:08Z

On this page I see https://ubiquibot-production.netlify.app/.netlify/functions/webhooks

wannacfuture · 2023-12-06T13:06:04Z

Thats correct url

Their ubiquibot-config.yml should have a personalAccessToken property that substitutes for this.

Not so sure about that.
Let's assume that we deployed our app.
Where would you put the PAT token for our config repo since all the resources should be public for all contributors.

0x4007 · 2023-12-06T13:10:00Z

Thats correct url

Their ubiquibot-config.yml should have a personalAccessToken property that substitutes for this.

Not so sure about that. Let's assume that we deployed our app. Where would you put the PAT token for our config repo since all the resources should be public for all contributors.

GitHub App permissions. I'm pretty sure this is already implemented or else it wouldn't be able to work with private config repos now.

When they add the UbiquiBot to their organization, they should allow access to their private ubiquibot-config repository.

I'm reviewing the environment variables because Netlify is making me enter them again. Some of them are never referenced in the code. Are you 100% sure they are all necessary?

wannacfuture · 2023-12-06T13:13:03Z

For now we are just making API request to the config repo with the token so even the app doesn't have any permission, it will access to the private config repo.

Yeap, all of them are needed.

0x4007 · 2023-12-06T13:16:54Z

so even the app doesn't have any permission, it will access to the private config repo.

This seems like a bad approach. We already had it working with the proper configuration (allow access to all the repositories from the app.)

0x4007 · 2023-12-06T13:29:56Z

@wannacfuture update the deploy documentation based on this conversation and 0x4007/ubiquibot-sandbox#1 when I have it working. Add to the README and then I want @whilefoo to try a deployment using these instructions.

0x4007 · 2023-12-06T16:47:37Z

I enabled Contents: Read-only Metadata: Read-only under Repository permissions

Just updated my PAT cause it has insufficient privileges. It broke here.

Made sure to also select "All repositories"

wannacfuture added 7 commits December 4, 2023 20:54

Update webhooks.ts

96aebdd

Merge branch 'refactor/general' of https://github.com/wannacfuture/ub…

ea077a0

…iquibot into refactor/general

feat: update

daa134a

feat: update

5e2500f

feat: update

8b92e0c

feat: update

a83b91e

feat: update

570f63b

0x4007 reviewed Dec 5, 2023

View reviewed changes

src/adapters/supabase/helpers/tables/logs.ts Outdated Show resolved Hide resolved

0x4007 mentioned this pull request Dec 5, 2023

Netlify Git Commit Hash ubiquity/ubiquibot#900

Closed

0x4007 reviewed Dec 5, 2023

View reviewed changes

src/handlers/shared/structured-metadata.ts Outdated Show resolved Hide resolved

0x4007 reviewed Dec 5, 2023

View reviewed changes

src/tests/commands-test.ts Outdated Show resolved Hide resolved

wannacfuture added 3 commits December 5, 2023 11:38

feat: add app deploy documentation

8d1def4

feat: delete

7346df1

feat: update

f65270a

0x4007 approved these changes Dec 6, 2023

View reviewed changes

Apply suggestions from code review

8678be9

0x4007 merged commit 705238b into 0x4007:refactor/general Dec 6, 2023
2 of 5 checks passed

0x4007 mentioned this pull request Dec 6, 2023

Revert "Refactor/general deploy" #59

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor/general deploy #58

Refactor/general deploy #58

wannacfuture commented Dec 5, 2023

wannacfuture commented Dec 5, 2023 •

edited

Loading

0x4007 commented Dec 6, 2023 •

edited

Loading

0x4007 commented Dec 6, 2023 •

edited

Loading

wannacfuture commented Dec 6, 2023 •

edited

Loading

0x4007 commented Dec 6, 2023 •

edited

Loading

0x4007 commented Dec 6, 2023 •

edited

Loading

wannacfuture commented Dec 6, 2023 •

edited

Loading

0x4007 commented Dec 6, 2023 •

edited

Loading

wannacfuture commented Dec 6, 2023 •

edited

Loading

0x4007 commented Dec 6, 2023

wannacfuture commented Dec 6, 2023

wannacfuture commented Dec 6, 2023

0x4007 commented Dec 6, 2023

wannacfuture commented Dec 6, 2023 •

edited

Loading

0x4007 commented Dec 6, 2023 •

edited

Loading

wannacfuture commented Dec 6, 2023 •

edited

Loading

0x4007 commented Dec 6, 2023

0x4007 commented Dec 6, 2023

0x4007 commented Dec 6, 2023 •

edited

Loading

Refactor/general deploy #58

Refactor/general deploy #58

Conversation

wannacfuture commented Dec 5, 2023

wannacfuture commented Dec 5, 2023 • edited Loading

Documentation for App Deploy

1.Register your Github App

2. Deploy the app

0x4007 commented Dec 6, 2023 • edited Loading

0x4007 commented Dec 6, 2023 • edited Loading

wannacfuture commented Dec 6, 2023 • edited Loading

0x4007 commented Dec 6, 2023 • edited Loading

0x4007 commented Dec 6, 2023 • edited Loading

wannacfuture commented Dec 6, 2023 • edited Loading

0x4007 commented Dec 6, 2023 • edited Loading

wannacfuture commented Dec 6, 2023 • edited Loading

0x4007 commented Dec 6, 2023

wannacfuture commented Dec 6, 2023

wannacfuture commented Dec 6, 2023

0x4007 commented Dec 6, 2023

wannacfuture commented Dec 6, 2023 • edited Loading

0x4007 commented Dec 6, 2023 • edited Loading

wannacfuture commented Dec 6, 2023 • edited Loading

0x4007 commented Dec 6, 2023

0x4007 commented Dec 6, 2023

0x4007 commented Dec 6, 2023 • edited Loading

wannacfuture commented Dec 5, 2023 •

edited

Loading

0x4007 commented Dec 6, 2023 •

edited

Loading

0x4007 commented Dec 6, 2023 •

edited

Loading

wannacfuture commented Dec 6, 2023 •

edited

Loading

0x4007 commented Dec 6, 2023 •

edited

Loading

0x4007 commented Dec 6, 2023 •

edited

Loading

wannacfuture commented Dec 6, 2023 •

edited

Loading

0x4007 commented Dec 6, 2023 •

edited

Loading

wannacfuture commented Dec 6, 2023 •

edited

Loading

wannacfuture commented Dec 6, 2023 •

edited

Loading

0x4007 commented Dec 6, 2023 •

edited

Loading

wannacfuture commented Dec 6, 2023 •

edited

Loading

0x4007 commented Dec 6, 2023 •

edited

Loading