Cannot add self-hosted Gitlab repos as a Gitpod Project #7781

shaal · 2022-01-23T06:10:53Z

Bug description

When I want to add a new project in Gitpod (in https://gitpod.io/new)

and I choose a self-hosted Gitlab server (last option in the screenshot above)

It doesn't show any repos available

Steps to reproduce

Create a user in Drupal.org - https://www.drupal.org/user/register
Open https://gitpod.io/#https://git.drupalcode.org/project/drupalpod
You'll see this screen
Click on the button Authorize with git.drupalcode.org
(You can ignore the workspace that opens, the steps above was just to get the authorization in Gitpod)
Go to https://gitpod.io/new
Click on (change) button
Choose the 4th option in the screen above (Continue with git.drupalcode.org)
You'll see that there are no repos to choose

Workspace affected

No response

Expected behavior

I was expecting to see all the repos under https://git.drupalcode.org/project/*
Specifically - https://git.drupalcode.org/project/drupalpod

Example repository

No response

Anything else?

This is what I have in Gitpod's Integrations page -

The text was updated successfully, but these errors were encountered:

AlexTugarev · 2022-02-09T07:23:06Z

@shaal, please help us to sort out issues with the project integration of self-managed GitLabs. It seems to be some sort of incompatibility in the API, which blocks the reading of projects here. We've seen similar with a very old installation of GitLab, thus would you mind to provide the version number?

Also, I just tried to find relevant log entries, but couldn't spot any related errors. Would you mind trying again and posting the time, please?

jankeromnes · 2022-02-09T09:01:55Z

Alternatively, maybe you could invite one of us to your Self-Managed GitLab instance? I think this could make debugging easier.

AlexTugarev · 2022-02-09T09:55:13Z

@jankeromnes, thanks for the hint!

Just signed up successfully to https://www.drupal.org/u/alextugarev

Unfortunately, I'm not allowed to create any projects there. Trying with existing ones.

AlexTugarev · 2022-02-09T10:23:07Z

I should be able to reproduce now.

AlexTugarev · 2022-02-09T11:41:20Z

@shaal, please check with this URL https://git.drupalcode.org/api/v4/projects?min_access_level=40 and let me know, how many projects are accessible to your account having Maintainer (40) permissions. See https://docs.gitlab.com/ee/api/members.html#valid-access-levels for more details on the access levels in GitLab.

For me this is empty, actually I'm not able to see any in there.

GItLab projects/repos can only be added to Gitpod if you have Maintainer (40) permissions, because that's a requirement for adding the webook for prebuilds and servers also a natural authorization level.

shaal · 2022-02-09T13:11:56Z

@AlexTugarev i can see projects listed on that url https://git.drupalcode.org/api/v4/projects?min_access_level=40

But I could see them only after I went to https://git.drupalcode.org and logged in (with my Drupal account)

AlexTugarev · 2022-02-09T13:14:02Z

But I could see them only after I went to https://git.drupalcode.org/ and logged in (with my Drupal account)

that is expected.

Also, I just tried to find relevant log entries, but couldn't spot any related errors. Would you mind trying again and posting the time, please?

☝🏻 could you do this, please?

shaal · 2022-02-09T13:24:56Z

@AlexTugarev sure, relevant log entries for what? What should I try doing?

I just tried adding again any project that is hosted on drupal.org (empty list)

If it's easier we can also discuss this on Gitpod discord.

shaal · 2022-02-22T18:20:00Z

@AlexTugarev please see my question above ^

shaal · 2022-04-15T19:53:43Z

@AlexTugarev @jankeromnes Is there any update on this issue? Anything I can do to help?

jankeromnes · 2022-04-19T07:13:09Z

Thanks for the ping @shaal!

@AlexTugarev is the issue status still up-to-date? (I.e. you are assigned, project says "In Progress")

If you're still actively working on this, how can I help? 😊
If you don't currently have time for this, could you please un-assign yourself, and reset the project status to "Scheduled", so that someone else can pick this up? 🙏

AlexTugarev · 2022-04-26T09:21:40Z

There were actually two issues in here, and one got resolved with integrating Bitbucket Server into the New Project page.

The remaining issue seems to be that the getProviderRepositoriesForUser returns empty response.

@shaal, what does https://git.drupalcode.org/api/v4/projects?min_access_level=40 return for you? For me, it's an empty array, which is expected as I'm not maintainer of any of those projects. Does the response contain the wanted projects for you?

shaal · 2022-04-27T01:15:04Z

@AlexTugarev yeah, I am getting an array of 3 items, which are the 3 projects I have write-access to.

gin_dashboard
drupalpod
rector

[
  {
    "id": 77335,
    "description": "For more information about this repository, visit the project page at https://www.drupal.org/project/gin_dashboard",
    "name": "gin_dashboard",
    "name_with_namespace": "project / gin_dashboard",
    "path": "gin_dashboard",
    "path_with_namespace": "project/gin_dashboard",
    "created_at": "2021-10-11T18:10:08.878Z",
    "default_branch": "main",
    "tag_list": [],
    "topics": [],
    "ssh_url_to_repo": "git@git.drupal.org:project/gin_dashboard.git",
    "http_url_to_repo": "https://git.drupalcode.org/project/gin_dashboard.git",
    "web_url": "https://git.drupalcode.org/project/gin_dashboard",
    "readme_url": null,
    "avatar_url": null,
    "forks_count": 0,
    "star_count": 0,
    "last_activity_at": "2021-10-11T18:10:08.878Z",
    "namespace": {
      "id": 2,
      "name": "project",
      "path": "project",
      "kind": "group",
      "full_path": "project",
      "parent_id": null,
      "avatar_url": null,
      "web_url": "https://git.drupalcode.org/groups/project"
    },
    "_links": {
      "self": "https://git.drupalcode.org/api/v4/projects/77335",
      "merge_requests": "https://git.drupalcode.org/api/v4/projects/77335/merge_requests",
      "repo_branches": "https://git.drupalcode.org/api/v4/projects/77335/repository/branches",
      "labels": "https://git.drupalcode.org/api/v4/projects/77335/labels",
      "events": "https://git.drupalcode.org/api/v4/projects/77335/events",
      "members": "https://git.drupalcode.org/api/v4/projects/77335/members"
    },
    "packages_enabled": true,
    "empty_repo": true,
    "archived": false,
    "visibility": "public",
    "resolve_outdated_diff_discussions": false,
    "container_expiration_policy": {
      "cadence": "1d",
      "enabled": false,
      "keep_n": 10,
      "older_than": "90d",
      "name_regex": ".*",
      "name_regex_keep": null,
      "next_run_at": "2021-10-12T18:10:08.910Z"
    },
    "issues_enabled": false,
    "merge_requests_enabled": true,
    "wiki_enabled": false,
    "jobs_enabled": false,
    "snippets_enabled": false,
    "container_registry_enabled": true,
    "service_desk_enabled": false,
    "can_create_merge_request_in": true,
    "issues_access_level": "disabled",
    "repository_access_level": "enabled",
    "merge_requests_access_level": "enabled",
    "forking_access_level": "enabled",
    "wiki_access_level": "disabled",
    "builds_access_level": "disabled",
    "snippets_access_level": "disabled",
    "pages_access_level": "private",
    "operations_access_level": "disabled",
    "analytics_access_level": "enabled",
    "container_registry_access_level": "enabled",
    "security_and_compliance_access_level": "private",
    "emails_disabled": null,
    "shared_runners_enabled": false,
    "lfs_enabled": false,
    "creator_id": 2,
    "import_status": "none",
    "ci_default_git_depth": 50,
    "ci_forward_deployment_enabled": true,
    "ci_job_token_scope_enabled": false,
    "public_jobs": true,
    "build_timeout": 3600,
    "auto_cancel_pending_pipelines": "enabled",
    "build_coverage_regex": null,
    "ci_config_path": "",
    "shared_with_groups": [],
    "only_allow_merge_if_pipeline_succeeds": false,
    "allow_merge_on_skipped_pipeline": null,
    "restrict_user_defined_variables": false,
    "request_access_enabled": false,
    "only_allow_merge_if_all_discussions_are_resolved": false,
    "remove_source_branch_after_merge": true,
    "printing_merge_request_link_enabled": true,
    "merge_method": "ff",
    "squash_option": "default_on",
    "suggestion_commit_message": null,
    "merge_commit_template": null,
    "squash_commit_template": null,
    "auto_devops_enabled": false,
    "auto_devops_deploy_strategy": "continuous",
    "autoclose_referenced_issues": true,
    "keep_latest_artifact": true,
    "runner_token_expiration_interval": null,
    "approvals_before_merge": 0,
    "mirror": false,
    "external_authorization_classification_label": null,
    "marked_for_deletion_at": null,
    "marked_for_deletion_on": null,
    "requirements_enabled": true,
    "requirements_access_level": "enabled",
    "security_and_compliance_enabled": true,
    "compliance_frameworks": [],
    "merge_requests_template": null,
    "merge_pipelines_enabled": true,
    "merge_trains_enabled": true,
    "permissions": {
      "project_access": {
        "access_level": 40,
        "notification_level": 3
      },
      "group_access": null
    }
  },
  {
    "id": 76369,
    "description": "For more information about this repository, visit the project page at https://www.drupal.org/project/drupalpod",
    "name": "drupalpod",
    "name_with_namespace": "project / drupalpod",
    "path": "drupalpod",
    "path_with_namespace": "project/drupalpod",
    "created_at": "2021-09-29T15:59:40.376Z",
    "default_branch": "main",
    "tag_list": [],
    "topics": [],
    "ssh_url_to_repo": "git@git.drupal.org:project/drupalpod.git",
    "http_url_to_repo": "https://git.drupalcode.org/project/drupalpod.git",
    "web_url": "https://git.drupalcode.org/project/drupalpod",
    "readme_url": "https://git.drupalcode.org/project/drupalpod/-/blob/main/README.md",
    "avatar_url": null,
    "forks_count": 0,
    "star_count": 0,
    "last_activity_at": "2022-04-15T19:44:51.728Z",
    "namespace": {
      "id": 2,
      "name": "project",
      "path": "project",
      "kind": "group",
      "full_path": "project",
      "parent_id": null,
      "avatar_url": null,
      "web_url": "https://git.drupalcode.org/groups/project"
    },
    "_links": {
      "self": "https://git.drupalcode.org/api/v4/projects/76369",
      "merge_requests": "https://git.drupalcode.org/api/v4/projects/76369/merge_requests",
      "repo_branches": "https://git.drupalcode.org/api/v4/projects/76369/repository/branches",
      "labels": "https://git.drupalcode.org/api/v4/projects/76369/labels",
      "events": "https://git.drupalcode.org/api/v4/projects/76369/events",
      "members": "https://git.drupalcode.org/api/v4/projects/76369/members"
    },
    "packages_enabled": true,
    "empty_repo": false,
    "archived": false,
    "visibility": "public",
    "resolve_outdated_diff_discussions": false,
    "container_expiration_policy": {
      "cadence": "1d",
      "enabled": false,
      "keep_n": 10,
      "older_than": "90d",
      "name_regex": ".*",
      "name_regex_keep": null,
      "next_run_at": "2021-09-30T15:59:40.446Z"
    },
    "issues_enabled": false,
    "merge_requests_enabled": true,
    "wiki_enabled": true,
    "jobs_enabled": true,
    "snippets_enabled": true,
    "container_registry_enabled": true,
    "service_desk_enabled": false,
    "can_create_merge_request_in": true,
    "issues_access_level": "disabled",
    "repository_access_level": "enabled",
    "merge_requests_access_level": "enabled",
    "forking_access_level": "enabled",
    "wiki_access_level": "enabled",
    "builds_access_level": "enabled",
    "snippets_access_level": "enabled",
    "pages_access_level": "private",
    "operations_access_level": "disabled",
    "analytics_access_level": "enabled",
    "container_registry_access_level": "enabled",
    "security_and_compliance_access_level": "private",
    "emails_disabled": null,
    "shared_runners_enabled": true,
    "lfs_enabled": false,
    "creator_id": 2,
    "import_status": "none",
    "ci_default_git_depth": 50,
    "ci_forward_deployment_enabled": true,
    "ci_job_token_scope_enabled": false,
    "public_jobs": true,
    "build_timeout": 3600,
    "auto_cancel_pending_pipelines": "enabled",
    "build_coverage_regex": null,
    "ci_config_path": "",
    "shared_with_groups": [],
    "only_allow_merge_if_pipeline_succeeds": false,
    "allow_merge_on_skipped_pipeline": null,
    "restrict_user_defined_variables": false,
    "request_access_enabled": false,
    "only_allow_merge_if_all_discussions_are_resolved": false,
    "remove_source_branch_after_merge": true,
    "printing_merge_request_link_enabled": true,
    "merge_method": "ff",
    "squash_option": "default_on",
    "suggestion_commit_message": null,
    "merge_commit_template": null,
    "squash_commit_template": null,
    "auto_devops_enabled": false,
    "auto_devops_deploy_strategy": "continuous",
    "autoclose_referenced_issues": true,
    "keep_latest_artifact": true,
    "runner_token_expiration_interval": null,
    "approvals_before_merge": 0,
    "mirror": false,
    "external_authorization_classification_label": null,
    "marked_for_deletion_at": null,
    "marked_for_deletion_on": null,
    "requirements_enabled": true,
    "requirements_access_level": "enabled",
    "security_and_compliance_enabled": true,
    "compliance_frameworks": [],
    "merge_requests_template": null,
    "merge_pipelines_enabled": true,
    "merge_trains_enabled": true,
    "permissions": {
      "project_access": {
        "access_level": 40,
        "notification_level": 3
      },
      "group_access": null
    }
  },
  {
    "id": 62395,
    "description": "For more information about this repository, visit the project page at https://www.drupal.org/project/rector",
    "name": "rector",
    "name_with_namespace": "project / rector",
    "path": "rector",
    "path_with_namespace": "project/rector",
    "created_at": "2020-02-24T18:39:00.037Z",
    "default_branch": "main",
    "tag_list": [],
    "topics": [],
    "ssh_url_to_repo": "git@git.drupal.org:project/rector.git",
    "http_url_to_repo": "https://git.drupalcode.org/project/rector.git",
    "web_url": "https://git.drupalcode.org/project/rector",
    "readme_url": null,
    "avatar_url": null,
    "forks_count": 0,
    "star_count": 0,
    "last_activity_at": "2020-04-14T17:59:51.414Z",
    "namespace": {
      "id": 2,
      "name": "project",
      "path": "project",
      "kind": "group",
      "full_path": "project",
      "parent_id": null,
      "avatar_url": null,
      "web_url": "https://git.drupalcode.org/groups/project"
    },
    "_links": {
      "self": "https://git.drupalcode.org/api/v4/projects/62395",
      "merge_requests": "https://git.drupalcode.org/api/v4/projects/62395/merge_requests",
      "repo_branches": "https://git.drupalcode.org/api/v4/projects/62395/repository/branches",
      "labels": "https://git.drupalcode.org/api/v4/projects/62395/labels",
      "events": "https://git.drupalcode.org/api/v4/projects/62395/events",
      "members": "https://git.drupalcode.org/api/v4/projects/62395/members"
    },
    "packages_enabled": true,
    "empty_repo": true,
    "archived": false,
    "visibility": "public",
    "resolve_outdated_diff_discussions": false,
    "container_expiration_policy": {
      "cadence": "7d",
      "enabled": false,
      "keep_n": null,
      "older_than": null,
      "name_regex": null,
      "name_regex_keep": null,
      "next_run_at": "2020-03-02T18:39:00.087Z"
    },
    "issues_enabled": false,
    "merge_requests_enabled": true,
    "wiki_enabled": false,
    "jobs_enabled": false,
    "snippets_enabled": false,
    "container_registry_enabled": true,
    "service_desk_enabled": false,
    "can_create_merge_request_in": true,
    "issues_access_level": "disabled",
    "repository_access_level": "enabled",
    "merge_requests_access_level": "enabled",
    "forking_access_level": "enabled",
    "wiki_access_level": "disabled",
    "builds_access_level": "disabled",
    "snippets_access_level": "disabled",
    "pages_access_level": "private",
    "operations_access_level": "disabled",
    "analytics_access_level": "enabled",
    "container_registry_access_level": "enabled",
    "security_and_compliance_access_level": "private",
    "emails_disabled": null,
    "shared_runners_enabled": false,
    "lfs_enabled": false,
    "creator_id": 2,
    "import_status": "none",
    "ci_default_git_depth": 50,
    "ci_forward_deployment_enabled": null,
    "ci_job_token_scope_enabled": false,
    "public_jobs": true,
    "build_timeout": 3600,
    "auto_cancel_pending_pipelines": "enabled",
    "build_coverage_regex": null,
    "ci_config_path": null,
    "shared_with_groups": [],
    "only_allow_merge_if_pipeline_succeeds": false,
    "allow_merge_on_skipped_pipeline": null,
    "restrict_user_defined_variables": false,
    "request_access_enabled": false,
    "only_allow_merge_if_all_discussions_are_resolved": false,
    "remove_source_branch_after_merge": true,
    "printing_merge_request_link_enabled": true,
    "merge_method": "ff",
    "squash_option": "default_on",
    "suggestion_commit_message": null,
    "merge_commit_template": null,
    "squash_commit_template": null,
    "auto_devops_enabled": false,
    "auto_devops_deploy_strategy": "continuous",
    "autoclose_referenced_issues": true,
    "keep_latest_artifact": true,
    "runner_token_expiration_interval": null,
    "approvals_before_merge": 0,
    "mirror": false,
    "external_authorization_classification_label": null,
    "marked_for_deletion_at": null,
    "marked_for_deletion_on": null,
    "requirements_enabled": true,
    "requirements_access_level": "enabled",
    "security_and_compliance_enabled": true,
    "compliance_frameworks": [],
    "merge_requests_template": null,
    "merge_pipelines_enabled": true,
    "merge_trains_enabled": true,
    "permissions": {
      "project_access": {
        "access_level": 40,
        "notification_level": 3
      },
      "group_access": null
    }
  }
]

AlexTugarev · 2022-04-27T07:30:58Z

~~@shaal, thanks very much for the details. There was one thing we didn't see in tests apparently.~~

"namespace": {
      "id": 2,
      "name": "project",
      "path": "project",
      "kind": "group",
      "full_path": "project",
      "parent_id": null,
      "avatar_url": null,
      "web_url": "https://git.drupalcode.org/groups/project"
    },

This group kind is the key to the problem. While the API returns all projects from Gitlab using the current session auth (via browser), it's not showing projects in groups when authorizing with OAuth 2.0.

~~We need to check and clarify with Gitlab on the expectations here.~~

edit

Oh no! That was a trap. I managed to mix up my gitlab.com accounts for testing. So, in the end group projects will be shown, when using the right account.

AlexTugarev · 2022-04-27T07:43:05Z

@shaal, just one question on the UI. You are sure to have clicked the drop down to select the project group?

shaal · 2022-04-27T12:21:09Z

@AlexTugarev
When I choose a git provider like Github itself, I can choose an organization, and then I see this list

When I choose Drupal Gitlab self-hosted as git provider, I see this

When I choose shaal, the list is empty, and nothing else that I can select -

shaal · 2022-04-27T12:33:18Z

@AlexTugarev, I opened Devtools -> Network -> WS,
The first orange arrow shows the result when I select Github, the response is about 130k.
The second orange arrow shows the result when I select Drupal self-hosted Gitlab, the response is an empty array.

AlexTugarev · 2022-04-28T09:19:02Z

@shaal, getting closer 🤞🏻

It seems to be an issue with GitLab after all.

For some reasons the authorization of the /projects endpoint is different between the web session and the oauth token. One can verify this by using curl and the current token issued for Gitpod.

Here you see the curl command for me:

curl --header "Authorization: Bearer TOKEN" "https://git.drupalcode.org/api/v4/projects?per_page=1&page=1"
[{"id":86354,"description":"For more information about this repository, visit the project page at https://www.drupal.org/project/belgian_postal_code","name":"belgian_postal_code","name_with_namespace":"project / belgian_postal_code","path":"belgian_postal_code","path_with_namespace":"project/belgian_postal_code","created_at":"2022-04-28T08:49:35.146Z","default_branch":"1.x","tag_list":[],"topics":[],"ssh_url_to_repo":"git@git.drupal.org:project/belgian_postal_code.git","http_url_to_repo":"https://git.drupalcode.org/project/belgian_postal_code.git","web_url":"https://git.drupalcode.org/project/belgian_postal_code","readme_url":null,"avatar_url":null,"forks_count":0,"star_count":0,"last_activity_at":"2022-04-28T08:49:35.146Z","namespace":{"id":2,"name":"project","path":"project","kind":"group","full_path":"project","parent_id":null,"avatar_url":null,"web_url":"https://git.drupalcode.org/groups/project"}}]

vs. the browsers output for GET https://git.drupalcode.org/api/v4/projects?per_page=1&page=1

[{"id":86354,"description":"For more information about this repository, visit the project page at https://www.drupal.org/project/belgian_postal_code","name":"belgian_postal_code","name_with_namespace":"project / belgian_postal_code","path":"belgian_postal_code","path_with_namespace":"project/belgian_postal_code","created_at":"2022-04-28T08:49:35.146Z","default_branch":"1.x","tag_list":[],"topics":[],"ssh_url_to_repo":"git@git.drupal.org:project/belgian_postal_code.git","http_url_to_repo":"https://git.drupalcode.org/project/belgian_postal_code.git","web_url":"https://git.drupalcode.org/project/belgian_postal_code","readme_url":null,"avatar_url":null,"forks_count":0,"star_count":0,"last_activity_at":"2022-04-28T08:49:35.146Z","namespace":{"id":2,"name":"project","path":"project","kind":"group","full_path":"project","parent_id":null,"avatar_url":null,"web_url":"https://git.drupalcode.org/groups/project"},"_links":{"self":"https://git.drupalcode.org/api/v4/projects/86354","merge_requests":"https://git.drupalcode.org/api/v4/projects/86354/merge_requests","repo_branches":"https://git.drupalcode.org/api/v4/projects/86354/repository/branches","labels":"https://git.drupalcode.org/api/v4/projects/86354/labels","events":"https://git.drupalcode.org/api/v4/projects/86354/events","members":"https://git.drupalcode.org/api/v4/projects/86354/members"},"packages_enabled":true,"empty_repo":false,"archived":false,"visibility":"public","resolve_outdated_diff_discussions":false,"container_expiration_policy":{"cadence":"1d","enabled":false,"keep_n":10,"older_than":"90d","name_regex":".*","name_regex_keep":null,"next_run_at":"2022-04-29T08:49:35.258Z"},"issues_enabled":false,"merge_requests_enabled":true,"wiki_enabled":false,"jobs_enabled":false,"snippets_enabled":false,"container_registry_enabled":true,"service_desk_enabled":false,"can_create_merge_request_in":true,"issues_access_level":"disabled","repository_access_level":"enabled","merge_requests_access_level":"enabled","forking_access_level":"enabled","wiki_access_level":"disabled","builds_access_level":"disabled","snippets_access_level":"disabled","pages_access_level":"private","operations_access_level":"disabled","analytics_access_level":"enabled","container_registry_access_level":"enabled","security_and_compliance_access_level":"private","emails_disabled":null,"shared_runners_enabled":false,"lfs_enabled":false,"creator_id":2,"import_status":"none","ci_default_git_depth":20,"ci_forward_deployment_enabled":true,"ci_job_token_scope_enabled":false,"public_jobs":true,"build_timeout":3600,"auto_cancel_pending_pipelines":"enabled","build_coverage_regex":null,"ci_config_path":"","shared_with_groups":[],"only_allow_merge_if_pipeline_succeeds":false,"allow_merge_on_skipped_pipeline":null,"restrict_user_defined_variables":false,"request_access_enabled":false,"only_allow_merge_if_all_discussions_are_resolved":false,"remove_source_branch_after_merge":true,"printing_merge_request_link_enabled":true,"merge_method":"ff","squash_option":"default_on","suggestion_commit_message":null,"merge_commit_template":null,"squash_commit_template":null,"auto_devops_enabled":false,"auto_devops_deploy_strategy":"continuous","autoclose_referenced_issues":true,"keep_latest_artifact":true,"runner_token_expiration_interval":null,"approvals_before_merge":0,"mirror":false,"external_authorization_classification_label":null,"marked_for_deletion_at":null,"marked_for_deletion_on":null,"requirements_enabled":true,"requirements_access_level":"enabled","security_and_compliance_enabled":false,"compliance_frameworks":[],"merge_requests_template":null,"merge_pipelines_enabled":true,"merge_trains_enabled":true,"permissions":{"project_access":null,"group_access":null}}]

Reading on https://docs.gitlab.com/ee/api/projects.html#list-all-projects it seems the authorization is not applied/accepted.

Just verified with gitlab.com, and it works as expected, i.e. /projects returns authorized results and everything is found properly.

gtsiolis · 2022-04-28T10:30:59Z

In case it helps, the only changes that were made to the /projects API endpoint from the release of the 14.9.3[1] in April 12, 2022 to the next major 15.0 release which is already deployed on GitLab.com are the following two commits but not sure if these are related to the issue here.

https://gitlab.com/gitlab-org/gitlab/-/commit/57691c52d878a93c4acfb1b8a88b2185d78062b8

https://gitlab.com/gitlab-org/gitlab/-/commit/b7396cd27824c7e994794de9fbe66455b659b141

shaal · 2022-04-28T11:51:06Z

@AlexTugarev Anything I can help with?

AlexTugarev · 2022-04-28T13:50:34Z

@shaal, can you check if there are any restrictions configured in the GitLab instance to the permission model?

I'll try to verify with a test instance of GitLab, maybe starting same version of GitLab in a workspace. But I don't yet see a way to proceed 😢

shaal · 2022-04-28T15:30:25Z

@AlexTugarev Are you available for a chat / call on Gitpod Discord?
I don't know how to get access token to run curl commands like you did.
How can I tell if there are any restrictions configured in the GitLab instance to the permission model?

Also, we could ask/discuss some of these things with the people who are in charge of Drupal's infrastructure / Self-hosted Gitlab.

shaal · 2022-04-28T21:18:54Z

@AlexTugarev, great news!
Neil Drumm (Drupal Association) was able to resolve the issue and wrote -

We had been stripping authentication headers for that path, we are no longer doing that. Since drupal.org is the canonical list of projects, we are careful about what we allow.

So now I can see DrupalPod (from Drupal's self-hosted Gitlab) and add it to my projects.
I do get 1 more error when I do that (although it seems that everything actually works as expected)

After clicking on the green Select button, I get a popup in the browser:

gitpod.io says:
Request createProject failed with message: Cannot read properties of undefined (reading 'project_access')

I asked Neil Drumm if he knew what it was, and he responded -

That error message doesn’t tell me what path they are requesting, if it is another similar thing. I will need to know what it’s requesting next. And ideally a list of all subsequent requests they expect to work, if possible.

AlexTugarev · 2022-05-03T14:28:02Z

@shaal, that makes sense now!

Neil Drumm (Drupal Association) was able to resolve the issue and wrote -
We had been stripping authentication headers for that path, we are no longer doing that. Since drupal.org is the canonical list of projects, we are careful about what we allow.

Well the next error seems to be of the same kind. Could it be that requests to subpaths are still modified?

You can verify it yourself, the wanted repo to be added as project, for which the error from above is thrown is printed as simple (i.e. the unauthorized format)

curl --header "Authorization: Bearer TOKEN" "https://git.drupalcode.org/api/v4/projects/76369"

But if you change the header to include the cookie from browser, you'll see the full representation, which is expected as response.

AlexTugarev · 2022-05-04T11:59:10Z

@shaal, oh I missed the last paragraph of your last message.

That error message doesn’t tell me what path they are requesting, if it is another similar thing. I will need to know what it’s requesting next. And ideally a list of all subsequent requests they expect to work, if possible.

That's a pity, that basically means, we'd need to scan the API surface used in Gitpod to enable it, right?
Just to make sure, is allow-listing of GET /api/v4/* no option at all?

AlexTugarev · 2022-05-04T12:08:06Z

From reading https://github.com/gitpod-io/gitpod/blob/main/components/server/src/gitlab/api.ts#L101-L112, I can tell it's at least the following list of request (query params omitted)

GET,PUT,DELETE /projects
GET,PUT,DELETE /projects/:id
GET,PUT,DELETE /projects/:id/hooks
GET /projects/:id/repository/files/:file_path
GET /projects/:id/repository/commits
GET /projects/:id/repository/tags
GET /projects/:id/merge_requests

AlexTugarev · 2022-05-09T12:50:35Z

@shaal, can we close this issue now?

I think, once the web application firewall is setup correctly on Drupal's self-hosted Gitlab it should be good. We've verified with 2 different GitLab installations by now.

Please reopen or create another more specific follow up.

shaal · 2023-06-08T19:51:27Z

I don't think this is resolved yet...
Previously (#7781 (comment)) I was able to see list of projects in self-hosted Gitlab.

But now, in https://gitpod.io/projects/new after choosing Drupal git here (the last option)

I get this screen

with this error in console.

Error: Request getProviderRepositoriesForUser failed with message: Response code 401 (Unauthorized)
    at main.js:449:48
    at main.js:276:17
    at main.js:260:13
    at setImmediate.js:40:13
    at l (setImmediate.js:69:21)
    at n (setImmediate.js:109:17)

The git repo I am trying to add to my personal user in Gitpod, is https://git.drupalcode.org/project/drupalpod

tyler36 · 2023-08-04T07:27:56Z

Just came across this issue and it seems to be working for git.drupalcode.org now:

Steps to reproduce

Open https://gitpod.io/#https://git.drupalcode.org/project/drupalpod
It said I was unauthorized, but I click to authorize and it open an oauth permission popup.
After giving permissing, I was able to start a project host on git.drupalcode.org

Now, in https://gitpod.io/projects/new after choosing Drupal git here (the last option), I can see 1 project that I am currently the maintainer for.

shaal · 2024-02-08T04:53:44Z

@AlexTugarev @gtsiolis @axonasif

It is now possible to open https://gitpod.io/#https://git.drupalcode.org/project/drupalpod, but there seems to be a bug. When the warning displays that you are not authorized, and you click to authorize, you are brought back to the page with the warning that you are still not authorized. However, reloading the page solves this issue.

When opening the above link, you only have read permissions. When I try to push changes, I get an error that I am not allowed to do so. When I check https://gitpod.io/user/integrations, I see that I am missing write permissions. When I try to add write permissions, I get a new error.

You can see the thread below for all the details of what I tried doing and what the Drupal organization tried doing, following Gitpod's documentation:

https://www.drupal.org/project/infrastructure/issues/3404441#comment-15396581

Could you please clarify what is missing so that users can open the Drupal-hosted GitLab repository in Gitpod and get write permissions?

github-actions · 2024-05-22T15:26:55Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

axonasif added git provider: gitlab self-hosted team: webapp Issue belongs to the WebApp team type: bug Something isn't working labels Jan 23, 2022

shaal mentioned this issue Feb 8, 2022

Remove deprecated '#prebuild/' context URL prefix #4353

Closed

3 tasks

jldec added this to 🍎 WebApp Team Feb 9, 2022

jankeromnes removed the self-hosted label Feb 9, 2022

AlexTugarev self-assigned this Feb 9, 2022

JanKoehnlein moved this to Scheduled in 🍎 WebApp Team Feb 10, 2022

AlexTugarev moved this from Scheduled to In Progress in 🍎 WebApp Team Feb 17, 2022

mrsimonemms mentioned this issue Mar 2, 2022

Projects support for Gitpod Self-hosted Instances #8536

Open

AlexTugarev closed this as completed May 9, 2022

Repository owner moved this from In Progress to Done in 🍎 WebApp Team May 9, 2022

axonasif reopened this Jun 9, 2023

github-project-automation bot moved this to Scheduled in 🍎 WebApp Team Jun 9, 2023

AlexTugarev removed their assignment Sep 11, 2023

github-actions bot added the meta: stale This issue/PR is stale and will be closed soon label May 22, 2024

github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Jun 2, 2024

github-project-automation bot moved this from Scheduled to In Validation in 🍎 WebApp Team Jun 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot add self-hosted Gitlab repos as a Gitpod Project #7781

Cannot add self-hosted Gitlab repos as a Gitpod Project #7781

shaal commented Jan 23, 2022

AlexTugarev commented Feb 9, 2022 •

edited

Loading

jankeromnes commented Feb 9, 2022

AlexTugarev commented Feb 9, 2022 •

edited

Loading

AlexTugarev commented Feb 9, 2022

AlexTugarev commented Feb 9, 2022

shaal commented Feb 9, 2022

AlexTugarev commented Feb 9, 2022

shaal commented Feb 9, 2022

shaal commented Feb 22, 2022

shaal commented Apr 15, 2022

jankeromnes commented Apr 19, 2022 •

edited

Loading

AlexTugarev commented Apr 26, 2022

shaal commented Apr 27, 2022

AlexTugarev commented Apr 27, 2022 •

edited

Loading

AlexTugarev commented Apr 27, 2022

shaal commented Apr 27, 2022

shaal commented Apr 27, 2022

AlexTugarev commented Apr 28, 2022

gtsiolis commented Apr 28, 2022

shaal commented Apr 28, 2022

AlexTugarev commented Apr 28, 2022

shaal commented Apr 28, 2022

shaal commented Apr 28, 2022

AlexTugarev commented May 3, 2022

AlexTugarev commented May 4, 2022

AlexTugarev commented May 4, 2022

AlexTugarev commented May 9, 2022

shaal commented Jun 8, 2023

tyler36 commented Aug 4, 2023

shaal commented Feb 8, 2024

github-actions bot commented May 22, 2024

Cannot add self-hosted Gitlab repos as a Gitpod Project #7781

Cannot add self-hosted Gitlab repos as a Gitpod Project #7781

Comments

shaal commented Jan 23, 2022

Bug description

Steps to reproduce

Workspace affected

Expected behavior

Example repository

Anything else?

AlexTugarev commented Feb 9, 2022 • edited Loading

jankeromnes commented Feb 9, 2022

AlexTugarev commented Feb 9, 2022 • edited Loading

AlexTugarev commented Feb 9, 2022

AlexTugarev commented Feb 9, 2022

shaal commented Feb 9, 2022

AlexTugarev commented Feb 9, 2022

shaal commented Feb 9, 2022

shaal commented Feb 22, 2022

shaal commented Apr 15, 2022

jankeromnes commented Apr 19, 2022 • edited Loading

AlexTugarev commented Apr 26, 2022

shaal commented Apr 27, 2022

AlexTugarev commented Apr 27, 2022 • edited Loading

AlexTugarev commented Apr 27, 2022

shaal commented Apr 27, 2022

shaal commented Apr 27, 2022

AlexTugarev commented Apr 28, 2022

gtsiolis commented Apr 28, 2022

shaal commented Apr 28, 2022

AlexTugarev commented Apr 28, 2022

shaal commented Apr 28, 2022

shaal commented Apr 28, 2022

AlexTugarev commented May 3, 2022

AlexTugarev commented May 4, 2022

AlexTugarev commented May 4, 2022

AlexTugarev commented May 9, 2022

shaal commented Jun 8, 2023

tyler36 commented Aug 4, 2023

Steps to reproduce

shaal commented Feb 8, 2024

github-actions bot commented May 22, 2024

AlexTugarev commented Feb 9, 2022 •

edited

Loading

AlexTugarev commented Feb 9, 2022 •

edited

Loading

jankeromnes commented Apr 19, 2022 •

edited

Loading

AlexTugarev commented Apr 27, 2022 •

edited

Loading