Git dependencies fail to clone with HTTPS->SSH insteadOf in git config

[ghost]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Debian 11
• Poetry version: 1.2.0
• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/nyuszika7h/da9d938a8043a58e175bfd9e09f8bf0d

Issue

I have the following in ~/.config/git/config:

[url "git@github.com:"]
        insteadOf = https://github.com/

Poetry 1.2.0 seems to have introduced a regression where this causes git clones to fail:

❯ poetry init -n
❯ poetry add -vvv git+https://github.com/nyuszika7h/m3u8.git
Loading configuration file /home/nyuszika7h/.config/pypoetry/config.toml
Creating virtualenv test in /home/nyuszika7h/test/.venv
Using virtualenv: /home/nyuszika7h/test/.venv

  Stack trace:

  3  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/dulwich/client.py:1121 in fetch_pack
      1119│         with proto:
      1120│             try:
    → 1121│                 refs, server_capabilities = read_pkt_refs(proto.read_pkt_seq())
      1122│             except HangupException:
      1123│                 raise _remote_error_from_stderr(stderr)

  2  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/dulwich/client.py:240 in read_pkt_refs
       238│     refs = {}
       239│     # Receive refs from server
    →  240│     for pkt in pkt_seq:
       241│         (sha, ref) = pkt.rstrip(b"\n").split(None, 1)
       242│         if sha == b"ERR":

  1  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/dulwich/protocol.py:287 in read_pkt_seq
      285│             flush-pkt.
      286│         """
    → 287│         pkt = self.read_pkt_line()
      288│         while pkt:
      289│             yield pkt

  HangupException

  The remote server unexpectedly closed the connection.

  at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/dulwich/protocol.py:232 in read_pkt_line
      228│
      229│         try:
      230│             sizestr = read(4)
      231│             if not sizestr:
    → 232│                 raise HangupException()
      233│             size = int(sizestr, 16)
      234│             if size == 0:
      235│                 if self.report_activity:
      236│                     self.report_activity(4, "read")

The following error occurred when trying to handle this error:


  Stack trace:

  20  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:329 in run
       327│
       328│             try:
     → 329│                 exit_code = self._run(io)
       330│             except Exception as e:
       331│                 if not self._catch_exceptions:

  19  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/console/application.py:185 in _run
       183│         self._load_plugins(io)
       184│
     → 185│         exit_code: int = super()._run(io)
       186│         return exit_code
       187│

  18  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:423 in _run
       421│             io.input.set_stream(stream)
       422│
     → 423│         exit_code = self._run_command(command, io)
       424│         self._running_command = None
       425│

  17  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:465 in _run_command
       463│
       464│         if error is not None:
     → 465│             raise error
       466│
       467│         return event.exit_code

  16  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:449 in _run_command
       447│
       448│             if event.command_should_run():
     → 449│                 exit_code = command.run(io)
       450│             else:
       451│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

  15  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/commands/base_command.py:119 in run
       117│         io.input.validate()
       118│
     → 119│         status_code = self.execute(io)
       120│
       121│         if status_code is None:

  14  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/commands/command.py:83 in execute
        81│
        82│         try:
     →  83│             return self.handle()
        84│         except KeyboardInterrupt:
        85│             return 1

  13  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/console/commands/add.py:154 in handle
       152│             return 0
       153│
     → 154│         requirements = self._determine_requirements(
       155│             packages,
       156│             allow_prereleases=self.option("allow-prereleases"),

  12  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/console/commands/init.py:357 in _determine_requirements
       355│
       356│         result = []
     → 357│         for requirement in self._parse_requirements(requires):
       358│             if "git" in requirement or "url" in requirement or "path" in requirement:
       359│                 result.append(requirement)

  11  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/console/commands/init.py:416 in _parse_requirements
       414│             cwd = Path.cwd()
       415│
     → 416│         return [
       417│             parse_dependency_specification(
       418│                 requirement=requirement,

  10  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/console/commands/init.py:417 in <listcomp>
       415│
       416│         return [
     → 417│             parse_dependency_specification(
       418│                 requirement=requirement,
       419│                 env=self.env if isinstance(self, EnvCommand) else None,

   9  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/dependency_specification.py:216 in parse_dependency_specification
       214│
       215│     specification = (
     → 216│         _parse_dependency_specification_url(requirement, env=env)
       217│         or _parse_dependency_specification_path(requirement, cwd=cwd)
       218│         or _parse_dependency_specification_simple(requirement)

   8  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/dependency_specification.py:68 in _parse_dependency_specification_url
        66│
        67│     if url_parsed.scheme in ["git+https", "git+ssh"]:
     →  68│         return _parse_dependency_specification_git_url(requirement, env)
        69│
        70│     if url_parsed.scheme in ["http", "https"]:

   7  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/dependency_specification.py:49 in _parse_dependency_specification_git_url
        47│
        48│     source_root = env.path.joinpath("src") if env else None
     →  49│     package = Provider.get_package_from_vcs(
        50│         "git",
        51│         url=url.url,

   6  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/puzzle/provider.py:342 in get_package_from_vcs
       340│             raise ValueError(f"Unsupported VCS dependency {vcs}")
       341│
     → 342│         return _get_package_from_git(
       343│             url=url,
       344│             branch=branch,

   5  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/puzzle/provider.py:96 in _get_package_from_git
        94│     source_root: Path | None = None,
        95│ ) -> Package:
     →  96│     source = Git.clone(
        97│         url=url,
        98│         source_root=source_root,

   4  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/vcs/git/backend.py:426 in clone
       424│         try:
       425│             if not cls.is_using_legacy_client():
     → 426│                 local = cls._clone(url=url, refspec=refspec, target=target)
       427│                 cls._clone_submodules(repo=local)
       428│                 return local

   3  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/vcs/git/backend.py:258 in _clone
       256│             local = Repo(str(target))
       257│
     → 258│         remote_refs = cls._fetch_remote_refs(url=url, local=local)
       259│
       260│         logger.debug(

   2  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/vcs/git/backend.py:201 in _fetch_remote_refs
       199│
       200│         with local:
     → 201│             result: FetchPackResult = client.fetch(
       202│                 path,
       203│                 local,

   1  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/dulwich/client.py:824 in fetch
        822│             f, commit, abort = target.object_store.add_pack()
        823│         try:
     →  824│             result = self.fetch_pack(
        825│                 path,
        826│                 determine_wants,

  HangupException

  ssh: Could not resolve hostname https: Name or service not known

  at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/dulwich/client.py:1123 in fetch_pack
      1119│         with proto:
      1120│             try:
      1121│                 refs, server_capabilities = read_pkt_refs(proto.read_pkt_seq())
      1122│             except HangupException:
    → 1123│                 raise _remote_error_from_stderr(stderr)
      1124│             (
      1125│                 negotiated_capabilities,
      1126│                 symrefs,
      1127│                 agent,

This works fine, however:

[url "ssh://git@github.com/"]
        insteadOf = https://github.com/

[dimbleby]

You'll want to report this at dulwich

Meanwhile - per the docs - there's an escape hatch https://python-poetry.org/docs/configuration/#experimentalsystem-git-client

[jelmer]

This is now fixed in dulwich master FWIW.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.