Feature: expose repo as composer package.

[jeppech]

I've seen over at GitLab that someone is in the making of Composer Service, that makes it possible to expose a repository, as a Composer package for PHP.

That would be a pretty cool feature, if you'd like to host your own private packages. Atm, it's possible to have a static package repo, by using the static repository generator Satis.

Furthermore, support for npm packages or ruby gems, could also be possible.

Actually when I'm thinking about it, these features might not belong to the Gogs core, but maybe as some sort of plugins/add-ons, as mentioned in #2438

[unknwon]

Indeed, sounds like a good use case of #2438.

BTW, how does composer package of PHP work? Never used PHP... sorry.

[jeppech]

Heh, no need to be sorry :)

Basically every, project kan be a composer package. It just need to have composer.json file in the root folder structure. The composer file, describes the project and its dependencies. The very basics for requiring some dependencies, could be something like this:

{
    "repositories": [
        {
            "type": "composer",
            "url": "http://try.gogs.io"
        }
    ],
    "require": {
        "monolog/monolog": "~1.17.0"
    }
}

To install the above dependency, you'll type composer install, and the Composer CLI tool, will look for a package.json in the root of http://try.gogs.io, this file tells which packages are available and where they're located. A very basic version of this could be

{
    "packages": {
        "monolog/monolog": {
            "v1.17.0": {
                "name": "monolog/monolog",
                "version": "v1.17.0",
                "version_normalized": "1.17.0.0",
                "source": {
                    "type": "git",
                    "url": "http://try.gogs.io/monolog/monolog",
                    "reference": "d7c2fa6f5aa9ac73724c234c62f7f2d83035a7bd"
                },
                "dist": {
                    "type": "tar",
                    "url": "http://try.gogs.io/monolog/monolog/archive/v1.17.0.tar.gz"
                },
                "time": "2015-02-10 19:36:36",
                "type": "library"
            }
        }
    }
}

Actually at this point, I'm kinda uncertain how this file is generated, because the official repo, Packagist, has a search key with a value of /search.json?q=%query% in their packages.json which is undocumented.

What I've found so far, is that if you for instance search for monolog https://packagist.org/search.json?q=monolog it returns a list of packages i.e

{
    "results": [
        {
            "name": "monolog/monolog",
            "description": "Sends your logs to files, sockets, inboxes, databases and various web services",
            "url": "https://packagist.org/packages/monolog/monolog",
            "repository": "https://github.com/Seldaek/monolog",
            "downloads": 22744596,
            "favers": 4443
        }
    ]
}

This result set, contains a key url, which points to the location of the package in packagist. And I noticed if you prepend .json to that url like this https://packagist.org/packages/monolog/monolog.json it actually shows, what would correspond to an entry of the before mentioned packages.json

Further reading for the packages.json can be found here

[bkcsoft]

This would indeed be a valid use-case for plugins. The /search.json you mention is (at least) referenced here https://packagist.org/apidoc . Though it isn't much help 😒

[jeppech]

I think I figured out what it's used for, when executing composer -vvv search monolog/monolog you can see that the composer app actually looks up /search.json?q=monolog/monolog, to retrieve some info:

jc@mbp $ composer -vvv search monolog/monolog
You are running composer with xdebug enabled. This has a major impact on runtime performance. See https://getcomposer.org/xdebug
Loading config file /Users/jc/.composer/auth.json
No composer.json found in the current directory, showing packages from packagist
Downloading https://packagist.org/packages.json
Writing /Users/jc/.composer/cache/repo/https---packagist.org/packages.json into cache
Downloading https://packagist.org/search.json?q=monolog/monolog
monolog/monolog Sends your logs to files, sockets, inboxes, databases and various web services
monolog/monolog
kdyby/monolog Integration of Monolog into Nette Framework
stackify/monolog Stackify logs and errors for Monolog
thinframe/monolog Monolog Application
cakephp/monolog CakePHP Monolog Plugin
flowpack/monolog Monolog integration for Flow
symfony/monolog-bundle Symfony MonologBundle
symfony/monolog-bridge Symfony Monolog Bridge
kamisama/monolog-init Very basic and light Dependency Injector Container for Monolog
logentries/logentries-monolog-handler A handler for Monolog that sends messages to Logentries.com.
m6web/monolog-extra-bundle Provide extra features for Monolog
wazaari/monolog-mysql A handler for Monolog that sends messages to MySQL
enlitepro/enlite-monolog Monolog integration to Zend Framework 2
flynsarmy/slim-monolog Monolog logging support Slim Framework

But when requiring the actual package it does not use the search method

jc@mbp $ composer -vvv search monolog/monolog
Reading ./composer.json
Loading config file /Users/jc/.composer/auth.json
Loading config file ./composer.json
Executing command (/Users/jc): git describe --exact-match --tags
Executing command (/Users/jc): git branch --no-color --no-abbrev -v
Executing command (/Users/jc): hg branch
Executing command (/Users/jc): svn info --xml
Failed to initialize global composer: Composer could not find the config file: /Users/jc/.composer/composer.json
To initialize a project, please create a composer.json file as described in the https://getcomposer.org/ "Getting Started" section
Downloading https://packagist.org/packages.json
Writing /Users/jc/.composer/cache/repo/https---packagist.org/packages.json into cache
Downloading http://packagist.org/p/provider-2013%24ff90a5caee747457233c7927b1a81797850de6779661815ac4c7663b3b7d6eda.json
Writing /Users/jc/.composer/cache/repo/https---packagist.org/p-provider-2013.json into cache
Downloading http://packagist.org/p/provider-2014%248776bf44e4162545d15610ae7b6d085218b489d66f5192a8725a59a3bab9f13a.json
Writing /Users/jc/.composer/cache/repo/https---packagist.org/p-provider-2014.json into cache
Downloading http://packagist.org/p/provider-2015%24353f1f241024bc840a505bc1281aee94ae0d7953a8e8e2cab24154d06f996cf0.json
Writing /Users/jc/.composer/cache/repo/https---packagist.org/p-provider-2015.json into cache
Downloading http://packagist.org/p/provider-2015-04%24d238240864b8325773ac496b3daaf9bc4481376eb2d5d55715fdb9ee50b4c8f8.json
Writing /Users/jc/.composer/cache/repo/https---packagist.org/p-provider-2015-04.json into cache
Downloading http://packagist.org/p/provider-2015-07%24d295bda30c49ee368869484582a264579d1a7f33095651b999b5d599277967bd.json
Writing /Users/jc/.composer/cache/repo/https---packagist.org/p-provider-2015-07.json into cache
Downloading http://packagist.org/p/provider-2015-10%243cc3c0be7fab051132a873b60ffb687f3eb403a63058a893f087295dc3599552.json
Writing /Users/jc/.composer/cache/repo/https---packagist.org/p-provider-2015-10.json into cache
Downloading http://packagist.org/p/provider-2016-01%24fe5a870932496d05a6cf523b842f9497898ba9cad673d10d6e6a7330392cc9fd.json
Writing /Users/jc/.composer/cache/repo/https---packagist.org/p-provider-2016-01.json into cache
Downloading http://packagist.org/p/provider-archived%24a1dda04e44e120babb82f2449ab01bc68a2a2bc0b1483498b4959bf1305e2b4b.json
Writing /Users/jc/.composer/cache/repo/https---packagist.org/p-provider-archived.json into cache
Downloading http://packagist.org/p/provider-latest%24e32f2df453895917a4dc11b74275f18b62255230d0a20464b50f87dc44dfcd81.json
Writing /Users/jc/.composer/cache/repo/https---packagist.org/p-provider-latest.json into cache
Downloading http://packagist.org/p/monolog/monolog%24590f46ce86d4f72467a792cf3a4fadcad266eb527ac9aabf9cb2048737fd07b2.json
Writing /Users/jc/.composer/cache/repo/https---packagist.org/provider-monolog$monolog.json into cache
Using version ^1.17 for monolog/monolog
./composer.json has been updated

You'll see that it downloads all those references it can find in the packages.json, and probably parsing them until it finds a match.

[unknwon]

Thanks for the great details!

So... what exactly is missing from Gogs side?

[unknwon]

ping @jeppech

[jeppech]

Well, I think this should be implemented as a plugin. So this might, should be put on hold, until the plugin module for Gogs is ready?

[unknwon]

@jeppech yes. But hmm, from my zero knowledge of PHP, what is missing from Gogs side right now?

[jeppech]

Oh, like that. Well, you don't have to know any PHP at all, this is just a package manager for PHP. I don't see Gogs missing anything, when speaking of the composer integration, besides the plugin system of course.

As of now, I'm not very familiar with the composer internal system, so I'll have to read up on that. I'm unsure whether the composer system should be accessible to Gogs, in order to make the 'package.json', or if we can generate that our self.

So, for now, nothing further is needed in Gogs, besides the plugin module.

[unknwon]

I'm getting more confused....