Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix some RPM registry flaws #28782

Merged
merged 3 commits into from
Jan 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 35 additions & 14 deletions docs/content/usage/packages/rpm.en-us.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,24 +24,34 @@ The following examples use `dnf`.

## Configuring the package registry

To register the RPM registry add the url to the list of known apt sources:
To register the RPM registry add the url to the list of known sources:

```shell
dnf config-manager --add-repo https://gitea.example.com/api/packages/{owner}/rpm/{group}.repo
```

| Placeholder | Description |
| ----------- |----------------------------------------------------|
| `owner` | The owner of the package. |
| `group` | Everything, e.g. `el7`, `rocky/el9` , `test/fc38`.|
| Placeholder | Description |
| ----------- | ----------- |
| `owner` | The owner of the package. |
| `group` | Optional: Everything, e.g. empty, `el7`, `rocky/el9`, `test/fc38`. |

Example:

```shell
# without a group
dnf config-manager --add-repo https://gitea.example.com/api/packages/testuser/rpm.repo

# with the group 'centos/el7'
dnf config-manager --add-repo https://gitea.example.com/api/packages/testuser/rpm/centos/el7.repo
```

If the registry is private, provide credentials in the url. You can use a password or a [personal access token](development/api-usage.md#authentication):

```shell
dnf config-manager --add-repo https://{username}:{your_password_or_token}@gitea.example.com/api/packages/{owner}/rpm/{group}.repo
```

You have to add the credentials to the urls in the `rpm.repo` file in `/etc/yum.repos.d` too.
You have to add the credentials to the urls in the created `.repo` file in `/etc/yum.repos.d` too.

## Publish a package

Expand All @@ -54,11 +64,17 @@ PUT https://gitea.example.com/api/packages/{owner}/rpm/{group}/upload
| Parameter | Description |
| --------- | ----------- |
| `owner` | The owner of the package. |
| `group` | Everything, e.g. `el7`, `rocky/el9` , `test/fc38`.|
| `group` | Optional: Everything, e.g. empty, `el7`, `rocky/el9`, `test/fc38`. |

Example request using HTTP Basic authentication:

```shell
# without a group
curl --user your_username:your_password_or_token \
--upload-file path/to/file.rpm \
https://gitea.example.com/api/packages/testuser/rpm/upload

# with the group 'centos/el7'
curl --user your_username:your_password_or_token \
--upload-file path/to/file.rpm \
https://gitea.example.com/api/packages/testuser/rpm/centos/el7/upload
Expand All @@ -83,17 +99,22 @@ To delete an RPM package perform a HTTP DELETE operation. This will delete the p
DELETE https://gitea.example.com/api/packages/{owner}/rpm/{group}/package/{package_name}/{package_version}/{architecture}
```

| Parameter | Description |
|-------------------|----------------------------|
| `owner` | The owner of the package. |
| `group` | The package group . |
| `package_name` | The package name. |
| `package_version` | The package version. |
| `architecture` | The package architecture. |
| Parameter | Description |
| ----------------- | ----------- |
| `owner` | The owner of the package. |
| `group` | Optional: The package group. |
| `package_name` | The package name. |
| `package_version` | The package version. |
| `architecture` | The package architecture. |

Example request using HTTP Basic authentication:

```shell
# without a group
curl --user your_username:your_token_or_password -X DELETE \
https://gitea.example.com/api/packages/testuser/rpm/package/test-package/1.0.0/x86_64

# with the group 'centos/el7'
curl --user your_username:your_token_or_password -X DELETE \
https://gitea.example.com/api/packages/testuser/rpm/centos/el7/package/test-package/1.0.0/x86_64
```
Expand Down
14 changes: 7 additions & 7 deletions models/packages/package.go
Original file line number Diff line number Diff line change
Expand Up @@ -191,18 +191,18 @@ type Package struct {
func TryInsertPackage(ctx context.Context, p *Package) (*Package, error) {
e := db.GetEngine(ctx)

key := &Package{
OwnerID: p.OwnerID,
Type: p.Type,
LowerName: p.LowerName,
}
existing := &Package{}

has, err := e.Get(key)
has, err := e.Where(builder.Eq{
"owner_id": p.OwnerID,
"type": p.Type,
"lower_name": p.LowerName,
}).Get(existing)
if err != nil {
return nil, err
}
if has {
return key, ErrDuplicatePackage
return existing, ErrDuplicatePackage
}
if _, err = e.Insert(p); err != nil {
return nil, err
Expand Down
12 changes: 10 additions & 2 deletions models/packages/package_blob.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,12 +41,20 @@ type PackageBlob struct {
func GetOrInsertBlob(ctx context.Context, pb *PackageBlob) (*PackageBlob, bool, error) {
e := db.GetEngine(ctx)

has, err := e.Get(pb)
existing := &PackageBlob{}

has, err := e.Where(builder.Eq{
"size": pb.Size,
"hash_md5": pb.HashMD5,
"hash_sha1": pb.HashSHA1,
"hash_sha256": pb.HashSHA256,
"hash_sha512": pb.HashSHA512,
}).Get(existing)
if err != nil {
return nil, false, err
}
if has {
return pb, true, nil
return existing, true, nil
}
if _, err = e.Insert(pb); err != nil {
return nil, false, err
Expand Down
26 changes: 13 additions & 13 deletions models/packages/package_file.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,18 +46,18 @@ type PackageFile struct {
func TryInsertFile(ctx context.Context, pf *PackageFile) (*PackageFile, error) {
e := db.GetEngine(ctx)

key := &PackageFile{
VersionID: pf.VersionID,
LowerName: pf.LowerName,
CompositeKey: pf.CompositeKey,
}
existing := &PackageFile{}

has, err := e.Get(key)
has, err := e.Where(builder.Eq{
"version_id": pf.VersionID,
"lower_name": pf.LowerName,
"composite_key": pf.CompositeKey,
}).Get(existing)
if err != nil {
return nil, err
}
if has {
return pf, ErrDuplicatePackageFile
return existing, ErrDuplicatePackageFile
}
if _, err = e.Insert(pf); err != nil {
return nil, err
Expand Down Expand Up @@ -93,13 +93,13 @@ func GetFileForVersionByName(ctx context.Context, versionID int64, name, key str
return nil, ErrPackageFileNotExist
}

pf := &PackageFile{
VersionID: versionID,
LowerName: strings.ToLower(name),
CompositeKey: key,
}
pf := &PackageFile{}

has, err := db.GetEngine(ctx).Get(pf)
has, err := db.GetEngine(ctx).Where(builder.Eq{
"version_id": versionID,
"lower_name": strings.ToLower(name),
"composite_key": key,
}).Get(pf)
if err != nil {
return nil, err
}
Expand Down
12 changes: 6 additions & 6 deletions models/packages/package_version.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,17 +39,17 @@ type PackageVersion struct {
func GetOrInsertVersion(ctx context.Context, pv *PackageVersion) (*PackageVersion, error) {
e := db.GetEngine(ctx)

key := &PackageVersion{
PackageID: pv.PackageID,
LowerVersion: pv.LowerVersion,
}
existing := &PackageVersion{}

has, err := e.Get(key)
has, err := e.Where(builder.Eq{
"package_id": pv.PackageID,
"lower_version": pv.LowerVersion,
}).Get(existing)
if err != nil {
return nil, err
}
if has {
return key, ErrDuplicatePackageVersion
return existing, ErrDuplicatePackageVersion
}
if _, err = e.Insert(pv); err != nil {
return nil, err
Expand Down
23 changes: 23 additions & 0 deletions models/packages/rpm/search.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
// Copyright 2024 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT

package rpm

import (
"context"

packages_model "code.gitea.io/gitea/models/packages"
rpm_module "code.gitea.io/gitea/modules/packages/rpm"
)

// GetGroups gets all available groups
func GetGroups(ctx context.Context, ownerID int64) ([]string, error) {
return packages_model.GetDistinctPropertyValues(
ctx,
packages_model.TypeRpm,
ownerID,
packages_model.PropertyTypeFile,
rpm_module.PropertyGroup,
nil,
)
}
5 changes: 4 additions & 1 deletion modules/packages/rpm/metadata.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,10 @@ import (
)

const (
PropertyMetadata = "rpm.metadata"
PropertyMetadata = "rpm.metadata"
PropertyGroup = "rpm.group"
PropertyArchitecture = "rpm.architecture"

SettingKeyPrivate = "rpm.key.private"
SettingKeyPublic = "rpm.key.public"

Expand Down
5 changes: 0 additions & 5 deletions modules/templates/util_string.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
package templates

import (
"regexp"
"strings"

"code.gitea.io/gitea/modules/base"
Expand All @@ -26,10 +25,6 @@ func (su *StringUtils) Contains(s, substr string) bool {
return strings.Contains(s, substr)
}

func (su *StringUtils) ReplaceAllStringRegex(s, regex, new string) string {
return regexp.MustCompile(regex).ReplaceAllString(s, new)
}

func (su *StringUtils) Split(s, sep string) []string {
return strings.Split(s, sep)
}
Expand Down
8 changes: 8 additions & 0 deletions modules/util/slice.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
package util

import (
"cmp"
"slices"
"strings"
)
Expand Down Expand Up @@ -45,3 +46,10 @@ func SliceSortedEqual[T comparable](s1, s2 []T) bool {
func SliceRemoveAll[T comparable](slice []T, target T) []T {
return slices.DeleteFunc(slice, func(t T) bool { return t == target })
}

// Sorted returns the sorted slice
// Note: The parameter is sorted inline.
func Sorted[S ~[]E, E cmp.Ordered](values S) S {
slices.Sort(values)
return values
}
3 changes: 3 additions & 0 deletions options/locale/locale_en-US.ini
Original file line number Diff line number Diff line change
Expand Up @@ -3414,6 +3414,9 @@ rpm.registry = Setup this registry from the command line:
rpm.distros.redhat = on RedHat based distributions
rpm.distros.suse = on SUSE based distributions
rpm.install = To install the package, run the following command:
rpm.repository = Repository Info
rpm.repository.architectures = Architectures
rpm.repository.multiple_groups = This package is available in multiple groups.
rubygems.install = To install the package using gem, run the following command:
rubygems.install2 = or add it to the Gemfile:
rubygems.dependencies.runtime = Runtime Dependencies
Expand Down
Loading