Support repo license

[yp05327]

Close #278
Close #24076

Solutions:

• Use google/licenseclassifier
  Test result between google/licensecheck and go-license-detector:
  Support repo license #24872 (comment)
  Test result between google/licensecheck and google/licenseclassifier:
  Support repo license #24872 (comment)
• Generate License Convert Name List to avoid import license templates with same contents
  Gitea automatically get latest license data from spdx/license-list-data.
  But unfortunately, some license templates have same contents. Duplicate license options with identical contents #20915 click here to see the list
  So we will generate a list of these license templates with same contents and create a new file to save the result when using make generate-license. (Need to decide the save path)
• Save License info into a new table repo_license
  Can easily support searching repo by license in the future.

Screen shot

Single License:

Multiple Licenses:

Triggers:

• Push commit to default branch
• Create repo
• Mirror repo
• When Default Branch is changed, licenses should be updated

Todo:

• Save Licenses info in to DB when there's a change to license file in the commit
• DB Migration
• A nominal test?
• Select which library to use(Support repo license #24872 (comment))
• API Support
• Add repo license table
• Select license in settings if there are several licenses(Not recommended)
• License board(later, not in this PR)
  

[silverwind]

Did you compare https://github.com/go-enry/go-license-detector vs. https://github.com/google/licensecheck?

[JakobDev]

The last commit on google/licensecheck is from one Year ago, so it looks like it is not developed anymore. I would suggest using go-enry/go-license-detector.

[silverwind]

The last commit on google/licensecheck is from one Year ago, so it looks like it is not developed anymore. I would suggest using go-enry/go-license-detector.

They should be compared by more than just that, ideally by feeding them a corpus of oddly licensed repos. One thing i'd be interested in if any of the two can detect license field in package.json for example.

[silverwind]

Select license in settings if there are several licenses?

I wouldn't. License detection should be fully automatic and not overrideable, so that the git repo remains portable and is not tied to gitea-specific license override that can not be migrated.

Save Licenses info in to DB when there's a change to license file in the commit?

Some form of cache would be nice to not calculate license every render, but instead only on push to the branch.

License board?

Seems nice but I'd say something for later, and only if we can outsource the license metadata.

[techknowlogick]

Thanks for this PR <3

Could you add at least a nominal test, as while upstream is tested we should ensure that our use of the library is acceptable.

I do think the storage of information in the database is a good idea as that's what we do with language stats.

As for which library we use, there are pros/cons for both. The google one is used by pkg.go.dev, and for enry we already use their library for language stats. Although I suspect that they both are somewhat close to getting what the actual license issue. so I think unless there is some significant performance impact or one library is wildly inaccurate either is fine.

[yp05327]

I tried the CLI tool of go-license-detector.

There are two problems:

• Performance problem
  go-license-detector can automatically search license files which seems good,
  but if an unrelated big size file's name starts with license, it will takes lots of time to get the result.
  For example, in this folder:
  
  Run ./license-detector ./1 is very fast, but ./license-detector ./ will take a lot of time.
  
  
  (It is too slow, so I canceled the process)

  And, for google/licensecheck, it only takes about 10s to finish the detection.
  
  So it seems that google/licensecheck has higher performance.
  (Is it caused by different IO process? The test of license-detector used it's own IO process, the test of licensecheck used charset? Not sure about this now.)
  Test code:
  
  Test Result:
  
  (Stopped at licensedb.InvestigateLicenseText for long time)

• Detect multi licenses.
  It seems that go-license-detector can not detect multi licenses in one file.
  In https://github.com/yp05327/test, there are two licenses in LICENSE
  In https://github.com/yp05327/test2, there is only one license in LICENSE
  but the result is same:
  
  
  google/licensecheck can detect all of them.
  

dimiss changr request