WIP autoproj versioning functionality #28
Conversation
|
NOTE: you'll need to apply the misc_cleanup branch on your autobuild checkout |
|
While there probably are bugs, the state of the code as-is seem fine to me and I have not experienced any problem in the day-to-day operations (provided the latest commit is applied ... which fixes usage of overrides.yml). I'd like to merge this in master. @jakobs: what do you think ? |
|
I don't have a problem in general with working on master. However, I am currently a bit confused to the state of the branches. There are a lot of different branches for e.g. autobuild. Which one is the right one to use here? Also, it would be nice to discuss and describe the functionality of the new commands in a wiki. |
|
Yeah ... Right. Fair enough. So, this currently requires rock-core/autobuild#14. As for the wiki, just created this https://rock.opendfki.de/wiki/WikiStart/OngoingWork/RockFlavorsOrReleases. I thought I published it on the rock-dev ML but it seems that I did not. I'm going to send an email there. |
|
and rebased on current master |
|
autoproj commit does not work with bash and sh: |
|
I also had another bug related to the cachinfo. Created a pull request for that (#33) |
That acd is not available on bash and sh would be a bug in itself. I have some bash machines here where it works, though, so I don't know what happens. In any case, autoproj commit should definitely NOT depend on acd being present. |
|
I've rebased the branch on the new master (I've merged some smaller PRs that were sitting around for a month or so). I had also to rebase the improve_handling_of_detached_head branch on autobuild. These two branches are all that we need now |
expand_path does not like given nil, which leads to an error before the actual error message can be displayed.
this is to reflect that an autoproj command does not necessarily need to be a build command.
before, the snapshot functionality was used, but this is often too much, just to recover the version of the package. Changed the snapshot to use the version method.
similar to snapshot, this will return a list of packages and their respective version information. Unlike snapshot it does not actually snapshot any files.
mainly for testing
The files need to be placed in an overrides folder. This only applies to the LocalPackageSet. The old overrides.yml is still used at the end of the overrides chain.
still not what it should be. missing is handling of package sets.
Given that the most common operation is to merge two full snapshots, it makes sense IMO
In addition, the dir is renamed to overrides.d to match the current pattern on Linux tooling (sudoers.d, system.d, ...)
This commit separates the version generation from the package sets from the one from packages (as most probably users will want to control this). Moreover, it separates the work previously done by #commit into the version generation (snapshot_*) and saving the new versions (#versions) The renaming of #versions into #snapshot_* is by consistency with the Autobuild::Package API in which the method is called snapshot.
The separation between 'version_control' and 'overrides' was done in autoproj-snapshot because it was overwriting the overrides.yml file. Since we don't have such problem with the files in overrides.d/, we only need to define overrides (which greatly simplifies things). In addition, it is now assumed that the files in overrides.d only contains overrides information, and remove the toplevel hash. The 'overrides' field in autoproj/overrides.yml is still parsed as-is. snapshot_package_sets now uses the package set's importer repository ID (e.g. github:rock-core/package_set) as name.
…ge_sets identical Package set snapshotting was until now adding the whole VCS definition while the package snapshot was only adding the pinning information. Make snapshot_package_set have the same behaviour than snapshot_packages
This is to make clear what they actually are. The repository ID by itself is confusing
This is to ensure that it properly overrides all version info set in the package sets (and thus ensure that the commit is available)
This ensures that we can easily compare two versions files from different point in time.
in case ruby version is less than 2.1.0, the cachinfo parameters have to presented as individual arguments. Passing the array directly resulted in a bad syntax for the git command line.
acd is a shell function, which might not be present. Since we only need to go to the installation root, using autoproj locate directly is good enough
The user would be shown the raw error before. He now gets a more descriptive message as well as in which file the error is.
Ruby 1.9's YAML engine saves non-UTF8 strings as !binary and then uuencodes them (as yaml requires utf8). 2.0+ seems to be nicer and checks if the string is a valid utf8 string before doing that. Encode all snapshot strings to utf8 before feeding them to YAML.
pkg here is just a string
This tests for exclusion even before we import the package, as the autobuild code might already contain relevant information (e.g. build tool, package type, ...)
WIP autoproj versioning functionality
IMPORTANT: this requires the misc_cleanup branch of autobuild to be present. I'm trying to get it reviewed and merged bit-by-bit ...
This pull request implements the support in autoproj for versioning, that is the ability to replicate an installation 1:1 without having to fear for changing software or package sets.
The following subcommands are planned: