Next: investigate gringo/aspcud performances

[Drup]

% gringo --version
gringo version 4.5.4
Address model: 64-bit
Configuration: with Python 3.5.1, with Lua 5.3.2
% aspcud --version
aspcud version 1.9.

With opam 2.0, gringo/aspcud takes much longer than it used to do. I'm not sure if it's a regression in gringo (that I upgraded not-so-long ago) or the new compiler-as-package that is much more stressful for the checker, but it's probably worth investigating either way.

I don't have a specific query to propose, it's rather systematic. Trying to upgrade the compiler (in particular in a switch where it's not authorized) seemed much worse.

[Drup]

Also, it tends to take indecent amounts of RAM, up to 2Go.

[AltGr]

Yes, I have seen that as well and started investigating; the main cost (~80%) seems to be due simply to the too many versions of ocaml -- you can attempt to see the difference by doing opam pin ocaml <cur-version>, which will make all other versions invisible to the solver. I intend to attempt some optimisations by e.g. removing all other versions of the compiler packages in general when they are not needed (the effect should be roughly the same as the `Keep_version flag that we pass through cudf, but which doesn't seem to help).

There is also a huge cost on opam list --rec --required-by, when building the dependency graph -- but this is related to dose, not to the solver.

If you're interested, you can also try with mccs or packup, which should be now much better supported (http://opam.ocaml.org/doc/Install.html#ExternalSolvers)

[Drup]

I'm going to try packup and report if it's any better.

An alternative solution, which is related to another feature request, is to never upgrade the compiler unless explicitly asked for. We could mark a package as not to be upgraded (that would be for compilers, but also for coq and other very costly package to recompile). A bit like pinning a version, but automatic, with easier upgrade and a UX that tells you when you can upgrade (but doesn't do it).

[AltGr]

An alternative solution, which is related to another feature request, is to never upgrade the compiler unless explicitly asked for. We could mark a package as not to be upgraded (that would be for compilers, but also for coq and other very costly package to recompile). A bit like pinning a version, but automatic, with easier upgrade and a UX that tells you when you can upgrade (but doesn't do it).

That's actually what's currently done (explicit changes will be refused unless the package is pinned; and the solver is given a Keep_version flag that has this semantics). It's thecompiler:field in~/.opam/SWITCH/.opam-switch/opam-state`. Opam will also try to reinstall a base package that got removed (or couldn't get installed) for some reason.

[Drup]

Ah, right, so it's indeed purely a solver issue then as you said.

[AltGr]

Some thoughts on this: the fact that only a single version of a given package can be installed at a time is expressed in cudf through a self-conflict: in opam, we always have

package: foo
conflicts: foo

Conflicts are, as I understand it, what makes the raw problem difficult to solve. And given how it's encoded in cudf, I suspect that the solvers don't have special handling for self-conflicts (it's OK in debian because they never have many conflicting versions). With the ocaml package and its 160 versions, I think we get a clue about the combinatory explosion. There may also be inefficiencies in the way the fact that conflicts are symmetric (but their declarations needn't be) is handled.

cc @rdicosmo @abate

[abate]

the problem here, I think, is related to the growing number of versions of each package/compiler present in an opam repository. Aspcud was created to efficiently solve problems where the number of available versions for each package is small or one (as in software distribution a la debian). As the number of different versions for each package grows, aspcud, that is not optimized for these kind of problems, will start to consume more memory and it performances might degrade. Reducing the number of available version might work in the mean time, but this is just a hack. Setting up a QA policy to regularly cleaning up the repository, removing old, broken packages could be a more viable long term strategy.

On our side, we should work with the aspcud people to adapt the model to this class of problems, or many using a different solver. mccs could be a good candidate. I discourage the use of pack-up as it is known to provide wrong answers from time to time.

[AltGr]

I have indeed experienced with mccs for a while, and it's not as bad as aspcud on our current case, but still takes too long. You are right about cleaning the repository and encouraging the solvers to get better optimised for us, but on the shorter term, we need to avoid having 160 versions of ocaml concurrently available when possible. Putting all the dev versions in a separate repository could also be a very easy short-term fix, if acceptable.

[abate]

Instead of creating a different repository, maybe it might be worth exploring the possibility of filtering the packages versions that you pass to the solver. This could be configured by the user. For example, by default, only the latest 3 stable compiler releases are considered and only the last 3 major stable releases for each package. This would greatly speed up the solver without too much work.

[let-def]

Still suffering from the issue.
I am doing an opam upgrade after migration to opam 2.0.
gringo is using 4.6 GiB of memory, more than my computer has :).

[rdicosmo]

Hi Frederic,
package installation is an NP complete problem, so it is pretty well possible
to hit explosive configurations.

As a (very rough) rule of thumb, the more conflicts and disjunctions are present
in the universe passed to aspcud, the more likely it is to step on a complexity
bomb.

Aspcud was designed to work well on Debian-like instances where one has 2/3
versions of each package maximum... working with dozens of versions of packages
is unchartered territory and would require more (theoretical as well as
practical) investigation

Roberto

On Thu, Nov 17, 2016 at 01:52:06AM -0800, Frédéric Bour wrote:

Still suffering from the issue.
I am doing an opam upgrade after migration to opam 2.0.
gringo is using 4.6 GiB of memory, more than my computer has :).

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.*

Roberto

[let-def]

@rdicosmo thanks for your answer.
Note that this is not a complain. Opam 2.0 is not released yet, I choose to try the beta version and report an issue which makes it unusable on my setup.
And my test was not completely useless as @AltGr was expecting long time, but not that much in memory apparently.

PS 1: I remember seeing you talking about the mancoosi project :)

[rdicosmo]

Hi Frederic, no problem at all : as you know, I'm now fully busy on
Software Heritage, and as much as I would be delighted to look into the
issue in detail, I'm out of available time... so I'm just trying to convey
some high level information hoping it may be useful to those who have the
time to look into this ... :-)

2016-11-17 22:07 GMT+01:00 Frédéric Bour notifications@github.com:

@rdicosmo https://github.com/rdicosmo thanks for your answer.
Note that this is not a complain. Opam 2.0 is not released yet, I choose
to try the beta version and report an issue which makes it unusable on my
setup.
And my test was not completely useless as @AltGr
https://github.com/AltGr was expecting long time, but not that much in
memory apparently.

PS 1: I remember seeing you talking about the mancoosi project :)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#2534 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAp-v7chw_3LbFFVT9w9FbVee01XdQrVks5q_MH0gaJpZM4IP4fC
.

Roberto Di Cosmo

---

Professeur (on leave at/detache a INRIA Roquencourt)
IRIF email : roberto@dicosmo.org
Universite Paris Diderot web : http://www.dicosmo.org
Case 7014 Twitter : http://twitter.com/rdicosmo
5, Rue Thomas Mann

F-75205 Paris Cedex 13 FRANCE

Office location:

Paris Diderot INRIA

Bureau 3020 (3rd floor) Bureau C123
Batiment Sophie Germain Batiment C
8 place Aurélie Nemours 2, Rue Simone Iff
Tel: +33 1 57 27 92 20 Tel: +33 1 80 49 44 42

Metro
Bibliotheque F. Mitterrand Ligne 6: Dugommier

ligne 14/RER C Ligne 14/RER A: Gare de Lyon

GPG fingerprint 2931 20CE 3A5A 5390 98EC 8BFC FCCA C3BE 39CB 12D3

[rdicosmo]

By the way, it would be very useful to keep a copy of the CUDF files corresponding to the explosive cases: these will make fantastic benchmarks for the next solver competition. Indeed, for the case of Debian packages we have solvers that behave well and we stopped running solver comptetitions, but the Opam use case is different and may require to revamp the compeitions if there are enough new challenging benchmarks.

[AltGr]

@rdicosmo good point; do you have a preferred place where they should be stored, or should we mail them directly to you ?

[rdicosmo]

@AltGr: just create a github repo, and add the .cudf file there (ideally, also
add a .txt file with any extra relevant explanations) :-)

On Mon, Nov 21, 2016 at 12:58:37AM -0800, Louis Gesbert wrote:

@rdicosmo good point; do you have a preferred place where they should be
stored, or should we mail them directly to you ?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.*

Roberto Di Cosmo

---

Professeur (on leave at/detache a INRIA Roquencourt)
IRIF E-mail : roberto@dicosmo.org
Universite Paris Diderot Web : http://www.dicosmo.org
Case 7014 Twitter : http://twitter.com/rdicosmo
5, Rue Thomas Mann

F-75205 Paris Cedex 13 France

Office location:

Paris Diderot INRIA

Bureau 3020 (3rd floor) Bureau C123
Batiment Sophie Germain Batiment C
8 place Aurélie Nemours 2, Rue Simone Iff
Tel: +33 1 57 27 92 20 Tel: +33 1 80 49 44 42

Metro
Bibliotheque F. Mitterrand Ligne 6: Dugommier

ligne 14/RER C Ligne 14/RER A: Gare de Lyon

GPG fingerprint 2931 20CE 3A5A 5390 98EC 8BFC FCCA C3BE 39CB 12D3

[Drup]

@AltGr The performance is still not fabulous, in particular in switches where switch set-base was not called.

[RalfJung]

the problem here, I think, is related to the growing number of versions of each package/compiler present in an opam repository.

Ouch. We've been suffering from extremely slow opam performance (30s and more to compute a solution) more and more recently (this is all on opam 1.2.2), and this would explain it. To work around opam's lack of dependencies on a particular git commit (#2762), we automatically publish every single commit (actually: every single head that gets pushed and passes CI) of our project into an opam repo. So, we have lots of versions -- probably close to 100 by now.

Not that this should make any difficulty, the dependency is always of the form "I want this exact version". But somehow the solver doesn't seem smart enough to use that to its benefit?

I haven't tried opam 2.0 yet; does this issue being closed mean that performance has significantly improved?

[bobpoekert]

I just started having this problem. running opam install is causing aspcud/gringo to use all of my 4Gb of memory and then die

[GoPavel]

I just started having this problem. running opam install is causing aspcud/gringo to use all of my 4Gb of memory and then die

You can try another solver. For instance, mccs helped me, while aspcud consumes all of my 12Gb.

p. s.
my invironment:

$ uname -rsomv
Linux 5.0.0-27-generic #28-Ubuntu SMP Tue Aug 20 19:53:07 UTC 2019 x86_64 GNU/Linux
$ opam --version
2.0.3
$ aspcud -v
aspcud version 1.9.4

[AltGr]

@bobpoekert what version of opam is that on ?

[bobpoekert]

@AltGr 2.0.3