sage-loaded libraries are slowing down cypari2 a lot (on x86_64 glibc) #34850

videlec · 2022-12-14T20:10:39Z

import timeit

prog="""
from cypari2 import Pari
pari = Pari(size={size})
pari("{cmd}")
"""

prog = prog.format(size=2**24, cmd="quadclassunit(1 - 2^100)")

print("python environment with cypari2:          ",
      timeit.timeit(prog, number=3))

setup="""
from sage.ext.memory import init_memory_functions
init_memory_functions()
"""
print("with sage custom GMP allocation functions:",
      timeit.timeit(prog, setup=setup, number=3))

setup="""
import sage.all
"""
print("with the whole sagemath library lodaded:  ",
      timeit.timeit(prog, setup=setup, number=3))

gives

$ python pari_timing.py 
python environment with cypari2:           4.871978694998688
with sage custom GMP allocation functions: 4.941096214999561
with the whole sagemath library lodaded:   7.39602135700261

See also

this sage-devel thread
a similar problem with pari_jupyter pari-jupyter/#27

CC: @edgarcosta @mkoeppe @JohnCremona @fredrik-johansson @zimmermann6

Component: performance

Issue created by migration from https://trac.sagemath.org/ticket/34850

The text was updated successfully, but these errors were encountered:

dimpase · 2022-12-15T00:09:04Z

comment:3

Could it be just a different memory layout as soon as sage.all is imported, causing the memory consumption of the process to increase about 6-fold ?

mezzarobba · 2022-12-15T10:22:44Z

comment:5

FWIW, I don't see the issue on my system:

python environment with cypari2:           3.9759535739940475
with sage custom GMP allocation functions: 4.250082928003394
with the whole sagemath library lodaded:   3.8536450660030823

(edit: I previously posted a version where I was running the test code under sage, not ipython, but the results above are with ipython)

videlec · 2022-12-15T10:26:05Z

comment:6

Thanks Marc for running the test. What is your configuration? eg what is the system and does sage use system pari?

mezzarobba · 2022-12-15T10:33:35Z

comment:7

I was going to add more details after doit sage -f pari—which is still running at the moment. But the example above is already using sage's pari (I also have a systemwide pari, but it is a different version). This is on Debian sid with an Intel cpu. Note that I am using Python 3.11 via #33842.

mantepse · 2022-12-15T11:30:00Z

comment:8

on ubuntu 22.04.01 LTS:

┌────────────────────────────────────────────────────────────────────┐
│ SageMath version 9.8.beta5, Release Date: 2022-12-11               │
│ Using Python 3.10.6. Type "help()" for help.                       │
└────────────────────────────────────────────────────────────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Warning: this is a prerelease version, and it may be unstable.     ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
sage: import timeit
....: 
....: prog="""
....: from cypari2 import Pari
....: pari = Pari(size={size})
....: pari("{cmd}")
....: """
....: 
....: prog = prog.format(size=2**24, cmd="quadclassunit(1 - 2^100)")
....: 
....: print("python environment with cypari2:          ",
....:       timeit.timeit(prog, number=3))
....: 
....: setup="""
....: from sage.ext.memory import init_memory_functions
....: init_memory_functions()
....: """
....: print("with sage custom GMP allocation functions:",
....:       timeit.timeit(prog, setup=setup, number=3))
....: 
....: setup="""
....: import sage.all
....: """
....: print("with the whole sagemath library lodaded:  ",
....:       timeit.timeit(prog, setup=setup, number=3))
python environment with cypari2:           5.757997545006219
with sage custom GMP allocation functions: 5.887209259002702
with the whole sagemath library lodaded:   5.1330207900027744

dimpase · 2022-12-15T12:01:31Z

comment:9

If you replace

    import sage.all

with

    import sympy

you'll get the same slowdown. Bingo! Sympy probably changes FPU status, it seems, according to
build/pkgs/sympow/email-exchange.txt

mezzarobba · 2022-12-15T12:35:30Z

comment:10

Replying to Dima Pasechnik:

Sympy probably changes FPU status, it seems, according to
build/pkgs/sympow/email-exchange.txt

Did you accidentally look in sympow/ instead of sympy/, or do you mean sympow also has something to do with it?

I (obviously) don't see any slowdown here with import sympy either, even after reinstalling sympy with sage -f.

dimpase · 2022-12-15T12:49:19Z

comment:11

You know that the greatest discoveries are made by accident ;-)

Indeed, I went looking for FPU in sage source repo, and found that sympow thing, which got swapped to sympy in my brain... Anyhow, for me sympy causes the same slowdown:

$ ./sage --python p.py
python environment with cypari2:           4.7912631159997545
with sage custom GMP allocation functions: 4.886824406014057
with sympy loaded:   6.885674357006792
$ cat p.py
import timeit

prog="""
from cypari2 import Pari
pari = Pari(size={size})
pari("{cmd}")
"""

prog = prog.format(size=2**24, cmd="quadclassunit(1 - 2^100)")

print("python environment with cypari2:          ",
      timeit.timeit(prog, number=3))

setup="""
from sage.ext.memory import init_memory_functions
init_memory_functions()
"""
print("with sage custom GMP allocation functions:",
      timeit.timeit(prog, setup=setup, number=3))

setup="""
import sympy
"""
print("with sympy loaded:  ",
      timeit.timeit(prog, setup=setup, number=3))

dimpase · 2022-12-15T13:06:45Z

comment:12

I see this sympy difference on the latest Sage beta on 3 boxes: Fedora 34 with a newish CPU, and on Gentoo with a much older CPU, and on Debian Bullseye (stable) with an even older and slower CPU.

I see a bit of a speedup, by about 15%, with either sage.all or sympy imported on macOS 13 (x86_64, as well as arm64, aka M1)

videlec · 2022-12-15T15:15:04Z

comment:13

Replying to Dima Pasechnik:

I see this sympy difference on the latest Sage beta on 3 boxes: Fedora 34 with a newish CPU, and on Gentoo with a much older CPU, and on Debian Bullseye (stable) with an even older and slower CPU.

I see a bit of a speedup, by about 15%, with either sage.all or sympy imported on macOS 13 (x86_64, as well as arm64, aka M1)

Interesting.

on system sage on archlinux : no difference between no setup and import sympy
on the compiled sage : same slowdown with import sympy and import sage.all

dimpase · 2022-12-15T16:56:02Z

comment:14

narrowing it down further, with taking all of Sage out:

# p1.py
import timeit

prog="""
from cypari2 import Pari
pari = Pari(size={size})
pari("{cmd}")
"""

prog = prog.format(size=2**24, cmd="quadclassunit(1 - 2^100)")

print("python environment with cypari2:         ",
      timeit.timeit(prog, number=3))

setup="""
import sympy
"""
print("with sympy loaded:                       ",
      timeit.timeit(prog, setup=setup, number=3))

On Gentoo with system python3.10.8, system-wide pari 2.15.1, running this p1.py

"naked" system python, pip-installed cypari2 and sympy - no time difference
venv'ed system python, pip-installed cypari2 and sympy - no time difference
Sage's-venv wrapped system python, Sage-installed cypari and sympy - sympy produces a slowdown.

So this is down to interaction between python, sympy, and cypari2 (and Sage's venv?)

dimpase · 2022-12-15T17:04:39Z

comment:15

Matthias, there is something funny in Sage's venv, causing such a slowdown on Linux.

mkoeppe · 2022-12-15T17:48:59Z

comment:16

To narrow it down further, you can try installing the Sage-built wheels (from venv/var/lib/sage/wheels/) in a separate venv. Specifically, take cypari2 from the sage wheel rather than from pypi.

dimpase · 2022-12-15T20:53:20Z

comment:17

Replying to Matthias Köppe:

To narrow it down further, you can try installing the Sage-built wheels (from venv/var/lib/sage/wheels/) in a separate venv. Specifically, take cypari2 from the sage wheel rather than from pypi.

This way, no difference in timing, so the wheels themselves are OK.

mkoeppe · 2022-12-15T21:15:54Z

comment:18

Next you could check whether it makes a difference to run the separate venv inside or outside of the sage environment set up by sage -sh.

dimpase · 2022-12-15T22:02:07Z

comment:19

if I start sage -sh, cd somewhere outside Sage directory, run python -m venv swheels,
activate swheels, and try doing things there, pip does not look too healthy:

(swheels) (sage-sh) dima@hilbert:swheels$ pip cache purge
An error occurred during configuration: option format: invalid choice: 'columns' (choose from 'human', 'abspath')

regardless, pip install works there, and I can run the test - again, no difference in time.
So, somehow, putting Sage's python into venv cures this time discrepancy.

(swheels) (sage-sh) dima@hilbert:swheels$ python p.py
python environment with cypari2:          5.116253892017994
with sympy loaded:                        5.216335576027632
(swheels) (sage-sh) dima@hilbert:swheels$ deactivate 
(sage-sh) dima@hilbert:swheels$ which python
/mnt/opt/Sage/sage-dev/local/var/lib/sage/venv-python3.10/bin/python
(sage-sh) dima@hilbert:swheels$ python p.py 
python environment with cypari2:          5.446458007965703
with sympy loaded:                        8.667378643993288

mkoeppe · 2022-12-15T22:05:31Z

comment:20

Replying to Dima Pasechnik:

if I start sage -sh, cd somewhere outside Sage directory, run python -m venv swheels,
activate swheels, and try doing things there, pip does not look too healthy:
(swheels) (sage-sh) dima@hilbert:swheels$ pip cache purge
An error occurred during configuration: option format: invalid choice: 'columns' (choose from 'human', 'abspath')

Indeed. I've opened #34853 for this unrelated issue.

mkoeppe · 2022-12-15T22:07:39Z

comment:21

I'd suggest to look whether mpmath is somehow involved, see #25445

dimpase · 2022-12-15T22:20:29Z

comment:22

Replying to Matthias Köppe:

I'd suggest to look whether mpmath is somehow involved, see #25445

In case, it's exactly the same wheels, mpmath included, that are involved here. As to #25445, indeed, import mpmath has the same side effect on performance as import sympy.

So we can now test instead the following:

# p1.py
import timeit

prog="""
from cypari2 import Pari
pari = Pari(size={size})
pari("{cmd}")
"""

prog = prog.format(size=2**24, cmd="quadclassunit(1 - 2^100)")

print("python environment with cypari2:         ",
      timeit.timeit(prog, number=3))

setup="""
import mpmath
"""
print("with mpmath loaded:                       ",
      timeit.timeit(prog, setup=setup, number=3))

mkoeppe · 2022-12-15T22:26:31Z

comment:23

Try if the various MPMATH_... environment variables (https://github.com/mpmath/mpmath/blob/master/mpmath/libmp/backend.py#L66) have an effect

dimpase · 2022-12-15T22:42:47Z

comment:24

Oops, we're back to import sage.all here, as your link points few lines below at

if ('MPMATH_NOSAGE' not in os.environ and 'SAGE_ROOT' in os.environ or
        'MPMATH_SAGE' in os.environ):
    try:
        import sage.all
        import sage.libs.mpmath.utils as _sage_utils
        sage = sage.all
        sage_utils = _sage_utils
        BACKEND = 'sage'
        MPZ = sage.Integer
    except:
        pass

and skipping it indeed cures the discrepancy.

(sage-sh) dima@hilbert:swheels$ MPMATH_NOSAGE=yes python p.py
python environment with cypari2:          5.126071550010238
with sympy loaded:                        5.132636028982233
(sage-sh) dima@hilbert:swheels$ python p.py
python environment with cypari2:          5.0600332279573195
with sympy loaded:                        8.450407178024761

sorry for the bogus lead :-)

The question is now - can we import less than sage.all pieces?

mkoeppe · 2022-12-15T23:08:24Z

comment:25

Can you reproduce it if you replace import mpmath by import sage.libs.pari.all or import.sage.cpython.all etc?

dimpase · 2022-12-15T23:43:34Z

comment:26

Replying to Matthias Köppe:

Can you reproduce it if you replace import mpmath by import sage.libs.pari.all or import.sage.cpython.all etc?

Success with sage.libs.eclib.mat or sage.libs.eclib.homspace - they are still a serious chunk of sage.all, about 40%, by some measure, but not all of it:

$ ./sage --python
>>> import sys
>>> import sage.libs.eclib.mat
>>> len(sys.modules.keys())
673
>>>

vs

$ ./sage --python
Python 3.10.8 (main, Nov 29 2022, 20:55:11) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> len(sys.modules.keys())
74
>>> import sage.all
>>> len(sys.modules.keys())
1712

for sage.libs.eclib.homspace one has 674 keys
(but these for sage.libs.eclib.mat are a subset of these, so we can go with the latter)

dimpase · 2022-12-15T23:52:00Z

comment:27

>>> import sage.libs.eclib.mwrank
>>> len(sys.modules.keys())
346

and import sage.libs.eclib.mwrank does not reproduce. so we're down to ~330 modules to check :-)

zimmermann6 · 2023-01-05T16:12:57Z

comment:103

I guess the glibc patch got lost because it was not sent at the proper place. I have put a comment on the bugzilla page.

dimpase · 2023-01-05T16:27:57Z

comment:104

I can confirm that the workaround provided in https://sourceware.org/bugzilla/show_bug.cgi?id=19924
(load a library with TLS, and access a TLS variable there)
also works here, so it's really the same bug.

dimpase · 2023-01-05T17:09:01Z

comment:105

That's my experiment:

git clone https://git.sr.ht/~dimpase/tls_workaround

and hit make. (we need to create a dummy dynamic library with TLS, etc, so it's a bit of setup)

zimmermann6 · 2023-01-06T06:35:27Z

comment:106

with the following simple change in pari_time.c, there is no more issue:

$ diff pari_time.c.orig pari_time.c
4a5
> #include <mpfr.h>
8a10
>     mpfr_exp_t e = mpfr_get_emin ();

However you might have to do the same for any library that has TLS variables.

mezzarobba · 2023-01-06T09:20:01Z

comment:107

Replying to Paul Zimmermann:

I guess the glibc patch got lost because it was not sent at the proper place. I have put a comment on the bugzilla page.

It has been sent to libc-alpha too:
https://patchwork.ozlabs.org/project/glibc/patch/b116855de71098ef7dd2875dd3237f8f3ecc12c2.1618301209.git.szabolcs.nagy@arm.com/

zimmermann6 · 2023-01-06T10:01:54Z

comment:108

It has been sent to libc-alpha too: [...]

right, I will ask Szabolcs if he can post a new version as requested.

vinc17fr · 2023-01-06T10:11:31Z

comment:109

Hi Marc,

Replying to Marc Mezzarobba:

In any case, it appears that loading new libraries (specifically, FLINT) after MPFR makes pari work at normal speed again. Regardless where the issue really comes from, this suggests that there is at least a workaround that MPFR could implement.

I don't see what workaround MPFR could implement. If I understand correctly, there are 2 things to avoid the bug: either change the order of the library loading, but this is something that can be done only at the application or user level, or access an MPFR thread-local variable, but I don't think that this is possible at library load time on the MPFR side, i.e. this is also something that can be done only at the application level (see Paul's suggestion, consisting in calling mpfr_get_emin()).

dimpase · 2023-01-06T10:15:46Z

comment:110

Replying to Paul Zimmermann:

with the following simple change in pari_time.c, there is no more issue:
$ diff pari_time.c.orig pari_time.c
4a5
> #include <mpfr.h>
8a10
>     mpfr_exp_t e = mpfr_get_emin ();
However you might have to do the same for any library that has TLS variables.

alternatively, and certainly easier w.r.t. software complexity (vendors of libraries don't have to do anything), before starting any computations, one can dlopen a dummy library

__thread int dummy_tls_for_lib = 23;

void dummy_access()
{
    dummy_tls_for_lib++;
}

and access a TLS variable, as in https://git.sr.ht/~dimpase/tls_workaround/tree/main/item/p.c#L34

      void* handle = dlopen("./libdummy.so", RTLD_NOW);
      void (*dummy_access)(void);
      *(void**)(&dummy_access) = dlsym(handle, "dummy_access");
      dummy_access();

dimpase · 2023-01-06T12:08:43Z

comment:111

Replying to Paul Zimmermann:

It has been sent to libc-alpha too: [...]

right, I will ask Szabolcs if he can post a new version as requested.

frankly speaking, it looks as if glibc is stalling on this bug since, like, forever...
I also don't get what Carlos asks from Szabolcs (1.5 YEARS after the submission of the patches) - it doesn't look as if anything in that patch series was touched.

mezzarobba · 2023-01-06T12:31:35Z

comment:112

Replying to Vincent Lefèvre:

I don't think that this is possible at library load time on the MPFR side, i.e. this is also something that can be done only at the application level

Okay, I assumed to could be done at load time, but you definitely know better :-)

tornaria · 2023-01-06T13:17:46Z

comment:113

Replying to Paul Zimmermann:

with the following simple change in pari_time.c, there is no more issue:
$ diff pari_time.c.orig pari_time.c
4a5
> #include <mpfr.h>
8a10
>     mpfr_exp_t e = mpfr_get_emin ();
However you might have to do the same for any library that has TLS variables.

Linking with -lmpfr certainly fixes the issue, no need to change the program, and the order is unimportant:

$ cc pari_time.c -lpari -o pari_time && ./pari_time libmpfr.so
dlopen("libmpfr.so", RTLD_LAZY): ok
elapsed: 1.650579
$ cc pari_time.c -lpari -lmpfr -o pari_time && ./pari_time libmpfr.so
dlopen("libmpfr.so", RTLD_LAZY): ok
elapsed: 1.016185
$ cc pari_time.c -lmpfr -lpari -o pari_time && ./pari_time libmpfr.so
dlopen("libmpfr.so", RTLD_LAZY): ok
elapsed: 1.015236

Here the dlopen() call is a nop since libmpfr has already been loaded before the program started.

tornaria · 2023-01-06T13:59:35Z

comment:114

Replying to Vincent Lefèvre:

Hi Marc,

Replying to Marc Mezzarobba:

In any case, it appears that loading new libraries (specifically, FLINT) after MPFR makes pari work at normal speed again. Regardless where the issue really comes from, this suggests that there is at least a workaround that MPFR could implement.

I don't see what workaround MPFR could implement. If I understand correctly, there are 2 things to avoid the bug: either change the order of the library loading, but this is something that can be done only at the application or user level, or access an MPFR thread-local variable, but I don't think that this is possible at library load time on the MPFR side, i.e. this is also something that can be done only at the application level (see Paul's suggestion, consisting in calling mpfr_get_emin()).

Just for fun I rebuilt mpfr-4.1.1 with the following silly patch and voilà, the issue goes away:

--- a/src/exceptions.c	2022-01-06 12:27:38.000000000 -0300
+++ b/src/exceptions.c	2023-01-06 10:32:49.845632284 -0300
@@ -34,6 +34,12 @@
   return __gmpfr_emin;
 }
 
+static void __attribute__ ((constructor))
+__dummy_access_tls (void)
+{
+	__gmpfr_emin = MPFR_EMIN_DEFAULT;
+}
+
 #undef mpfr_set_emin
 
 int

vinc17fr · 2023-01-06T14:24:44Z

comment:115

Replying to Gonzalo Tornaría:

Just for fun I rebuilt mpfr-4.1.1 with the following silly patch and voilà, the issue goes away:

--- a/src/exceptions.c	2022-01-06 12:27:38.000000000 -0300
+++ b/src/exceptions.c	2023-01-06 10:32:49.845632284 -0300
@@ -34,6 +34,12 @@
   return __gmpfr_emin;
 }
 
+static void __attribute__ ((constructor))
+__dummy_access_tls (void)
+{
+	__gmpfr_emin = MPFR_EMIN_DEFAULT;
+}
+
 #undef mpfr_set_emin
 
 int

But note that __attribute__ ((constructor)) is a GNU extension.

tornaria · 2023-01-06T14:37:29Z

comment:116

Replying to Vincent Lefèvre:

But note that __attribute__ ((constructor)) is a GNU extension.

Fortunately, the bug is also a GNU extension.

Just guard the whole thing inside MPFR_HAVE_CONSTRUCTOR_ATTR and some GLIBC defined macro.

dimpase · 2023-01-06T15:32:09Z

comment:117

Replying to Gonzalo Tornaría:

Fortunately, the bug is also a GNU extension.

GNU inaction, rather... Time for another eglibc fork?

Just guard the whole thing inside MPFR_HAVE_CONSTRUCTOR_ATTR and some GLIBC defined macro.

and it's x86_64 - only, too, right?

vinc17fr · 2023-01-06T15:50:43Z

comment:118

Replying to Gonzalo Tornaría:

Replying to Vincent Lefèvre:

But note that __attribute__ ((constructor)) is a GNU extension.

Fortunately, the bug is also a GNU extension.

Things are more complex. Such GNU extensions at compiler level are available on various systems, and it is not clear that they are working correctly on other systems. In theory, this should have no effect on such systems, but it is not impossible to trigger a bug yielding an incorrect behavior. In particular, this feature is probably used very little at library load time. So I'm wondering whether it has been fully tested on various systems. Enabling such a workaround via a configure option should be safer.

zimmermann6 · 2023-01-06T17:47:45Z

comment:119

anyway, the issue is clearly on the glibc side, and it is not a good idea to try to workaround any compiler/library issue on the mpfr side. But of course Sage developers can patch their mpfr version if they like.

zimmermann6 · 2023-01-06T17:52:08Z

comment:120

a new version of the patch: https://patchwork.ozlabs.org/project/glibc/patch/20221019111439.63501-1-szabolcs.nagy@arm.com/

and the corresponding discussion:

https://sourceware.org/pipermail/libc-alpha/2023-January/144533.html

If someone out there wants this to be included in glibc 2.37 (which will be released in February), feel free to review the patch on the libc-alpha list.

tornaria · 2023-01-07T19:44:47Z

comment:121

Thanks Paul. I tried the v3 version of the patch that was posted yesterday (https://sourceware.org/pipermail/libc-alpha/2023-January/144535.html).

It does fix the slowdown:

$ cat pari_time.c 
#include <stdio.h>
#include <time.h>
#include <dlfcn.h>

#include <pari/pari.h>

void run()
{
    clock_t t = clock();
    setrand(stoi(1)); /* make it deterministic */
    GEN D = subui(1,powuu(2,100));
    GEN ans = quadclassunit0(D, 0, NULL, 38);
    printf("elapsed: %.3f\n", (double)(clock()-t)/CLOCKS_PER_SEC);
}


int main(int argc, char *argv[])
{
    pari_init(1<<24, 0);
    run();

    while(--argc) {
        char * name = *++argv;
        void * h = dlopen(name, RTLD_LAZY);
        printf("dlopen(\"%s\", RTLD_LAZY): %s\n", name, h ? "ok" : "fail");
        run();
    }
}

$ cc pari_time.c -lpari -o pari_time

$ ./pari_time libmpfr.so.6 libgomp.so.1
elapsed: 1.291
dlopen("libmpfr.so.6", RTLD_LAZY): ok
elapsed: 1.977
dlopen("libgomp.so.1", RTLD_LAZY): ok
elapsed: 1.253

$ glibc/build/elf/ld.so ./pari_time libmpfr.so.6 libgomp.so.1
elapsed: 1.263
dlopen("libmpfr.so.6", RTLD_LAZY): ok
elapsed: 1.238
dlopen("libgomp.so.1", RTLD_LAZY): ok
elapsed: 1.229

mezzarobba · 2023-02-07T09:06:56Z

alternatively, and certainly easier w.r.t. software complexity (vendors of libraries don't have to do anything), before starting any computations, one can dlopen a dummy library
[...]
and access a TLS variable, as in https://git.sr.ht/~dimpase/tls_workaround/tree/main/item/p.c#L34

So do you think we should do something like that at the end of sage.all? Even if the bug gets fixed in glibc, the fix will take a while to propagate, and this looks like a simple enough workaround...

zimmermann6 · 2023-02-27T14:47:36Z

for the record at today's glibc weekly review meeting I asked the v3 patch (https://patchwork.sourceware.org/project/glibc/patch/20230106185250.2936935-1-szabolcs.nagy@arm.com/)
to be reviewed.

zimmermann6 · 2023-09-01T07:35:12Z

after several pings from myself, this issue should now be fixed in glibc development branch, thus should be in 2.39
that will be release in February 2024. Can someone check?

tornaria · 2024-04-12T23:09:13Z

@zimmermann6 @videlec
This seems to be fixed with glibc 2.39:

$ ./pari_time libmpfr.so.6 libgomp.so.1
elapsed: 1.504
dlopen("libmpfr.so.6", RTLD_LAZY): ok
elapsed: 1.493
dlopen("libgomp.so.1", RTLD_LAZY): ok
elapsed: 1.496

For comparision (the first one is just the same as the system glibc):

$ LD_LIBRARY_PATH=glibc-2.39_2 glibc-2.39_2/ld-linux-x86-64.so.2 ./pari_time libmpfr.so.6 libgomp.so.1
elapsed: 1.512
dlopen("libmpfr.so.6", RTLD_LAZY): ok
elapsed: 1.492
dlopen("libgomp.so.1", RTLD_LAZY): ok
elapsed: 1.496
$ LD_LIBRARY_PATH=glibc-2.38_6 glibc-2.38_6/ld-linux-x86-64.so.2 ./pari_time libmpfr.so.6 libgomp.so.1
elapsed: 1.488
dlopen("libmpfr.so.6", RTLD_LAZY): ok
elapsed: 2.477
dlopen("libgomp.so.1", RTLD_LAZY): ok
elapsed: 1.465
$ LD_LIBRARY_PATH=glibc-2.36_2:pari-2.15.1_1:gomp-12.2.0_3 glibc-2.36_2/ld-linux-x86-64.so.2 ./pari_time libmpfr.so.6 libgomp.so.1
elapsed: 1.503
dlopen("libmpfr.so.6", RTLD_LAZY): ok
elapsed: 2.383
dlopen("libgomp.so.1", RTLD_LAZY): ok
elapsed: 1.517

As for the original report:

$ python pari_timing.py 
python environment with cypari2:           4.868679889012128
with sage custom GMP allocation functions: 4.992306913016364
with the whole sagemath library lodaded:   4.356488843972329

zimmermann6 · 2024-04-15T08:18:31Z

thank you for checking, we can now close this ticket I guess.

videlec added this to the sage-9.8 milestone Dec 14, 2022

videlec added t: performance labels Dec 14, 2022

This comment has been minimized.

Sign in to view

mkoeppe modified the milestones: sage-9.8, sage-9.9 Feb 11, 2023

mkoeppe removed this from the sage-10.0 milestone Apr 30, 2023

dimpase mentioned this issue Sep 4, 2023

Always enable pthread flags gap-system/gap#5196

Closed

dimpase closed this as completed Jun 21, 2024

mkoeppe added this to the sage-10.4 milestone Jul 8, 2024

sage-loaded libraries are slowing down cypari2 a lot (on x86_64 glibc) #34850

sage-loaded libraries are slowing down cypari2 a lot (on x86_64 glibc) #34850

Comments

videlec commented Dec 14, 2022

This comment has been minimized.

This comment has been minimized.

dimpase commented Dec 15, 2022

mezzarobba commented Dec 15, 2022

videlec commented Dec 15, 2022

mezzarobba commented Dec 15, 2022

mantepse commented Dec 15, 2022

dimpase commented Dec 15, 2022

mezzarobba commented Dec 15, 2022

dimpase commented Dec 15, 2022

dimpase commented Dec 15, 2022

videlec commented Dec 15, 2022

dimpase commented Dec 15, 2022

dimpase commented Dec 15, 2022

mkoeppe commented Dec 15, 2022

dimpase commented Dec 15, 2022

mkoeppe commented Dec 15, 2022

dimpase commented Dec 15, 2022

mkoeppe commented Dec 15, 2022

mkoeppe commented Dec 15, 2022

dimpase commented Dec 15, 2022

mkoeppe commented Dec 15, 2022

dimpase commented Dec 15, 2022

mkoeppe commented Dec 15, 2022

dimpase commented Dec 15, 2022

dimpase commented Dec 15, 2022

zimmermann6 commented Jan 5, 2023

dimpase commented Jan 5, 2023

dimpase commented Jan 5, 2023

zimmermann6 commented Jan 6, 2023

mezzarobba commented Jan 6, 2023

zimmermann6 commented Jan 6, 2023

vinc17fr commented Jan 6, 2023

dimpase commented Jan 6, 2023

dimpase commented Jan 6, 2023

mezzarobba commented Jan 6, 2023

tornaria commented Jan 6, 2023

tornaria commented Jan 6, 2023

vinc17fr commented Jan 6, 2023

tornaria commented Jan 6, 2023

dimpase commented Jan 6, 2023

vinc17fr commented Jan 6, 2023

zimmermann6 commented Jan 6, 2023

zimmermann6 commented Jan 6, 2023

tornaria commented Jan 7, 2023

mezzarobba commented Feb 7, 2023

zimmermann6 commented Feb 27, 2023

zimmermann6 commented Sep 1, 2023

tornaria commented Apr 12, 2024

zimmermann6 commented Apr 15, 2024