Build: require SQLite 3.11 (refs #1718)

[rouault]

PROJ can build and run against older version (3.7 for example), but it has
been found that performance is horrible. With 3.11 (Ubuntu 16.04), it is
fine, so sets this as the minimum version.

[kbevers]

Thanks. Just a reminder from RFC3:

A change in programming language standard can only be introduced with a new major version release of PROJ. Changes for software package dependencies can be introduced with minor version releases of PROJ. Changing the version requirements for a dependency needs to be approved by the PSC.

So, we can introduce this with PROJ 6.3 provided that the PSC accepts this change. SQLite 3.11 was released in 2016 so we are clear of the older-than-two-years requirement for package dependencies.

I would suggest that RFC3 be amended with the new requirements of SQLite.

[rouault]

RFC3 be amended with the new requirements of SQLite.

As I read it, RFC3 is general policy and bootstrapping. Does it need to be a living document ? Probably not the best place to capture requirements that might be per PROJ version

[rouault]

Motion sent to the list

[kbevers]

Does it need to be a living document ?

It doesn't have to but I figure that it might be useful. Maybe it's too much admin work to keep up to date, I dont know... I was just thinking a history section that keeps track of when new requirements are introduced.

Where do we keep the list of requirements now, that isn't RFC3? The README? Somewhere on the website? I think this is a place where we can improve things a bit at least.

[rouault]

Where do we keep the list of requirements now

In the checks of build systems :-)

[kbevers]

In the checks of build systems :-)

I'll go out on a limb here and say that that isn't good enough :)

[rouault]

I'll go out on a limb here and say that that isn't good enough :)

it turns out I discovered there was already more or less something for that. I've tweaked it in b2873d7

[rouault]

Motion passed

[devrimgunduz]

Can you please give me a hint about compiling Proj with a non-standard sqlite version (I mean, which is outside $PATH, etc?)

I'm having trouble doing it. Proj successfully finds my new sqlite, but then I get:

libtool: link: g++ -std=c++11 -Wall -Wextra -Winit-self -Wunused-parameter -Wformat -Werror=format-security -Wno-format-nonliteral -Wlogical-op -Wshadow -Werror=vla -Wmissing-declarations -Wnon-virtual-dtor -Woverloaded-virtual -Wdeprecated -Wzero-as-null-pointer-constant -g -O2 -fvisibility=hidden -Wl,-rpath -Wl,/usr/proj62/lib64 -o .libs/geod apps/geod.o apps/geod_set.o apps/geod_interface.o apps/emess.o  ./.libs/libproj.so -lpthread -lm -Wl,-rpath -Wl,/usr/proj62/lib
./.libs/libproj.so: undefined reference to `sqlite3_column_text'
./.libs/libproj.so: undefined reference to `sqlite3_close'

...(and this goes on a bit more)

Any ideas? Thanks!

[rouault]

Any ideas? Thanks!

How do you invoke ./configure ? Do you use pkg-config or explicitly set SQLITE3_CFLAGS and SQLITE3_LIBS ?

[devrimgunduz]

Argh, thanks! I was using them, but apparently in a wrong way. Here is what I did, which worked. %{sqlite33dir } is an RPM macro, and it's replaced by /usr/sqlite330 in my case:

export SQLITE3_LIBS="-L%{sqlite33dir}/lib -lsqlite3"
export SQLITE3_INCLUDE_DIR='%{sqlite33dir}/include'
export SQLITE3_CFLAGS="$CFLAGS -I%{sqlite33dir}/include"
export PATH=%{sqlite33dir}/bin/:$PATH

Thanks! Just a FYI: I'm working on custom sqlite3 packages on RHEL7, and Proj packages in the PostgreSQL RPM repository will depend of this sqlite version, which will solve the issues.

[rouault]

Just a FYI: I'm working on custom sqlite3 packages on RHEL7, and Proj packages in the PostgreSQL RPM repository will depend of this sqlite version, which will solve the issues.

Yes, @hobu made me aware of that. Much appreciated ! I've posted about your work on the PROJ mailing list too in https://lists.osgeo.org/pipermail/proj/2019-November/009002.html

So, you're no replacing the system libsqlite3 ? How does that work with packages such as GDAL which themselves link to sqlite3 (do you build it againt that /usr/sqlite330 too ?). Otherwise that might potentially cause runtime clashes

[devrimgunduz]

Hi @rouault ,

I'm not replacing the system sqlite. The package I created install things under /usr/sqlite330 directory. I rebuilt proj, and with rpath magic, made sure that libproj.so is linked to the new sqlite:
-bash-4.2$ ldd /usr/proj62/lib/libproj.so|grep sqlite
libsqlite3.so.0 => /usr/sqlite330/lib/libsqlite3.so.0 (0x00007f6c46200000)
I'll do the same for GDAL, libspatialite, libgeotiff, and then rebuild PostGIS against these.

Do you see any potential problems with this plan?

Thanks!

Regards, Devrim

[rouault]

Do you see any potential problems with this plan?

No, that sounds good. Downstream users of those packages having an explicit sqlite3 dependency should make sure to link against sqlite330 too.

Before you attempted this, I tried a quick & dirty approach by having an internal copy of sqlite inside PROJ, but I also tried to avoid any potential symbol clash by having a #define sqlite330_sqlite3_XXXX sqlite3_XXXX for each sqlite3 symbol. That way you can link both against PROJ and the system sqlite3 without risks
A solution to do that easily is from the amalgation sqlite3.c to

gcc -shared -fPIC sqlite3.c -shared -o temp.so
(for i in `objdump -T temp.so | grep sqlite3 | awk '{print $7}'`; do echo "#define $i sqlite330_$i"; done) > rename.h
cat rename.h sqlite3.h > sqlite3_with_rename.h
mv sqlite3_with_rename.h sqlite3.h

That way only the PROJ package would have to been built against this hacked sqlite3 lib, and all other packages could link against the regular sqlite3 without risk of conflict