Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ponyc rpm installs but compiler doesnt work libtinfo.so.5 error (ncurses package re-org) #1208

Closed
Triangle345 opened this issue Sep 10, 2016 · 41 comments
Assignees
Labels
triggers release Major issue that when fixed, results in an "emergency" release

Comments

@Triangle345
Copy link

Using fedora 24 kernel 4.6.6

I am able to install the rpm just fine. However; when running ponyc I get the following error

ponyc
ponyc: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory

Is this a known issue or am I missing some other dependency?

@SeanTAllen
Copy link
Member

@Triangle345 i'm not aware of this being an issue. Sounds like a missing dependency of some sort. If you are able to track it down so we can fix for others, that would be awesome.

@Triangle345
Copy link
Author

Triangle345 commented Sep 10, 2016

So the issue is that on new versions of linux (like fedora 24) libtinfo.so.6 exists but 5 does not. So one remedy for this is to trick pony into thinking libtinfo.so5 exists. To my understanding libtinfo is part of the ncurses package. Simply create a sym link in /usr/lib and /usr/lib64 (depending on architecture):

sudo ln -s libtinfo.so.6 libtinfo.so.5

Not sure how you guys want to handle this during rpm install though

@SeanTAllen
Copy link
Member

I'm not familiar with how the RPM builds happen at this point. @killerswan can you weigh in?

@killerswan
Copy link
Member

The RPMs that get uploaded are built one of Travis's Ubuntu systems, using this make command:
https://github.com/ponylang/ponyc/blob/master/.travis.yml#L200

That is defined here:
https://github.com/ponylang/ponyc/blob/master/Makefile#L669

So the real magic is that FPM call at 697...

Is there another package we can make the RPM require that'll provide the version of ncurses, or that library, we need?

Or is this kinda cross-compiling from Ubuntu just not going to work very well here?

@Triangle345
Copy link
Author

Maybe just package version 5 of ncurses with the rpm/deb?

On Sep 10, 2016 3:41 PM, "Kevin Cantú" notifications@github.com wrote:

The RPMs that get uploaded are built one of Travis's Ubuntu systems, using
this make command:
https://github.com/ponylang/ponyc/blob/master/.travis.yml#L200

That is defined here:
https://github.com/ponylang/ponyc/blob/master/Makefile#L669

So the real magic is that FPM call at 697...

Is there another package we can make the RPM require that'll provide the
version of ncurses, or that library, we need?

Or is this kinda cross-compiling from Ubuntu just not going to work very
well here?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1208 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEEHvr2ylHryMFXBCW1NT94Ch0LgDWV_ks5qowfRgaJpZM4J5niW
.

@SeanTAllen SeanTAllen added difficulty: 1 - easy triggers release Major issue that when fixed, results in an "emergency" release labels Sep 14, 2016
@sylvanc
Copy link
Contributor

sylvanc commented Sep 14, 2016

This seems to be a problem for other compilers as well:

commercialhaskell/stack#257

commercialhaskell/stack#2300

commercialhaskell/stack#2302

@Triangle345
Copy link
Author

Why isnt pony built statically to include everything? Sure the binary would
be larger but not by that much.

On Wed, Sep 14, 2016 at 4:07 PM, Sylvan Clebsch notifications@github.com
wrote:

This seems to be a problem for other compiles as well:

commercialhaskell/stack#257
commercialhaskell/stack#257

commercialhaskell/stack#2300
commercialhaskell/stack#2300

commercialhaskell/stack#2302
commercialhaskell/stack#2302


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1208 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEEHvhtDYwJ4IMTmnBNk3YdlptML9Infks5qqFQcgaJpZM4J5niW
.

@Jbbouille
Copy link

We could add a dependency to "ncurses-devel" in the rpm package with -d flag ?

Like :

fpm -s dir -t rpm -C $(package) -p build/bin --name $(package_name) --conflicts $(package_conflicts) --version $(package_base_version) --iteration "$(package_iteration)" --description "The Pony Compiler" -d "ncurses-devel"

@Jbbouille
Copy link

Also I think that ponyc needs "openssl" and "pcre2" libraries. We could add these two dependencies to the rpm package with the same '-d' flag ?

@Triangle345
Copy link
Author

Even if adding a dependency to the rpm as you suggest, wont this still be
an issue if the distribution does not have version 5 of ncurses. For
example, you require ncurses-devel, so fedora package manager installs the
only version it has, which is 6 , which brings us back to the same
predicament. I think the only true way to solve this is, is to build the
binary statically or include the specific libraries within the rpm.

On Thu, Sep 15, 2016 at 4:31 AM, Jean-Baptiste Petit <
notifications@github.com> wrote:

Also I think that ponyc needs "openssl" and "pcre2" libraries. We could
add these two dependencies to the rpm package with the same '-d' flag ?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1208 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEEHvoIffnzgKFcVbm4ZMeM8MzYLxAB3ks5qqQJNgaJpZM4J5niW
.

@Triangle345
Copy link
Author

Another option that i touched on a little bit, but is dirty and easy is to
create a symbolic link to 6 but name it with 5. Even if we put this right
next to ponyc executable it should work. This is provided that the
interface from 5 to 6 did not change. This can be done through logic
inside the pre or post install in the rpm/deb.

On Thu, Sep 15, 2016 at 6:31 AM, Eugene Feldman efeldman21@gmail.com
wrote:

Even if adding a dependency to the rpm as you suggest, wont this still be
an issue if the distribution does not have version 5 of ncurses. For
example, you require ncurses-devel, so fedora package manager installs the
only version it has, which is 6 , which brings us back to the same
predicament. I think the only true way to solve this is, is to build the
binary statically or include the specific libraries within the rpm.

On Thu, Sep 15, 2016 at 4:31 AM, Jean-Baptiste Petit <
notifications@github.com> wrote:

Also I think that ponyc needs "openssl" and "pcre2" libraries. We could
add these two dependencies to the rpm package with the same '-d' flag ?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1208 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEEHvoIffnzgKFcVbm4ZMeM8MzYLxAB3ks5qqQJNgaJpZM4J5niW
.

@Jbbouille
Copy link

Well it seems that there is "libtinfo.so.5" in "ncurses-compat-libs" so instead of using -d "ncurses-devel" we could use -d "ncurses-compat-libs".

@Triangle345
Copy link
Author

that will work for fedora as of this moment but what happens when we have a
similar situation on other distributions? We would need to find the package
that provides compat libs if one exists.

Building it statically solves all these issues.

On Thu, Sep 15, 2016 at 7:19 AM, Jean-Baptiste Petit <
notifications@github.com> wrote:

Well it seems that there is "libtinfo.so.5" in "ncurses-compat-libs" so
instead of using -d "ncurses-devel" we could use -d "ncurses-compat-libs".


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1208 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEEHvvV9paPK-GEEBe9NYNAzRbieanmWks5qqSnfgaJpZM4J5niW
.

@SeanTAllen
Copy link
Member

And building statically open up an entire new host of issues as well.

@Triangle345
Copy link
Author

what sorts of issues are you thinking about, if I may ask?

On Thu, Sep 15, 2016 at 11:31 AM, Sean T Allen notifications@github.com
wrote:

And building statically open up an entire new host of issues as well.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1208 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEEHvkiDznH0zaj0ZoboGoucxETkL7Ywks5qqWTggaJpZM4J5niW
.

@SeanTAllen
Copy link
Member

statically linking all dependencies becomes a real issue when minor bug fix releases are done in the dependencies. its really problematic when it prevents easily upgrading to incorporate security fixes.

i've worked at more than 1 place that had a policy against deploying applications that statically linked standard libraries like ncurses.

@Triangle345
Copy link
Author

I guess adding ncurses-compat-libs as a dep should work for awhile until
curses 6 catches on. Should be a straight forward change.

On Fri, Sep 16, 2016 at 6:55 AM, Sean T Allen notifications@github.com
wrote:

statically linking all dependencies becomes a real issue when minor bug
fix releases are done in the dependencies. its really problematic when it
prevents easily upgrading to incorporate security fixes.

i've worked at more than 1 place that had a policy against deploying
applications that statically linked standard libraries like ncurses.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1208 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEEHvsOoSmZvl7bAbj18txbwVO_E7GO1ks5qqnWWgaJpZM4J5niW
.

@sylvanc
Copy link
Contributor

sylvanc commented Sep 21, 2016

Does anyone know how to make ncurses-compat-libs a dependency only if it's actually available? Is that a normal Linux thing? We can't require it everywhere, as ncurses-5 systems have no such package.

@sylvanc
Copy link
Contributor

sylvanc commented Sep 21, 2016

Also, yes, ncurses-compat-libs is The Right Thing (tm) to do :)

@Triangle345
Copy link
Author

hmm
try using Recommends: tag as this is a weak dependency. It will require a
package but not throw errors if its not installed.

another option:
A lot people just distribute for different versions like .deb for ubuntu
1404 or .deb for ubuntu < 10.04 etc. Same thing with rpms. We may have to
build rpms for different versions of the same operating system if huge
differences like that will break installation.

On Wed, Sep 21, 2016 at 4:26 PM, Sylvan Clebsch notifications@github.com
wrote:

Also, yes, ncurses-compat-libs is The Right Thing (tm) to do :)


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1208 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEEHvpCy97bCF-5nncuM5aoPD4FRLDrlks5qsZLvgaJpZM4J5niW
.

@Jbbouille
Copy link

So we should replace L657 by:

$(SILENT)fpm -s dir -t rpm -C $(package) -p build/bin --name $(package_name) --conflicts $(package_conflicts) --version $(package_base_version) --iteration "$(package_iteration)" --description "Package of the Pony Compiler, for Fedora > 24"
$(SILENT)fpm -s dir -t rpm -C $(package) -p build/bin --name "$(package_name)-old_fedora" --conflicts $(package_conflicts) --version $(package_base_version) --iteration "$(package_iteration)" --description "Package of the Pony Compiler, for Fedora < 24" -d "ncurses-compat-libs"

Something like that ?

@Triangle345
Copy link
Author

I've never used fpm before but the tag -d seems correct. However; dont you
have those two backwards? shoudnt compat lib be required only for fedora >
24? You have compat lib set up for < 24

On Thu, Sep 22, 2016 at 6:09 PM, Jean-Baptiste Petit <
notifications@github.com> wrote:

So we should replace L657
https://github.com/ponylang/ponyc/blob/master/Makefile#L657 by:

$(SILENT)fpm -s dir -t rpm -C $(package) -p build/bin --name $(package_name) --conflicts $(package_conflicts) --version $(package_base_version) --iteration "$(package_iteration)" --description "Package of the Pony Compiler, for Fedora > 24"
$(SILENT)fpm -s dir -t rpm -C $(package) -p build/bin --name "$(package_name)-old_fedora" --conflicts $(package_conflicts) --version $(package_base_version) --iteration "$(package_iteration)" --description "Package of the Pony Compiler, for Fedora < 24" -d "ncurses-compat-libs"

Something like that ?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1208 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEEHvkOIUgvYJ5sFxbAcjxIZSBoTXp-2ks5qsvyugaJpZM4J5niW
.

@killerswan
Copy link
Member

killerswan commented Oct 26, 2016

If I understand this correctly, now's a good time to figure out the version compat for RedHat and CentOS, and think about the naming a bit more. I'd like to get these new packages into the package conflicts, that is, so that only one of the following packages can be installed:

  • ponyc-release,
  • ponyc-release-legacy-ncurses,
  • ponyc-master,
  • ponyc-master-legacy-ncurses.

@killerswan
Copy link
Member

Although is the usability of these multiple packages really an improvement?

@killerswan
Copy link
Member

killerswan commented Oct 26, 2016

While I'm kicking ideas around, is there a virtual package provided by both older distros' ncurses-libs and new distros' ncurses-compat-libs, I wonder? Can we specify a dependency on "*/libtinfo.so.5"?

If there is no such virtual package, maybe we could set up the following 😜 :

  • ponyc-release (requiring ponydep-ncurses)
  • ponyc-master (requiring ponydep-ncurses)
  • ponydep-ncurses (a virtual package / capability)
  • ponydep-ncurses-libs-5 (requiring actual ncurses-libs < 6, providing ponydep-ncurses)
  • ponydep-ncurses-libs-5-compat (requiring actual ncurses-compat-libs >= 6, providing ponydep-ncurses)

Then on any of, say, Fedora 18, Fedora 24, CentOS 7, or RHEL 7 you'd install "ponyc-release" and get whichever dependencies Yum finds available... It seems like it may be a while before the latest RHEL, CentOS, and Fedora all provide the same version of ncurses-libs.

@killerswan
Copy link
Member

Note: I'll come back to this in a little while. I've just stubbed my toe trying to take a CentOS install of rubygems from 2.0.14 to 2.0.15 so I can install FPM...

@killerswan
Copy link
Member

I've now got the FPM stuff sorted out so I'm going to have another stab this afternoon. 😇

@killerswan
Copy link
Member

Hmm.

@sylvanc
Copy link
Contributor

sylvanc commented Nov 24, 2016

@killerswan does the "Recommends" approach not work? That is, having a single package that tries to install ncurses-compat but doesn't strictly require it?

@gumpt
Copy link

gumpt commented Mar 12, 2017

I'm looking at taking since since it's marked as an "easy" bug.

What's wrong with producing two or more rpms with different dependencies depending on the distribution targeted? It looks like .bintray.sh will find and upload both, so the change is limited to the makefile.

Is everyone really set on having one package?

@killerswan
Copy link
Member

Hmm, time for some more VM juggling to see if a single package which has the dependencies is even possible, like I was trying a while back. 😅

@SeanTAllen
Copy link
Member

@gumpt thanks! perhaps ping @killerswan here and see what needs doing and what you can help with.

@gumpt
Copy link

gumpt commented Mar 14, 2017

@killerswan AFAIK rpms don't have the same "recommends" option that debs do, or at least FPM doesn't have a way to use it. I was going to use --rpm-dist el7 and --rpm-dist fc24 -d ncurses-compat-libs for two separate rpms. Let me know if I can do anything.

I'll go looking for another way to contribute.

@SeanTAllen
Copy link
Member

@gumpt if you are looking for things to take on, drop me an email. plenty of things that need doing, i'm sure we can find something that interests you. thanks!

@killerswan
Copy link
Member

killerswan commented Mar 14, 2017

Regarding @sylvanc's "Recommends" suggestion: if I understand correctly (per old Stack Overflow conversations and the FPM docs), RPM doesn't support weak dependencies like that.

There is, however, FPM's --provides flag, which will make RPM virtual packages (which multiple things can provide). Hence the hierarchy like I was suggesting above:

  • ponyc-release (requiring ponydep-ncurses)
    • ponydep-ncurses (a virtual package / capability)
      • ponydep-ncurses-libs-5 (requiring actual ncurses-libs < 6, providing ponydep-ncurses)
      • ponydep-ncurses-libs-5-compat (requiring actual ncurses-compat-libs >= 6, providing ponydep-ncurses)

If I understand correctly, we can make installation of all these work with a single install command (for ponyc-release), if our repository (aha, see the meaning on Bintray?) here -- https://bintray.com/pony-language/ponyc-rpm -- contains all three real packages:

  1. ponyc-release
  2. ponydep-ncurses-libs-5
  3. ponydep-ncurses-libs-5-compat

It's time for me to go sleep now, but these FPM commands make the packages I haven't yet tested on a Fedora install 😅 :

fpm -s dir -t rpm -C build/ponyc-release-0.11.1-8888.922bed4 -p build/bin \
--name ponyc-release \
--provides ponyc \
--conflicts ponyc-master \
--version 0.11.1 \
--iteration "8889.922bed4" \
--description "The Pony Compiler" \
--depends ponydep-ncurses

fpm -s empty -t rpm -p build/bin \
--name ponydep-ncurses-libs-5-compat \
--conflicts ponydep-ncurses-libs-5 \
--provides ponydep-ncurses \
--version 2017.1 \
--iteration "4" \
--description "Virtual package for ncurses-compat" \
--depends "ncurses-compat-libs>=6"

fpm -s empty -t rpm -p build/bin \
--name ponydep-ncurses-libs-5 \
--conflicts ponydep-ncurses-libs-5-compat \
--provides ponydep-ncurses \
--version 2017.1 \
--iteration "4" \
--description "Virtual package for ncurses" \
--depends "ncurses<6"

Once I've set up a repo and poked at these some more, what I imagine doing is making a quick ponylang/ponydep-ncurses repo which contains a Travis build for the two ncurses packages which we can build once to bintray and then ignore, like, forever. But like I said, need to finish my experiment.

@gumpt, you're right: what we probably should have done is made two packages a couple months ago, just to move things along. But here we are and I'm feeling motivated to learn how to do it the pretty way...

@killerswan
Copy link
Member

killerswan commented Mar 14, 2017

It seems that I spoke too soon about weak dependencies: they can be specified now, although RPM ignores them, I'm not sure FPM can be used to add them, and I'm not sure either Yum or DNF does anything with them yet.

But anyways using the "provides" feature like I was planning above, I've now got a pair of packages to play with at https://github.com/killerswan/ponydep-ncurses (building here, deploying here). That allows me to say I'm not sure this is working as I hoped. 😿 Both Yum and DNF choke here when installing on Fedora 25 (which needs the compat lib):

[kevin@fed-pony-2 ~]$ sudo dnf install ponydep-ncurses
...
Error: nothing provides ncurses<6 needed by ponydep-ncurses-libs-5-0.1-11.8cc2023.x86_64
...

Maybe they'll use some smarter resolution logic when installing a package which depends on the virtual package than they do when installing literally that virtual package? But if not, then splitting the package (ponyc-release and ponyc-release-ncurses5) might be best for today.

@killerswan
Copy link
Member

Somewhat tangentially to all this, now I'm currently looking for advice about how basic Fedora package management is supposed to work.

Behold this insanity (with my personal bintray repo which now has a ponyc-release 0.11):

[kevin@fed-pony-2 ~]$ sudo dnf info ponyc-release
Last metadata expiration check: 0:17:10 ago on Wed Mar 15 01:01:04 2017.
Available Packages
Name        : ponyc-release
Arch        : x86_64
Epoch       : 0
Version     : 0.11.1
Release     : 78.992314d
...
[kevin@fed-pony-2 ~]$ dnf info ponyc-release
Last metadata expiration check: 0:05:41 ago on Wed Mar 15 01:12:43 2017.
Available Packages
Name        : ponyc-release
Arch        : x86_64
Epoch       : 0
Version     : 0.2.1
Release     : 71.4a2c9cd
...
[kevin@fed-pony-2 ~]$ sudo dnf install ponyc-release
Last metadata expiration check: 0:17:54 ago on Wed Mar 15 01:01:04 2017.
Dependencies resolved.
======================================================================================
 Package         Arch     Version               Repository                       Size
======================================================================================
Installing:
 ponyc-release   x86_64   0.2.1-71.4a2c9cd      bintray--killerswan-ponyc-rpm    12 M

Transaction Summary
======================================================================================
...

Consistently sudo dnf info finds the 0.11 version, dnf info finds the 0.2 version, and sudo dnf install finds the 0.2 version. HOW MANY PACKAGE CACHES DOES FEDORA HAVE?

I'm sure in like an hour all this will make sense but right now I'm puzzled! 😸

@killerswan
Copy link
Member

Yep, dnf --refresh ... makes it stop using old caches. 🍺

@killerswan
Copy link
Member

Victory is mine!

On Fedora 25:

[kevin@fed-pony-2 ~]$ sudo dnf --refresh install ponyc-release
bintray--killerswan-ponyc-rpm                                        14 kB/s | 8.5 kB     00:00    
Dependencies resolved.
====================================================================================================
 Package                        Arch    Version                Repository                      Size
====================================================================================================
Installing:
 ncurses-compat-libs            x86_64  6.0-6.20160709.fc25    fedora                         302 k
 ponyc-release                  x86_64  0.11.1-78.992314d      bintray--killerswan-ponyc-rpm   14 M
 ponydep-ncurses-libs-5-compat  x86_64  0.1-17.dbf25ec         bintray--killerswan-ponyc-rpm  1.6 k

Transaction Summary
====================================================================================================
Install  3 Packages

Total download size: 14 M
Installed size: 43 M
Is this ok [y/N]: 

On CentOS 7:

[kevin@centos-pony ~]$ sudo yum clean all
...
[kevin@centos-pony ~]$ sudo yum install ponyc-release
...
Resolving Dependencies
--> Running transaction check
---> Package ponyc-release.x86_64 0:0.11.1-78.992314d will be installed
--> Processing Dependency: ponydep-ncurses for package: ponyc-release-0.11.1-78.992314d.x86_64
--> Running transaction check
---> Package ponydep-ncurses-libs-5.x86_64 0:0.1-17.dbf25ec will be installed
--> Finished Dependency Resolution

Dependencies Resolved

====================================================================================================
 Package                  Arch     Version              Repository                             Size
====================================================================================================
Installing:
 ponyc-release            x86_64   0.11.1-78.992314d    bintraybintray-killerswan-ponyc-rpm    14 M
Installing for dependencies:
 ponydep-ncurses-libs-5   x86_64   0.1-17.dbf25ec       bintraybintray-killerswan-ponyc-rpm   1.5 k

Transaction Summary
====================================================================================================
Install  1 Package (+1 Dependent package)

Total download size: 14 M
Installed size: 42 M
Is this ok [y/d/N]: 

Hooray!! 🍾

@killerswan
Copy link
Member

killerswan commented Mar 15, 2017

TODO:

  1. Move killerswan/ponydep-ncurses repository into our ponylang org. (I lack permission.) Thanks @SeanTAllen! Now here.
  2. Finish Replace credentials for GitHub and Bintray ponydep-ncurses#1 to update Bintray credentials. (I may be the only one who can get the pony-buildbot-2 API key.)
  3. Add new packages for ponydep-ncurses-libs-5 and ponydep-ncurses-libs-5-compat, here on Bintray.
  4. Start Travis builds for this GitHub repo. See here.
  5. Confirm that the builds are getting deployed correctly. 😅
  6. Confirm that the ponydep-ncurses package installs as expected.
  7. Merge ponyc Add an RPM dependency on ponydep-ncurses #1680, adding that dependency to ponyc-release RPM packages.

@killerswan killerswan changed the title ponyc rpm installs but compiler doesnt work libtinfo.so.5 error ponyc rpm installs but compiler doesnt work libtinfo.so.5 error (ncurses package re-org) Mar 15, 2017
@SeanTAllen
Copy link
Member

@killerswan I believe that my merging #1680 closes this. If I was incorrect to close, please reopen.

@Triangle345 when we do our next release, can you verify that the problem was fixed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
triggers release Major issue that when fixed, results in an "emergency" release
Projects
None yet
Development

No branches or pull requests

7 participants