Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issue building from source using sbt rpm:packageBin #22

Open
oscarrobinson opened this issue Jul 4, 2017 · 11 comments
Open

Issue building from source using sbt rpm:packageBin #22

oscarrobinson opened this issue Jul 4, 2017 · 11 comments

Comments

@oscarrobinson
Copy link

Having pulled the latest source from github, running sbt rpm:packageBin results in the following output:

[warn] Executing in batch mode.
[warn]   For better performance, hit [ENTER] to switch to interactive mode, or
[warn]   consider launching sbt without any commands, or explicitly passing 'shell'
[info] Loading project definition from /Users/Oscar/Desktop/pillar/project
[info] Updating {file:/Users/Oscar/Desktop/pillar/project/}pillar-build...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Set current project to pillar (in build file:/Users/Oscar/Desktop/pillar/)
[error] error: Group field must be present in package: (main package)
[error] error: License field must be present in package: (main package)
[error] error: Package has no %description: de.kaufhof-4.1.0-1.noarch
[info] Building target platforms: noarch-kaufhof-Linux
java.lang.RuntimeException: Unable to run rpmbuild, check output for details. Errorcode 1
	at scala.sys.package$.error(package.scala:27)
	at com.typesafe.sbt.packager.rpm.RpmHelper$$anonfun$buildPackage$1.apply(RpmHelper.scala:110)
	at com.typesafe.sbt.packager.rpm.RpmHelper$$anonfun$buildPackage$1.apply(RpmHelper.scala:87)
	at sbt.IO$.withTemporaryDirectory(IO.scala:358)
	at com.typesafe.sbt.packager.rpm.RpmHelper$.buildPackage(RpmHelper.scala:87)
	at com.typesafe.sbt.packager.rpm.RpmHelper$.buildRpm(RpmHelper.scala:39)
	at com.typesafe.sbt.packager.rpm.RpmPlugin$$anonfun$projectSettings$47.apply(RpmPlugin.scala:158)
	at com.typesafe.sbt.packager.rpm.RpmPlugin$$anonfun$projectSettings$47.apply(RpmPlugin.scala:158)
	at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
	at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
	at sbt.std.Transform$$anon$4.work(System.scala:63)
	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
	at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
	at sbt.Execute.work(Execute.scala:237)
	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
	at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
	at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
[error] (rpm:packageBin) Unable to run rpmbuild, check output for details. Errorcode 1
[error] Total time: 0 s, completed 04-Jul-2017 09:13:14
@el-dom
Copy link

el-dom commented Jul 5, 2017

Gonna need your help here. I added the missing settings but my system is not capable of building a rpm to begin with, so i'm not quite sure if this is sufficient now. Could you pull again and try if you get further?

@oscarrobinson
Copy link
Author

oscarrobinson commented Jul 5, 2017 •
edited
Loading

I can get further now, although it still doesn't work. Error is:

[info] Building target platforms: noarch-kaufhof-Linux
[info] Executing(%install): /bin/sh -e /var/folders/kx/hmdvm1hd4qd4qvygj1cvs6zr0000gn/T/sbt_db969b50/rpm-tmp.45072
[error] + umask 022
[error] + cd /Users/Oscar/Desktop/pillar/target/rpm/BUILD
[error] + /bin/rm -rf /Users/Oscar/Desktop/pillar/target/rpm/buildroot
[error] + /bin/mkdir -p /Users/Oscar/Desktop/pillar/target/rpm/buildroot
[error] + '[' -e /Users/Oscar/Desktop/pillar/target/rpm/buildroot ']'
[error] + mv '/Users/Oscar/Desktop/pillar/target/rpm/tmp-buildroot/*' /Users/Oscar/Desktop/pillar/target/rpm/buildroot
[error] mv: rename /Users/Oscar/Desktop/pillar/target/rpm/tmp-buildroot/* to /Users/Oscar/Desktop/pillar/target/rpm/buildroot/*: No such file or directory
[error] error: Bad exit status from /var/folders/kx/hmdvm1hd4qd4qvygj1cvs6zr0000gn/T/sbt_db969b50/rpm-tmp.45072 (%install)
[info] 
[info] 
[info] RPM build errors:
[error]     Bad exit status from /var/folders/kx/hmdvm1hd4qd4qvygj1cvs6zr0000gn/T/sbt_db969b50/rpm-tmp.45072 (%install)
[trace] Stack trace suppressed: run last rpm:packageBin for the full output.
[error] (rpm:packageBin) Unable to run rpmbuild, check output for details. Errorcode 1
[error] Total time: 0 s, completed 05-Jul-2017 16:29:50

I am building on a Mac so possible this isn't going to work anyway.

@isaias-b
Copy link

isaias-b commented Jul 5, 2017

Stack traces look look the same on the first sight:

> rpm:packageBin
[info] Building target platforms: noarch-kaufhof-Linux
[info] Executing(%install): /bin/sh -e /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/sbt_fc5f2fc8/rpm-tmp.8964
[error] + umask 022
[error] + cd /Users/isaias/Projects/pillar2/target/rpm/BUILD
[error] + /bin/rm -rf /Users/isaias/Projects/pillar2/target/rpm/buildroot
[error] + /bin/mkdir -p /Users/isaias/Projects/pillar2/target/rpm/buildroot
[error] + '[' -e /Users/isaias/Projects/pillar2/target/rpm/buildroot ']'
[error] + mv '/Users/isaias/Projects/pillar2/target/rpm/tmp-buildroot/*' /Users/isaias/Projects/pillar2/target/rpm/buildroot
[error] mv: rename /Users/isaias/Projects/pillar2/target/rpm/tmp-buildroot/* to /Users/isaias/Projects/pillar2/target/rpm/buildroot/*: No such file or directory
[error] error: Bad exit status from /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/sbt_fc5f2fc8/rpm-tmp.8964 (%install)
[info]
[info]
[error]     Bad exit status from /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/sbt_fc5f2fc8/rpm-tmp.8964 (%install)
[info] RPM build errors:
java.lang.RuntimeException: Unable to run rpmbuild, check output for details. Errorcode 1
	at scala.sys.package$.error(package.scala:27)
	at com.typesafe.sbt.packager.rpm.RpmHelper$$anonfun$buildPackage$1.apply(RpmHelper.scala:110)
	at com.typesafe.sbt.packager.rpm.RpmHelper$$anonfun$buildPackage$1.apply(RpmHelper.scala:87)
[trace] Stack trace suppressed: run last rpm:packageBin for the full output.
[error] (rpm:packageBin) Unable to run rpmbuild, check output for details. Errorcode 1
[error] Total time: 0 s, completed Jul 5, 2017 7:12:26 PM
> last rpm:packageBin
java.lang.RuntimeException: Unable to run rpmbuild, check output for details. Errorcode 1
	at scala.sys.package$.error(package.scala:27)
	at com.typesafe.sbt.packager.rpm.RpmHelper$$anonfun$buildPackage$1.apply(RpmHelper.scala:110)
	at com.typesafe.sbt.packager.rpm.RpmHelper$$anonfun$buildPackage$1.apply(RpmHelper.scala:87)
	at sbt.IO$.withTemporaryDirectory(IO.scala:358)
	at com.typesafe.sbt.packager.rpm.RpmHelper$.buildPackage(RpmHelper.scala:87)
	at com.typesafe.sbt.packager.rpm.RpmHelper$.buildRpm(RpmHelper.scala:39)
	at com.typesafe.sbt.packager.rpm.RpmPlugin$$anonfun$projectSettings$47.apply(RpmPlugin.scala:158)
	at com.typesafe.sbt.packager.rpm.RpmPlugin$$anonfun$projectSettings$47.apply(RpmPlugin.scala:158)
	at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
	at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
	at sbt.std.Transform$$anon$4.work(System.scala:63)
	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
	at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
	at sbt.Execute.work(Execute.scala:237)
	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
	at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
	at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
[error] (rpm:packageBin) Unable to run rpmbuild, check output for details. Errorcode 1

I also tried reducing the overall dependencies to generate a rpm enabled minimal working example with this build.sbt file:

import sbt._
import Keys._

fork in Test := true

val rpmSettings = {
  Seq(
    rpmVendor := "kaufhof",
    rpmUrl := Some("https://github.com/Galeria-Kaufhof/pillar"),
    rpmGroup := Some("Development/Others"),
    linuxPackageMappings in Rpm := linuxPackageMappings.value,
    packageName in Rpm := "de.kaufhof",
    packageDescription in Rpm := """Pillar manages migrations for your Cassandra data stores.""",
//    packageArchitecture in Rpm := "noarch",
    version in Rpm := version.value,
    rpmRelease in Rpm := "1",
    packageSummary in Rpm := "Pillar manages migrations for your Cassandra data stores.",
    rpmLicense := Some("http://www.opensource.org/licenses/mit-license.php")
  )
}

lazy val root = Project(
  id = "pillar",
  base = file("."),
  settings = Defaults.coreDefaultSettings
).settings(
    traceLevel := 0,
    name := "pillar",
    organization := "de.kaufhof",
    version := "4.1.0",
    scalaVersion := "2.12.2"
  )
  .settings(
    parallelExecution in Test := false
  )
  .settings(rpmSettings)
  .enablePlugins(RpmPlugin)

//rpmBrpJavaRepackJars := false

I also removed all the code in src and replaced it with a HelloWorld application.
Still the same output.

I could reproduce this with:

$ rpmbuild --version
rpmbuild (RPM) 5.4.15

I will need to check version 4 due to the information given kafka manager's issue 53. But first, @scarrobin can you confirm that you are using version 5.x?

Due to the stacktrace given by @puneetsaha i looked up the sources of RpmHelper.scala at Line 89 but the packageArchitecture := "noarch" didn't seem to influence the problem.

@el-dom
Copy link

el-dom commented Jul 5, 2017

sbt/sbt-native-packager#266 also claims that things are only tested against rpm 4
@isaias-b Might be worth giving rpmBrpJavaRepackJars := true a shot

@oscarrobinson
Copy link
Author

@isaias-b Yes, I am using rpm v5.4.15

@isaias-b
Copy link

isaias-b commented Jul 6, 2017

@el-dom I have tested rpmBrpJavaRepackJars yesterday already with true and false, but forgot to mention that it doesn't affect the stack traces as well. Just retested it, but still same result.

@scarrobin I tried downgrading rpm to a 4.x with brew yesterday but didn't succeed. I am unable to use this homebrew thing :D. I will try later in the afternoon again or maybe you can give it a try?

@el-dom
Copy link

el-dom commented Jul 10, 2017

Since the native packager doesn't seem to work too well when it comes to building RPMs outside of a redhat based distribution, i removed it and reactivated the old rhPackage task. So given that fpm is installed, it should now be possible to build the rpm with sbt rhPackage again. But i still need confirmation for that, since fpm is also not working on my machine due to a too new dependency (and no way to downgrade)

 fpm --version
/usr/lib/ruby/2.4.0/rubygems/dependency.rb:310:in `to_specs': Could not find 'clamp' (~> 1.0.0) - did find: [clamp-1.1.2] (Gem::MissingSpecVersionError)
Checked in 'GEM_PATH=/home/dom/.gem/ruby/2.4.0:/usr/lib/ruby/gems/2.4.0', execute `gem env` for more information
	from /usr/lib/ruby/2.4.0/rubygems/specification.rb:1442:in `block in activate_dependencies'
	from /usr/lib/ruby/2.4.0/rubygems/specification.rb:1431:in `each'
	from /usr/lib/ruby/2.4.0/rubygems/specification.rb:1431:in `activate_dependencies'
	from /usr/lib/ruby/2.4.0/rubygems/specification.rb:1413:in `activate'
	from /usr/lib/ruby/2.4.0/rubygems.rb:299:in `block in activate_bin_path'
	from /usr/lib/ruby/2.4.0/rubygems.rb:299:in `synchronize'
	from /usr/lib/ruby/2.4.0/rubygems.rb:299:in `activate_bin_path'
	from /usr/bin/fpm:22:in `<main>'

@isaias-b
Copy link

Seems to be our best option ATM

@davidnadeau
Copy link

davidnadeau commented Jul 14, 2017 •
edited
Loading

sbt rhPackage builds the rpm. However when I run sudo rpm -i target/pillar-4.1.0-DEV.noarch.rpm I get

error: error(22) getting keys from Filepaths index
error: error(1) getting records from Filepaths index
error: error(22) getting keys from Filepaths index
error: error(1) getting records from Filepaths index
error: error(22) getting keys from Filepaths index
error: error(1) getting records from Filepaths index
error: Failed dependencies:
	/opt/pillar/bin is needed by pillar-4.1.0-DEV.noarch
	/opt/pillar/conf is needed by pillar-4.1.0-DEV.noarch
	/opt/pillar/lib is needed by pillar-4.1.0-DEV.noarch

Am I missing something?

fpm --version
1.8.1
rpmbuild --version
rpmbuild (RPM) 5.4.15

@isaias-b
Copy link

Whatever the missing bit might be, i am running into the same issue. But i saw some suspicious warnings in the sbt log about things being discarded. I am no expert in this topic but maybe someone more experienced has some suggestions according to the log. However, @davidnadeau can you confirm that the sbt log on your side looks the same?

$ sbt rhPackage
[warn] Executing in batch mode.
[warn]   For better performance, hit [ENTER] to switch to interactive mode, or
[warn]   consider launching sbt without any commands, or explicitly passing 'shell'
[info] Loading global plugins from /Users/isaias/.sbt/0.13/plugins
[info] Loading project definition from /Users/isaias/Projects/pillar/project
[info] Set current project to pillar (in build file:/Users/isaias/Projects/pillar/)
[info] Including from cache: netty-transport-4.0.37.Final.jar
[info] Including from cache: netty-buffer-4.0.37.Final.jar
[info] Including from cache: netty-handler-4.0.37.Final.jar
[info] Including from cache: asm-analysis-5.0.3.jar
[info] Including from cache: jffi-1.2.10.jar
[info] Including from cache: asm-5.0.3.jar
[info] Including from cache: asm-util-5.0.3.jar
[info] Including from cache: asm-commons-5.0.3.jar
[info] Including from cache: netty-codec-4.0.37.Final.jar
[info] Including from cache: cassandra-driver-core-3.1.4.jar
[info] Including from cache: jffi-1.2.10-native.jar
[info] Including from cache: netty-common-4.0.37.Final.jar
[info] Including from cache: jnr-x86asm-1.0.2.jar
[info] Including from cache: asm-tree-5.0.3.jar
[info] Including from cache: jnr-posix-3.0.27.jar
[info] Including from cache: slf4j-api-1.7.7.jar
[info] Including from cache: metrics-core-3.1.2.jar
[info] Including from cache: config-1.3.1.jar
[info] Including from cache: jnr-constants-0.9.0.jar
[info] Including from cache: jnr-ffi-2.0.7.jar
[info] Including from cache: guava-16.0.1.jar
[info] Including from cache: scala-library-2.12.2.jar
[info] Checking every *.class/*.jar file's SHA-1.
[info] Merging files...
[warn] Merging 'META-INF/INDEX.LIST' with strategy 'discard'
[warn] Merging 'META-INF/MANIFEST.MF' with strategy 'discard'
[warn] Merging 'META-INF/io.netty.versions.properties' with strategy 'last'
[warn] Merging 'META-INF/maven/com.datastax.cassandra/cassandra-driver-core/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.datastax.cassandra/cassandra-driver-core/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.github.jnr/jffi/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.github.jnr/jffi/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.github.jnr/jnr-constants/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.github.jnr/jnr-constants/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.github.jnr/jnr-ffi/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.github.jnr/jnr-ffi/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.github.jnr/jnr-posix/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.github.jnr/jnr-posix/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.github.jnr/jnr-x86asm/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.github.jnr/jnr-x86asm/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.google.guava/guava/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.google.guava/guava/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/io.dropwizard.metrics/metrics-core/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/io.dropwizard.metrics/metrics-core/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/io.netty/netty-buffer/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/io.netty/netty-buffer/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/io.netty/netty-codec/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/io.netty/netty-codec/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/io.netty/netty-common/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/io.netty/netty-common/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/io.netty/netty-handler/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/io.netty/netty-handler/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/io.netty/netty-transport/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/io.netty/netty-transport/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/org.slf4j/slf4j-api/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/org.slf4j/slf4j-api/pom.xml' with strategy 'discard'
[warn] Strategy 'discard' was applied to 30 files
[warn] Strategy 'last' was applied to a file
[info] Assembly up to date: /Users/isaias/Projects/pillar/target/scala-2.12/pillar-assembly-4.1.0.jar
{:timestamp=>"2017-07-14T19:25:23.336520+0200", :message=>"Force flag given. Overwriting package at /Users/isaias/Projects/pillar/target/pillar-4.1.0-DEV.noarch.rpm", :level=>:warn}
{:timestamp=>"2017-07-14T19:25:24.324394+0200", :message=>"Created package", :path=>"/Users/isaias/Projects/pillar/target/pillar-4.1.0-DEV.noarch.rpm"}
[success] Total time: 4 s, completed Jul 14, 2017 7:25:24 PM

Running this with rpm leads then to the exact same output as yours:

$ rpm -i target/pillar-4.1.0-DEV.noarch.rpm
error: error(22) getting keys from Filepaths index
error: error(1) getting records from Filepaths index
error: error(22) getting keys from Filepaths index
error: error(1) getting records from Filepaths index
error: error(22) getting keys from Filepaths index
error: error(1) getting records from Filepaths index
error: Failed dependencies:
  /opt/pillar/bin is needed by pillar-4.1.0-DEV.noarch
  /opt/pillar/conf is needed by pillar-4.1.0-DEV.noarch
  /opt/pillar/lib is needed by pillar-4.1.0-DEV.noarch

@davidnadeau
Copy link

@isaias-b sbt rhPackage gives me the same output.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@davidnadeau @oscarrobinson @el-dom @isaias-b