Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build fail #127

Closed
ghost opened this issue Oct 24, 2016 · 9 comments
Closed

Build fail #127

ghost opened this issue Oct 24, 2016 · 9 comments

Comments

@ghost
Copy link

ghost commented Oct 24, 2016

Attempting build on OS X 10.11.5 (15F34).
Followed all steps without warnings or errors, until make.
make failed with the following output.

$make
/Applications/Xcode.app/Contents/Developer/usr/bin/make  all-recursive
Making all in src
Making all in sparkey
make[3]: Nothing to be done for `all'.
./libpostal_data download all /applications/libpostal
Checking for new libpostal data file...
libpostal data file up to date
Checking for new libpostal geodb data file...
New libpostal geodb data file available
Downloading multipart: http://libpostal.s3.amazonaws.com/geodb.tar.gz, size=486746092, num_chunks=7
Downloading part 1: filename=/applications/libpostal/geodb.tar.gz.1, offset=0, max=67108863
Downloading part 2: filename=/applications/libpostal/geodb.tar.gz.2, offset=67108864, max=134217727
Downloading part 3: filename=/applications/libpostal/geodb.tar.gz.3, offset=134217728, max=201326591
Downloading part 4: filename=/applications/libpostal/geodb.tar.gz.4, offset=201326592, max=268435455
Downloading part 5: filename=/applications/libpostal/geodb.tar.gz.5, offset=268435456, max=335544319
Downloading part 6: filename=/applications/libpostal/geodb.tar.gz.6, offset=335544320, max=402653183
Downloading part 7: filename=/applications/libpostal/geodb.tar.gz.7, offset=402653184, max=486746092
make[3]: *** [all-local] Error 1
make[2]: *** [all-recursive] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

I googled the error and found identical output in openvenues/node-postal#6(closed). @jeffriesen 'got it running' on Aug 25. In this repo, the last four lines of my output match #13(closed). I followed chamnap's solution in #5(closed), i.e. I did brew upgrade, with result pkg-config 0.29.1; then, I ran make again, but had the same output, except Downloading part 3 and Downloading part 4 were reversed - Downloading part 4: filename=/applications/libpostal/geodb.tar.gz.4, offset=201326592, max=268435455 Downloading part 3: filename=/applications/libpostal/geodb.tar.gz.3, offset=134217728, max=201326591.

@albarrentine
Copy link
Contributor

Does the dir you specified for --datadir when running configure have enough space? It needs approximately 2.2G free to build the current version of libpostal (that number may change in subsequent updates to the models).

That what it sounds like to me. At that point in the build it's just trying to download chunks of a file in parallel using curl, so probably either lack of disk space or some kind S3 connection weirdness.

@ghost
Copy link
Author

ghost commented Oct 28, 2016

The disk (Macintosh HD) that houses the dir I specified for --datadir had approximately 4G free. I freed up an order of magnitude more space and ran it again; It generated the same output.

@albarrentine
Copy link
Contributor

Ok, and the datadir you want is /applications/libpostal, correct? That's what's being used, so if you moved it somewhere along the way, you'll have to re-run configure. That's also probably a root-owned directory, might need to chown it to your user. That doesn't seem likely though because It looks like it did download one of the files (libpostal_data.tar.gz, which is about 10M) successfully but failed on geodb.tar.gz, which is ~400M.

You can also try manually downloading the files. If you have awscli installed, it's aws s3 cp s3://libpostal/geodb.tar.gz $LIBPOSTAL_DATA_DIR. Otherwise it's curl http://libpostal.s3.amazonaws.com/geodb.tar.gz -O $LIBPOSTAL_DATA_DIR/geodb.tar.gz. If downloading that way works, then maybe there's a problem with the data download script, but I very much suspect it's space/permissions.

@ghost
Copy link
Author

ghost commented Oct 29, 2016

Yes, the libpostal dir. For me it's /desktop/libpostal.

I ran curl http://libpostal.s3.amazonaws.com/geodb.tar.gz -O $LIBPOSTAL_DATA_DIR/geodb.tar.gz

Output:
0curl: (56) Recv failure: Connection reset by peer
curl: (3) <url> malformed

@albarrentine
Copy link
Contributor

Oh, my typo. The curl command is -o lowercase, not -O. Will want to set LIBPOSTAL_DATA_DIR=/desktop/libpostal before running that.

Also, I can see in the make output that the dir being used is /applications/libpostal, so you'll need to run configure --datadir=/desktop/libpostal to set it to the correct location as far as make is concerned.

@ghost
Copy link
Author

ghost commented Oct 31, 2016

curl http://libpostal.s3.amazonaws.com/geodb.tar.gz -o /desktop/libpostal/geodb.tar.gz succeeded.

I moved the libpostal dir to /Applications, and ran make there. As discussed, make fails at downloading part 7 of geodb.tar.gz. I tried:

/Applications/libpostal$ src/address_parser
dyld: Library not loaded: /usr/local/lib/libpostal.0.dylib
  Referenced from: /Applications/libpostal/src/.libs/address_parser
  Reason: image not found
Trace/BPT trap: 5

@albarrentine
Copy link
Contributor

Ok, I'm not clear on where you want the datadir to be, /desktop or /Applicaitons. When you say "I moved the libpostal dir" is that with mv or did you run ./configure again?

Try nuking the entire libpostal checkout and start over following the exact steps in the Readme using whichever datadir you chose.

@ghost
Copy link
Author

ghost commented Oct 31, 2016

I apologize for the possibly unnecessary length.

I moved the libpostal dir with mv and then ran ./configure again:

/users/samuelputnam/desktop$ mv libpostal /applications
/users/samuelputnam/desktop$ cd /applications/libpostal
/applications/libpostal$ ./bootstrap.sh
glibtoolize: putting auxiliary files in '.'.
glibtoolize: copying file './ltmain.sh'
glibtoolize: Consider adding 'AC_CONFIG_MACRO_DIRS([m4])' to configure.ac,
glibtoolize: and rerunning glibtoolize and aclocal.
glibtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am.
configure.ac:8: installing './compile'
configure.ac:6: installing './missing'
src/Makefile.am: installing './depcomp'

/applications/libpostal$ ./configure --datadir=/applications/libpostal
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
# I am leaving out output lines to save space in this comment
config.status: executing depfiles commands
config.status: executing libtool commands

/applications/libpostal$ make
/Applications/Xcode.app/Contents/Developer/usr/bin/make  all-recursive
Making all in src
Making all in sparkey
make[3]: Nothing to be done for `all'.
./libpostal_data download all /applications/libpostal/libpostal
Checking for new libpostal data file...
libpostal data file up to date
Checking for new libpostal geodb data file...
New libpostal geodb data file available
Downloading multipart: http://libpostal.s3.amazonaws.com/geodb.tar.gz, size=486746092, num_chunks=7
Downloading part 1: filename=/applications/libpostal/libpostal/geodb.tar.gz.1, offset=0, max=67108863
Downloading part 3: filename=/applications/libpostal/libpostal/geodb.tar.gz.3, offset=134217728, max=201326591
Downloading part 4: filename=/applications/libpostal/libpostal/geodb.tar.gz.4, offset=201326592, max=268435455
Downloading part 2: filename=/applications/libpostal/libpostal/geodb.tar.gz.2, offset=67108864, max=134217727
Downloading part 5: filename=/applications/libpostal/libpostal/geodb.tar.gz.5, offset=268435456, max=335544319
Downloading part 6: filename=/applications/libpostal/libpostal/geodb.tar.gz.6, offset=335544320, max=402653183
Downloading part 7: filename=/applications/libpostal/libpostal/geodb.tar.gz.7, offset=402653184, max=486746092
^Z
[3]+  Stopped                 make

I stopped it because it did not continue for an amount of time similar to that when it had failed at this step.

Then, nuking the entire libpostal checkout and start over following the exact steps in the Readme using whichever datadir you chose.

/applications/libpostal$ cd ..
/Applications$ rm -rf libpostal
/Applications$ brew uninstall snappy autoconf automake libtool pkg-config
Uninstalling /usr/local/Cellar/snappy/1.1.3... (20 files, 411.9K)
Uninstalling /usr/local/Cellar/autoconf/2.69... (70 files, 3.0M)
Uninstalling /usr/local/Cellar/automake/1.15... (130 files, 2.9M)
Uninstalling /usr/local/Cellar/libtool/2.4.6... (70 files, 3.7M)
Uninstalling /usr/local/Cellar/pkg-config/0.29.1... (10 files, 627.2K)

/Applications$ brew install snappy autoconf automake libtool pkg-config
==> Downloading https://homebrew.bintray.com/bottles/snappy-1.1.3.el_capitan.bot
Already downloaded: /Library/Caches/Homebrew/snappy-1.1.3.el_capitan.bottle.tar.gz
==> Pouring snappy-1.1.3.el_capitan.bottle.tar.gz
🍺  /usr/local/Cellar/snappy/1.1.3: 20 files, 411.9K
==> Downloading https://homebrew.bintray.com/bottles/autoconf-2.69.el_capitan.bo
Already downloaded: /Library/Caches/Homebrew/autoconf-2.69.el_capitan.bottle.4.tar.gz
==> Pouring autoconf-2.69.el_capitan.bottle.4.tar.gz
==> Caveats
Emacs Lisp files have been installed to:
  /usr/local/share/emacs/site-lisp/autoconf
==> Summary
🍺  /usr/local/Cellar/autoconf/2.69: 70 files, 3.0M
==> Downloading https://homebrew.bintray.com/bottles/automake-1.15.el_capitan.bo
Already downloaded: /Library/Caches/Homebrew/automake-1.15.el_capitan.bottle.2.tar.gz
==> Pouring automake-1.15.el_capitan.bottle.2.tar.gz
🍺  /usr/local/Cellar/automake/1.15: 130 files, 2.9M
==> Downloading https://homebrew.bintray.com/bottles/libtool-2.4.6.el_capitan.bo
Already downloaded: /Library/Caches/Homebrew/libtool-2.4.6.el_capitan.bottle.tar.gz
==> Pouring libtool-2.4.6.el_capitan.bottle.tar.gz
==> Caveats
In order to prevent conflicts with Apple's own libtool we have prepended a "g"
so, you have instead: glibtool and glibtoolize.
==> Summary
🍺  /usr/local/Cellar/libtool/2.4.6: 70 files, 3.7M
==> Downloading https://homebrew.bintray.com/bottles/pkg-config-0.29.1.el_capita
Already downloaded: /Library/Caches/Homebrew/pkg-config-0.29.1.el_capitan.bottle.tar.gz
==> Pouring pkg-config-0.29.1.el_capitan.bottle.tar.gz
🍺  /usr/local/Cellar/pkg-config/0.29.1: 10 files, 627.2K

/Applications$ git clone https://github.com/openvenues/libpostal
Cloning into 'libpostal'...
remote: Counting objects: 21231, done.
remote: Compressing objects: 100% (24/24), done.
error: RPC failed; curl 56 SSLRead() return error -365.00 KiB/s   
fatal: The remote end hung up unexpectedly
fatal: early EOF
fatal: index-pack failed
/Applications$ 

Trying git clone again:

/Applications$ git clone https://github.com/openvenues/libpostal
Cloning into 'libpostal'...
remote: Counting objects: 21231, done.
remote: Compressing objects: 100% (24/24), done.
remote: Total 21231 (delta 6), reused 0 (delta 0), pack-reused 21207
Receiving objects: 100% (21231/21231), 32.36 MiB | 328.00 KiB/s, done.
Resolving deltas: 100% (12741/12741), done.
Checking connectivity... done.

Continuing:

/Applications$ cd libpostal
/Applications/libpostal$ ./bootstrap.sh
glibtoolize: putting auxiliary files in '.'.
glibtoolize: copying file './ltmain.sh'
glibtoolize: Consider adding 'AC_CONFIG_MACRO_DIRS([m4])' to configure.ac,
glibtoolize: and rerunning glibtoolize and aclocal.
glibtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am.
configure.ac:8: installing './compile'
configure.ac:8: installing './config.guess'
configure.ac:8: installing './config.sub'
configure.ac:6: installing './install-sh'
configure.ac:6: installing './missing'
src/Makefile.am: installing './depcomp'
parallel-tests: installing './test-driver'

Applications/libpostal$ ./configure --datadir=/applications/libpostal
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
# I am leaving out output lines to save space in this comment
config.status: executing depfiles commands
config.status: executing libtool commands

/Applications/libpostal$ make
/Applications/Xcode.app/Contents/Developer/usr/bin/make  all-recursive
Making all in src
Making all in sparkey
depbase=`echo buf.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
    /bin/sh ../../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I../..     -I/usr/local/include -O2 -Wall -Wextra -Wfloat-equal -Wshadow -Wpointer-arith -Werror -pedantic -MT buf.lo -MD -MP -MF $depbase.Tpo -c -o buf.lo buf.c &&\
    mv -f $depbase.Tpo $depbase.Plo
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../.. -I/usr/local/include -O2 -Wall -Wextra -Wfloat-equal -Wshadow -Wpointer-arith -Werror -pedantic -MT buf.lo -MD -MP -MF .deps/buf.Tpo -c buf.c  -fno-common -DPIC -o .libs/buf.o
# I am leaving out output lines to save space in this comment
language_classifier_test-string_utils.o language_classifier_test-file_utils.o utf8proc/language_classifier_test-utf8proc.o language_classifier_test-unicode_scripts.o  -L/usr/local/lib ./.libs/libscanner.a -lsnappy
./libpostal_data download all /applications/libpostal/libpostal
Checking for new libpostal data file...
New libpostal data file available
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 10.1M  100 10.1M    0     0   300k      0  0:00:34  0:00:34 --:--:--  313k
x address_expansions/
x address_expansions/address_dictionary.dat
x numex/
x numex/numex.dat
x transliteration/
x transliteration/transliteration.dat
Checking for new libpostal geodb data file...
New libpostal geodb data file available
Downloading multipart: http://libpostal.s3.amazonaws.com/geodb.tar.gz, size=486746092, num_chunks=7
Downloading part 2: filename=/applications/libpostal/libpostal/geodb.tar.gz.2, offset=67108864, max=134217727
Downloading part 3: filename=/applications/libpostal/libpostal/geodb.tar.gz.3, offset=134217728, max=201326591
Downloading part 1: filename=/applications/libpostal/libpostal/geodb.tar.gz.1, offset=0, max=67108863
Downloading part 4: filename=/applications/libpostal/libpostal/geodb.tar.gz.4, offset=201326592, max=268435455
Downloading part 5: filename=/applications/libpostal/libpostal/geodb.tar.gz.5, offset=268435456, max=335544319
Downloading part 6: filename=/applications/libpostal/libpostal/geodb.tar.gz.6, offset=335544320, max=402653183
Downloading part 7: filename=/applications/libpostal/libpostal/geodb.tar.gz.7, offset=402653184, max=486746092

This time, make moved beyond the Downloading part 7: filename=/applications/libpostal/libpostal/geodb.tar.gz.7, offset=402653184, max=486746092 step after 30 mins and failed at Downloading part 10: filename=/applications/libpostal/libpostal/language_classifier.tar.gz.10, offset=603979776, max=672728402.

x geodb/
x geodb/geodb.spi
x geodb/geodb.spl
x geodb/geodb_feature_graph.dat
x geodb/geodb_features.trie
x geodb/geodb_names.trie
x geodb/geodb_postal_codes.dat
Checking for new libpostal parser data file...
New libpostal parser data file available
Downloading multipart: http://libpostal.s3.amazonaws.com/parser.tar.gz, size=190250553, num_chunks=2
Downloading part 1: filename=/applications/libpostal/libpostal/parser.tar.gz.1, offset=0, max=67108863
Downloading part 2: filename=/applications/libpostal/libpostal/parser.tar.gz.2, offset=67108864, max=190250553
x address_parser/
x address_parser/address_parser.dat
x address_parser/address_parser_vocab.trie
x address_parser/address_parser_phrases.trie
Checking for new libpostal language classifier data file...
New libpostal language classifier data file available
Downloading multipart: http://libpostal.s3.amazonaws.com/language_classifier.tar.gz, size=672728402, num_chunks=10
Downloading part 1: filename=/applications/libpostal/libpostal/language_classifier.tar.gz.1, offset=0, max=67108863
Downloading part 2: filename=/applications/libpostal/libpostal/language_classifier.tar.gz.2, offset=67108864, max=134217727
Downloading part 3: filename=/applications/libpostal/libpostal/language_classifier.tar.gz.3, offset=134217728, max=201326591
Downloading part 4: filename=/applications/libpostal/libpostal/language_classifier.tar.gz.4, offset=201326592, max=268435455
Downloading part 5: filename=/applications/libpostal/libpostal/language_classifier.tar.gz.5, offset=268435456, max=335544319
Downloading part 6: filename=/applications/libpostal/libpostal/language_classifier.tar.gz.6, offset=335544320, max=402653183
Downloading part 7: filename=/applications/libpostal/libpostal/language_classifier.tar.gz.7, offset=402653184, max=469762047
Downloading part 8: filename=/applications/libpostal/libpostal/language_classifier.tar.gz.8, offset=469762048, max=536870911
Downloading part 9: filename=/applications/libpostal/libpostal/language_classifier.tar.gz.9, offset=536870912, max=603979775
Downloading part 10: filename=/applications/libpostal/libpostal/language_classifier.tar.gz.10, offset=603979776, max=672728402
make[3]: *** [all-local] Error 1
make[2]: *** [all-recursive] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

Based on the content of libpostal_data,

fi

LIBPOSTAL_S3_BUCKET_NAME="libpostal"
LIBPOSTAL_S3_KEY="s3://$LIBPOSTAL_S3_BUCKET_NAME"
LIBPOSTAL_S3_BUCKET_URL="http://$LIBPOSTAL_S3_BUCKET_NAME.s3.amazonaws.com"
LIBPOSTAL_DATA_FILE="libpostal_data.tar.gz"
LIBPOSTAL_GEODB_FILE="geodb.tar.gz"
LIBPOSTAL_PARSER_FILE="parser.tar.gz"
LIBPOSTAL_LANG_CLASS_FILE="language_classifier.tar.gz"

COMMAND=$1

it looks like language_classifier.tar.gz could be the final download during make, so:

/Applications/libpostal$ curl http://libpostal.s3.amazonaws.com/language_classifier.tar.gz -o /applications/libpostal/language_classifier.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  641M  100  641M    0     0   204k      0  0:53:37  0:53:37 --:--:--  134k
/Applications/libpostal$ open language_classifier.tar.gz

Trying it out:

/applications/libpostal$ src/language_classifier "One-hundred twenty E 96th St"
ERR   transliteration table is NULL. Call libpostal_setup() or transliteration_module_setup()
   at transliterate (transliterate.c:674) errno: None
Languages:

en (0.999715)

And address_parser:

/applications/libpostal$ src/address_parser
Loading models...

Welcome to libpostal's address parser.

Type in any address to parse and print the result.

Special commands:

.language [code] to specify a language
.country [code] to specify a country
.exit to quit the program

> "Quatre vingt douze Ave des Champs-Elysees"

Result:

{
  "house": "\" quatre vingt",
  "road": "douze ave des",
  "house": "champs-elysees \""
}

> 

Seems all is good here. Thank you.

@ghost ghost closed this as completed Oct 31, 2016
@wnasich
Copy link

wnasich commented Sep 8, 2021

What are the steps if I want to remove it from my system ?

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants