make: Add macos M1 support

[cdecker]

The M1 Macs support both x86_64 and arm64 architectures, which forced
homebrew to use a different path for its storage (/opt/homebrew/
instead of /usr/local). If we don't adjust the path we'd mix x86_64
and arm64 libraries which can lead to weird compiler and linker
errors.

This patch just introduces CPATH and LIBRARY_PATH as suggested by
the homebrew team, and detects the current architecture automatically.

Changelog-Added: macos: Added m1 architecture support for macos

[vincenzopalazzo]

LGTM, but have no possibility to test.

ack b76c377

[cdecker]

I'll likely be adding a few more fixes to this PR before undrafting it, but it should be ready for the next release.

[valentinewallace]

FYI, this is what ended up working for me to build on the M1:

1. Checkout v0.10.2
2. Cherry-pick commits from make: Add macos M1 support #4988
   1. HEAD at the time: 0b26ae370d319555769a60f407e71706d22b9713
3. python3.9 -m venv venv
4. source venv/bin/activate
5. export CPATH=/opt/homebrew/include
6. export LIBRARY_PATH=/opt/homebrew/lib
7. export CFLAGS=-Wno-error=implicit-function-declaration
8. pip3 install -r requirements.txt
9. PG_CONFIG='' ./configure
10. make -j8 # This will fail
11. pip3 install MarkupSafe==2.0.1
12. make -j8 # This will fail again
13. pip3 install mistune==0.8.4

cc @dongcarl

[vincenzopalazzo]

@valentinewallace have you tried the pip3 install -r requirements.lock may fix your issue

[dongcarl]

@vincenzopalazzo

Right, that solves all the specific pip install problems. There are a few extra things needed in addition to @valentinewallace's instructions on a fresh M1 mac (before running pip3 install -r requirements.lock)

1. You need to brew install postgres since otherwise setup.py for psycopg2-binary will compain that:

  Error: pg_config executable not found.

  pg_config is required to build psycopg2 from source.  Please add the directory
  containing pg_config to the $PATH or specify the full executable path with the
  option:

      python setup.py build_ext --pg-config /path/to/pg_config build ...

  or with the pg_config option in 'setup.cfg'.

  If you prefer to avoid building psycopg2 from source, please install the PyPI
  'psycopg2-binary' package instead.

  For further information please check the 'doc/src/install.rst' file (also at
  <https://www.psycopg.org/docs/install.html>).

2. You need to brew install openssl and set export LDFLAGS="-L/opt/homebrew/opt/openssl@3/lib" since otherwise setup.py for psycopg2-binary will compain that: ld: library not found for -lssl. Note that export LIBRARY_PATH=/opt/homebrew/lib is not sufficient here because openssl is not symlinked into /opt/homebrew/lib. See brew info openssl for more details

Overall, I think a good solution might be to have ./configure generate contrib/pyln-testing/requirements.txt from a contrib/pyln-testing/requirements.txt.in, so that installing postgres isn't a hard requirement for devving on C-Lightning.

[cdecker]

Ok, while it didn't address the failing tests yet, this should be merged so we can at least compile for M1s.

[ddustin]

LGTM but I don't have an M1 to test on either.

ACK 5c3e07e

[cdecker]

It's been a while and this has been working ok on my M1, so let's merge it.

[jb55]

@cdecker I am getting

make distclean
./configure
...
*** We need a database, but neither sqlite3 nor postgres found

on my linux machine

and I bisected it to:

e3f53e0

[justinmoon]

This worked on my m1 a month or so ago, btw.

[cdecker]

@cdecker I am getting

make distclean ./configure ... *** We need a database, but neither sqlite3 nor postgres found

on my linux machine

and I bisected it to:

e3f53e0

What linker flags are being passed to the compiler? You can make compilation verbose via make V=1 iirc. Seems the linker path and/or include paths may be wrongly identified.

[jb55]

On Tue, Jul 12, 2022 at 04:56:58AM -0700, Christian Decker wrote: > @cdecker I am getting > > `make distclean` `./configure` `...` `*** We need a database, but neither sqlite3 nor postgres found` > > on my linux machine > > and I bisected it to: > > [e3f53e0](e3f53e0) What linker flags are being passed to the compiler? You can make compilation verbose via `make V=1` iirc. Seems the linker path and/or include paths may be wrongly identified.

looks like I was just missing pkg-config from my local dev dependencies.