macOS build docs need refresh (2019)

[jaraco]

BPO	35905
Nosy	@warsaw, @jaraco, @ned-deily

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ned-deily'
closed_at = None
created_at = <Date 2019-02-06.00:12:22.643>
labels = ['3.8', 'docs']
title = 'macOS build docs need refresh (2019)'
updated_at = <Date 2022-03-24.16:40:40.166>
user = 'https://github.com/jaraco'

bugs.python.org fields:

activity = <Date 2022-03-24.16:40:40.166>
actor = 'Battant'
assignee = 'ned.deily'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2019-02-06.00:12:22.643>
creator = 'jaraco'
dependencies = []
files = []
hgrepos = []
issue_num = 35905
keywords = []
message_count = 4.0
messages = ['334895', '334898', '334943', '415959']
nosy_count = 5.0
nosy_names = ['barry', 'jaraco', 'ned.deily', 'docs@python', 'Battant']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue35905'
versions = ['Python 3.8']

[jaraco]

In python/devguide#453 (comment), I understand that Ned wishes to update the macOS build docs prior to linking to them from the dev guide.

[warsaw]

All I know is that for 3.7 and 3.8 (3.6 is different), I have this little helper script to build against Homebrew libraries.

#!/bin/sh

export CPPFLAGS="-I$(brew --prefix sqlite3)/include -I$(brew --prefix zlib)/include"
export LDFLAGS="-L$(brew --prefix sqlite3)/lib -L$(brew --prefix zlib)/lib"
export PKG_CONFIG_PATH="/usr/local/opt/openssl/lib/pkgconfig"

./configure

[jaraco]

I also have a script that does something very similar (https://github.com/jaraco/jaraco.develop/blob/master/jaraco/develop/macos-build-python.py), invoked with python -m jaraco.develop.macos-build-python (or pip-run -m jaraco.develop -- -m jaraco.develop.macos-build-python if you don't have it installed).

[Battant]

Hello,

I tried to configure my installation according your instructions

Her's step I follow
export CPPFLAGS="-I$(brew --prefix sqlite3)/include -I$(brew --prefix zlib)/include"export LDFLAGS="-L$(brew --prefix sqlite3)/lib -L$(brew --prefix zlib)/lib"
export PKG_CONFIG_PATH="/usr/local/opt/openssl/lib/pkgconfig"
python3.10 pip-run -m jaraco.develop -- -m jaraco.develop.macos-build-python
/opt/homebrew/Cellar/python@3.10/3.10.2/bin/python3.10: can't open file ' .../cpytonpip-run': [Errno 2] No such file or directory

in the cpython directory :
run commands

export CPPFLAGS="-I$(brew --prefix sqlite3)/include -I$(brew --prefix zlib)/include"
export LDFLAGS="-L$(brew --prefix sqlite3)/lib -L$(brew --prefix zlib)/lib
export PKG_CONFIG_PATH="/usr/local/opt/openssl/lib/pkgconfig"
./configure -with-pydebug --with-tcltk-includes="-I/Library/Frameworks/Tcl.framework/Headers -I/Library/Frameworks/Tk.framework/Headers" --with-tcltk-libs="/Library/Frameworks/Tcl.framework/Tcl /Library/Frameworks/Tk.framework/Tk" --enable-optimizations
make clean
make -j8

gcc -L/opt/homebrew/opt/zlib/lib -L/opt/homebrew/opt/zlib/lib -fno-semantic-interposition -fprofile-instr-generate -Wl,-stack_size,1000000 -framework CoreFoundation -o python.exe Programs/python.o libpython3.10d.a -ldl -framework CoreFoundation
gcc -L/opt/homebrew/opt/zlib/lib -L/opt/homebrew/opt/zlib/lib -fno-semantic-interposition -fprofile-instr-generate -Wl,-stack_size,1000000 -framework CoreFoundation -o Programs/_testembed Programs/_testembed.o libpython3.10d.a -ldl -framework CoreFoundation
Undefined symbols for architecture arm64:
"_libintl_bindtextdomain", referenced from:
Undefined symbols for architecture arm64:
"_libintl_bindtextdomain", referenced from:
__locale_bindtextdomain_impl in libpython3.10d.a(_localemodule.o)
__locale_bindtextdomain_impl in libpython3.10d.a(_localemodule.o)
"_libintl_dcgettext", referenced from:
"_libintl_dcgettext", referenced from:
__locale_dcgettext_impl in libpython3.10d.a(_localemodule.o)
__locale_dcgettext_impl in libpython3.10d.a(_localemodule.o)
"_libintl_dgettext", referenced from:
"_libintl_dgettext", referenced from:
__locale_dgettext_impl in libpython3.10d.a(_localemodule.o)
__locale_dgettext_impl in libpython3.10d.a(_localemodule.o)
"_libintl_gettext", referenced from:
"_libintl_gettext", referenced from:
__locale_gettext_impl in libpython3.10d.a(_localemodule.o)
__locale_gettext_impl in libpython3.10d.a(_localemodule.o)
"_libintl_setlocale", referenced from:
"_libintl_setlocale", referenced from:
__locale_setlocale_impl in libpython3.10d.a(_localemodule.o)
_locale_decode_monetary in libpython3.10d.a(_localemodule.o)
__locale_setlocale_impl in libpython3.10d.a(_localemodule.o)
_locale_decode_monetary in libpython3.10d.a(_localemodule.o)
"_libintl_textdomain", referenced from:
"_libintl_textdomain", referenced from:
__locale_textdomain_impl in libpython3.10d.a(_localemodule.o)
__locale_textdomain_impl in libpython3.10d.a(_localemodule.o)
ld: symbol(s) not found for architecture arm64
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [Programs/_testembed] Error 1
make[3]: *** Waiting for unfinished jobs....
make[3]: *** [python.exe] Error 1
make[2]: *** [build_all_generate_profile] Error 2
make[1]: *** [profile-gen-stamp] Error 2
make: *** [profile-run-stamp] Error 2

could you help me please to fixe this issus

Best regards

Battant

[bodkan]

I have been having minor issues with compiling the recent versions of CPython on the latest macOS (& Homebrew) according to the compilation instructions in the Python Developer’s Guide.

I have seen various recent issues reported on the similar topic (#92895 appears to be exactly the same) but haven’t found a place where updates to the guide are being tracked (apparently a process which is already underway).

What follows are a couple of simple steps which resulted in a successful compilation of CPython on my Mac, including fixing #92895. I'm sharing my notes here, hoping it might help another newbie interested in starting hacking with CPython internals and contributing to CPython.

Setup

• M1 MBP
• latest macOS 12.5
• latest XCode 13.4.1
• Homebrew dependencies as described in the guide (i.e. brew install pkg-config openssl@1.1 xz gdbm tcl-tk)
• git clone git@github.com:python/cpython.git; cd cpython

Original compilation instructions

Running the following as described in the dev guide:

PKG_CONFIG_PATH="$(brew --prefix tcl-tk)/lib/pkgconfig" \
  ./configure --with-pydebug --with-openssl=$(brew --prefix openssl)

make -s -j2

The above compiles successfully, but gives the following warning:

Note: Deepfreeze may have added some global objects,
      so run 'make regen-global-objects' if necessary.

The necessary bits to build these optional modules were not found:
_gdbm                                                          
To find the necessary bits, look in configure.ac and config.log.

I wasn't really satisfied with this because gdbm is installed on my system (having installed all necessary dependencies via Homebrew as shown above), so there is no reason why this shouldn't work.

Solution

Looking around at how Homebrew itself compiles Python, I ended up with the following trivial addition to the ./configure step from the official docs:

CFLAGS=-I${HOMEBREW_PREFIX}/include \
LDFLAGS=-L${HOMEBREW_PREFIX}/lib \
PKG_CONFIG_PATH="$(brew --prefix tcl-tk)/lib/pkgconfig" \
./configure --with-pydebug \
                   --with-openssl=$(brew --prefix openssl)
                   
make -j8 -s

Note that for whatever reason, the paths to include/ and lib/ obtained viabrew --prefix gdbm (which is sometimes recommended elsewhere, and also actually used to link to tcl-tk) actually don’t give what’s needed. The correct paths to include/ and lib/ are under $HOMEBREW_PREFIX. Well, at least on my Mac and my Homebrew they are, both of which are at their latest versions.

I can confirm that this works on the latest dev version (i.e. git checkout b8c528694 at the time of writing), which is the version I'm interested in using for hacking/learning purposes. In any case, if older CPython versions depend on other libraries, the same recipe could/should apply as well.