Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stim v1.7.0 installation fails for Apple Silicon #179

Closed
oscarhiggott opened this issue Dec 14, 2021 · 10 comments
Closed

Stim v1.7.0 installation fails for Apple Silicon #179

oscarhiggott opened this issue Dec 14, 2021 · 10 comments

Comments

@oscarhiggott
Copy link

oscarhiggott commented Dec 14, 2021 •
edited
Loading

Unfortunately the Python interpreter crashes (using an M1-based mac) when importing stim after installing v1.7.0 with the prebuilt binaries:

Python 3.9.6 (default, Jun 29 2021, 06:20:32) 
[Clang 12.0.0 (clang-1200.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import stim
zsh: illegal hardware instruction  python

I tried instead installing from source with the pip --no-binary flag which raises this exception instead:

  src/stim.cc:17:10: fatal error: 'stim.h' file not found
  #include "stim.h"
           ^~~~~~~~
  1 error generated.
  error: command '/usr/bin/clang' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for stim

since the stim.h file is no longer in the source distribution. From a comparison with the v1.6.0 source distribution it looks like all the header files are missing in the v1.7.0 PyPI source distribution.
@Strilanc
Copy link
Collaborator

Thanks for reporting this so quickly.

I'll fix the missing header today and drop the mac prebuilts and look into if ciwheelbuilder can do M1 stuff...

@oscarhiggott
Copy link
Author

Thanks! It looks like there was a similar issue last year with Tensorflow: https://stackoverflow.com/questions/65383338/zsh-illegal-hardware-instruction-python-when-installing-tensorflow-on-macbook.

@oscarhiggott
Copy link
Author

I'll leave the crash report here too in case it helps

Process:               Python [62849]
Path:                  /usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python
Identifier:            org.python.python
Version:               3.9.6 (3.9.6)
Code Type:             X86-64 (Translated)
Parent Process:        zsh [24288]
Responsible:           Terminal [10033]
User ID:               501

Date/Time:             2021-12-14 18:05:05.6343 +0000
OS Version:            macOS 12.0.1 (21A559)
Report Version:        12

Time Awake Since Boot: 160000 seconds
Time Since Wake:       52970 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_INSTRUCTION (SIGILL)
Exception Codes:       0x0000000000000001, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Reason:    Namespace SIGNAL, Code 4 Illegal instruction: 4
Terminating Process:   exc handler [62849]

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   _stim_march_avx2.cpython-39-darwin.so	       0x10c24de5c stim::GateDataMap::add_gate_data_annotations(bool&) + 76
1   _stim_march_avx2.cpython-39-darwin.so	       0x10c24bbc6 stim::GateDataMap::GateDataMap() + 38
2   _stim_march_avx2.cpython-39-darwin.so	       0x10c24de00 _GLOBAL__sub_I_gate_data.cc + 16
3   dyld                          	       0x203125b49 invocation function for block in dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const + 182
4   dyld                          	       0x20314c0ff invocation function for block in dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const + 129
5   dyld                          	       0x203143893 invocation function for block in dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const + 566
6   dyld                          	       0x203112d91 dyld3::MachOFile::forEachLoadCommand(Diagnostics&, void (load_command const*, bool&) block_pointer) const + 129
7   dyld                          	       0x20314361b dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const + 179
8   dyld                          	       0x20314bb30 dyld3::MachOAnalyzer::forEachInitializerPointerSection(Diagnostics&, void (unsigned int, unsigned int, unsigned char const*, bool&) block_pointer) const + 118
9   dyld                          	       0x20314bda2 dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const + 386
10  dyld                          	       0x203125a7c dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const + 144
11  dyld                          	       0x203125c08 dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 178
12  dyld                          	       0x203125cac dyld4::Loader::runInitializersBottomUpPlusUpwardLinks(dyld4::RuntimeState&) const + 108
13  dyld                          	       0x2031349b5 dyld4::APIs::dlopen_from(char const*, int, void*) + 563
14  Python                        	       0x10b8e1815 _imp_create_dynamic + 1276
15  Python                        	       0x10b7fea7b cfunction_vectorcall_FASTCALL + 95
16  Python                        	       0x10b8aeea9 _PyEval_EvalFrameDefault + 45899
17  Python                        	       0x10b8a2a18 _PyEval_EvalCode + 397
18  Python                        	       0x10b7b4324 _PyFunction_Vectorcall + 376
19  Python                        	       0x10b8b139f call_function + 164
20  Python                        	       0x10b8aec71 _PyEval_EvalFrameDefault + 45331
21  Python                        	       0x10b7b426c _PyFunction_Vectorcall + 192
22  Python                        	       0x10b8b139f call_function + 164
23  Python                        	       0x10b8aebd4 _PyEval_EvalFrameDefault + 45174
24  Python                        	       0x10b7b426c _PyFunction_Vectorcall + 192
25  Python                        	       0x10b8b139f call_function + 164
26  Python                        	       0x10b8aecad _PyEval_EvalFrameDefault + 45391
27  Python                        	       0x10b7b426c _PyFunction_Vectorcall + 192
28  Python                        	       0x10b8b139f call_function + 164
29  Python                        	       0x10b8aecad _PyEval_EvalFrameDefault + 45391
30  Python                        	       0x10b7b426c _PyFunction_Vectorcall + 192
31  Python                        	       0x10b8b139f call_function + 164
32  Python                        	       0x10b8aecad _PyEval_EvalFrameDefault + 45391
33  Python                        	       0x10b7b426c _PyFunction_Vectorcall + 192
34  Python                        	       0x10b7b5770 object_vacall + 313
35  Python                        	       0x10b7b59d1 _PyObject_CallMethodIdObjArgs + 252
36  Python                        	       0x10b8dcb75 PyImport_ImportModuleLevelObject + 1028
37  Python                        	       0x10b8ac669 _PyEval_EvalFrameDefault + 35595
38  Python                        	       0x10b8a2a18 _PyEval_EvalCode + 397
39  Python                        	       0x10b89d797 builtin_exec + 389
40  Python                        	       0x10b7fea7b cfunction_vectorcall_FASTCALL + 95
41  Python                        	       0x10b8aeea9 _PyEval_EvalFrameDefault + 45899
42  Python                        	       0x10b8a2a18 _PyEval_EvalCode + 397
43  Python                        	       0x10b7b4324 _PyFunction_Vectorcall + 376
44  Python                        	       0x10b8b139f call_function + 164
45  Python                        	       0x10b8aec71 _PyEval_EvalFrameDefault + 45331
46  Python                        	       0x10b7b426c _PyFunction_Vectorcall + 192
47  Python                        	       0x10b8b139f call_function + 164
48  Python                        	       0x10b8aebd4 _PyEval_EvalFrameDefault + 45174
49  Python                        	       0x10b7b426c _PyFunction_Vectorcall + 192
50  Python                        	       0x10b8b139f call_function + 164
51  Python                        	       0x10b8aecad _PyEval_EvalFrameDefault + 45391
52  Python                        	       0x10b7b426c _PyFunction_Vectorcall + 192
53  Python                        	       0x10b8b139f call_function + 164
54  Python                        	       0x10b8aecad _PyEval_EvalFrameDefault + 45391
55  Python                        	       0x10b7b426c _PyFunction_Vectorcall + 192
56  Python                        	       0x10b7b5770 object_vacall + 313
57  Python                        	       0x10b7b59d1 _PyObject_CallMethodIdObjArgs + 252
58  Python                        	       0x10b8dcb75 PyImport_ImportModuleLevelObject + 1028
59  Python                        	       0x10b8ac669 _PyEval_EvalFrameDefault + 35595
60  Python                        	       0x10b8a2a18 _PyEval_EvalCode + 397
61  Python                        	       0x10b905b45 run_eval_code_obj + 126
62  Python                        	       0x10b905a95 run_mod + 96
63  Python                        	       0x10b906476 PyRun_InteractiveOneObjectEx + 1242
64  Python                        	       0x10b902a62 PyRun_InteractiveLoopFlags + 191
65  Python                        	       0x10b902970 PyRun_AnyFileExFlags + 60
66  Python                        	       0x10b921c1c pymain_run_stdin + 141
67  Python                        	       0x10b921134 Py_RunMain + 729
68  Python                        	       0x10b9222d1 pymain_main + 35
69  Python                        	       0x10b9225a7 Py_BytesMain + 42
70  dyld                          	       0x2031164fe start + 462
71  ???                           	               0x3 ???

Thread 1:: com.apple.rosetta.exceptionserver
0   runtime                       	    0x7ff7fff378e4 0x7ff7fff33000 + 18660
1   runtime                       	    0x7ff7fff44928 0x7ff7fff33000 + 71976
2   runtime                       	    0x7ff7fff460a4 0x7ff7fff33000 + 77988


Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x000000010c2b2030  rbx: 0x000000010c35c840  rcx: 0x0000000306f647f0  rdx: 0x0000000306f646f0
  rdi: 0x000000010c35c840  rsi: 0x0000000306f6103f  rbp: 0x0000000306f61020  rsp: 0x0000000306f60ee0
   r8: 0x00007ff844f254f0   r9: 0x0000000000000000  r10: 0x000000010c20d000  r11: 0x000000020314c07e
  r12: 0x000000010c35c840  r13: 0x000000010c2bde70  r14: 0x0000000306f6103f  r15: 0x000000010b2d2060
  rip: 0x000000010c24de5c  rfl: 0x0000000000000202
 tmp0: 0x000000010c24de5c tmp1: 0xffffff488510f8c5 tmp2: 0x00c510244411f8c5


Binary Images:
       0x10c20d000 -        0x10c358fff _stim_march_avx2.cpython-39-darwin.so (*) <449636e2-6c8d-30b9-bc79-b7231504fb1b> /usr/local/lib/python3.9/site-packages/stim/_stim_march_avx2.cpython-39-darwin.so
       0x203111000 -        0x20317cfff dyld (*) <1a6ae033-9438-33c0-8077-988fd885250a> /usr/lib/dyld
       0x10b749000 -        0x10ba20fff org.python.python (3.9.6, (c) 2001-2019 Python Software Foundation.) <8ac37f5c-9f54-3c08-b839-b36904648acd> /usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/Python
               0x0 - 0xffffffffffffffff ??? (*) <00000000-0000-0000-0000-000000000000> ???
    0x7ff7fff33000 -     0x7ff7fff62fff runtime (*) <9f5d65be-d8d0-3979-bb05-e651a67e785c> /usr/libexec/rosetta/runtime

@Strilanc
Copy link
Collaborator

That's such a weird place for it to fail. It's initializing the gate data map? There's nothing particularly simd-ish happening there...

I wonder if it's just a compiler optimization that used avx without me realizing, and it comes down to the fact that the architecture detection is using the avx version initially instead of the non-avx version...

Could you find the stim/__init__.py file in your virtual environment and replace

import stim._stim_march_avx2 as _tmp

at the top with

import stim._stim_march_polyfill as _tmp

and see if that works?

In the meantime I'm gonna try to get my hands on an M1 macbook...

@oscarhiggott
Copy link
Author

Sure, just made that change in the stim/__init__.py file and the import works now

@Strilanc
Copy link
Collaborator

Okay, that's an easy fix then. I'll look into the header issue (since I want the fallback to work too!) and then make a release.

Thanks this was really helpful.

@oscarhiggott
Copy link
Author

Great, thanks!

@Strilanc Strilanc mentioned this issue Dec 14, 2021
Merged
Strilanc added a commit that referenced this issue Dec 15, 2021
@Strilanc
Fix package installation issues (#180)
af59b92 
- For sdist, fix specifying `data_files=` on each Extension instead of on the package itself
- For sdist, fix headers being given as `data_files=` instead of `headers=`
- For wheels, fix using the avx2 instead of the vanilla package to determine CPU capabilities
    - It was touching avx instructions along the way, causing crashes on M1 macs

First part of fixing #179
@Strilanc
Copy link
Collaborator

Okay, I have uploaded v1.7.1.

Could you confirm that pip install stim now works on your machine? (may have to say stim==1.7.1 if certain indexes haven't updated yet)

Also, could you confirm that pip install --no-binary stim stim doesn't still crash with the same missing header? (No need to wait for it to finish, it takes quite awhile; the header failure was quite early before right?)

@oscarhiggott
Copy link
Author

Sure, both the prebuilt and source installations work nicely now with v1.7.1

@Strilanc
Copy link
Collaborator

Great, closing the issue as fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Strilanc @oscarhiggott