Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

make sharedinstall does not create lib-dynload in DESTDIR if lib-dynload exists in system already #100221

Closed
mgorny opened this issue Dec 13, 2022 · 3 comments
Closed

make sharedinstall does not create lib-dynload in DESTDIR if lib-dynload exists in system already #100221

mgorny opened this issue Dec 13, 2022 · 3 comments
Labels
build The build process and cross-build type-bug An unexpected behavior, bug, or error

Comments

@mgorny
Copy link
Contributor

mgorny commented Dec 13, 2022 •
edited by bedevere-bot
Loading

Bug report

The Makefile.pre.in rule for creating directories is specified as:

$(DESTSHARED):
		@for i in $(DESTDIRS); \
		do \
			if test ! -d $(DESTDIR)$$i; then \
				echo "Creating directory $$i"; \
				$(INSTALL) -d -m $(DIRMODE) $(DESTDIR)$$i; \
			else    true; \
			fi; \
		done

This means that if $(DESTSHARED) exists already (note: missing $(DESTDIR)!), i.e. Python 3.12 has been installed into the system, the directories in $(DESTDIR) won't be created and the successive sharedinstall rules fail due to the missing destination, e.g.:

/usr/bin/install -c -m 755 Modules/array.cpython-312-x86_64-linux-gnu.so /usr/lib/python3.12/lib-dynload/array.cpython-312-x86_64-linux-gnu.so
/usr/bin/install: cannot create regular file '/var/tmp/portage/dev-lang/python-3.12.0_alpha3/image/usr/lib/python3.12/lib-dynload/array.cpython-312-x86_64-linux-gnu.so': No such file or directory
/usr/bin/install -c -m 755 Modules/_asyncio.cpython-312-x86_64-linux-gnu.so /usr/lib/python3.12/lib-dynload/_asyncio.cpython-312-x86_64-linux-gnu.so
/usr/bin/install: cannot create regular file '/var/tmp/portage/dev-lang/python-3.12.0_alpha3/image/usr/lib/python3.12/lib-dynload/_asyncio.cpython-312-x86_64-linux-gnu.so': No such file or directory
/usr/bin/install -c -m 755 Modules/_bisect.cpython-312-x86_64-linux-gnu.so /usr/lib/python3.12/lib-dynload/_bisect.cpython-312-x86_64-linux-gnu.so
/usr/bin/install: cannot create regular file '/var/tmp/portage/dev-lang/python-3.12.0_alpha3/image/usr/lib/python3.12/lib-dynload/_bisect.cpython-312-x86_64-linux-gnu.so': No such file or directory

Full log: dev-lang:python-3.12.0_alpha3:20221207-142002.log

Your environment

  • CPython versions tested on: 3.12.0a3
  • Operating system and architecture: Gentoo Linux/amd64

Linked PRs
@mgorny mgorny added the type-bug An unexpected behavior, bug, or error label Dec 13, 2022
mgorny added a commit to mgorny/cpython that referenced this issue Dec 18, 2022
@mgorny
pythongh-100221: Fix creating dirs in make sharedinstall
de72019 
Fix creating install directories in `make sharedinstall` if they exist
already outside `DESTDIR`.  The previous make rules assumed that
the directories would be created via a dependency on a rule
for `$(DESTSHARED)` that did not fire if the directory did exist outside
`$(DESTDIR)`.

While technically `$(DESTDIR)` could be prepended to the rule name,
moving the rules for creating directories straight into
the `sharedinstall` rule seems to fit the common practices better.
Since the rule explicitly checks whether the individual directories
exist anyway, there seems to be no reason to rely on make determining
that implicitly as well.
@bedevere-bot bedevere-bot mentioned this issue Dec 18, 2022
Merged
@erlend-aasland erlend-aasland added the build The build process and cross-build label Jan 11, 2023
miss-islington pushed a commit that referenced this issue Feb 8, 2023
@mgorny
gh-100221: Fix creating dirs in make sharedinstall (GH-100329)
2a8bf25 
Fix creating install directories in `make sharedinstall` if they exist already outside `DESTDIR`.  The previous make rules assumed that the directories would be created via a dependency on a rule for `$(DESTSHARED)` that did not fire if the directory did exist outside `$(DESTDIR)`.

While technically `$(DESTDIR)` could be prepended to the rule name, moving the rules for creating directories straight into the `sharedinstall` rule seems to fit the common practices better. Since the rule explicitly checks whether the individual directories exist anyway, there seems to be no reason to rely on make determining that implicitly as well.
@zware
Copy link
Member

zware commented Feb 8, 2023

Is this only an issue on 3.12, or is 3.11 also affected?

@mgorny
Copy link
Contributor Author

mgorny commented Feb 8, 2023

I think the relevant code technically exists in 3.11 too but it's not being used since setup.py is used to install the extensions.

@zware
Copy link
Member

zware commented Feb 8, 2023

Ok, we'll leave 3.11 as is for now, then.

Thanks for the patch!

@zware zware closed this as completed Feb 8, 2023
carljm added a commit to carljm/cpython that referenced this issue Feb 9, 2023
@carljm
Merge branch 'main' into inlinecomp2
be3becc 
* main: (82 commits)
  pythongh-101670: typo fix in PyImport_ExtendInittab() (python#101723)
  pythonGH-99293: Document that `Py_TPFLAGS_VALID_VERSION_TAG` shouldn't be used. (#pythonGH-101736)
  no-issue: Add Dong-hee Na as the cjkcodecs codeowner (pythongh-101731)
  pythongh-101678: Merge math_1_to_whatever() and math_1() (python#101730)
  pythongh-101678: refactor the math module to use special functions from c11 (pythonGH-101679)
  pythongh-85984: Remove legacy Lib/pty.py code. (python#92365)
  pythongh-98831: Use opcode metadata for stack_effect() (python#101704)
  pythongh-101283: Version was just released, so should be changed in 3.11.3 (pythonGH-101719)
  pythongh-101283: Fix use of unbound variable (pythonGH-101712)
  pythongh-101283: Improved fallback logic for subprocess with shell=True on Windows (pythonGH-101286)
  pythongh-101277: Port more itertools static types to heap types (python#101304)
  pythongh-98831: Modernize CALL and family (python#101508)
  pythonGH-101696: invalidate type version tag in `_PyStaticType_Dealloc` (python#101697)
  pythongh-100221: Fix creating dirs in `make sharedinstall` (pythonGH-100329)
  pythongh-101670: typo fix in PyImport_AppendInittab() (pythonGH-101672)
  pythongh-101196: Make isdir/isfile/exists faster on Windows (pythonGH-101324)
  pythongh-101614: Don't treat python3_d.dll as a Python DLL when checking extension modules for incompatibility (pythonGH-101615)
  pythongh-100933: Improve `check_element` helper in `test_xml_etree` (python#100934)
  pythonGH-101578: Normalize the current exception (pythonGH-101607)
  pythongh-47937: Note that Popen attributes are read-only (python#93070)
  ...
@zware zware mentioned this issue Feb 23, 2023
Closed
@indygreg indygreg mentioned this issue Sep 24, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
build The build process and cross-build type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mgorny @zware @erlend-aasland