Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove outdated Tools/scripts/ example scripts #97669

Closed
vstinner opened this issue Sep 30, 2022 · 11 comments
Closed

Remove outdated Tools/scripts/ example scripts #97669

vstinner opened this issue Sep 30, 2022 · 11 comments
Labels
type-bug An unexpected behavior, bug, or error

Comments

@vstinner
Copy link
Member

The Tools/scripts/ directory contains more than 70+ example scripts. Most of them are outdated and short scripts Python which are simple to rewrite. When I asked other core developers, they suggest to remove all of them, see the discussion:

https://discuss.python.org/t/remove-outdated-tools-scripts-scripts/19571

I created this issue to look into these scripts and see what to do with them.

@vstinner vstinner added the type-bug An unexpected behavior, bug, or error label Sep 30, 2022
@vstinner
Copy link
Member Author

analyze_dxp.py: I created issue #97670 to remove the sys.getdxp() function and the Tools/scripts/analyze_dxp.py script.

@vstinner
Copy link
Member Author

See also issue #97649: Do not install Tools directory on Windows.

@vstinner
Copy link
Member Author

I created PR #97675 to remove a first batch of outdated example scripts.

@vstinner
Copy link
Member Author

vstinner commented Oct 3, 2022

@warsaw @gpshead @gvanrossum: Would you mind to approve formally this PR since you are in favor of removing Tools/scripts/ scripts according to your messages on https://discuss.python.org/t/remove-outdated-tools-scripts-scripts/19571?

vstinner added a commit that referenced this issue Oct 4, 2022
Remove outdated example scripts of the Tools/scripts/ directory. A
copy can be found in the old-demos project:
https://github.com/gvanrossum/old-demos

Removed scripts (39):

* byext.py
* byteyears.py
* cleanfuture.py
* copytime.py
* crlf.py
* db2pickle.py
* dutree.doc
* dutree.py
* find-uname.py
* find_recursionlimit.py
* finddiv.py
* findlinksto.py
* findnocoding.py
* fixcid.py
* fixdiv.py
* fixheader.py
* fixnotice.py
* fixps.py
* get-remote-certificate.py
* google.py
* highlight.py
* ifdef.py
* import_diagnostics.py
* lfcr.py
* linktree.py
* lll.py
* mailerdaemon.py
* make_ctype.py
* mkreal.py
* objgraph.py
* pdeps.py
* pickle2db.py
* pindent.py
* pysource.py
* reindent-rst.py
* rgrep.py
* suff.py
* texi2html.py
* which.py

Changes:

* Remove test_fixcid, test_lll, test_pdeps and test_pindent
  of test.test_tools.
* Remove get-remote-certificate.py changelog entry, since the script
  was removed.

Note: there is a copy of crlf.py in Lib/test/test_lib2to3/data/.
@vstinner
Copy link
Member Author

vstinner commented Oct 6, 2022

I created a draft PR #97963 to create a new Tools/build/ sub-directory which contains scripts used to build Python. I moved 13 scripts there. Not sure if it's a good idea or not.

With PR #97963 and PR #97964, the remaining scripts in Tools/scripts/ are:

Programs (3):

  • 2to3
  • idle3
  • pydoc3

Scripts (19):

  • abitype.py
  • checkpip.py
  • combinerefs.py
  • eptags.py
  • gprof2html.py
  • md5sum.py
  • nm2def.py
  • patchcheck.py
  • pathfix.py
  • pep384_macrocheck.py
  • ptags.py
  • reindent.py
  • run_tests.py
  • startuptime.py
  • summarize_stats.py
  • untabify.py: used by patchcheck.py
  • var_access_benchmark.py
  • verify_ensurepip_wheels.py
  • win_add2path.py

patchcheck.py, run_tests.py and verify_ensurepip_wheels.py are used by the Python workflow. Does it deserve its own sub-directory?

@vstinner
Copy link
Member Author

vstinner commented Oct 6, 2022

I created PR #97964 to move diff.py and ndiff.py scripts to the difflib documentation (to Doc/includes/).

ambv pushed a commit that referenced this issue Oct 7, 2022
Remove diff.py and ndiff.py scripts of Tools/scripts/: move them to
Doc/includes/.

* diff.py and ndiff.py files are no longer executable. Remove also
  their shebang ("#!/usr/bin/env python3").
* Remove the -profile command from ndiff.py to simply the code.
* Remove ndiff.py copyright and history command. The Python
  documentation examples are distributed under the "Zero Clause BSD
  License".
@vstinner
Copy link
Member Author

Oh nice, 4ed00be removed eptags.py and ptags.py.

@vstinner
Copy link
Member Author

PR to remove 5 more scripts: PR #98167

PR to remove abitype.py and pep384_macrocheck.py: PR #98165

mpage pushed a commit to mpage/cpython that referenced this issue Oct 11, 2022
Remove diff.py and ndiff.py scripts of Tools/scripts/: move them to
Doc/includes/.

* diff.py and ndiff.py files are no longer executable. Remove also
  their shebang ("#!/usr/bin/env python3").
* Remove the -profile command from ndiff.py to simply the code.
* Remove ndiff.py copyright and history command. The Python
  documentation examples are distributed under the "Zero Clause BSD
  License".
vstinner added a commit that referenced this issue Oct 11, 2022
Remove outdated example scripts of the Tools/scripts/ directory:

* gprof2html.py
* md5sum.py
* nm2def.py
* pathfix.py
* win_add2path.py

Remove test_gprof2html, test_md5sum and test_pathfix of test_tools.
vstinner added a commit that referenced this issue Oct 11, 2022
Remove abitype.py and pep384_macrocheck.py scripts of Tools/scripts/.
vstinner added a commit that referenced this issue Oct 12, 2022
Move patchcheck.py, reindent.py and untabify.py scripts to a new
Tools/patchcheck/ directory.
vstinner added a commit that referenced this issue Oct 12, 2022
test_tools.test_sundry() now uses an unittest mock to prevent the
logging module to register a real "atfork" function which kept the
logging module dictionary alive. So the logging module can be
properly unloaded. Previously, the logging module was loaded before
test_sundry(), but it's no longer the case since recent test_tools
sub-tests removals.
vstinner added a commit that referenced this issue Oct 12, 2022
The "pyperf command" tool be used instead. Example:

    $ python3 -m pyperf command -- python3 -c pass
    .....................
    command: Mean +- std dev: 17.8 ms +- 0.4 ms

pyperf also computes the standard deviation which gives an idea of
the benchmark looks reliable or not.
@vstinner
Copy link
Member Author

vstinner commented Oct 12, 2022

Updated list of Tools/scripts/ scripts. With PR #97963, there are only 8 remaining scripts.

Programs (3):

  • 2to3
  • idle3
  • pydoc3

Misc (5):

Update: I moved verify_ensurepip_wheels.py to Tools/build/ in my PR #97963.

carljm added a commit to carljm/cpython that referenced this issue Oct 14, 2022
* main: (38 commits)
  pythongh-98251: Allow venv to pass along PYTHON* variables to pip and ensurepip when they do not impact path resolution (pythonGH-98259)
  Bpo-41246: IOCP Proactor avoid callback code duplication (python#21399)
  bpo-46364: Use sockets for stdin of asyncio only on AIX (python#30596)
  pythongh-98178: syslog() is not thread-safe on macOS (python#98213)
  Mark all targets in `Doc/Makefile` as `PHONY` (pythonGH-98189)
  pythongh-97982: Factorize PyUnicode_Count() and unicode_count() code (python#98025)
  pythongh-96265: Formatting changes for faq/general (python#98129)
  tutorial: remove "with single quotes" (python#98204)
  pythongh-97669: Remove Tools/scripts/startuptime.py (python#98214)
  signalmodule.c uses _PyErr_WriteUnraisableMsg() (python#98217)
  pythongh-97669: Fix test_tools reference leak (python#98216)
  pythongh-97669: Create Tools/patchcheck/ directory (python#98186)
  pythongh-65046: Link to logging cookbook from asyncio docs (python#98207)
  Formatting fixes in contextlib docs (python#98111)
  pythongh-95276: Add callable entry to the glossary (python#95738)
  pythongh-96130: Rephrase use of "typecheck" verb for clarity (python#98144)
  Fix some incorrect indentation around the main switch (python#98177)
  pythongh-98172: Fix formatting in `except*` docs (python#98173)
  pythongh-97982: Remove asciilib_count() (python#98164)
  pythongh-95756: Free and NULL-out code caches when needed (pythonGH-98181)
  ...
carljm added a commit to carljm/cpython that referenced this issue Oct 14, 2022
* main: (37 commits)
  pythongh-98251: Allow venv to pass along PYTHON* variables to pip and ensurepip when they do not impact path resolution (pythonGH-98259)
  Bpo-41246: IOCP Proactor avoid callback code duplication (python#21399)
  bpo-46364: Use sockets for stdin of asyncio only on AIX (python#30596)
  pythongh-98178: syslog() is not thread-safe on macOS (python#98213)
  Mark all targets in `Doc/Makefile` as `PHONY` (pythonGH-98189)
  pythongh-97982: Factorize PyUnicode_Count() and unicode_count() code (python#98025)
  pythongh-96265: Formatting changes for faq/general (python#98129)
  tutorial: remove "with single quotes" (python#98204)
  pythongh-97669: Remove Tools/scripts/startuptime.py (python#98214)
  signalmodule.c uses _PyErr_WriteUnraisableMsg() (python#98217)
  pythongh-97669: Fix test_tools reference leak (python#98216)
  pythongh-97669: Create Tools/patchcheck/ directory (python#98186)
  pythongh-65046: Link to logging cookbook from asyncio docs (python#98207)
  Formatting fixes in contextlib docs (python#98111)
  pythongh-95276: Add callable entry to the glossary (python#95738)
  pythongh-96130: Rephrase use of "typecheck" verb for clarity (python#98144)
  Fix some incorrect indentation around the main switch (python#98177)
  pythongh-98172: Fix formatting in `except*` docs (python#98173)
  pythongh-97982: Remove asciilib_count() (python#98164)
  pythongh-95756: Free and NULL-out code caches when needed (pythonGH-98181)
  ...
vstinner added a commit that referenced this issue Oct 17, 2022
Create Tools/build/ directory. Move the following scripts from
Tools/scripts/ to Tools/build/:

* check_extension_modules.py
* deepfreeze.py
* freeze_modules.py
* generate_global_objects.py
* generate_levenshtein_examples.py
* generate_opcode_h.py
* generate_re_casefix.py
* generate_sre_constants.py
* generate_stdlib_module_names.py
* generate_token.py
* parse_html5_entities.py
* smelly.py
* stable_abi.py
* umarshal.py
* update_file.py
* verify_ensurepip_wheels.py

Update references to these scripts.
@vstinner
Copy link
Member Author

I moved 16 scripts to a new Tools/build/ directory: 1863302

There are 9 remaining files in Tools/scripts/, including the README file (so 8 scripts):

2to3
checkpip.py
combinerefs.py
idle3
pydoc3
README
run_tests.py
summarize_stats.py
var_access_benchmark.py

carljm added a commit to carljm/cpython that referenced this issue Oct 17, 2022
* main: (31 commits)
  pythongh-95913: Move subinterpreter exper removal to 3.11 WhatsNew (pythonGH-98345)
  pythongh-95914: Add What's New item describing PEP 670 changes (python#98315)
  Remove unused arrange_output_buffer function from zlibmodule.c. (pythonGH-98358)
  pythongh-98174: Handle EPROTOTYPE under macOS in test_sendfile_fallback_close_peer_in_the_middle_of_receiving (python#98316)
  pythonGH-98327: Reduce scope of catch_warnings() in _make_subprocess_transport (python#98333)
  pythongh-93691: Compiler's code-gen passes location around instead of holding it on the global compiler state (pythonGH-98001)
  pythongh-97669: Create Tools/build/ directory (python#97963)
  pythongh-95534: Improve gzip reading speed by 10% (python#97664)
  pythongh-95913: Forward-port int/str security change to 3.11 What's New in main (python#98344)
  pythonGH-91415: Mention alphabetical sort ordering in the Sorting HOWTO (pythonGH-98336)
  pythongh-97930: Merge with importlib_resources 5.9 (pythonGH-97929)
  pythongh-85525: Remove extra row in doc (python#98337)
  pythongh-85299: Add note warning about entry point guard for asyncio example (python#93457)
  pythongh-97527: IDLE - fix buggy macosx patch (python#98313)
  pythongh-98307: Add docstring and documentation for SysLogHandler.createSocket (pythonGH-98319)
  pythongh-94808: Cover `PyFunction_GetCode`, `PyFunction_GetGlobals`, `PyFunction_GetModule` (python#98158)
  pythonGH-94597: Deprecate child watcher getters and setters (python#98215)
  pythongh-98254: Include stdlib module names in error messages for NameErrors (python#98255)
  Improve speed. Reduce auxiliary memory to 16.6% of the main array. (pythonGH-98294)
  [doc] Update logging cookbook with an example of custom handling of levels. (pythonGH-98290)
  ...
@vstinner
Copy link
Member Author

vstinner commented Nov 3, 2022

I removed enough scripts, I close the issue :-)

@vstinner vstinner closed this as completed Nov 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

1 participant