Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove outdated Tools/scripts/ example scripts #97669

Closed
vstinner opened this issue Sep 30, 2022 · 11 comments
Closed

Remove outdated Tools/scripts/ example scripts #97669

vstinner opened this issue Sep 30, 2022 · 11 comments
Labels
type-bug An unexpected behavior, bug, or error

Comments

@vstinner
Copy link
Member

The Tools/scripts/ directory contains more than 70+ example scripts. Most of them are outdated and short scripts Python which are simple to rewrite. When I asked other core developers, they suggest to remove all of them, see the discussion:

https://discuss.python.org/t/remove-outdated-tools-scripts-scripts/19571

I created this issue to look into these scripts and see what to do with them.
@vstinner vstinner added the type-bug An unexpected behavior, bug, or error label Sep 30, 2022
@vstinner
Copy link
Member Author

analyze_dxp.py: I created issue #97670 to remove the sys.getdxp() function and the Tools/scripts/analyze_dxp.py script.

@vstinner
Copy link
Member Author

See also issue #97649: Do not install Tools directory on Windows.

@vstinner vstinner mentioned this issue Sep 30, 2022
Closed
This was referenced Sep 30, 2022
Closed
Merged
@vstinner
Copy link
Member Author

I created PR #97675 to remove a first batch of outdated example scripts.

@vstinner vstinner mentioned this issue Sep 30, 2022
Closed
@vstinner
Copy link
Member Author

vstinner commented Oct 3, 2022

@warsaw @gpshead @gvanrossum: Would you mind to approve formally this PR since you are in favor of removing Tools/scripts/ scripts according to your messages on https://discuss.python.org/t/remove-outdated-tools-scripts-scripts/19571?

vstinner added a commit that referenced this issue Oct 4, 2022
@vstinner
gh-97669: Remove outdated example scripts (#97675)
6cbbc26 
Remove outdated example scripts of the Tools/scripts/ directory. A
copy can be found in the old-demos project:
https://github.com/gvanrossum/old-demos

Removed scripts (39):

* byext.py
* byteyears.py
* cleanfuture.py
* copytime.py
* crlf.py
* db2pickle.py
* dutree.doc
* dutree.py
* find-uname.py
* find_recursionlimit.py
* finddiv.py
* findlinksto.py
* findnocoding.py
* fixcid.py
* fixdiv.py
* fixheader.py
* fixnotice.py
* fixps.py
* get-remote-certificate.py
* google.py
* highlight.py
* ifdef.py
* import_diagnostics.py
* lfcr.py
* linktree.py
* lll.py
* mailerdaemon.py
* make_ctype.py
* mkreal.py
* objgraph.py
* pdeps.py
* pickle2db.py
* pindent.py
* pysource.py
* reindent-rst.py
* rgrep.py
* suff.py
* texi2html.py
* which.py

Changes:

* Remove test_fixcid, test_lll, test_pdeps and test_pindent
  of test.test_tools.
* Remove get-remote-certificate.py changelog entry, since the script
  was removed.

Note: there is a copy of crlf.py in Lib/test/test_lib2to3/data/.
@bedevere-bot bedevere-bot mentioned this issue Oct 6, 2022
Merged
@vstinner
Copy link
Member Author

vstinner commented Oct 6, 2022 •
edited
Loading

I created a draft PR #97963 to create a new Tools/build/ sub-directory which contains scripts used to build Python. I moved 13 scripts there. Not sure if it's a good idea or not.

With PR #97963 and PR #97964, the remaining scripts in Tools/scripts/ are:

Programs (3):

  • 2to3
  • idle3
  • pydoc3

Scripts (19):

  • abitype.py
  • checkpip.py
  • combinerefs.py
  • eptags.py
  • gprof2html.py
  • md5sum.py
  • nm2def.py
  • patchcheck.py
  • pathfix.py
  • pep384_macrocheck.py
  • ptags.py
  • reindent.py
  • run_tests.py
  • startuptime.py
  • summarize_stats.py
  • untabify.py: used by patchcheck.py
  • var_access_benchmark.py
  • verify_ensurepip_wheels.py
  • win_add2path.py

patchcheck.py, run_tests.py and verify_ensurepip_wheels.py are used by the Python workflow. Does it deserve its own sub-directory?

@bedevere-bot bedevere-bot mentioned this issue Oct 6, 2022
Merged
@vstinner
Copy link
Member Author

vstinner commented Oct 6, 2022

I created PR #97964 to move diff.py and ndiff.py scripts to the difflib documentation (to Doc/includes/).

ambv pushed a commit that referenced this issue Oct 7, 2022
@vstinner
gh-97669: Move difflib examples to Doc/includes/ (#97964)
002252c 
Remove diff.py and ndiff.py scripts of Tools/scripts/: move them to
Doc/includes/.

* diff.py and ndiff.py files are no longer executable. Remove also
  their shebang ("#!/usr/bin/env python3").
* Remove the -profile command from ndiff.py to simply the code.
* Remove ndiff.py copyright and history command. The Python
  documentation examples are distributed under the "Zero Clause BSD
  License".
@vstinner
Copy link
Member Author

Oh nice, 4ed00be removed eptags.py and ptags.py.

This was referenced Oct 10, 2022
Merged
Merged
@vstinner
Copy link
Member Author

PR to remove 5 more scripts: PR #98167

PR to remove abitype.py and pep384_macrocheck.py: PR #98165

mpage pushed a commit to mpage/cpython that referenced this issue Oct 11, 2022
@vstinner @mpage
pythongh-97669: Move difflib examples to Doc/includes/ (python#97964)
97ae2e9 
Remove diff.py and ndiff.py scripts of Tools/scripts/: move them to
Doc/includes/.

* diff.py and ndiff.py files are no longer executable. Remove also
  their shebang ("#!/usr/bin/env python3").
* Remove the -profile command from ndiff.py to simply the code.
* Remove ndiff.py copyright and history command. The Python
  documentation examples are distributed under the "Zero Clause BSD
  License".
vstinner added a commit that referenced this issue Oct 11, 2022
@vstinner
gh-97669: Remove outdated example scripts (#97675) (#98167)
e0ae9dd 
Remove outdated example scripts of the Tools/scripts/ directory:

* gprof2html.py
* md5sum.py
* nm2def.py
* pathfix.py
* win_add2path.py

Remove test_gprof2html, test_md5sum and test_pathfix of test_tools.
vstinner added a commit that referenced this issue Oct 11, 2022
@vstinner
gh-97669: Remove abitype.py and pep384_macrocheck.py (#98165)
454a6d6 
Remove abitype.py and pep384_macrocheck.py scripts of Tools/scripts/.
@bedevere-bot bedevere-bot mentioned this issue Oct 11, 2022
Merged
vstinner added a commit that referenced this issue Oct 12, 2022
@vstinner
gh-97669: Create Tools/patchcheck/ directory (#98186)
0895c2a 
Move patchcheck.py, reindent.py and untabify.py scripts to a new
Tools/patchcheck/ directory.
@bedevere-bot bedevere-bot mentioned this issue Oct 12, 2022
Merged
@vstinner vstinner mentioned this issue Oct 12, 2022
Open
@bedevere-bot bedevere-bot mentioned this issue Oct 12, 2022
Merged
vstinner added a commit that referenced this issue Oct 12, 2022
@vstinner
gh-97669: Fix test_tools reference leak (#98216)
a8c8526 
test_tools.test_sundry() now uses an unittest mock to prevent the
logging module to register a real "atfork" function which kept the
logging module dictionary alive. So the logging module can be
properly unloaded. Previously, the logging module was loaded before
test_sundry(), but it's no longer the case since recent test_tools
sub-tests removals.
vstinner added a commit that referenced this issue Oct 12, 2022
@vstinner
gh-97669: Remove Tools/scripts/startuptime.py (#98214)
d09d2c7 
The "pyperf command" tool be used instead. Example:

    $ python3 -m pyperf command -- python3 -c pass
    .....................
    command: Mean +- std dev: 17.8 ms +- 0.4 ms

pyperf also computes the standard deviation which gives an idea of
the benchmark looks reliable or not.
@vstinner
Copy link
Member Author

vstinner commented Oct 12, 2022 •
edited
Loading

Updated list of Tools/scripts/ scripts. With PR #97963, there are only 8 remaining scripts.

Programs (3):

  • 2to3
  • idle3
  • pydoc3

Misc (5):

Update: I moved verify_ensurepip_wheels.py to Tools/build/ in my PR #97963.

carljm added a commit to carljm/cpython that referenced this issue Oct 14, 2022
@carljm
Merge branch 'main' into typewatch
6e53514 
* main: (38 commits)
  pythongh-98251: Allow venv to pass along PYTHON* variables to pip and ensurepip when they do not impact path resolution (pythonGH-98259)
  Bpo-41246: IOCP Proactor avoid callback code duplication (python#21399)
  bpo-46364: Use sockets for stdin of asyncio only on AIX (python#30596)
  pythongh-98178: syslog() is not thread-safe on macOS (python#98213)
  Mark all targets in `Doc/Makefile` as `PHONY` (pythonGH-98189)
  pythongh-97982: Factorize PyUnicode_Count() and unicode_count() code (python#98025)
  pythongh-96265: Formatting changes for faq/general (python#98129)
  tutorial: remove "with single quotes" (python#98204)
  pythongh-97669: Remove Tools/scripts/startuptime.py (python#98214)
  signalmodule.c uses _PyErr_WriteUnraisableMsg() (python#98217)
  pythongh-97669: Fix test_tools reference leak (python#98216)
  pythongh-97669: Create Tools/patchcheck/ directory (python#98186)
  pythongh-65046: Link to logging cookbook from asyncio docs (python#98207)
  Formatting fixes in contextlib docs (python#98111)
  pythongh-95276: Add callable entry to the glossary (python#95738)
  pythongh-96130: Rephrase use of "typecheck" verb for clarity (python#98144)
  Fix some incorrect indentation around the main switch (python#98177)
  pythongh-98172: Fix formatting in `except*` docs (python#98173)
  pythongh-97982: Remove asciilib_count() (python#98164)
  pythongh-95756: Free and NULL-out code caches when needed (pythonGH-98181)
  ...
carljm added a commit to carljm/cpython that referenced this issue Oct 14, 2022
@carljm
Merge branch 'main' into cached_property
0bb0c69 
* main: (37 commits)
  pythongh-98251: Allow venv to pass along PYTHON* variables to pip and ensurepip when they do not impact path resolution (pythonGH-98259)
  Bpo-41246: IOCP Proactor avoid callback code duplication (python#21399)
  bpo-46364: Use sockets for stdin of asyncio only on AIX (python#30596)
  pythongh-98178: syslog() is not thread-safe on macOS (python#98213)
  Mark all targets in `Doc/Makefile` as `PHONY` (pythonGH-98189)
  pythongh-97982: Factorize PyUnicode_Count() and unicode_count() code (python#98025)
  pythongh-96265: Formatting changes for faq/general (python#98129)
  tutorial: remove "with single quotes" (python#98204)
  pythongh-97669: Remove Tools/scripts/startuptime.py (python#98214)
  signalmodule.c uses _PyErr_WriteUnraisableMsg() (python#98217)
  pythongh-97669: Fix test_tools reference leak (python#98216)
  pythongh-97669: Create Tools/patchcheck/ directory (python#98186)
  pythongh-65046: Link to logging cookbook from asyncio docs (python#98207)
  Formatting fixes in contextlib docs (python#98111)
  pythongh-95276: Add callable entry to the glossary (python#95738)
  pythongh-96130: Rephrase use of "typecheck" verb for clarity (python#98144)
  Fix some incorrect indentation around the main switch (python#98177)
  pythongh-98172: Fix formatting in `except*` docs (python#98173)
  pythongh-97982: Remove asciilib_count() (python#98164)
  pythongh-95756: Free and NULL-out code caches when needed (pythonGH-98181)
  ...
vstinner added a commit that referenced this issue Oct 17, 2022
@vstinner
gh-97669: Create Tools/build/ directory (#97963)
1863302 
Create Tools/build/ directory. Move the following scripts from
Tools/scripts/ to Tools/build/:

* check_extension_modules.py
* deepfreeze.py
* freeze_modules.py
* generate_global_objects.py
* generate_levenshtein_examples.py
* generate_opcode_h.py
* generate_re_casefix.py
* generate_sre_constants.py
* generate_stdlib_module_names.py
* generate_token.py
* parse_html5_entities.py
* smelly.py
* stable_abi.py
* umarshal.py
* update_file.py
* verify_ensurepip_wheels.py

Update references to these scripts.
@vstinner
Copy link
Member Author

I moved 16 scripts to a new Tools/build/ directory: 1863302

There are 9 remaining files in Tools/scripts/, including the README file (so 8 scripts):

2to3
checkpip.py
combinerefs.py
idle3
pydoc3
README
run_tests.py
summarize_stats.py
var_access_benchmark.py

carljm added a commit to carljm/cpython that referenced this issue Oct 17, 2022
@carljm
Merge branch 'main' into typewatch
cdbe93b 
* main: (31 commits)
  pythongh-95913: Move subinterpreter exper removal to 3.11 WhatsNew (pythonGH-98345)
  pythongh-95914: Add What's New item describing PEP 670 changes (python#98315)
  Remove unused arrange_output_buffer function from zlibmodule.c. (pythonGH-98358)
  pythongh-98174: Handle EPROTOTYPE under macOS in test_sendfile_fallback_close_peer_in_the_middle_of_receiving (python#98316)
  pythonGH-98327: Reduce scope of catch_warnings() in _make_subprocess_transport (python#98333)
  pythongh-93691: Compiler's code-gen passes location around instead of holding it on the global compiler state (pythonGH-98001)
  pythongh-97669: Create Tools/build/ directory (python#97963)
  pythongh-95534: Improve gzip reading speed by 10% (python#97664)
  pythongh-95913: Forward-port int/str security change to 3.11 What's New in main (python#98344)
  pythonGH-91415: Mention alphabetical sort ordering in the Sorting HOWTO (pythonGH-98336)
  pythongh-97930: Merge with importlib_resources 5.9 (pythonGH-97929)
  pythongh-85525: Remove extra row in doc (python#98337)
  pythongh-85299: Add note warning about entry point guard for asyncio example (python#93457)
  pythongh-97527: IDLE - fix buggy macosx patch (python#98313)
  pythongh-98307: Add docstring and documentation for SysLogHandler.createSocket (pythonGH-98319)
  pythongh-94808: Cover `PyFunction_GetCode`, `PyFunction_GetGlobals`, `PyFunction_GetModule` (python#98158)
  pythonGH-94597: Deprecate child watcher getters and setters (python#98215)
  pythongh-98254: Include stdlib module names in error messages for NameErrors (python#98255)
  Improve speed. Reduce auxiliary memory to 16.6% of the main array. (pythonGH-98294)
  [doc] Update logging cookbook with an example of custom handling of levels. (pythonGH-98290)
  ...
@vstinner
Copy link
Member Author

vstinner commented Nov 3, 2022

I removed enough scripts, I close the issue :-)

@vstinner vstinner closed this as completed Nov 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@vstinner