bpo-37936: Systematically distinguish rooted vs. unrooted in .gitignore (GH-15823)

A root cause of bpo-37936 is that it's easy to write a .gitignore
rule that's intended to apply to a specific file (e.g., the
`pyconfig.h` generated by `./configure`) but actually applies to all
similarly-named files in the tree (e.g., `PC/pyconfig.h`.)

Specifically, any rule with no non-trailing slashes is applied in an
"unrooted" way, to files anywhere in the tree.  This means that if we
write the rules in the most obvious-looking way, then

 * for specific files we want to ignore that happen to be in
   subdirectories (like `Modules/config.c`), the rule will work
   as intended, staying "rooted" to the top of the tree; but

 * when a specific file we want to ignore happens to be at the root of
   the repo (like `platform`), then the obvious rule (`platform`) will
   apply much more broadly than intended: if someone tries to add a
   file or directory named `platform` somewhere else in the tree, it
   will unexpectedly get ignored.

That's surprising behavior that can make the .gitignore file's
behavior feel finicky and unpredictable.

To avoid it, we can simply always give a rule "rooted" behavior when
that's what's intended, by systematically using leading slashes.

Further, to help make the pattern obvious when looking at the file and
minimize any need for thinking about the syntax when adding new rules:
separate the rules into one group for each type, with brief comments
identifying them.

For most of these rules it's clear whether they're meant to be rooted
or unrooted, but in a handful of cases I've only guessed.  In that
case the safer default (the choice that won't hide information) is the
narrower, rooted meaning, with a leading slash.  If for some of these
the unrooted meaning is desired after all, it'll be easy to move them
to the unrooted section at the top.

Author: gnprice

Diff for: .gitignore

@@ -1,11 +1,14 @@
-# Two-trick pony for OSX and other case insensitive file systems:
-# Ignore ./python binary on Unix but still look into ./Python/ directory.
-/python
-!/Python/
+#####
+# First, rules intended to apply in all subdirectories.
+# These contain no slash, or only a trailing slash.
 
 *.cover
 *.iml
 *.o
+*.a
+*.so*
+*.dylib
+*.dll
 *.orig
 *.pyc
 *.pyd
@@ -18,6 +21,31 @@
 *.profraw
 *.dyn
 .gdb_history
+.purify
+__pycache__
+.hg/
+.svn/
+.idea/
+tags
+TAGS
+.vs/
+.vscode/
+gmon.out
+.coverage
+.mypy_cache/
+
+*.exe
+!Lib/distutils/command/*.exe
+
+# Ignore core dumps... but not Tools/msi/core/ or the like.
+core
+!core/
+
+
+#####
+# Then, rules meant for a specific location relative to the repo root.
+# These must contain a non-trailing slash (and may also have a trailing slash.)
+
 Doc/build/
 Doc/venv/
 Doc/.venv/
@@ -29,7 +57,7 @@ Lib/lib2to3/*.pickle
 Lib/test/data/*
 !Lib/test/data/README
 /Makefile
-Makefile.pre
+/Makefile.pre
 Misc/python.pc
 Misc/python-embed.pc
 Misc/python-config.sh
@@ -38,12 +66,9 @@ Modules/Setup.local
 Modules/config.c
 Modules/ld_so_aix
 Programs/_freeze_importlib
-Programs/_freeze_importlib.exe
 Programs/_testembed
-Programs/_testembed.exe
 PC/python_nt*.h
 PC/pythonnt_rc*.h
-PC/*/*.exe
 PC/*/*.exp
 PC/*/*.lib
 PC/*/*.bsc
@@ -62,53 +87,35 @@ PCbuild/*-pgi
 PCbuild/*-pgo
 PCbuild/*.VC.db
 PCbuild/*.VC.opendb
-PCbuild/.vs/
 PCbuild/amd64/
 PCbuild/arm32/
 PCbuild/arm64/
 PCbuild/obj/
 PCbuild/win32/
 Tools/unicode/data/
-.purify
-__pycache__
-autom4te.cache
-build/
-buildno
-config.cache
-config.log
-config.status
-config.status.lineno
-core
-!Tools/msi/core/
-db_home
-.hg/
-.idea/
-ipch/
-libpython*.a
-libpython*.so*
-libpython*.dylib
-libpython*.dll
-platform
-pybuilddir.txt
+/autom4te.cache
+/build/
+/config.cache
+/config.log
+/config.status
+/config.status.lineno
+/platform
+/pybuilddir.txt
 /pyconfig.h
-python-config
-python-config.py
-python.bat
-python.exe
-python-gdb.py
-python.exe-gdb.py
-reflog.txt
-.svn/
-tags
-TAGS
-.coverage
-coverage/
-externals/
-htmlcov/
+/python-config
+/python-config.py
+/python.bat
+/python-gdb.py
+/python.exe-gdb.py
+/reflog.txt
+/coverage/
+/externals/
+/htmlcov/
 Tools/msi/obj
 Tools/ssl/amd64
 Tools/ssl/win32
-.vs/
-.vscode/
-gmon.out
-.mypy_cache/
+
+# Two-trick pony for OSX and other case insensitive file systems:
+# Ignore ./python binary on Unix but still look into ./Python/ directory.
+/python
+!/Python/

Diff for: Lib/test/libregrtest/runtest.py

@@ -313,9 +313,7 @@ def cleanup_test_droppings(test_name, verbose):
     # since if a test leaves a file open, it cannot be deleted by name (while
     # there's nothing we can do about that here either, we can display the
     # name of the offending test, which is a real help).
-    for name in (support.TESTFN,
-                 "db_home",
-                ):
+    for name in (support.TESTFN,):
         if not os.path.exists(name):
             continue
 

Diff for: Misc/NEWS.d/next/Build/2019-09-10-00-54-48.bpo-37936.E7XEwu.rst

@@ -0,0 +1,5 @@
+The :file:`.gitignore` file systematically keeps "rooted", with a
+non-trailing slash, all the rules that are meant to apply to files in a
+specific place in the repo.  Previously, when the intended file to ignore
+happened to be at the root of the repo, we'd most often accidentally also
+ignore files and directories with the same name anywhere in the tree.