Crash on docstring with tabs

[foozzi]

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

1. Take this file (in text)
2. Run Black on it without any arguments
3. See error

Expected behavior Work

Environment (please complete the following information):

error: cannot format /home/user/cilantropy/dist_worker.py: INTERNAL ERROR: Black produced code that is not equivalent to the source.  Please report a bug on https://github.com/psf/black/issues.  This diff might be helpful: /tmp/blk_rifw8eif.log
Oh no! 💥 💔 💥
10 files left unchanged, 1 file failed to reformat.

• Version: 19.10b0
• OS and Python version: Linux (arch linux)/Python 3.8.3

Does this bug also happen on master? To answer this, you have two options:
pip install git+git://github.com/psf/black

Additional context:
Log:

--- src
+++ dst
@@ -62,11 +62,11 @@
       body=
         Expr(
           value=
             Constant(
               value=
-                'main worker for run subproccess and return result or error\n\n:param cmd: full command for run\t\n:return: result string and status error',  # str
+                'main worker for run subproccess and return result or error\n\n:param cmd: full command for run\n:return: result string and status error',  # str
             )  # /Constant
         )  # /Expr
         Try(
           body=
             Assign(

and my code:

def main_worker(cmd):
	""" 
	main worker for run subproccess and return result or error

	:param cmd: full command for run	
	:return: result string and status error
	"""	

	try:
		cmd_response = subprocess.run(
			cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=True
		)
	except subprocess.CalledProcessError as e:
		return "The pip command did not succeed: {stderr}".format(
				stderr=e.stderr.decode("utf-8")
			), True
		
	return cmd_response.stdout.decode("utf-8").replace(" "*6,"\n"), False

[hugovk]

This is caused by the tab at the end of a line in the docstring:

---

Here's a minimal example (initial tabs replaced with spaces):

def thing():
    """ 
    line ending in a tab	
    line ending with no whitespace
    """

---

Error does not occur with a tab at the end of the docstring:

def thing():
    """ 
    line ending in a tab	
    """

And does not occur with a space at the end:

def thing():
    """ 
    line ending in a space 
    line ending with no whitespace
    """

[foozzi]

@hugovk thanks!

[hugovk]

You're welcome!

This is still a bug in Black, so we can keep it open to be investigated :)

[hugovk]

git bisect points to a4c11a7 as the first bad commit, from "Re-indent the contents of docstrings" (#1053).

[JelleZijlstra]

We can probably fix this by loosening the constraints on docstrings when we do the AST comparison, similar to what I did in #1593.

[peterjc]

Updated (again), sorry for the noise.

Confirming a duplicate like #1638 (which may be a different but related issue in black 20.8b1)

[aureliojargas]

Note that the bug also occurs when there's a literal tab in the middle of a line. It seems black converts that to spaces and errors out.

def foo():
    """
    a	b  # it's a literal tab between a and b
    """
    pass

$ black foo.py
error: cannot format foo.py: INTERNAL ERROR: Black produced code that is not equivalent to the source.  Please report a bug on https://github.com/psf/black/issues.  This diff might be helpful: /tmp/blk_q5uiqxs8.log
Oh no! 💥 💔 💥
1 file failed to reformat.

The contents of /tmp/blk_q5uiqxs8.log

--- src
+++ dst
@@ -23,11 +23,11 @@
       body=
         Expr(
           value=
             Constant(
               value=
-                "a\tb  # it's a literal tab between a and b",  # str
+                "a   b  # it's a literal tab between a and b",  # str
             )  # /Constant
         )  # /Expr
         Pass(
         )  # /Pass
       decorator_list=

[manics]

Note also the bug only occurs with a multiline docstring:
This fails:

def f(p):
    """p:	Parameter
    """
    pass

This passes:

def f(p):
    """p:	Parameter"""
    pass

[jparise]

This appears to be caused by the intentional decision (introduced in #1053) to expand tabs to space (using `tabsize=8``, which is the expandtabs() default):

black/src/black/__init__.py

Lines 6745 to 6747 in 6dddbd7

	# Convert tabs to spaces (following the normal Python rules)
	# and split into a list of lines:
	lines = docstring.expandtabs().splitlines()