No luigi.six.{u,b} (#2848)

* Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * no six.{u,b} * joined * Update .travis.yml * "except ImportError" removed (#2845) * py2/py3 import error eliminated * E402 module level import not at top of file * E303 too many blank lines (3) * snakebite removed (#2833) * snamebite removed * no minicluster * no minicluster * flake8 * branches - master * no snakebite in tox.ini * [luigi.six.itervalues] usage removed (#2835) * no itervalues * no exception * [luigi.six.assertCountEqual] removed (#2841) * Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * six.assertCountEqual -> self.assertCountEqual * no ununsed import * E128 continuation line under-indented for visual indent * [luigi.six.next] removed (#2842) * Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * no six.next * yield from instead of loop * too-many-blank-lines * `luigi.six.add_metaclass` removed (#2843) * Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * six.add_metaclass -> metaclass= * six.add_metaclass -> metaclass= * no `from __future__ import ...` (#2846) * Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * no from __future__ import * [luigi.six.string_types] and [luigi.six.binary_type] removed (#2838) * no string_types * no-binary_type * Update .travis.yml * Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * original * no ununsed import * Update .travis.yml * Fix params hashing (#2540) * move params freezing to separated module * freeze dict for toml parser * FIX FrozenOrderedDict name * Freeze only params of config, not all config * +docstring * [luigi.mock.MockFile] removed (#2839) * no-mock-file * MockFile removed * unused import * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * new string_types * no string_types * no-binary_type * original * no ununsed import * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * new string_types * snakebite * no-unused-import * utf-8 in auth * no unused imports * py27 -> py37 in CONTRIBUTING.rst * E231 missing whitespace after ',' * [luigi.six.moves] removed (#2844) * Update .travis.yml * Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * no-six-moves * del attr removed * del attr removed * exec_ removed * py 2.7 mentions removed from README and setup.py (#2852) * 2.7 mentions removed from README and setup.py * py27 -> py37 in CONTRIBUTING.rst * abstractproperty -> property + abstractmethod (#2847) * Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * abstractproperty -> property + abstractmethod * Update beam_dataflow.py * Update beam_dataflow.py * no-if-six.py* (#2840) * no-if-six.py* * no unused imports * Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * postgres toxenv for py3 * postgres toxenv for py3 * postgres toxenv for py3 * \n * no-unused-import * test_multiple_workers back * F401 'luigi.six' imported but unused * test_url_without_basic_auth fix * [luigi.six.iteritems] usage removed (#2834) * iteritems usage removed * no unused imports * bmo * bmo * iteritems removed * no doc for iteritems * most_common from collections.Counter * returning tuple * postgres toxenv for py3 * original * no-unused-import * F401 'luigi.six' imported but unused * blank line * F401 'luigi.six' imported but unused * six.b('...') -> b'...' * blank line
spotify · Dec 3, 2019 · 0ac6458 · 0ac6458
1 parent 8b0fb12
commit 0ac6458
Show file tree

Hide file tree

Showing 7 changed files with 46 additions and 75 deletions.
diff --git a/luigi/batch_notifier.py b/luigi/batch_notifier.py
@@ -7,8 +7,8 @@
 from datetime import datetime
 import time
 
+
 import luigi
-from luigi import six
 from luigi.notifications import send_email, email
 import luigi.parameter
 
@@ -76,18 +76,18 @@ def _key(self, task_name, family, unbatched_args):
         elif self._config.batch_mode == 'family':
             return family
         elif self._config.batch_mode == 'unbatched_params':
-            param_str = six.u(', ').join(six.u('{}={}').format(*kv) for kv in unbatched_args.items())
-            return six.u('{}({})').format(family, param_str)
+            param_str = ', '.join('{}={}'.format(k, v) for k, v in unbatched_args.items())
+            return '{}({})'.format(family, param_str)
         else:
             raise ValueError('Unknown batch mode for batch notifier: {}'.format(
                 self._config.batch_mode))
 
     def _format_expl(self, expl):
         lines = expl.rstrip().split('\n')[-self._config.error_lines:]
         if self._email_format == 'html':
-            return six.u('<pre>{}</pre>').format('\n'.join(lines))
+            return '<pre>{}</pre>'.format('\n'.join(lines))
         else:
-            return six.u('\n{}').format(six.u('\n').join(map(six.u('      {}').format, lines)))
+            return '\n{}'.format('\n'.join(map('      {}'.format, lines)))
 
     def _expl_body(self, expls):
         lines = [self._format_expl(expl) for expl in expls]
@@ -102,15 +102,15 @@ def _format_task(self, task_tuple):
             _plural_format('{} disable{}', disable_count),
             _plural_format('{} scheduling failure{}', scheduling_count),
         ]
-        count_str = six.u(', ').join(filter(None, counts))
-        return six.u('{} ({})').format(task, count_str)
+        count_str = ', '.join(filter(None, counts))
+        return '{} ({})'.format(task, count_str)
 
     def _format_tasks(self, tasks):
         lines = map(self._format_task, sorted(tasks, key=self._expl_key))
         if self._email_format == 'html':
-            return six.u('<li>{}').format(six.u('\n<br>').join(lines))
+            return '<li>{}'.format('\n<br>'.join(lines))
         else:
-            return six.u('- {}').format(six.u('\n  ').join(lines))
+            return '- {}'.format('\n  '.join(lines))
 
     def _owners(self, owners):
         return self._default_owner | set(owners)
@@ -163,9 +163,9 @@ def _email_body(self, fail_counts, disable_counts, scheduling_counts, fail_expls
         for tasks, msg in expl_groups:
             body_lines.append(self._format_tasks(tasks))
             body_lines.append(msg)
-        body = six.u('\n').join(filter(None, body_lines)).rstrip()
+        body = '\n'.join(filter(None, body_lines)).rstrip()
         if self._email_format == 'html':
-            return six.u('<ul>\n{}\n</ul>').format(body)
+            return '<ul>\n{}\n</ul>'.format(body)
         else:
             return body
 

diff --git a/luigi/rpc.py b/luigi/rpc.py
@@ -26,7 +26,6 @@
 import time
 import base64
 
-from luigi import six
 from urllib.parse import urljoin, urlencode, urlparse
 from urllib.request import urlopen, Request
 from urllib.error import URLError
@@ -78,9 +77,8 @@ def _create_request(self, full_url, body=None):
         url = urlparse(full_url)
         if url.username:
             # base64 encoding of username:password
-            auth = base64.b64encode(six.b('{}:{}'.format(url.username, url.password or '')))
+            auth = base64.b64encode('{}:{}'.format(url.username, url.password or '').encode('utf-8'))
             auth = auth.decode('utf-8')
-
             # update full_url and create a request object with the auth header set
             full_url = url._replace(netloc=url.netloc.split('@', 1)[-1]).geturl()
             req = Request(full_url)

diff --git a/luigi/six.py b/luigi/six.py
@@ -456,11 +456,6 @@ def iterlists(d, **kw):
 
 
 if PY3:
-    def b(s):
-        return s.encode("latin-1")
-
-    def u(s):
-        return s
     unichr = chr
     import struct
     int2byte = struct.Struct(">B").pack
@@ -478,12 +473,6 @@ def u(s):
         _assertRaisesRegex = "assertRaisesRegex"
         _assertRegex = "assertRegex"
 else:
-    def b(s):
-        return s
-    # Workaround for standalone backslash
-
-    def u(s):
-        return unicode(s.replace(r'\\', r'\\\\'), "unicode_escape")
     unichr = unichr
     int2byte = chr
 
@@ -497,8 +486,6 @@ def indexbytes(buf, i):
     StringIO = BytesIO = StringIO.StringIO
     _assertRaisesRegex = "assertRaisesRegexp"
     _assertRegex = "assertRegexpMatches"
-_add_doc(b, """Byte literal""")
-_add_doc(u, """Text literal""")
 
 
 

diff --git a/test/batch_notifier_test.py b/test/batch_notifier_test.py
@@ -4,8 +4,6 @@
 import mock
 import unittest
 
-import six
-
 import luigi.batch_notifier
 
 
@@ -489,105 +487,96 @@ def test_batch_identical_expls_html(self):
 
     def test_unicode_error_message(self):
         bn = BatchNotifier(error_messages=1)
-        bn.add_failure('Task()', 'Task', {}, six.u('Érror'), [])
+        bn.add_failure('Task()', 'Task', {}, 'Érror', [])
         bn.send_email()
         self.check_email_send(
             'Luigi: 1 failure in the last 60 minutes',
-            six.u(
-                '- Task() (1 failure)\n'
-                '\n'
-                '      Érror'
-            )
+            '- Task() (1 failure)\n'
+            '\n'
+            '      Érror'
         )
 
     def test_unicode_error_message_html(self):
         self.email().format = 'html'
         bn = BatchNotifier(error_messages=1)
-        bn.add_failure('Task()', 'Task', {}, six.u('Érror'), [])
+        bn.add_failure('Task()', 'Task', {}, 'Érror', [])
         bn.send_email()
         self.check_email_send(
             'Luigi: 1 failure in the last 60 minutes',
-            six.u(
-                '<ul>\n'
-                '<li>Task() (1 failure)\n'
-                '<pre>Érror</pre>\n'
-                '</ul>'
-            ),
+            '<ul>\n'
+            '<li>Task() (1 failure)\n'
+            '<pre>Érror</pre>\n'
+            '</ul>'
         )
 
     def test_unicode_param_value(self):
         for batch_mode in ('all', 'unbatched_params'):
             self.send_email.reset_mock()
             bn = BatchNotifier(batch_mode=batch_mode)
-            bn.add_failure(six.u('Task(a=á)'), 'Task', {'a': six.u('á')}, 'error', [])
+            bn.add_failure('Task(a=á)', 'Task', {'a': 'á'}, 'error', [])
             bn.send_email()
             self.check_email_send(
                 'Luigi: 1 failure in the last 60 minutes',
-                six.u('- Task(a=á) (1 failure)')
+                '- Task(a=á) (1 failure)'
             )
 
     def test_unicode_param_value_html(self):
         self.email().format = 'html'
         for batch_mode in ('all', 'unbatched_params'):
             self.send_email.reset_mock()
             bn = BatchNotifier(batch_mode=batch_mode)
-            bn.add_failure(six.u('Task(a=á)'), 'Task', {'a': six.u('á')}, 'error', [])
+            bn.add_failure('Task(a=á)', 'Task', {'a': 'á'}, 'error', [])
             bn.send_email()
             self.check_email_send(
                 'Luigi: 1 failure in the last 60 minutes',
-                six.u(
-                    '<ul>\n'
-                    '<li>Task(a=á) (1 failure)\n'
-                    '</ul>'
-                )
+                '<ul>\n'
+                '<li>Task(a=á) (1 failure)\n'
+                '</ul>'
             )
 
     def test_unicode_param_name(self):
         for batch_mode in ('all', 'unbatched_params'):
             self.send_email.reset_mock()
             bn = BatchNotifier(batch_mode=batch_mode)
-            bn.add_failure(six.u('Task(á=a)'), 'Task', {six.u('á'): 'a'}, 'error', [])
+            bn.add_failure('Task(á=a)', 'Task', {'á': 'a'}, 'error', [])
             bn.send_email()
             self.check_email_send(
                 'Luigi: 1 failure in the last 60 minutes',
-                six.u('- Task(á=a) (1 failure)')
+                '- Task(á=a) (1 failure)'
             )
 
     def test_unicode_param_name_html(self):
         self.email().format = 'html'
         for batch_mode in ('all', 'unbatched_params'):
             self.send_email.reset_mock()
             bn = BatchNotifier(batch_mode=batch_mode)
-            bn.add_failure(six.u('Task(á=a)'), 'Task', {six.u('á'): 'a'}, 'error', [])
+            bn.add_failure('Task(á=a)', 'Task', {'á': 'a'}, 'error', [])
             bn.send_email()
             self.check_email_send(
                 'Luigi: 1 failure in the last 60 minutes',
-                six.u(
-                    '<ul>\n'
-                    '<li>Task(á=a) (1 failure)\n'
-                    '</ul>'
-                )
+                '<ul>\n'
+                '<li>Task(á=a) (1 failure)\n'
+                '</ul>'
             )
 
     def test_unicode_class_name(self):
         bn = BatchNotifier()
-        bn.add_failure(six.u('Tásk()'), six.u('Tásk'), {}, 'error', [])
+        bn.add_failure('Tásk()', 'Tásk', {}, 'error', [])
         bn.send_email()
         self.check_email_send(
             'Luigi: 1 failure in the last 60 minutes',
-            six.u('- Tásk() (1 failure)')
+            '- Tásk() (1 failure)'
         )
 
     def test_unicode_class_name_html(self):
         self.email().format = 'html'
         bn = BatchNotifier()
-        bn.add_failure(six.u('Tásk()'), six.u('Tásk'), {}, 'error', [])
+        bn.add_failure('Tásk()', 'Tásk', {}, 'error', [])
         bn.send_email()
         self.check_email_send(
             'Luigi: 1 failure in the last 60 minutes',
-            six.u(
-                '<ul>\n'
-                '<li>Tásk() (1 failure)\n'
-                '</ul>'
-            ),
+            '<ul>\n'
+            '<li>Tásk() (1 failure)\n'
+            '</ul>'
+
         )
diff --git a/test/contrib/hive_test.py b/test/contrib/hive_test.py
@@ -21,7 +21,6 @@
 import tempfile
 from helpers import unittest
 
-from luigi import six
 import luigi.contrib.hive
 import mock
 from luigi import LocalTarget
@@ -36,7 +35,7 @@ class HiveTest(unittest.TestCase):
     def mock_hive_cmd(self, args, check_return=True):
         self.last_hive_cmd = args
         self.count += 1
-        return six.u("statement{0}".format(self.count))
+        return 'statement{}'.format(self.count)
 
     def setUp(self):
         self.run_hive_cmd_saved = luigi.contrib.hive.run_hive
@@ -271,7 +270,7 @@ def test_run_hive_command(self, popen):
         # I'm testing this again to check the return codes
         # I didn't want to tear up all the existing tests to change how run_hive is mocked
         comm = mock.Mock(name='communicate_mock')
-        comm.return_value = six.b("some return stuff"), ""
+        comm.return_value = b'some return stuff', ''
 
         preturn = mock.Mock(name='open_mock')
         preturn.returncode = 0
@@ -284,7 +283,7 @@ def test_run_hive_command(self, popen):
         preturn.returncode = 17
         self.assertRaises(luigi.contrib.hive.HiveCommandError, luigi.contrib.hive.run_hive, ["blah", "blah"])
 
-        comm.return_value = six.b(""), "some stderr stuff"
+        comm.return_value = b'', 'some stderr stuff'
         returned = luigi.contrib.hive.run_hive(["blah", "blah"], False)
         self.assertEqual("", returned)
 

diff --git a/test/notifications_test.py b/test/notifications_test.py
@@ -25,7 +25,6 @@
 from luigi.notifications import generate_email
 from luigi.scheduler import Scheduler
 from luigi.worker import Worker
-from luigi import six
 import luigi
 
 
@@ -149,8 +148,8 @@ def testEmailRecipientsNoConfig(self):
     def test_generate_unicode_email(self):
         generate_email(
             sender='test@example.com',
-            subject=six.u('sübjéct'),
-            message=six.u("你好"),
+            subject='sübjéct',
+            message="你好",
             recipients=['receiver@example.com'],
             image_png=None,
         )

diff --git a/test/rpc_test.py b/test/rpc_test.py
@@ -29,7 +29,6 @@
 import socket
 from multiprocessing import Process, Queue
 import requests
-from luigi import six
 
 
 class RemoteSchedulerTest(unittest.TestCase):
@@ -206,7 +205,7 @@ def test_body_encoding(self):
 
         # with body
         req = fetcher._create_request('http://localhost', body={'foo': 'bar baz/test'})
-        self.assertEqual(req.data, six.b('foo=bar+baz%2Ftest'))
+        self.assertEqual(req.data, b'foo=bar+baz%2Ftest')
 
         # without body
         req = fetcher._create_request('http://localhost')