enum34 be gone

[huguesb]

• Fallback process command name in lock.getpcmd
• Moves task id computation into a separate function in task.py
• Speeds up get_work with a number of small tweaks
• Fix documentation generation
• Add DictParameter
• Use immutable and ordered dict in DictParameter
• Add test cases for DictParameter
• Update dict param docs with tag based exammple
• Add parse-serialize test case
• Add enbrite.ly to companies that use Luigi
• Added support for SFTP.
• Update import in parameter, revert setup.py
• Refactor s3 to s3 copying
• correct example spark_als
• Add s3 directory copy tests
• Scheduler UI: Fix python3 issues in resources view
• Scheduler: Remove backward compatibility code
• scheduler.py: Update comment
• Corrected usage of 'query' as it was defined as a property, but reference and used as if it were a method.
• Add DJ/WSJ to list of companies.
• Updated to use iteration (over recursion) and TempFiles to avoid segmentation faults due to massive recursion and extremely large datasets from salesforce.
• Don't make every response an ordereddict. The order of the dictionary fields don't matter here.What matters is that the list of dicts is in the appropriate order if one specifies an order by within their SOQL. Since lists maintain order, this is good."
• Attempt to give luigi s3 module assume role support
• Luigi tasks for google dataproc
• Fixed _ftp_exists(...) method. Previously called nlst(path) on full path including filename and extention. This only worked if the file existed.
• Updated the way asks get run and added more documentation
• scheduler.py: Refactor Scheduler.prune()
• Scheduler: Refactor and bugfix task pruning
• assistants bugfix: Also prune UNKNOWN tasks
• EnumParameter docs: Encourage singular enums
• EnumParameter docs: Use Enum not IntEnum
• Sets task.run to None instead of NotImplemented in ExternalTasks
• Fix spelling error in central_planner_test.py
• Return code for 'check_complete' exception
• Add an index on task_events.task_id.
• Return code for 'requires()' exception
• Re-raise exceptions to trigger "unhandled_exception"
• Add tests for unwanted return code 0
• Added ListParameter and TupleParameter.
• Added tests for List/Tuple Parameter serialize/parse and config read.
• Added comment explanation of parse() execution for TupleParameter.
• Added test for cli arg parse.
• 2.1.0
• Added module level documentation for a couple use cases of luigi.util
• Bug fixes for Hive CLI and Beeline compatibility
• Clarify wording of "clean slate" in scheduler logging
• Add several extra options to bigquery load configuration
• fix failing tests
• Add explicit whitelist of RPC commands for luigid
• Release version 2.1.1
• Add GUI-visible status messages for tasks
• Make luigid honor logging_conf_file
• Add status message endpoints to RPC handler list.
• parameter.py: Rename .__config to ._config_path
• Bugfix: Scheduler crashes for some configs
• Include sys.argv in error emails (Include sys.argv in error emails #1637)
• Updated grammer in target.AtomicLocalFile class
• Raise Queue.Empty for get() on empty queue
• scheduler: Remove unused prune_done_tasks option (scheduler: Remove unused prune_done_tasks option #1640)
• Changed task_history_override to task_history_impl (Update CentralPlannerScheduler docsting #1641)
• Fix bug in LocalFileSystem.move (Fix bug in LocalFileSystem.move #1643)
• luigid logging: Log less and change logger name (luigid logging: Log less and change logger name #1636)
• Make status_message optional in task serialization. (Make status_message optional in task serialization. #1645)
• Upgraded dagre-d3 to v0.4.17 to resolve issue Dependency graph visualization does not display arrows in chrome. #1604
• Add the archive parameters to HadoopJobRunner
• move archives paramter to the end. use instance variable directly.
• Warn user if Parameter serializes something other than a string.
• Set scheduler_url, logging_conf_file, and module to '' by default, rather than None. Made appropriate conditional checking updates.
• Update test which uses logging_conf_file to default its value to an empty string rather than None.
• bugfix: Don't crash when logging isn't specified
• Fix typo
• fix failing tests
• remove test_task_limit, to address in a separate PR
• name statuses more consistently
• use the same wording in email as in the log message
• work around erroring (order sensitive) test
• Add GitHub PR template file (Issue/PR Templates #1655)
• Fix test_dep_graph_missing_deps
• Fix test_connection_error
• Revert "work around erroring (order sensitive) test"
• Set namespace for FailingTask
• Default return code set to 0 for 'scheduling_error'
• Add contrib package for MSSQL (Add contrib package for MSSQL #1650)
• Fix running nose with xunit. (Fix running nose with xunit. #1659)
• changed "visualizer" to "visualiser" to be consistent with the majority of occurrences (changed "visualizer" to "visualiser" to be consistent with the majori… #1667)
• Fix formatting of example configuration snippet
• Added rename/copy/move functions to FileSystem and FileSystemTarget classes where applicable
• provide missing S3CopyJSONToTable INFO logging (provide missing S3CopyJSONToTable INFO logging #1674)
• Removed redundant function
• Fix typo in a log message.
• Compute the 'new_deps' variable only when it's needed.
• Added rename and copy to FileSystem, removed rename from children
• Allow GCSTargets for Hadoop streaming jobs (Allow GCSTarget's as input and GCSFlagTarget's as output for Hadoop jobs #1664)
• tox.ini: Remove # comment, fixing travis breakage
• allow sftp default port 22 to be overridden
• Add Hotels.com as user of Luigi to the README.rst
• Fix worker configuration headings. (Fix worker configuration headings. #1691)
• Add support for multiple results in salesforce batch jobs (#1685 Add support for multiple results in salesforce batch jobs #1686)
• Clarify file permission error message (Clarify file permission error message #1692)
• Uses more thorough pid check for locking
• Convert readthedocs link for their .org -> .io migration for hosted projects
• Fixes Sphinx version for docs to 1.3b1
• Accepts more arguments in overridden _warn_node function in doc conf
• [docs] Bump sphinx version for tox builds ([docs] Bump sphinx version for tox builds #1701)
• Allows package binaries to be passed to job runner (Allows package binaries to be passed to job runner #1565)
• docs: Fix typo (master #1703)
• Redshift: raise an error when no columns are found (raise an error when no columns are found #1676)
• Fix docs for @inherits decorator. (Fix inherits decorator docs. #1679)
• Adds AWS session token support for redshift copy statements (Aws session token #1702)
• Updatat six to version 1.10.0 (Updated six to version 1.10.0 #1709)
• Added a link to Custobar Luigi slides to Who uses Luigi list
• Can yield (not return) a list of tasks. (Can yield (not return) a list of tasks. #1713)
• Ensure DictParameter's are normalized to FrozenOrderedDict (Ensure DictParameter's are normalized to FrozenOrderedDict #1708)
• Implement pass-through parameters on Range tasks (Pass through parameters on Range tasks to of tasks #1675)
• DateParameter docs: Add example with interpolation (DateParameter docs: Add example with interpolation #1716)
• Correct docs for Range.of-params (Correct docs for Range.of-params #1715)
• Adding release step for Debian packages. (Adding release step for Debian packages. #1718)
• Add luigi-deps-tree visualising tool (Add "print task tree only" option to luigi #1680)
• Add combiner to docstrings in mrrunner
• Update example_top_artists.rst (Update example_top_artists.rst #1662)
• Consistent Luigi spelling in docs (Consistent Luigi spelling in docs #1723)
• Remove tracking url callback hack (Remove tracking url callback hack #1722)
• Add note about underscore in parameter names (Added note in documentation about underscore in parameter names #1729)
• Deprecated LocalTarget fn propery
• Remove MockTarget path property
• Rename MockTarget class variable _fn to path
• Fix salesforce default content type (Fix default content type (salesforce) #1724)
• Removes redundant function definitions from rpc and server (Removes redundant function definitions from rpc and server #1734)
• Reduce number of get_params calls in common_params.
• Add additional event handlers to tasks (Add timeout and process failure event handlers to tasks #1698)
• Fix bug with GCSFlagTarget
• Caches get_autoconfig_client on a per-thread basis
• Reset terminal colors in external_program ([MRG+2] Reset terminal colors in external_program ENVIRONMENT print #1742)
• Update call to iteritems in luigi/tools/deps: deprecated in Python 3 (Update call to iteritems in luigi/tools/deps: deprecated in Python 3 #1749)
• Add tests for hashing parameters (Added tests for hashing parameters except ListParameter #1719)
• Version 2.2.0
• Add option for list() and/or listdir() to return the boto.s3.key.Key object
• Add tests for list() and listdir()
• README: Add pypi version badge (README: Add pypi version badge #1757)
• Normalize ListParameter to be Immutable (Normalize ListParameter to be Immutable #1759)
• docs: Set minimum versions for sphinx (docs: Set minimum versions for sphinx #1760)
• docs: Install sphinx 1.4.4 in setup.py (docs: Install sphinx 1.4.4 in setup.py #1761)
• Fix "owner_email" log message. (Add missing space in log message. #1762)
• Disable codecov comments on GitHub PRs (Disable codecov comments on GitHub PRs #1754)
• Remove the confusing "dummy_test_module" directory (Remove the confusing "dummy_test_module" directory #1756)
• Update copy() to return number and size of files copied
• Fix exist method for ftp server
• Remove sitecustomize file (Remove sitecustomize file #1755)
• tests: Fix warning in remote_scheduler_test.py (tests: Fix warning in remote_scheduler_test.py #1774)
• Update retcodes to handle new cases (Update retcodes to handle new cases #1771)
• tests: Skip a inttermittently failing s3 test (tests: Skip a inttermittently failing s3 test #1777)
• Assistants: Don't affect longevity of tasks (Assistants: Don't affect longevity of tasks #1772)
• Remove abstract Scheduler class (Remove abstract Scheduler class #1778)
• Rename CentralPlannerScheduler to Scheduler (Rename CentralPlannerScheduler to Scheduler #1781)
• README: Remove monthly downloads badge
• Excludes .tox from flake8 to prevent checking third-party libraries (Excludes .tox from flake8 to prevent checking third-party libraries #1785)
• flake8: Unbreak travis build
• tox: Specify sphinx dependency better
• Add DateSecondParameter to parameter.py (Adding DateSecondParameter to parameter.py (close #1775) #1779)
• Marking as minimum upstream severity instead of max (Marking as minimum upstream severity instead of max #1789)
• Truncates long streaming job confs (Truncates long streaming job confs #1773)
• tests: Add a testcase confirming Marking as minimum upstream severity instead of max #1789 (tests: Add a testcase confirming #1789 #1790)
• Changed removing temporary scripts directory from shutil.rmtree() to a subprocess calling 'rm -rf'.
• Added option to run SGEJobTask locally.
• Added InputFileParameter and OutputFileParameter classes.
• Removed unused import.
• Fixed changes to match luigi standard.
• sge: Add pwd to python's path when running on cluster. (Added pwd to python's path when running on cluster. #1797)
• Parameterized poll_time for SGEJobTask. (Parameterized poll_time for SGEJobTask. #1795)
• Implement per task retry policies (Per Task Retry-Policy #1791)
• SGE: Add --dont-remove-tmp-dir parameter to SGEJobTask (Added --dont-remove-tmp-dir parameter to SGEJobTask. #1798)
• Revert "Added InputFileParameter and OutputFileParameter classes." (Revert "Added InputFileParameter and OutputFileParameter classes." #1803)
• Made creating a tarball of luigi optional to reduce I/O overhead of running an SGEJobTask.
• Added parameters to SGEJobTask to have control over the job name.
• Add NumericalParameter (Added NumericalParameter Parameter class that does automatic error ch… #1799)
• Add ChoiceParameter (Add ChoiceParameter to restrict parameter options to those specified. #1800)
• locking bugfix: make ps print full commands (Use ps command that prints full command lines without errors #1809)
• Add current directory to python path
• Added passthrough for PySFTP Connection keyword arguments to RemoteTarget constructor (Added passthrough for PySFTP Connection keyword arguments to RemoteTarget constructor #1815)
• Enables running of multiple tasks in batches (Enables running of multiple tasks in batches #1784)
• Version 2.3.0
• Update RELEASE-PROCESS.rst
• Limits resource usage bar fill to 100% (Limits resource usage bar fill to 100% #1818)
• Adds BATCH_RUNNING to the visualiser (Adds BATCH_RUNNING to the visualiser #1817)
• docs: Add instructions for seeing rendered docs (docs: Add instructions for seeing rendered docs #1814)
• bugfix: worker dies on smtp connection errors (Don't kill the worker when it cannot connect to smtp server #1821)
• Uses a sane default for pysftp_conn_kwargs (Uses a sane default for pysftp_conn_kwargs #1825)
• worker.py: Remove old and not useful comment (worker.py: Remove old and not useful comment #1792)
• Bug fix for spark task can not be pickled (Bug fix for spark task can not be pickled #1819)
• Version 2.3.1
• Allows breaking resource strings on commas in visualizer (Allows breaking resource strings on commas in visualizer #1828)
• Moves error messages in the task list to a popup (Moves error messages in the task list to a popup #1827)
• Add Blendle to list of companies using Luigi (Add Blendle to list of companies using Luigi #1837)
• Add and refactor tests for disabling workers (Add and refactor tests for disabling workers #1838)
• gcp: Add image_version support to Dataproc (Added image_version support to Dataproc #1833)
• pig: Properly use Python3's buffer interface (Properly use Python3's buffer interface. #1836)
• tests: Skip a inttermittently failing sqla test (tests: Skip a inttermittently failing sqla test #1840)
• tox: Remove last trace of not_imported.py (tox: Remove last trace of not_imported.py #1841)
• dataproc tests: Attempt to fix failing build (dataproc tests: Attempt to fix failing build #1843)
• Prevents batch tasks from running before their dependencies (Prevents batch tasks from running before their dependencies #1846)
• Allows clients to specify max_shown_tasks in task_list (Allows clients to specify max_shown_tasks in task_list #1847)
• Sets upstream status for BATCH_RUNNING jobs to UPSTREAM_RUNNING (Sets upstream status for BATCH_RUNNING jobs to UPSTREAM_RUNNING #1848)
• Filters tasks in second branch of Worker.get_pending_tasks (Filters tasks in second branch of Worker.get_pending_tasks #1849)
• Fails batch_running tasks if worker dies early (Fails batch_running tasks if worker dies early #1850)
• Update local locking logic (Update local locking logic #1842)
• Disables siginterrupt for SIGUSR1 (Disables siginterrupt for SIGUSR1 #1844)
• Adds time_running to batch running tasks (Adds time_running to batch running tasks #1851)
• worker_tests.py: Fix minor resource leak (worker_tests.py: Fix minor resource leak #1853)
• server.py: Remove old comment (server.py: Remove old comment #1854)
• Fix sample code in SGE package description
• Cleanup server.py file (Cleanup server.py file #1855)
• Tries using path for non-hdfs targets in hadoop_jar (Tries using path for non-hdfs targets in hadoop_jar #1824)
• Refine disabling of workers (Scheduler: don't give tasks to disabled assistants #1839)
• Version 2.3.2
• Handles errors when loading batch tasks from multiple ids
• Enables worker timeouts with only one worker
• Uses task_process instead of p as a variable name in _run_task
• Renames TaskProcess.random_seed to TaskProcess.use_multiprocessing
• Inherits WorkerTest from LuigiTestCase
• Refactors task load exception error notifications
• Adds a test that task load failures trackbacks are sent to the scheduler
• Updates LuigiTestCase to clear instance caches before and after runs
• Enables running with 0 workers to support assistants (Enables running with 0 workers to support assistants #1861)
• Add support to RangeByMinutes backfilling tasks (Add support to RangeByMinutes backfilling tasks #1863)
• Uses task name instead of task_id as job identifier for MR (Uses task name instead of task_id as job identifier for MR #1868)
• Prevents batched tasks from all getting marked RUNNING
• Add CORS headers to api requests (Add CORS headers to api requests #1870)
• Moves e-mail config into luigi.Config classes (Moves e-mail config into luigi.Config classes #1871)
• pyspark: Fix bug about overriden job name parameter (Fixed bug where DataprocPysparkTask overrides the job name parameter … #1878)
• [bigquery] use the same capitalization as Google does
• [bigquery] clean up some mess
• Add UDF support for BigQuery (Add UDF support for BigQuery #1881)
• docs: batch_mode => batch_method (fixed typo: batch_mode => batch_method #1890)
• Use the proc filesystem to gather process information (Use the proc filesystem to gather process information #1886)
• tests: Disable server_test.test_404 (tests: Disable server_test.test_404 #1880)
• Version 2.3.3
• LocalTarget: Allow moves across devices (Allow a file to be moved cross-device. #1877)
• make elasticsearch example run out of the box (make elasticsearch example run out of the box #1902)
• Make clone() carry along non-parameters (Clone from non-parameters #1630)
• docs: Update outdated links and config variables
• email: Cleanup email code and log less
• redshift: Correctly escape query used with UNLOAD (Correctly escape query used with Redshift UNLOAD #1907)
• file.py: Rename to local_target.py (file.py: Rename to local_target.py #1915)
• targets: Implement temporary_path decorator (targets: Implement temporary_path decorator #1909)
• Add location parameter to BigQueryTarget
• dataset_exists checks dataset regional location
• Add tests for BigQuery dataset location
• Separate BQ tests from GCS tests
• remove testfixtures
• use Python's None instead of 'undefined' when location is unspecified
• [bigquery] point out Avro support
• [bigquery] add BigQueryLoadAvro that adds schema documentation to the table
• add an integration test for BigQueryLoadAvro
• Test to verify description of schema is added.
• clean up tiny unused bits
• fix load_avro_dir test to load an actual directory; fix bugs
• sanitize exception handling, logging and documentation
• add tests for nullable and undocumented fields
• Sets wait_jitter in worker_test.py (Sets wait_jitter in worker_test.py #1927)
• Make temporary_path() decorator create directories (Make temporary_path() decorator create directories #1919)
• Cleanup code and adhere to E305 (Cleanup code by removing if __name__ == '__main__' pattern #1922)
• Add docs for DataParameter for default=today() (Add docs for DataParameter for default=today() #1928)
• Removes None default params in email config class (Checks for proper email configuration before sending error emails #1923)
• Add PyData Berlin 2015 presentation from TrustYou (Added PyData Berlin 2015 presentation from TrustYou #1930)
• travis: Don't run failing bigquery tests (travis: Don't run failing bigquery tests #1931)
• Adds a flag to turn off failure e-mails in the worker
• BatchNotifier class to handle sending batch e-mails
• Add batch notification e-mails to the scheduler
• Ensures a consistent pretty_id in scheduler.Task
• Improves documentation of batch_notifier config
• Handles unicode in batch notifier
• Fixes style issues in batch notifier
• Addresses Arash's review
• Revert "Marking as minimum upstream severity instead of max (Marking as minimum upstream severity instead of max #1789)"
• Counts FAILED tasks as PENDING in count_pending
• Refactors get_running_tasks and get_pending_tasks
• Version 2.4.0
• BigQuery jobs success verified using status.errorResult
• Add retries for 500 errors in BigQuery
• make RunAnywayTarget compatible with python 2.7
• Make task_namespace inherit from parent Task (Fix task_namespace inheritance. #1953)
• Make luigi.task.externalize return shallow copies (Make luigi.task.externalize return shallow copies  #1952)
• Remove the long-deprecated BooleanParameter (Remove the long-deprecated BooleanParameter #1959)
• emails: Increase logging if email is misconfigured (emails: Increase logging if email is misconfigured #1960)
• [bigquery] add support for non-legacy SQL ([bigquery] add support for non-legacy SQL #1896)
• externalize: Add a testcase similar to docs (externalize: Add a testcase similar to docs #1963)
• Update SF to cast header value to str (Update salesforce http header for Requests update #1912)
• Tests: Move namespace_tests into task_test.py
• namespaces: Fix changed behavior since Fix task_namespace inheritance. #1953
• Fix serialization for TimeDeltaParameter (Allow 0 time timedeltas, fix serialize to allow passing timedelta par… #1968)
• ChainFormat error message fix (ChainFormat error message fix #1971)
• Add missing commas to the contrib.sla module code examples. (Add missing commas to the contrib.sla module code examples. #1972)
• tests: Remove __name__ == '__main__' guards (tests: Remove __name__ == '__main__' guards #1973)
• flake8: Put configuration into standardized place (flake8: Put configuration into standardized place #1974)
• server_test.py: Fix crash in test_save_state (server_test.py: Fix crash in test_save_state #1977)
• externalize: Dont register the copied tasks (externalize: Dont register the copied tasks #1975)
• retcodes: Understand succesful retries (Improve retcodes and execution summary for retry success #1951)
• target_test.py: Use clean paths to avoid failures (target_test.py: Use clean paths to avoid failures #1978)
• redshift: Fix formatting of documentation (Fix formatting of documentation #1981)
• Version 2.5.0
• Set default log level via config (Set default log level via config #1987)
• docs: Update link to ConfigParser (docs: Update link to ConfigParser #1990)
• Keep datatable state in the browser history. (Keep datatable state in the browser history. #1986)
• minor, file.py: Fix deprecation date
• Remove deprecated backward-compatibility modules (Remove deprecated backward-compatibility modules #1995)
• Add redshift copy support for role-based credential string (Add redshift copy support for role-based credential string #1962)
• README - who uses luigi (README - who uses luigi #2004)
• use compress_response by default (use compress_response by default #2007)
• Fixed link to HDFS API docs in README (Fixed link to HDFS API docs in README #2006)
• Update s3 docstrings (Update s3 docstrings #2003)
• Track dependency graph fields values change.
• Task list fields values moved from query params to hash.
• Graph tab fields tracking tests added.
• Cosmetics.
• Task graph display fixed.
• Introduce rpc-attempt retry count and wait time in configuration (introduce rpc-attempt retry count and wait time in configuration #1994)
• add TaskStatusReporter class to fix Windows pickle issue (add TaskStatusReporter class to fix Windows pickle issue #1992)
• Move non-core like modules to luigi.contrib
• Make mrrunner module private to luigi
• Control worker processes via UI (scheduler → worker communication) (Control worker processes via UI (scheduler → worker communication) #1993)
• Allow namespaces to be set to python module (Allow namespaces to be set to python module #2000)
• Show number of unread worker messages in UI (Show number of unread worker messages in UI #2013)
• Fix required credential string name for key-based redshift copy credentials (Fix key-based credentials for redshift copy #2016)
• Set authors to "The Luigi Authors" (Set authors to "The Luigi Authors" #2009)
• Update TaskParmeter to use updated library code (Update TaskParmeter to use updated library code #2023)
• Version 2.6.0
• Refactored setup of google api clients (Refactored setup of google api clients #2018)
• Add the extra_arguments to easily pass additional arguments to streaming jobs (Add the extra_arguments to easily pass additional arguments to streaming jobs #1929)
• Introduce Python3 compatibility in example (Introduce Python3 compatibility in example #2032)
• Fix bad URI path (Repaired luigi build #2033)
• Support for Kubernetes Jobs (Support for Kubernetes Jobs #1906)
• salesforce: use 'wb' when opening output file (update contrib.salesforce to use 'wb' when opening output file #2034)
• Create ISSUE_TEMPLATE.md (Create ISSUE_TEMPLATE.md #2030)
• Update PULL_REQUEST_TEMPLATE.md (Update PULL_REQUEST_TEMPLATE.md #2037)
• parametrization -> parameterization (Fix a typo #2046)
• Specify default port to comply with change in init for psycopg 2.7 (Specify default port to comply with change in init for psycopg 2.7 #2049)
• Bugfix for retry policy (Fixed problem that the retry policy per task does not work. #2012)
• Address JS error in visualizer when displaying the dependency graph (Address JS error in visualizer when displaying the dependency graph #2042)
• Fix links to task graphs in visualizer. (Fix links to task graphs in visualizer. #2025)
• Test against Python 3.6 (Test against Python 3.6 #2053)
• Add method rename_dont_move to LocalFileSystem (Add method rename_dont_move to LocalFileSystem #2057)
• Make temporary_path work with paths not containing a directory (Make temporary_path work with paths not containing a directory #2058)
• Remove anything related to Debian
• Version 2.6.1
• Fix worker tool box in visualizer (Fix worker tool box in visualizer #2063)
• travis: Run less builds
• Fix CSS and JS imports
• Extend from default layout
• Do not raise HDFSCliError inside snakebite client.
• Remove unused class variable.
• Refactor: Simplify client initialisation.
• Add MongoDB Target Support (Add MongoDB Target Support #2062)
• add Red Hat to list of Luigi users (add Red Hat to list of Luigi users #2067)
• Update format of luigi users and their public content (Update format of luigi users and their public content #2072)
• Split up the list of users in two lists (Split up the list of users in two lists #2076)
• Add Squarespace to list of Luigi users (Add Squarespace to list of Luigi users #2081)
• Avoid reading global variables in worker. (luigi.worker.Worker(retry_external_tasks=True) has no effect #2077)
• Avoid warnings in unit tests. (Avoid warnings in unit tests. #2078)
• Document that Parameter now must be a str, to avoid warnings added by issue Dynamic Requires incorrectly treats all luigi.Parameter as strings #1607. (Document that Parameter() now must be a string. #2082)
• Add Newsela to list of companies using Luigi (Add Newsela to list of companies using Luigi #2065)
• Fix duplicate explicit target name warnings (Fix duplicate explicit target name warnings #2088)
• Changed postgres import on TopArtists example (Changed postgres import on TopArtists example #2090)
• Enable "forgiving failures" from webapp (Forgive Failures #2091)
• included Grovo name in README.rst (included Grovo name in README.rst #2102)
• Fix minor spelling mistakes in log and doc strings (Fix minor spelling mistakes in log and doc strings #2101)
• add oao to list of companies using luigi without slides (Add OAO's name as a company which uses Luigi #1976)
• Add 'IF NOT EXISTS' to prevent duplicate table exception (Add 'IF NOT EXISTS' to prevent duplicate table exception #2064)
• Defer S3 Client instantiation until needed (Defer S3 Client instantiation until needed #2079)
• sqla: only reflect relevant tables (Fix luigi.contrib.sqla performance bug; don't reflect all tables #2104)
• Add Weebly to list of Luigi users (add Weebly to list of Luigi users #2056)
• Parameter: Refactor out wrong type check
• Docs: Mention to avoid simultaneous updates to single file (Add section to the patterns doc about avoiding simultaneous updates to single file #2040)
• add listdir method to ftp remote file system (add listdir method to ftp remote file system #2114)
• Recursively normalize DictParameter (Recursively normalize DictParameter #2115)
• Don’t use cached autoconfig client in config tests (Don’t use cached autoconfig client in config tests #2120)
• Check whether renames worked in atomic hdfs pipes (Check whether renames worked in atomic hdfs pipes #2119)
• dataproc: more consistently get authentication arguments
• Fix "Show Upstream Dependencies" in visualiser (Fix "Show Upstream Dependencies" in visualiser #2092)
• Update tasks when filter on server is toggled
• Remove search query from url bar on empty query
• Set hideDone on in graph links (Set hideDone on in graph links #2125)
• Set hideDone in worker root task links (Set hideDone in worker root task links #2126)
• Remove length from state when it equals default (Remove length from state when it equals default #2128)
• Preserve options in svg graph links (Preserve options in svg graph links #2132)
• Fix resource graph links
• Store expanded resources in the URL bar
• Add task family to url hash
• Add status to url hash
• Retain container types from output() to input() (Make getpaths type-aware, add getpaths test. #2113)
• Preserve graph options when clicking d3 nodes (Preserve graph options when clicking d3 nodes #2139)
• Move visualiser graph type default to a constant
• Make svg the default graph type
• added getninjas to user list in readme (added getninjas to user list in readme #2146)
• Use list/dict comprehension (Use list and dict comprehension in server #2144)
• Version 2.6.2
• Move visualiser css into a separate file
• Pause and unpause the scheduler from the viz
• travis: Disable email notifications
• Prevent stowaway resources in scheduler
• Prevent changing resources from exceeding limits
• Set datatable order correctly on hash change (Set datatable order correctly on hash change #2159)
• Update check for table existence to be case insensitive (Update check for table existence to be case insensitive #2086)
• Remove superfluos project_id parameter in BigQuery job definition (Remove superfluos projectId parameter in BigQuery job definition #2117)
• Serialize statuses array for URL hash (Serialize statuses array for URL hash #2165)
• Scheduler API for updating a single resource
• Update individual resource counts from the vis
• hive: Use DictParameter in ExternalHiveTask (ExternalHiveTask use DictParameter #1999)
• Fix getpcmd on macOS
• Fix DeprecationWarning in tests
• Fix don't skip S3 test cases fixed upstream
• Fix test silence warnings
• Refactor rename MockFile -> MockTarget
• Fix tests to use new config format
• Refactor test to use assertTrue
• Fix tests to use new config format
• Reformat flake8 complaints
• Fix decode to utf8 when reading /proc/*/cmdline
• Fix decode to utf8 when reading /proc/*/cmdline
• Fix decode to utf8 when reading /proc/*/cmdline
• Travis: Remove minicluster builds (Travis: Remove minicluster builds #2141)
• Handle e-mail failures in batch notifier (Handle e-mail failures in batch notifier #2177)
• PySparkTask fix for bytes / str type error and import error (PySparkTask fix for bytes / str type error and import error #2168)
• Update mkdir method signature for CDH3 client. (Update mkdir method signature for CDH3 client. #2184)
• MongoDB target items count (MongoDB target items count #2186)
• hadoop: Fix CDH3 mkdir implementation (Fix CDH3 mkdir implementation #2188)
• travis: Remove ssh loopback trick
• Deprecate python 3.3 support for Luigi
• travis: Disable pypy scheduler tests
• Add worker config check_unfulfilled_deps. (Add worker config check_unfulfilled_deps. #2189)
• Fix emailing of unexpected error in Worker._announce_scheduling_failure. (Fix emailing of unexpected error in Worker._announce_scheduling_failure. #2191)
• Update _get_s3_config to only return s3 section with interpolation, excluding defaults (Update S3's _get_s3_config() to not include DEFAULT section #2182)
• [bug-fix] scheduler won't override running status incorrectly ([bug-fix] scheduler won't override running status incorrectly #2187)
• Support cluster mode in PySpark (Support cluster mode in PySpark #2197)
• Added Deloitte to the list of users (Added Deloitte to the list of users #2206)
• Prevent worker from running same task repeatedly (Prevent worker from running same task repeatedly #2207)
• Clear tasks on disabled workers (Clear tasks on disabled workers #2208)
• [docs] consistency fix for the month value (minor inconsistency fix for the month value #2217)
• Remove assert that verifies if tables were marked (Remove assert that verifies if tables were marked for PSQL integrations #2199)
• breaking: Remove deprecated contrib redirections (breaking: Remove deprecated contrib redirections #2181)
• Set utf-8 encoding in the MIMEText constructor (Set utf-8 encoding in the MIMEText constructor #2210)
• Progress bar in luigi visualizer (Progress bar in the status message modal + auto-refresh of the modal #2108)
• Add 'schema' attribute to CopyToTable and use for target table (Allow schema to be specified for target table in sqla.CopyToTable #2176)
• Rework the logic and the tests of the graph visualization type checkboxes. (Rework the logic and the tests of the graph visualization type checkboxes. #2193)
• Allow spark to load config from different version (Allow spark task to load config from different version #2196)
• Version 2.7.0
• Update the docstring of S3FlagTarget (Update the docstring of S3FlagTarget #2192)
• README: avoid here links; ack readthedocs (README: avoid _here_ links; ack readthedocs #2222)
• Standardize Redshift credential usage (REDSHIFT: standardize credential usage #2068)
• Update to handle complex avro types for Avro Schema Definition copying to Big Query Schema descriptions
• Fix typo in docstring (Fix typo in docstring #2225)
• Cleaned up comments. Refactored to have explicit primitive support and primitives inside of unions support. Updated Job description comments and removed unnecessary logging.
• updated testing suite to match the more complex schema doc loader
• updated testing suite to match the more complex schema doc loader
• flake8 formatting fixes
• flake8 formatting fixes
• Fix "IF NOT EXISTS" Postgres 8 incompatibility (Fix "IF NOT EXISTS" Postgres 8 incompatibility #2228)
• MongoDB contrib : Next replaced with for loop to avoid propagation of StopIteration (MongoDB contrib : Next replaced with for loop to avoid propagation of StopIteration #2221)
• hivevar option in hive task (hivevar option in hive query task #2185)
• Fix ListParameter Hashability (Fix ListParameter Hashability #2227)
• Prevent module shadowing in pyspark_runner.py (Prevent module shadowing in pyspark_runner.py #2232)
• Added support to specify the number of processes for parallel scheduling (Added support to specify the number of processes for parallel scheduling #2205)
• Accept kwargs for mysql connection (accept kwargs for mysql connection #2226)
• Fix os.makedirs issue in multithreading (Raise FileExistsError when running multiple workers #2039) (Fix os.makedirs issue in multithreading (#2039) #2093)
• Added cluster support for ECS and corrected status debug message (Added cluster support for ECS and corrected status debug message #2045)
• Silence Parameter type warning for WebHdfsClient (Silence Parameter type warning in WebHdfsClient #2231)
• Issue template: Mention it isn't for questions (Issue template: Mention it isn't for questions #2143)
• Add ExtractTask to luigi.contrib.BigQuery module (Add ExtractTask to luigi.contrib.BigQuery module #2134)
• Fixed a typo in docstring (Fixed a typo in docstring #2236)
• Adding autocommit property for Redshift queries for Vacuums, etc. (Adding autocommit property for Redshift queries for Vacuums, etc. #2242)
• Add HA functionality to WebHdfsClient (Add HA functionality to WebHdfsClient #2230)
• docs: Fix hypen vs underscore inconsistencies (Fix wrong configuration docs #2160)
• Use columns defined for COPY to Redshift so we never rely on order (Use columns defined for COPY to Redshift so we never rely on order #2245)
• Improve sentence structure in configuration.rst (update configuration.rst to fix confusing typo #2248)
• Version 2.7.1
• Adding voyages-sncf.com as a company using luigi (Adding voyages-sncf.com as a company using luigi #2251)
• Minor fix (for the sake of understanding) (Minor fix (for the sake of understanding) #2250)
• Remove invalid entrypoint: luigi.tools.migrate (Remove invalid entrypoint: luigi.tools.migrate #2257)
• changed the logging status of prune messages to debug (Fix 2234 #2254)
• fix typo suceeded -> succeeded (fix typo suceeded -> succeeded #2258)
• Support for the Docker Python SDK (Support for the Docker Python SDK #2158)
• Add Leipzig University Library as user (add Leipzig University Library as user #2262)
• Speed up task_list when beyond limit (Speed up task_list when beyond limit #2239)
• Remove Avro description copying for BigQuery
• Cleanup irrelevant comment, remove unused imports
• Except all Exceptions for flake8
• Fix bare except clause
• Rename ambiguous class/params for flake8
• Fix new flake8 tests (Fix new flake8 tests #2274)
• Preserve filter on server on reload (Preserve filter on server on reload #2273)
• Cleans up dangerous function default variables ([] and {}) (Cleans up dangerous function default variables ([] and {}) #2275)
• Python 3.6 missing from README #2279 Add Python 3.6 to list of supported Python versions. (#2279 Add Python 3.6 to README. #2280)
• Hide re-enable and forgive buttons on success (Hide re-enable and forgive buttons on success #2281)
• Add Synetiq to Who uses Luigi?. (Add Synetiq to Who uses Luigi?. #2287)
• Convert dates to datetimes for DateHourParameter (Convert dates to datetimes for DateHourParameter #2285)
• Small refactor to make intent clearer (Intent refactor #2255)
• Re-add loading the avro schema and copying the schema doc to bq table desc
• Fix Flake8 errors
• dynamic demo fix (dynamic demo fix #2300)
• Added show-error button to disabled tasks in visualizer (Added show-error button to disabled tasks in visualizer #2266)
• Handle multiprocessing with request sessions (Handle multiprocessing with request sessions #2290)
• Optional parameter (Optional parameter #2295)
• Fixes dangerous default parameter values to make them immutable (Fixes dangerous default parameter values to make them immutable #2302)
• Replace set calls with literals (Replace set calls with literals #2293)
• update_status -> update_status_message (TaskStatusReporter: update_status -> update_status_message #2292)
• vizualiser: Format js code (Code format #2309)
• Fix MRO on tasks using util.requires ( Fix MRO on tasks using util.requires #2054 #2307)
• Add Stacktome to companies who use Luigi (Add stacktome to companies list #2313)
• Add LINX+Neemu+Chaordic to the list of Luigi companies (Add LINX+Neemu+Chaordic to the list of Luigi companies #2320)
• Fix Python 3 TypeError in contrib.hive.HiveTableTarget.exists()
• Replaced param_args with dynamic property with deprecation message
• Check task equality using param_kwargs instead of param_args
• Update to 2.7.2
• Fix unicode formatting. (Fix unicode formatting. #2339)
• Show status message button in worker tab when only progress is set. (Show status message button in worker tab when only progress is set. #2344)
• More verbose exceptions for mistyped depenencies (Verbose worker error logging #2353)
• Add possibility to specify Redshift column compression. (Add possibility to specify Redshift column compression. #2343)
• Support for AWS Batch (Support for AWS Batch #2321)
• Allow NoneType columns in Redshift COPY job (Allow NoneType columns in Redshift COPY job #2303)
• fix contrib.docker_runner exit code check Master build broken #2355
• check that pods start successfully. fix [contrib.kubernetes] Hangs on 'Pod errors: Unschedulable' #2334
• delete pods and jobs on success
• handle ContainersNotReady situation
• print pod logs on exit
• add instructions to stream pod logs
• update based on PR comments
• detect pod start failure and return error
• handle ErrImagePull and add activeDeadlineSeconds
• don't fail on ContainerCreating
• parameterize active_deadline_seconds
• improve PR based on comments
• Fix a warning in googleapiclient usage (Fix a warning in googleapiclient usage in contrib.gcs #2306)
• Document ways for starting Luigi inside Python code (Documented way for starting Luigi tasks inside Python code #2301)
• Add data files to .gitignore (added generated data files to .gitignore #2367)
• Replace oauth2client by google-auth (Replace oauth2client by google-auth #2361)
• Only run code coverage cheks for core luigi
• Version 2.7.3
• codecov: Don't run coverage on unreasonable files
• Pass kwargs to discovery.build() when instantiating GSCClient. (Pass kwargs to discovery.build() when instantiating GSCClient. #2291)
• codecov: Don't expect coverage in kubrenetes.py
• kubernetes: Add parameterized backoff limit (Add parameterized backoff limit. #2375)
• Add google-auth-httplib2 as dependency
• Allow release of resources during task running. (Allow release of resources during task running. #2346)
• Removed message 'No Instance(s) Available.' in Windows when starting … (Removed message 'No Instance(s) Available.' in Windows when starting … #2294)
• Fix typo in luigi documentation (Typo in running luigi documentation #2387)
• Add new company user (Add new company user #2388)
• Version 2.7.4
• Fix xss vulnerability from usage of append (append is vulnerable of xss #2391)
• Updated to version 2.7.5 (Update to version 2.7.5 #2395)
• Adds batched_params which is a dictionary of lists of batched parameters indexed by parameter name
• missed the worker bit
• update setup.py
• backport of ParameterVisibilities
• Test fixes and version bump
• Luigi email region for 2.7.5
• Bump luigi version
• Luigi email region for 2.7.5
• Added new batched_params to the task
• Add and mod tests to test batched_params
• Add a superset/subset batch test
• Bump setup.py version number
• Fixes for batched_params (default and non-list)
• update tests for new batched_params
• fix RangeX classes to support param-name from the base class
• add a test pairing MixinNaiveBulkComplete and ranges
• added a reload for batch jobs as they are now dynamic
• Handle config python2/3 compatibility
• Version bump
• patch in LuigiRunResult and LuigiStatusCode with additional field for status num
• success should be 0
• only return status code num to keep interface similar to returning bool
• remove interpolation import thing
• version bump from 1.1.0 to 1.2.0
• remove enum34 requirement version bump since was accident
• bring in 2.7.6 scheduler changes, don't count tasks with UNKNOWN dependencies as pending
• fix interpolation import
• use getattr for param_visiblities
• bump version
• don't bring in 2.7.6 changes
• unit tests for unknown state handling
• worker keep alive for test
• wrap add multiprocess in fork lock
• version bump
• luigi bobapki http adapter
• bump version again
• fix enum34 dependency