bpo-28806: Improve the netrc library #127

zhangyangyu · 2017-02-16T04:18:36Z

No description provided.

dmerejkowsky · 2017-02-18T13:47:46Z

@zhangyangyu Nice work! I also wrote a tiny patch for netrc, so maybe you could integrate #123 in your PR as well?

zhangyangyu · 2017-02-20T05:05:15Z

@dmerejkowsky I wrote this patch months ago, so I have to spend some time picking it up to see if there is any possibility. Happy to see someone also interested in improving netrc.

dmerejkowsky · 2017-02-20T19:13:09Z

OK, then. Hopefully our PRs won't conflict with each other, so this not a big deal.

Keep up the good work!

…tor and traceback objects __setstate__ must accept the state returned by __reduce__. This was not the case for generator and trace-back objects. This commit fixes this. The next commit (merge of issue python#127) adds the relevant test cases. Additionally amends changelog.txt. https://bitbucket.org/stackless-dev/stackless/issues/107

…ling tests. This change is based on f5f98595c6cc63b from 2.7-slp, but it is heavily modified for Python 3. - Pickle 'callable-iterator' objects correctly. Previously the unpickled object had the type '_stackless._wrap.callable-iterator'. - Fix pickling of 'method-wrapper' objects. Previously pickling them caused a SystemError exception. - Fix copy.copy() for 'callable-iterator', 'method', 'dict_keys', 'dict_values' and 'dict_items' objects. Previously the copied object had the type '_stackless._wrap....'. - Fix Stackless pickling tests. The method StacklessTestCase.dumps() didn't pass the pickle protocol to the pickler. - Remove dead code in prickelpit.c. The code was used in older Stackless versions.

…iterator' Disable the Stackless specific code for pickling 'iterator' and 'callable_iterator' objects. C-Python 3.3 already pickles them. Add tests to ensure, that Stackless can still unpickle old pickles. https://bitbucket.org/stackless-dev/stackless/issues/127

raspy · 2018-04-17T10:06:06Z

Lib/netrc.py

                elif tt == 'login' or tt == 'user':
                    login = lexer.get_token()
                elif tt == 'account':
                    account = lexer.get_token()
                elif tt == 'password':
-                    if os.name == 'posix' and default_netrc:
+                    if os.name == 'posix' and default_netrc and login != "anonymous":


This assumes that login has already been parsed, but the netrc format does not require providing login token before password token. It would be better to perform this check on storing parsed data into self.hosts[entryname], when all machine-related data is parsed and available.

auvipy

also please re base

auvipy · 2018-10-09T14:45:33Z

Doc/library/netrc.rst

@@ -32,6 +32,12 @@ the Unix :program:`ftp` program and other FTP clients.

   .. versionchanged:: 3.4 Added the POSIX permission check.

+   .. versionchanged:: 3.7


you can retarget it for python 3.9

serhiy-storchaka

I'm not sure, but it may be worth to backport these changes to maintained versions. It fixes several bugs in the .netrc file parsing.

serhiy-storchaka · 2018-04-19T06:34:26Z

Doc/library/netrc.rst

@@ -32,6 +32,12 @@ the Unix :program:`ftp` program and other FTP clients.

   .. versionchanged:: 3.4 Added the POSIX permission check.

+   .. versionchanged:: 3.7


serhiy-storchaka · 2018-10-29T09:50:12Z

Lib/netrc.py

+                continue
+            if ch == "\"":
+                for ch in fiter:
+                    if ch != "\"":


I would revert this condition.

if ch == '"': return token

And removed continue.

serhiy-storchaka · 2018-10-29T09:54:14Z

Lib/netrc.py

+        self.lineno = 1
+        self.instream = fp
+        self.whitespace = "\n\t\r "
+        self._stack = []


It is not a stack, it is a queue. I suggest to rename it to pushback, as in shlex.lexer.

serhiy-storchaka · 2018-10-29T09:56:46Z

Lib/netrc.py

+        for ch in fiter:
+            if ch in self.whitespace:
+                continue
+            if ch == "\"":


serhiy-storchaka · 2018-10-29T09:57:25Z

Lib/netrc.py

+                    ch = self._read_char()
+                token += ch
+                for ch in fiter:
+                    if ch not in self.whitespace:


The same as above.

serhiy-storchaka · 2018-10-29T10:11:30Z

Lib/netrc.py

                    lexer.instream.readline()
                continue
            elif tt == 'machine':
                entryname = lexer.get_token()
            elif tt == 'default':
                entryname = 'default'
-            elif tt == 'macdef':                # Just skip to end of macdefs
+            elif tt == 'macdef':
                entryname = lexer.get_token()


You need to read and ignore the rest of the line after the macro name.

serhiy-storchaka · 2018-10-29T10:13:48Z

Lib/netrc.py

+                        raise NetrcParseError(
+                            "Macro definition missing null line terminator.",
+                            file, lexer.lineno)
+                    if line == '\n':


Add a comment.

serhiy-storchaka · 2018-10-29T10:17:54Z

Lib/netrc.py

@@ -127,10 +172,10 @@ def __repr__(self):
        rep = ""
        for host in self.hosts.keys():
            attrs = self.hosts[host]
-            rep = rep + "machine "+ host + "\n\tlogin " + repr(attrs[0]) + "\n"
+            rep = rep + "machine "+ host + "\n\tlogin " + attrs[0] + "\n"


Perhaps we need a special function which escapes whitespaces and other special characters?

serhiy-storchaka · 2018-10-29T10:21:01Z

Lib/netrc.py

+                    ch = self._read_char()
+                token += ch
+                for ch in fiter:
+                    if ch not in self.whitespace:


I suggest to add a flag for denoting whether the terminating whitespace was '\n'. It will be helpful for comments and macdef.

csabella · 2019-05-20T00:52:47Z

@zhangyangyu, would you be able to address Serhiy's review from October? Thanks!

zhangyangyu · 2019-05-20T02:15:23Z

@csabella I am busy in my job now and this issue has been a long time even myself needs to reacquaint with it. But October seems not that hard to me. Is 3.8 going to release on October?

csabella · 2019-05-31T13:12:30Z

@zhangyangyu, sorry, I meant that Serhiy had left a comment last October, not that this needs to be looked at before the next October. The cutoff for 3.8 is this coming Monday (2019-06-03). I hope you still find time to look at it in the next few months. Thanks!

eamanu · 2019-09-18T23:04:13Z

Hello I can continue this PR if your consider good.

Cc @csabella

eamanu · 2020-06-21T00:52:16Z

Hi @auvipy @csabella @zhangyangyu There is some plan with this PR?
Was open 3 years ago. If the author is still interested on this bugs
resolution we could open a new PR.

What's your opinion?

thedrow · 2020-09-02T12:07:53Z

@eamanu I think that at this point, you can proceed with fixing the problems uncovered during code review.

HighnessAtharva · 2021-05-24T12:26:53Z

So should this PR be closed?

thedrow · 2021-05-24T14:04:12Z

I think so.

nanjekyejoannah · 2021-05-24T22:20:38Z

Closing this PR since another PR #26330 continues and addresses comments on this thread. All reviews should move to the new PR.

Improve netrc library largely

601b3f7

zhangyangyu added the type-feature A feature request or enhancement label Feb 16, 2017

the-knights-who-say-ni added the CLA signed label Feb 16, 2017

zhangyangyu force-pushed the workspace branch from 8384aca to 601b3f7 Compare February 16, 2017 05:31

yan12125 mentioned this pull request Jun 19, 2017

.netrc file: add support for passwords containing spaces ytdl-org/youtube-dl#13434

Open

8 tasks

akruis referenced this pull request in stackless-dev/stackless Sep 5, 2017

merge 3.4-slp (Stackless #127, pickling fixes)

785c16c

akruis referenced this pull request in stackless-dev/stackless Sep 5, 2017

merge 3.4-slp (Stackless #127, use C-Python pickling)

1a49b66

akruis pushed a commit to akruis/cpython that referenced this pull request Sep 9, 2017

merge 3.3-slp (Stackless python#127, pickling fixes)

9994a00

akruis pushed a commit to akruis/cpython that referenced this pull request Sep 9, 2017

merge 3.3-slp (Stackless python#127, use C-Python pickling)

43b9010

ghost mentioned this pull request Sep 24, 2017

Improve frame and traceback pickling stackless-dev/stackless#107

Closed

Mariatta added needs rebase and removed needs rebase labels Oct 9, 2017

akruis pushed a commit to akruis/cpython that referenced this pull request Oct 10, 2017

Merge 3.5-slp (Stackless python#127, pickling fixes)

79de128

akruis pushed a commit to akruis/cpython that referenced this pull request Oct 10, 2017

Merge 3.5-slp (Stackless python#127, use C-Python pickling)

eae6aae

zhangyangyu added the awaiting review label Jan 18, 2018

jjjake mentioned this pull request Mar 14, 2018

Feature: Auth should use .netrc instead of a .cfg jjjake/internetarchive#154

Closed

raspy reviewed Apr 17, 2018

View reviewed changes

auvipy reviewed Oct 9, 2018

View reviewed changes

chrahunt mentioned this pull request Oct 27, 2018

Support credentials in configuration for private PyPI pypa/pip#4315

Closed

serhiy-storchaka reviewed Oct 29, 2018

View reviewed changes

jayvdb mentioned this pull request Jan 7, 2020

Store token in a dotfile fridex/githubcap#9

Open

eamanu mentioned this pull request May 24, 2021

bpo-28806: Continue work: improve the netrc library #26330

Merged

nanjekyejoannah closed this May 24, 2021

zhangyangyu mentioned this pull request Oct 29, 2022

Improve the netrc library #72992

Closed

gvanrossum mentioned this pull request Aug 22, 2023

heap-use-after-free in _PyFunction_LookupByVersion #108253

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bpo-28806: Improve the netrc library #127

bpo-28806: Improve the netrc library #127

zhangyangyu commented Feb 16, 2017

dmerejkowsky commented Feb 18, 2017

zhangyangyu commented Feb 20, 2017 •

edited

Loading

dmerejkowsky commented Feb 20, 2017

raspy Apr 17, 2018

auvipy left a comment

auvipy Oct 9, 2018

auvipy Sep 17, 2019

serhiy-storchaka left a comment

serhiy-storchaka Apr 19, 2018

serhiy-storchaka Oct 29, 2018

serhiy-storchaka Oct 29, 2018

serhiy-storchaka Oct 29, 2018

serhiy-storchaka Oct 29, 2018

serhiy-storchaka Oct 29, 2018

serhiy-storchaka Oct 29, 2018

serhiy-storchaka Oct 29, 2018

serhiy-storchaka Oct 29, 2018

csabella commented May 20, 2019

zhangyangyu commented May 20, 2019

csabella commented May 31, 2019

eamanu commented Sep 18, 2019

eamanu commented Jun 21, 2020

thedrow commented Sep 2, 2020

HighnessAtharva commented May 24, 2021

thedrow commented May 24, 2021

nanjekyejoannah commented May 24, 2021

		@@ -32,6 +32,12 @@ the Unix :program:`ftp` program and other FTP clients.

		.. versionchanged:: 3.4 Added the POSIX permission check.

		.. versionchanged:: 3.7

bpo-28806: Improve the netrc library #127

bpo-28806: Improve the netrc library #127

Conversation

zhangyangyu commented Feb 16, 2017

dmerejkowsky commented Feb 18, 2017

zhangyangyu commented Feb 20, 2017 • edited Loading

dmerejkowsky commented Feb 20, 2017

Choose a reason for hiding this comment

auvipy left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

serhiy-storchaka left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

csabella commented May 20, 2019

zhangyangyu commented May 20, 2019

csabella commented May 31, 2019

eamanu commented Sep 18, 2019

eamanu commented Jun 21, 2020

thedrow commented Sep 2, 2020

HighnessAtharva commented May 24, 2021

thedrow commented May 24, 2021

nanjekyejoannah commented May 24, 2021

zhangyangyu commented Feb 20, 2017 •

edited

Loading