Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

XYZ CSV skipinitialspace #15832

Merged
merged 1 commit into from
Jun 8, 2024
Merged

XYZ CSV skipinitialspace #15832

merged 1 commit into from
Jun 8, 2024

Conversation

w-e-w
Copy link
Collaborator

@w-e-w w-e-w commented May 18, 2024

Description

if there's a special character in side a CSV value. the value needs to be quoted
the issue is without skipinitialspace "quoting is finicky"

csv.Dialect.skipinitialspace

spaces immediately following the delimiter are ignored

skipinitialspace makes parseing CSV more consistent when quotes are involved

demo
without skipinitialspace (current)

aaa, "b,b",ccc
    ^
# notice that there is a space

will be parsed as 4 item

aaa
"b
b"
ccc

only the input without space is parsed as expected

aaa,"b,b", ccc

to

aaa
b,b
ccc

with skipinitialspace will be parsed as 3 it will consistently parsed as aaa,"b,b", ccc aaa, "b,b",ccc regardless of the extra Spaces

aaa
b,b
ccc

demo test script

from csv import reader
from itertools import chain
from io import StringIO


def csv_without_skipinitialspace(data_str):
    return list(map(str.strip, chain.from_iterable(reader(StringIO(data_str)))))


def csv_with_skipinitialspace(data_str):
    return list(map(str.strip, chain.from_iterable(reader(StringIO(data_str), skipinitialspace=True))))


def print_list(csv_list):
    for item in csv_list:
        print(item)
        print('-'*10)


if __name__ == '__main__':
    # input_string = '''aaa,"b,b", ccc'''
    input_string = '''aaa, "b,b", ccc'''
    print_list(csv_without_skipinitialspace(input_string))
    print('='*10)
    print_list(csv_with_skipinitialspace(input_string))
Details

output of aaa, "b,b", ccc'

aaa
----------
"b
----------
b"
----------
ccc
----------
==========
aaa
----------
b,b
----------
ccc
----------

aaa,"b,b", ccc'

aaa
----------
b,b
----------
ccc
----------
==========
aaa
----------
b,b
----------
ccc
----------

Checklist:

@AUTOMATIC1111 AUTOMATIC1111 merged commit 5977cb0 into dev Jun 8, 2024
6 checks passed
@AUTOMATIC1111 AUTOMATIC1111 deleted the xyz-csv-skipinitialspace branch June 8, 2024 07:46
@w-e-w w-e-w mentioned this pull request Jun 9, 2024
4 tasks
@lawchingman lawchingman mentioned this pull request Oct 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants