CSV validation: column headings foo.bar and foo.baz cause column to go missing

[candlerb]

NetBox version

v3.4.6

Python version

3.8

Steps to Reproduce

Go to /ipam/ip-addresses/import/ (IPAM > IP Addresses > upload)

Paste in the following data into the text area:

address,status,virtual_machine.site,virtual_machine.name,interface
192.168.1.1/30,active,SITE_A,Server_1,Net1

Click Submit.

(Example taken from discussion #11988)

Expected Behavior

Successful upload, or a meaningful error response.

Observed Behavior

The data is rejected with the following error:

Row 1: Expected 4 columns but found 5

The error message suggests that the submitter only included four column headings, which is incorrect.

Analysis

Ref: Discussion link.

Looking at test cases for CSV upload, I see this example:

        input = """
        address,status,vrf.name
...
        output = (
            {'address': None, 'status': None, 'vrf': 'name'},
                                              ^^^^^^^^^^^^^

I am guessing that if there's a column "foo.bar" and another column "foo.baz", the second column overwrites the dict key "foo" in this structure.

Resolution

The simplest resolution would be to detect duplicate/overlapping column names, and report them as such. (e.g. "Column 'foo.baz' conflicts with column 'foo.bar'")

This would also capture the simpler case:

address,status,status
192.168.1.1/30,active,active

(also wrongly reported as "Expected 2 columns but found 3")

A longer-term resolution would be to support the use case which the original poster wanted, which was to be able to refer to a related object via more than one attribute, as had been originally proposed here when CSV upload was reworked for v2.8.2. I think that would be a substantial change.

[jeremystretch]

The error message is expected: CSV import currently supports only one declaration of each column. However if you or @kevintedder would like to resubmit this as a feature request we can explore the use case and potential implementations to support specifying multiple attributes per field.