CSV Import Results is always Null Values for All Rows Using CSV Mapper

[KARINKISH1]

Prerequisites

• I read the Deployment and Setup section of the OpenCTI documentation as well as the Troubleshooting page and didn't find anything relevant to my problem.
• I went through old GitHub issues and couldn't find anything relevant.
• I googled the issue and didn't find anything relevant.

Description

I am trying to import a CSV file with 468,000 rows of data into OpenCTI. The file has columns such as name, domain, score, and description. I created a CSV Mapper and tested it, and it seemed to work fine. However, when I try to import data using that mapper, I only get "null/468,000" entries.

Environment

1. OS (where OpenCTI server runs): Ubuntu 22.04.2 LTS
2. OpenCTI version: 6.2.5

Reproducible Steps

Steps to create the smallest reproducible scenario:

1. Create a CSV Mapper for a file with columns name, domain, score, and description.
2. Test the mapper to ensure it works correctly.
3. Attempt to import the CSV file.
4. Observe that the import results in "null/468,000".

Additional information

I have attached screenshots of my CSV Mapper configuration and the resulting null import data.

[nino-filigran]

@KARINKISH1 could you maybe share your CSV here so that we can try it?

[KARINKISH1]

test_csv_5.csv
Here is short example of my file.

[nino-filigran]

@KARINKISH1 it might be 2 things:

• only a display issue (your entities are correctly imported, but there's an issue of the display of the result), let me know if you see the entities contained in your file being ingested in your platform.
• or a real issue which is known: Improve CSV parser to stream file to handle very large volume of data #7589. The issue is still being worked on for information.

[KARINKISH1]

So, I checked. And I saw that indeed, if you upload a small file (the one I left you as an example), there is an issue with display because it added them correctly. However, when I uploaded the old, full file of 80.4 MB, they did not get added at all.

[ThreatCTI]

Nothing at all is being added.

[KARINKISH1]

Hello! Are there any updates regarding this error?

[nino-filigran]

Can you provide me with your larger file please?

[KARINKISH1]

@nino-filigran ofc! You can download the file from this link:
https://drive.google.com/file/d/1ewk3zIH2e7Ea0l8IW-xiIYPxayZeGeST/view?usp=sharing

[KARINKISH1]

Hi @nino-filigran I wanted to know if you’ve used the file I provided (link above). Were you able to reproduce the issue? Thanks

[KARINKISH1]

guys? i really need help with this

[nino-filigran]

Sorry @KARINKISH1 so far I have not been able to reproduce no, so I don't have anything for you yet. I guess you're still encountering a problem, right?