Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Path issues - Ubuntu Server 20.04 + bad support for Windows #126

Closed
sefinek opened this issue Jul 14, 2024 · 9 comments · May be fixed by BiraruYT/LadiesAttireAPI#27, BiraruYT/LadiesAttireAPI#28, BiraruYT/LadiesAttireAPI#29 or BiraruYT/LadiesAttireAPI#30
Closed

Path issues - Ubuntu Server 20.04 + bad support for Windows #126

sefinek opened this issue Jul 14, 2024 · 9 comments · May be fixed by BiraruYT/LadiesAttireAPI#27, BiraruYT/LadiesAttireAPI#28, BiraruYT/LadiesAttireAPI#29 or BiraruYT/LadiesAttireAPI#30
Assignees
@kylefarris
Labels
bug

Comments

@sefinek
Copy link

sefinek commented Jul 14, 2024 •
edited
Loading

1

The scanDir function seems to be malfunctioning. It generates an incorrect /usr/bin/clamdscan command, which looks like this (for example):

/usr/bin/clamdscan --no-summary --fdpass --remove --multiscan
--log=/home/sefinek/logs/www/patrons.sefinek.net.log /mnt/samba/disk2/stella/uploaded_logs/8.0.1.19+889a9d2ee7d2470a179f81211ad77688f6772676/Polska/XXX/1720989200642_unzipped/Logs/Prepare\ Stella 
/mnt/samba/disk2/stella/uploaded_logs/8.0.1.19+889a9d2ee7d2470a179f81211ad77688f6772676/Polska/XXX/1720989200642_unzipped/Logs/Prepare\ Stella/2.0.1.0.log 
/mnt/samba/disk2/stella/uploaded_logs/8.0.1.19+889a9d2ee7d2470a179f81211ad77688f6772676/Polska/XXX/1720989200642_unzipped/DeviceInfo.sfn

As you can see, where there is a space, the clamscan module unexpectedly adds \, causing the path to be incorrect. I think I fixed it with this commit in my fork: https://github.com/sefinek24/clamscam-linwin/commit/8bef262898da0febabc3cc36b761e95756663e4a, but I don't remember.

2

On my Windows workstation, I use a TCP connection with clamdscan. Unfortunately, there is another issue here. The clamscan module uses the find command to locate all files in folders. For some reason, this didn't work on Windows; the module couldn't find files in the folder. In my fork, I used this solution: https://github.com/sefinek24/clamscam-linwin/commit/2b17542c0e3201353704d26274468dccbc4b7c77

[StellaLogs]: Error during file scan: NodeClamError: There was an issue scanning the path specified!
    at E:\Projects\www\api.sefinek.net\node_modules\clamscan\index.js:2017:33
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  data: {
    path: 'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.0.1.19+889a9d2ee7d2470a179f81211ad77688f6772676\\JZWXCJ9XHBS8X3HSMEX9A4SG649H1D4TW5V1MGGVRATYYB1MZVWG\\1720987386402_unzipped',
    err: Error: Command failed: find E:\Projects\other\stella-data\uploaded_logs\8.0.1.19+889a9d2ee7d2470a179f81211ad77688f6772676\JZWXCJ9XHBS8X3HSMEX9A4SG649H1D4TW5V1MGGVRATYYB1MZVWG\1720987386402_unzipped
    FIND: Parameter format not correct

        at genericNodeError (node:internal/errors:984:15)
        at wrappedFn (node:internal/errors:538:14)
        at ChildProcess.exithandler (node:child_process:422:12)
        at ChildProcess.emit (node:events:519:28)
        at maybeClose (node:internal/child_process:1105:16)
        at ChildProcess._handle.onexit (node:internal/child_process:305:5) {
      code: 2,
      killed: false,
      signal: null,
      cmd: 'find E:\\Projects\\other\\stella-data\\uploaded_logs\\8.0.1.19+889a9d2ee7d2470a179f81211ad77688f6772676\\JZWXCJ9XHBS8X3HSMEX9A4SG649H1D4TW5V1MGGVRATYYB1MZVWG\\1720987386402_unzipped',
      stdout: '',
      stderr: 'FIND: Parameter format not correct\r\n'
    }
  },
  date: 2024-07-14T20:03:06.468Z
}

Thanks
@kylefarris kylefarris self-assigned this Jul 22, 2024
@kylefarris kylefarris added the bug label Jul 22, 2024
@kylefarris
Copy link
Owner

I believe I have resolved your first issue in local branch. I'm working on writing the tests for the your second issue. I'm using pure JS to get the file listing now instead of using commands (file or cmd /c)--should be the better solution moving forward. That strategy was a holdover from a time when this module mostly just proxied calls to a unix shell.

Good catch and a great/thorough bug report. Thanks!

Fix should be out shortly.

@sefinek
Copy link
Author

sefinek commented Jul 22, 2024

No problem, and thanks @kylefarris! Let me know when you manage to fix it, and I'll test it on my end too. Take your time.

@kylefarris
Copy link
Owner

Hey! So, I think I've created a fix for your issue. Please check out v2.3.0 and see if it works for ya!

@sefinek
Copy link
Author

sefinek commented Jul 23, 2024 •
edited
Loading

Tested 😢 Somethint went wrong

Maybe it would be worth using this https://github.com/sefinek24/clamscan-linwin/commit/2b17542c0e3201353704d26274468dccbc4b7c77 solution for Windows?
Or simply use Get-ChildItem -Path from the PowerShell?

[StellaLogs]: ------------------------------- Starting the log file upload process by ::ffff:127.0.0.1 -------------------------------
=================================== 🔎 SCANNING THE ZIP FILE ===================================
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: using remote server: 192.168.0.111:7418
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK 
node-clam: File is OK!
=================================== 🔎 SCANNING UNZIPPED FILE ===================================
[FileDir]: Extraction completed
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
[StellaLogs]: Error during file scan: NodeClamError: There was an issue scanning the path specified!
    at E:\Projects\www\api.sefinek.net\node_modules\clamscan\index.js:2010:33 {
  data: {
    path: 'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721729622471_unzipped',
    err: Error: Command failed: find E:\Projects\other\stella-data\uploaded_logs\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\XXX\1721729622471_unzipped
    FIND: Parameter format not correct

        at genericNodeError (node:internal/errors:984:15)
        at wrappedFn (node:internal/errors:538:14)
        at ChildProcess.exithandler (node:child_process:422:12)
        at ChildProcess.emit (node:events:519:28)
        at maybeClose (node:internal/child_process:1105:16)
        at Socket.<anonymous> (node:internal/child_process:457:11)
        at Socket.emit (node:events:519:28)
        at Pipe.<anonymous> (node:net:338:12) {
      code: 2,
      killed: false,
      signal: null,
      cmd: 'find E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721729622471_unzipped',
      stdout: '',
      stderr: 'FIND: Parameter format not correct\r\n'
    }
  },
  date: 2024-07-23T10:13:42.507Z
}

@kylefarris
Copy link
Owner

Oh shoot... 🫠 I'm an idiot. Hang tight.

@kylefarris
Copy link
Owner

kylefarris commented Jul 23, 2024 •
edited
Loading

Apologies for that. The previous commit had the relevant lines removed during clean up. File discovery now happens entirely using Node APIs. I would have caught this if I was testing on Windows. I might be able to add that to the GitHub Actions for CI--not sure how to do that yet but I can look into it.

Try v2.3.1 and you should be good.

@sefinek
Copy link
Author

sefinek commented Jul 23, 2024

Thanks! You're amazing - everything is working correctly now. I'm also sending you the logs:

[StellaLogs]: ------------------------------- Starting the log file upload process by ::ffff:127.0.0.1 -------------------------------
=================================== 🔎 SCANNING THE ZIP FILE ===================================
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: using remote server: 192.168.0.111:7418
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
=================================== 🔎 SCANNING UNZIPPED FILE ===================================
[FileDir]: Extraction completed
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Scanning a list of 12 passed files. [
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\Configs\\FPS Unlocker.json',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\Configs\\ReShade.ini',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\DeviceInfo.sfn',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\dxdiag.sfn',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\Logs\\Genshin Stella Mod\\2.1.0.1.log',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\Logs\\Info 4842\\2.0.0.0.log',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\Logs\\Launcher\\8.1.0.0.log',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\Logs\\Launcher\\8.1.1.0.log',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\Logs\\Launcher\\8.1.2.0.log',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\Logs\\Launcher\\8.1.3.0.log',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\Logs\\Prepare Stella\\2.0.1.0.log',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\Logs\\Prepare Stella\\2.0.2.0.log'
]
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: using remote server: 192.168.0.111:7418
node-clam: using remote server: 192.168.0.111:7418
node-clam: using remote server: 192.168.0.111:7418
node-clam: using remote server: 192.168.0.111:7418
node-clam: using remote server: 192.168.0.111:7418
node-clam: using remote server: 192.168.0.111:7418
node-clam: using remote server: 192.168.0.111:7418
node-clam: using remote server: 192.168.0.111:7418
node-clam: using remote server: 192.168.0.111:7418
node-clam: using remote server: 192.168.0.111:7418
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: using remote server: 192.168.0.111:7418
node-clam: using remote server: 192.168.0.111:7418
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Scan Complete!
node-clam: Num Bad Files:  0
node-clam: Num Good Files:  12
node-clam: Num Viruses:  0
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
[StellaLogs]: 📃 The log file was successfully received and saved in E:\Projects\other\stella-data\uploaded_logs\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\XXX\1721754446497.zip
[StellaLogs]: deviceId: XXX; ip: ::ffff:127.0.0.1; country: unknown; city: unknown
[200 POST 920.970 ms] /api/v7/genshin-stella-mod/launcher/logs/upload Mozilla/5.0 (compatible; StellaTelemetry/1.0.1.0; +https://stella.sefinek.net) "-" -
[FileDir]: Removed directory: E:\Projects\other\stella-data\uploaded_logs\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\XXX\1721754446801_unzipped

@kylefarris
Copy link
Owner

Sweet! Glad we got it working :)

@sefinek
Copy link
Author

sefinek commented Jul 23, 2024

I have checked it more thoroughly now, and it works very well. Thanks again. Have a nice holiday. I'm not sure if the holidays are still ongoing in your country, but all the best anyway <:

node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: Win.Test.EICAR_HDB-1 FOUND
node-clam: Scan Response:  stream: Win.Test.EICAR_HDB-1 FOUND
node-clam: File is INFECTED!
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Scan Complete!
node-clam: Num Bad Files:  1
node-clam: Num Good Files:  12
node-clam: Num Viruses:  1
=================================== 🐛 THE SECOND FILE IS INFECTED! ===================================
 {
  goodFiles: [
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Configs\\FPS Unlocker.json',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Configs\\ReShade.ini',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\DeviceInfo.sfn',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\dxdiag.sfn',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Logs\\Genshin Stella Mod\\2.1.0.1.log',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Logs\\Info 4842\\2.0.0.0.log',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Logs\\Launcher\\8.1.0.0.log',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Logs\\Launcher\\8.1.1.0.log',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Logs\\Launcher\\8.1.2.0.log',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Logs\\Launcher\\8.1.3.0.log',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Logs\\Prepare Stella\\2.0.1.0.log',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Logs\\Prepare Stella\\2.0.2.0.log'
  ],
  badFiles: [
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Logs\\Launcher\\eicar.com.log'
  ],
  viruses: [ 'Win.Test.EICAR_HDB-1' ],
  errors: {}
}

This was referenced Sep 8, 2024
Open
Open
This was referenced Sep 16, 2024
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kylefarris kylefarris

Labels
bug
Projects
None yet
Milestone
No milestone
2 participants
@kylefarris @sefinek