Memory exhausted while uploading a large torrent

[bianyifan]

The torrent file uploaded is 1.2 MiB large. Because GitHub does not support attachments of issues, the related torrent file has been sent to the author in another way.

[Rhilip]

It may mistake to closed this issue in last commit which change our Bencode library.

Though this torrent (Large torent test.zip) May invalid.

Why I say invalid?

After change the run mode to HOT Reload,
I got another Error throw, (sandfoxme/bencodec/Engine/Decoder.php#L103) ,
and even when I change the bencode library to something like OPSnet/bencode-torrent ,
It still throw at those breakpoint (OPSnet/bencode-torrent/Bencode.php#L103_L107)
which verify if fully parse the string.

But the bencode library like NexusPHP , or python library kosqx/better-bencode don't verify, So this torrent can be upload success.

In last commit (0e1a6ab), I remove the verify part , And then this torrent can be accept by our TorrentUploadForm model.

But......

But last commit didn't solve this issue, since this issue is about the Memory exhausted.
I'm worried that this problem may cause by project's framework of memory leak.
So I will recheck the framework with the part of apps , and keep this issue open.

[povsister]

In fact, NexusPHP has every uploaded torrent file verified. It uses a set of recursive functions to encode/decode bencoded files.
One ill-formatted element encountered during the parse is enough to cause a NULL return val, which means something went wrong. And the server just simply reject such torrents with those erros.

It looks not that elegant but really works out on most occasion.

[bianyifan]

That specific torrent file is valid except trailing new line. That new line (0x0a, aka CR) is automatically added by vim on saving file after I changed the announce URL of that torrent. In fact, sandfoxme/bencodec/Engine/Decoder.php#L103 has a clear notice about that. It's not big deal and can be simply ignored.

Still working on memory exhausting issue.

[Rhilip]

It does not seem to reappear after commit 4413b46 , Though I think this issues fixed before the release v0.1.2-alpha

With Test Setting :

'worker_num'       => 1,        // 工作进程数
'max_request'      => 3000,   // 进程的最大任务数

After three times trys to upload this large torrent , The Memory used not exhausted , the error report show this value change like 4436288 -> 4625136 -> 4625136.
Also I notice When increase the setting of worker_num to default number 8, The fail tmp_file will be autocleaned after this worker next time when it recevie the new request.

[Rhilip]

I don't think this bug can reproduce after 5 alpha versions, since many change has been commited.
So I decide to closed it.