History
Just released v0.9.1-hotfix.1 (also latest) because I saw that Plex won't play error videos because the error video file size and original content file size is different and then it gets "stuck" for about a minute trying to read the faulty file (also rclone showing unexpected EOF errors because of the file size mismatch). It should no longer happen and Plex can scan expired files faster.
Also fixed repairs not triggering on unrestrictable links when I reworked the code there to fix the issue where zurg gets stuck when there are more expired torrents than concurrent_workers. It should now trigger properly.
v0.9.1 released! docker pull ghcr.io/debridmediamanager/zurg-testing:latest or :v0.9.1
Changelog:
- Fixed health check issue (if you still see zurg as unhealthy after upgrading, restart it)
- Before it throws you an error video, zurg's data fetcher will retry up to 5 times now
- zurg now runs on the main CPU thread to maximize performance
- Added
retain_rd_torrent_nameinconfig.yml, set totrueto make it truly compatible withrdt-client - Repairs should now prevent torrents from being renamed
- The config
on_library_updategets passed the updated directories now as arguments ` - Added
serve_from_rcloneinconfig.yml, set totrueif you want zurg to just pass the file URL to rclone for downloading - Added
force_ipv6inconfig.yml, set totrueif you want zurg to connect to Real-Debrid's IPv6 address - In progress torrents no longer gets listed in directories
- Adjusted the modification times displayed for torrent directories and files. Files will now show
endedproperty while directories showaddedproperty. - Timezone issue fixed for modification times
- The rclone
httpmount now displays modification times for files
Moving forward, unless a hotfix is needed for a critical bug in zurg, I am adjusting the release schedule to be every 2 weeks to give me more time to work on other projects. Enjoy!
🌈 IT'S BEEN A MONTH SINCE ZURG'S FIRST RELEASE! 🥳
You guys remember zurg-1000? 😅 That's exactly a month ago!
Today's special so here is a special release!
Binaries for all platforms
Check out the latest releases on https://github.com/debridmediamanager/zurg-testing/tree/main/releases/
Deletes now work on webdav
You can delete a torrent , and...
Deleting individual files also work
Just make sure you retain your data/ directory and delete info_cache_time_hours in your configs, it's not needed
If all files are deleted inside a torrent, the torrent is also deleted
Webdav now works as fast as http
Lots of optimizations were done and also, the key update:
Just add pacer_min_sleep = 0 in your rclone.conf
[zurg]
type = webdav
url = http://zurg:9999
vendor = other
pacer_min_sleep = 0
Note: be careful on editing rclone.conf as your mount might not work properly, best to stop Plex server first if you dont know what you are doing or your library might reset itself
Several new filters are added!
not_regex
any_file_inside_not_regex
any_file_inside_not_contains
any_file_inside_not_contains_strict
and possibly the best filter ever (can only be topped by anime: true)...
has_episodes: true
which is a detection algorithm of episodes found inside a torrent
my config.yml looks like this now
directories:
anime:
group_order: 10
group: media
filters:
- and:
- has_episodes: true
- any_file_inside_regex: /^\[/ # e.g. [SubsPlease]
- any_file_inside_not_regex: /s\d\de\d\d/i
shows:
group_order: 20
group: media
filters:
- has_episodes: true
movies:
group_order: 30
group: media
filters:
- regex: /.*/
Sponsors
THANK YOU!!! you know who you are Thanks to vanderruss for sponsoring me on github My patreon page is also live 🎇
Hey all! I had several fuckups on the releases recently. The latest tag was pointing to an unstable code 😨 for a long time and I think a lot of people tried it and obviously got a bad experience with setting up.
I've fixed this issue and did a ton of rework on v0-defuse to again resolve the very difficult programming problem of race conditions and hopefully the latest image should be at par if not better than the previous versions. ⛅
I've also complete reworked the repair functionality :tools: and it shouldn't recreate gazillions of torrents. The repairs happen on the hash level now (for maximum efficiency). If one file is broken inside a group of torrents with the same hash, it will check the whole directory for any other broken files and it will only redownload what is broken (it doesn't reinsert torrents like what others do because that is not 100% guaranteed to repair broken files). The health check has been removed because it was also causing issues for libraries that takes a long time to get complete information for.
FYI http is faster than webdav by a factor of 4.5 (i compare zurg by itself now, lol)
p.s
(I really need to start focusing on the fuse work or else zurg will never get to v1 aka climax)
I'm sleepy & tired but cannot forget to thank @godver3 @nometaisbeta @yeungdrizzy for being amazing project sponsors! Also first time I saw zurg being put on a guide written by @thebropuks which is just mind blowing for a month-old software to make that much impact. 🤯
Enjoy the rest of the weekend y'all!
New release! latest or v0-defuse doesn't mattery for now
I've reworked every part of the architecture
the repair function is fully rewritten, it should be behaving properly now :slight_smile: Added a proper http client with retry/unrestrict
this setting is now fixed retain_folder_name_extension , it is now working to make it more compatible with rdt-client
and I added preferred_hosts! As to how to determine what to put on preferred hosts list, check my script here https://github.com/debridmediamanager/real-debrid-network-test/blob/main/real-debrid-network-test.sh (you can put as many as you want)
I can finally work back again on the fuse branch. Time to sleep!
^ I've created a hotfix for this, please reconfigure your docker-compose files and use this image instead
docker pull ghcr.io/debridmediamanager/zurg-testing:v0-defuse
- It contains the logging improvements (no way to configure logging level yet)
- race condition fixes making it more stable
- the repair functionality apparently can't be disabled entirely so I've fixed it here as well
(no fuse mount yet, hence defuse because i made an effort to remove it as it's not yet ready)
Hi all, i've recovered and am back working on the project again :slight_smile:
Some updates before the weekend: i haven't reached the point of working on the renames and deletes yet, but the mounting works, and can see the directories and files. it is again magnitudes faster because no translation of data is happening between protocols as we don't need rclone as a middle layer anymore.
I've also improved the way logging works and log level should be configurable now.
There are also several race conditions reported by different users that I've resolved. Again the directory mapping for torrents should be more stable on the next release.
(the http and webdav ports were untouched so you could still use that as a "stable" fallback)
The problem I'm having now is the way bytes are read inside a file. If using http or webdav, large chunks are requested at a time. This is great because when the connection closes, zurg can immediately stop requesting the file from RD servers. No bytes wasted. If using a filesystem mount, the OS controls the read buffer and it seems to be just 1kb chunks? It's super inefficient so I'm currently working on my own implementation of a cache layer that divides the whole file into parts and parts are loaded as needed. This is the main blocker of an early release.
Anw, hope everyone had a nice weekend!
zurg progress update
eta on the rename/delete function is by end of this week it will only support linux for now, and you will no longer have to use rclone
i'm collecting non critical bugs that will also be included in the release
zurg will also be released as a binary apart from the docker image
Update for today:exclamation: I have fixed this issue on the latest release that's available to be pulled now
-
any_file_inside_regexand otherany_file_inside_*filters works now - added
network_buffer_sizein config.yml to let you configure the best buffer size that's most compatible with your internet speed. there's a perfect setting here that depends on MTU, connection speed etc. so I made this configurable. - the issues affecting playability of the files (the files being unplayable, etc.) should now be fixed
it should make things a lot more stable!
A new release should be available in the next 10 mins
- repaired files creating a different folder should not happen again (no need to delete anything here for this fix to work)
- if same filename and size, then it deduplicates the files inside a folder (shows you the latest file)
- directory mapping issue fixed, thanks @Deep-value !
I tried making an improvement on the responsiveness of the playback. If this is something you're interested on testing, pull the latest image :slight_smile:
This helps even if you're using filebot
New release
Add this on your config.yml if you want your Plex libraries to always be updated :slight_smile:
on_library_update: |
token="Your X-Plex-Token Here"
plex_url="http://plex.box"
sections_xml=$(curl -s "$plex_url/library/sections" -H "X-Plex-Token: $token")
section_ids=$(echo "$sections_xml" | grep -o 'key="[0-9]*"' | awk -F'"' '{print $2}')
for id in $section_ids; do
curl -s -X POST "$plex_url/library/sections/$id/refresh" -H "X-Plex-Token: $token"
done
echo "Plex library refresh triggered!"please pull the latest release that will be available in 10 minutes
docker image inspect ghcr.io/debridmediamanager/zurg-testing:latest | grep revision
should be 0fd2fa691e1c32aec9f7762274fc9941e9313925
auto repair is done :slight_smile:
https://github.com/debridmediamanager/zurg-testing new release available in a few mins
- directory mapping fix (hopefully no more weird issues!)
- new config doc https://github.com/debridmediamanager/zurg-testing/blob/main/config.md (optional) you might want to update your
config.ymlbased on what you see here in this doc
please test the latest release. unfortuntely i haven't created a proper docker release pipeline so every release uses the latest tag for now
another change again for rclone.conf https://github.com/debridmediamanager/zurg-testing/blob/main/rclone.conf no_slash = false
the new release includes repair functionality, tested on a few use cases
ok i think i have fixed the issues on the http mount mainly
- file metadata - file type and file sizes are now properly set (you need to update rclone config again and set
no_headtofalse - when updating the torrent list, all directories goes empty and then back to normal again (or sometimes the directory mapping just fails)
The docker image is building now and will be available in a few minutes
change from zurg-1000 to zurg-testing
it's built now, docker pull ghcr.io/debridmediamanager/zurg-testing:main
https://github.com/debridmediamanager/zurg-testing/blob/main/README.md
pushed a new release, i recommend everyone to pull that image and update your rclone configs
if you are using a dockerized rclone, the new config should look like this
[zurg]
type = http
url = http://zurg:9999/http
no_head = true
no_slash = true
i have now heavily tested this with the rclone usecase
benchmark based on https://github.com/debridmediamanager/zurg-testing/blob/main/experiment.md
it's around 5-6x faster
it seems davfs2 is crazy fast though based on @kakasanti 's tests :slight_smile: thank you!
@here https://github.com/debridmediamanager/zurg-1000
I think i will remove the stupid limit, it was supposed to send the message that this is a beta/test version so people will provide feedback, but i think it will make the test comparisons look silly