Bazaar may not recognize subtitles files without language code

[screamerbg]

Describe the bug
Bazaar may not recognize subtitles files without language code, apparently related to file naming.

As an example, I have 2 seasons of the same TV show with subtitles named exactly as the video file.

Working example

The subtitles are recognized correctly.

File structure

Bazaar page

Non-working example

The subtitles are not recognized, as if they don't exist. Apparently renaming the video and subtitle files altogether fixes the issue.

File structure

Bazaar page

To Reproduce
Steps to reproduce the behavior:

1. Add "The X-Files" TV series in Sonarr
2. Recreate the file structure as shown in the first attached image. (The X-Files - Season 11 folder)
3. Check Bazaar - it doesn't seem to recognize the subtitle files

Expected behavior
Bazarr should correctly identify the subtitles files.

Screenshots

Config
config.ini.zip

Software (please complete the following information):

• Bazarr: 0.8.3.4
• Radarr version 0.2.0.1450
• Sonarr version 2.0.0.5338
• OS: Windows 7 (latest updates)

Additional context
Add any other context about the problem here.

[morpheus65535]

Please empty log, enable debug logging in settings-->general, go to X-Files series page and click on scan from disk button then provide the bazarr.log file.

[screamerbg]

@morpheus65535 See attached

bazarr.log.zip

[morpheus65535]

Bazarr is not even aware that those file are there... we don't even get to the point where it guess the language from the content of the file. What are the permission of those srt files?

[screamerbg]

The folder permissions inherit the parent's folder permissions, e.g. nothing special to it. In fact, the existing subtitles have been downloaded using Bazarr previously.

I've just deleted them and triggered "Download Missing Subtitles" + "Scan DIsk for Subtitles". Attached is the debug log below. As you might notice, Bazarr downloads a fresh copy of the subtitles in the season folder, yet the scan process doesn't "see" them later on.

bazarr.log.zip

[morpheus65535]

Did you enable CHMOD option in settings? Maybe there's a value here that prevent Bazarr from reading the files.

[screamerbg]

I can't find any CHMOD option in settings, Bazarr Ver. 0.8.3.4. And I don't think that this has to do anything with permissions anyway.

Upon renaming all files (mkv + srt) to strip away .AVC.x264.MULTi.AC3-STM from the name, and then refreshing Sonarr + Bazarr sync w/ Sonarr + Bazarr file rescan, the subtitles magically appear.

File structure

Bazarr season page

Upon renaming all files to their original names (essentially adding .AVC.x264.MULTi.AC3-STM before the extension), the subtitles disappear again.

[morpheus65535]

How long is the complete path of one of those episode?

[screamerbg]

E:\TV Series\The X-Files\The X-Files - Season 11\The.X-Files.S11E01.1080p.BluRay.AVC.x264.MULTi.AC3-STM.mkv is 107 characters.

[morpheus65535]

Hi honestly have no clue what is the issue here... if you just add language code to subtitles filename, do they get recognized by Bazarr?

[screamerbg]

Hi. Apologies for the delay - pre-Christmas madness got in the way.

Adding language code does make the episodes appear. See below. Apparently the issue isn't related to the filesystem, but rather something in the scan logic.

Have you had the chance to reproduce the issue on your end, e.g. create a file structure like the one described and see what may be causing it? Would it help if I send you text file with all file names etc?

File structure

Bazarr season page

[morpheus65535]

I have episodes with way longer path than that but I cannot reproduce your issue. To be honest, I till don't know how to reproduce the issue.

[screamerbg]

Here is a guide how to reproduce the issue:

1. Create a folder under your TV Series root named "The X-Files" and then create a sub-folder "The X-Files - Season 11"

2. Copy a few MKV files from one of your existing TV series into the "The X-Files - Season 11" folder and name them as the following:

The.X-Files.S11E01.1080p.BluRay.AVC.x264.MULTi.AC3-STM.mkv
The.X-Files.S11E02.1080p.BluRay.AVC.x264.MULTi.AC3-STM.mkv
The.X-Files.S11E03.1080p.BluRay.AVC.x264.MULTi.AC3-STM.mkv
The.X-Files.S11E04.1080p.BluRay.AVC.x264.MULTi.AC3-STM.mkv
The.X-Files.S11E05.1080p.BluRay.AVC.x264.MULTi.AC3-STM.mkv
The.X-Files.S11E06.1080p.BluRay.AVC.x264.MULTi.AC3-STM.mkv
The.X-Files.S11E07.1080p.BluRay.AVC.x264.MULTi.AC3-STM.mkv
The.X-Files.S11E08.1080p.BluRay.AVC.x264.MULTi.AC3-STM.mkv
The.X-Files.S11E09.1080p.BluRay.AVC.x264.MULTi.AC3-STM.mkv
The.X-Files.S11E10.1080p.BluRay.AVC.x264.MULTi.AC3-STM.mkv

3. Assuming that you're familiar with Sonarr, open it and add new TV series "X-Files" and point it to the root folder you've created.

4. Once you're able to confirm that the MKV files are correctly recognized by Sonarr for X-Files Season 11, download the following zip archve containing sample subtitle files. You could use other subtitle files as well - it's important that they are named exactly as the sample files.

5. Go to Bazaar Settings and ensure that "Single Language" is enabled (it's under "Subtitles languages").

6. After you're successful with all of the above, go to Bazaar -> System and click the following "Update Series list from Sonarr" and then "Sync episodes with Sonarr".

7. Finally, open the "X-Files" in Bazaar and refresh the files from disk. Bazaar won't recognize the subtitles at first, but renaming the subs and adding a language code before the extension will "magically" make them appear in Bazaar upon refresh from disk.

Hope this helps. Let me know if something isn't well explained.

[morpheus65535]

@screamerbg that'S perfectly explained. Give a couple of hours and I'll test it and should come with a fix if I'm able to reproduce.

[morpheus65535]

@pannal if match_strictness="strict", those subtitles aren't returned by search_external_subtitles(). Could you take a look? If I use loose mode it works as expected.

[pannal]

Yep, AC3-STM gets removed from the subtitle filename for the strict filename comparison and thus it doesn't exactly match anymore. This is due to language code detection, which is a bit too lax.

Prototyping a solution right now.

[pannal]

@morpheus65535 can you try https://github.com/pannal/Sub-Zero.bundle/tree/bzr_%23703?
Essentially those changes: pannal/Sub-Zero.bundle@a92f3e2

Pushed a possible fix to development. Please try it out @screamerbg

[screamerbg]

@pannal Thanks for the patch, though it doesn't seem to fix the issue.

I've applied your changes with the following steps:

1. Clone the git repo
2. Check out development branch
3. Copy 2 files from from the git repo to the installed Bazarr location - libs\subzero\language.py and libs\subliminal_patch\language.py
4. Delete all .pyc files in the Bazarr tree
5. Restart Bazarr using the Windows service manager.
6. Going to the TV series and clicking Scan disk for subtitles.

Am I doing this correctly?

[morpheus65535]

In fact the patch is already available in Bazarr dev branch. You simply pull latest dev.

[screamerbg]

@morpheus65535 How do I switch to the dev branch? I couldn't find an option is Settings.

[morpheus65535]

@screamerbg using windows installer? You can’t. You have to be running from source.

[screamerbg]

@morpheus65535 I see. As I couldn't find how to switch to dev, I used the steps above to update just these 2 files that were patched. Unfortunately the issue still persists.

[morpheus65535]

If you want to test that way, at least update the files from those you can find on Bazarr github. I'm not sure it will work as master and dev branch are really different as we are migrating code base to python 3.x.

[screamerbg]

@morpheus65535, @pannal Just ran Bazarr from source and it works like a charm - Bazarr recognized the aforementioned X-Files subtitles and also for other series as well.

Thank you both for this!

How do I get rid of the Python 3 notice?

[morpheus65535]

Great news!

To get rid of the Python deprecation warning, just run Bazarr using Python >= 3.6 (3.8 works fine) and you win’t see it anymore.

[pannal]

Whahey! Merry Christmas! :)