Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incomplete anlysis, waveform overview and display, of a song file, after conducting library "Analyze" #11280

Open
nPrevail opened this issue Feb 16, 2023 · 33 comments

Comments

@nPrevail
Copy link

nPrevail commented Feb 16, 2023

Screenshot from 2023-02-15 23-56-44

I analyzed my library of files, and in some instances, some of my files would be analyzed but incompletely scanned. This image attached to this message is what I see.

When playing the incomplete song file, the audio cuts off before the song reaches the end and the deck stops playing.

I have tried "hiding" the song file, deleting the song file data from the library, and I have tried reanalyzing the song file, yet it rescans and still shows as incomplete at the same time stamp.

There is nothing wrong with the song file itself; it plays fine in a music player (ie. VLC)

I happen to have backups of my music files; back ups of the same songs in other hard drives. The only solution I found was to use the back up of the song file from the directory folder, and overwrite the original song from my Mixxx music folder.

I have tried duplicating the song file in Mixxx library/folder (copy and pasting the song file), and it does not fix the issue. But grabbing the song file, which had a different "created by" date, and a slightly different file size (unedited metadata?) seemed to fix the issue.

Perhaps this is a metadata song file issue?

@uklotzde
Copy link
Contributor

Decoding stops at the first unrecoverable error. Please check the logs for more information.

@nPrevail
Copy link
Author

nPrevail commented Feb 16, 2023

Please check the logs for more information.

What's the fastest way to check the logs? I'm look through the .mixxx folder, and I have about 10 log files to look at.

EDIT: How do I check logs?
Would it be something like this from the manual? https://manual.mixxx.org/2.3/en/chapters/appendix/commandline_dev_tools.html

@uklotzde
Copy link
Contributor

Start Mixxx, reset the waveform data of the affected song, then load it on a deck. The log of the current session is in mixxx.log.

@uklotzde
Copy link
Contributor

Starting mixxx from a terminal and simply watching the outputs is another option.

Please also send me one of the affected files. It would be interesting to investigate what's wrong and which decoder (libmad or ffmpeg) fails to decode it.

@ronso0
Copy link
Member

ronso0 commented Feb 16, 2023

@nPrevail Yes, could also run Mixxx from the command line with --developer argument. The only benefit is that you'd see the debug output in real time, not just when it's flushed to disk

@uklotzde
Copy link
Contributor

The --developer argument is only needed if Mixxx crashes. Otherwise it just causes unnecessary I/O.

@nPrevail
Copy link
Author

nPrevail commented Feb 16, 2023

Start Mixxx, reset the waveform data of the affected song, then load it on a deck.

I have tried resetting the waveform. I actually reset all the parameters, and had loaded onto the deck, and the issue would still persist.

It's unfortunate that none of these logs are timestamped, but I'll paste all the relevant log lines I have.

@uklotzde
Copy link
Contributor

Resetting the waveform enforces the analyzer to re-decode the whole file for getting the logs. The error will not be fixed magically.

@uklotzde
Copy link
Contributor

Bugs that fix themselves under unknown circumstances would be the worst case. Reproducible bugs are good bugs ;)

@nPrevail
Copy link
Author

Logs

Debug [AnalyzerThread 0 #1]: AnalyzerThread - Analyzing QFileInfo(/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac)
Debug [AnalyzerThread 0 #1]: AnalysisDAO fetched 2 analyses, 5252991 bytes for track 3147 in 121 ms
Debug [AnalyzerThread 0 #1]: Reading waveform from byte array: allSignalSize 610918 visualSampleRate 441 audioVisualRatio 100
Debug [AnalyzerThread 0 #1]: Reading waveform from byte array: allSignalSize 3842 visualSampleRate 2.77197 audioVisualRatio 15909.3
Debug [AnalyzerThread 0 #1]: AnalyzerWaveform - loadStored - Stored waveform loaded
Debug [AnalyzerThread 0 #1]: AnalyzerBeats preference settings:
Plugin: "qm-tempotracker:0"
Fixed tempo assumption: true
Re-analyze when settings change: true
Re-analyze imported from other software: true
Fast analysis: false
Debug [AnalyzerThread 0 #1]: AnalyzerKey preference settings:
Plugin: "keyfinder:2"
Re-analyze when settings change: true
Fast analysis: false
Debug [AnalyzerThread 0 #1]: Keys version/sub-version unchanged since previous analysis. Not analyzing.
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Debug [Main]: GLWaveformRendererSignalShader::loadShaders
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Warning [AnalyzerThread 0 #1]: SoundSourceFLAC - FLAC decoding error "STREAM_DECODER_ERROR_STATUS_FRAME_CRC_MISMATCH" in file "/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac"
Warning [AnalyzerThread 0 #1]: SoundSourceFLAC - FLAC decoding error "STREAM_DECODER_ERROR_STATUS_LOST_SYNC" in file "/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac"
Warning [AnalyzerThread 0 #1]: SoundSourceFLAC - Unexpected frame index 20172800 > 20168704 while decoding FLAC file "/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac"
Warning [AnalyzerThread 0 #1]: AudioSource - Failed to read sample frames: expected = [20168704 -> 20172800) , actual = [20168704 -> 20168704)
Info [AnalyzerThread 0 #1]: AudioSource - Shrinking readable frame index range: before = [0 -> 30545828) , after = [0 -> 20168704)
Debug [AnalyzerThread 0 #1]: ReplayGain 2.0 (libebur128) result is -9.37333 dB for QFileInfo(/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac)
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Resuming analysis
Debug [Main]: TrackAnalysisScheduler - Resuming
Info [Main]: AnalysisFeature - Resuming analysis
Debug [Main]: TrackAnalysisScheduler - Resuming
Debug [Main]: Successfully deserialized BeatGrid
Debug [Main]: Successfully deserialized KeyMap
Debug [Main]: BaseTrackCache(0x562e329e55a0) updateIndexWithQuery took 0 ms
Debug [Main]: BaseTrackPlayerImpl::slotLoadTrack "[Channel1]" Track(0x562e32fd33d0)
Debug [Main]: TrackAnalysisScheduler - Resuming
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Debug [Main]: TrackCollectionManager - Saving track "/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac" in internal collection
Debug [Main]: AnalyzerThread - Enqueueing next track 3146
Debug [AnalyzerThread 0 #1]: AnalyzerThread - Dequeued next track 3146
Debug [AnalyzerThread 0 #1]: AnalyzerThread - Analyzing QFileInfo(/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 01 Set It Out.flac)
Debug [AnalyzerThread 0 #1]: AnalysisDAO fetched 2 analyses, 3889488 bytes for track 3146 in 32 ms
Debug [AnalyzerThread 0 #1]: Reading waveform from byte array: allSignalSize 453256 visualSampleRate 441 audioVisualRatio 100
Debug [AnalyzerThread 0 #1]: Reading waveform from byte array: allSignalSize 3842 visualSampleRate 3.73618 audioVisualRatio 11803.5
Debug [AnalyzerThread 0 #1]: AnalyzerWaveform - loadStored - Stored waveform loaded
Debug [AnalyzerThread 0 #1]: AnalyzerBeats preference settings:
Plugin: "qm-tempotracker:0"
Fixed tempo assumption: true
Re-analyze when settings change: true
Re-analyze imported from other software: true
Fast analysis: false
Debug [AnalyzerThread 0 #1]: AnalyzerKey preference settings:
Plugin: "keyfinder:2"
Re-analyze when settings change: true
Fast analysis: false
Debug [AnalyzerThread 0 #1]: Keys version/sub-version unchanged since previous analysis. Not analyzing.
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Debug [AnalyzerThread 0 #1]: ReplayGain 2.0 (libebur128) result is -6.6405 dB for QFileInfo(/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 01 Set It Out.flac)
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Resuming analysis
Debug [Main]: TrackAnalysisScheduler - Resuming
Info [Main]: AnalysisFeature - Resuming analysis
Debug [Main]: TrackAnalysisScheduler - Resuming
Debug [Main]: Successfully deserialized BeatGrid
Debug [Main]: Successfully deserialized KeyMap
Debug [Main]: BaseTrackCache(0x562e329e55a0) updateIndexWithQuery took 0 ms
Debug [Main]: BaseTrackPlayerImpl::slotLoadTrack "[Channel1]" Track(0x562e48334340)
Debug [Main]: TrackAnalysisScheduler - Resuming
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Debug [Main]: TrackCollectionManager - Saving track "/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 01 Set It Out.flac" in internal collection
Debug [Main]: AnalyzerThread - Enqueueing next track 3147
Debug [AnalyzerThread 0 #1]: AnalyzerThread - Dequeued next track 3147
Debug [AnalyzerThread 0 #1]: AnalyzerThread - Analyzing QFileInfo(/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac)
Debug [AnalyzerThread 0 #1]: AnalysisDAO fetched 2 analyses, 5252991 bytes for track 3147 in 28 ms
Debug [AnalyzerThread 0 #1]: Reading waveform from byte array: allSignalSize 610918 visualSampleRate 441 audioVisualRatio 100
Debug [AnalyzerThread 0 #1]: Reading waveform from byte array: allSignalSize 3842 visualSampleRate 2.77197 audioVisualRatio 15909.3
Debug [AnalyzerThread 0 #1]: AnalyzerWaveform - loadStored - Stored waveform loaded
Debug [AnalyzerThread 0 #1]: AnalyzerBeats preference settings:
Plugin: "qm-tempotracker:0"
Fixed tempo assumption: true
Re-analyze when settings change: true
Re-analyze imported from other software: true
Fast analysis: false
Debug [AnalyzerThread 0 #1]: AnalyzerKey preference settings:
Plugin: "keyfinder:2"
Re-analyze when settings change: true
Fast analysis: false
Debug [AnalyzerThread 0 #1]: Keys version/sub-version unchanged since previous analysis. Not analyzing.
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Warning [AnalyzerThread 0 #1]: SoundSourceFLAC - FLAC decoding error "STREAM_DECODER_ERROR_STATUS_FRAME_CRC_MISMATCH" in file "/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac"
Warning [AnalyzerThread 0 #1]: SoundSourceFLAC - FLAC decoding error "STREAM_DECODER_ERROR_STATUS_LOST_SYNC" in file "/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac"
Warning [AnalyzerThread 0 #1]: SoundSourceFLAC - Unexpected frame index 20172800 > 20168704 while decoding FLAC file "/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac"
Warning [AnalyzerThread 0 #1]: AudioSource - Failed to read sample frames: expected = [20168704 -> 20172800) , actual = [20168704 -> 20168704)
Info [AnalyzerThread 0 #1]: AudioSource - Shrinking readable frame index range: before = [0 -> 30545828) , after = [0 -> 20168704)
Debug [AnalyzerThread 0 #1]: ReplayGain 2.0 (libebur128) result is -9.37333 dB for QFileInfo(/home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac)
Info [Main]: AnalysisFeature - Suspending analysis
Debug [Main]: TrackAnalysisScheduler - Suspending
Info [Main]: AnalysisFeature - Resuming analysis
Debug [Main]: TrackAnalysisScheduler - Resuming
Info [Main]: AnalysisFeature - Resuming analysis
Debug [Main]: TrackAnalysisScheduler - Resuming
Debug [AnalyzerThread 5 #10]: Writing waveform from byte array: dataSize 358134 allSignalSize 358134 visualSampleRate 441 audioVisualRatio 100
Debug [AnalyzerThread 3 #8]: Writing waveform from byte array: dataSize 238712 allSignalSize 238712 visualSampleRate 441 audioVisualRatio 108.844
Debug [AnalyzerThread 6 #11]: Writing waveform from byte array: dataSize 416126 allSignalSize 416126 visualSampleRate 441 audioVisualRatio 100
Debug [AnalyzerThread 5 #10]: AnalysisDAO saved analysis 20762 "2958701 (1380325 compressed)" bytes for track 6874 in 425 ms
Debug [AnalyzerThread 5 #10]: Saved waveform analysis for trackId 6874 analysisId 20762
Debug [AnalyzerThread 5 #10]: Writing waveform from byte array: dataSize 3842 allSignalSize 3842 visualSampleRate 4.72851 audioVisualRatio 9326.4
Debug [AnalyzerThread 5 #10]: AnalysisDAO saved analysis 20763 "31420 (15337 compressed)" bytes for track 6874 in 13 ms
Debug [AnalyzerThread 5 #10]: Saved waveform summary analysis for trackId 6874 analysisId 20763
Debug [AnalyzerThread 5 #10]: AnalyzerWaveform - Waveform generation for track 6874 done 13 s

@uklotzde
Copy link
Contributor

The FLAC decoder is trustworthy, the last bug has been fixed long ago. Unfortunately, some of your files seem to be corrupt.

If you start vlc from the command line you might see warning logs. Maybe they ignore the error and continue decoding, accepting a short discontinuity in the decoded audio that might be unnoticeable. Mixxx is more rigorous here.

@nPrevail
Copy link
Author

The main culprit file is: /home/nPrev/Music/DJ (LL)/Disco/disco edits and covers/Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac"

@nPrevail
Copy link
Author

Unfortunately, some of your files seem to be corrupt.

Ah I see... I wonder how that happened...

@uklotzde
Copy link
Contributor

Use the official FLAC tools to analyze the file contents. Should be possible somehow. If they report that the file is ok then we need to check our FLAC decoder.

@uklotzde
Copy link
Contributor

And don't delete or modify the file! Keep it until the case could be closed.

@nPrevail
Copy link
Author

I have the "tainted" file on another drive. I can probably tinker with FLAC tools to analyze the contents for corruption.

@nPrevail
Copy link
Author

nPrevail commented Feb 16, 2023

Can anyone point out which FLAC tools I can use?

I've installed mingw64-flac-tools and mingw64-flac, but I don't know how to use it...

Screenshot from 2023-02-16 14-01-22

@uklotzde

@nPrevail
Copy link
Author

nPrevail commented Feb 16, 2023

So, I ran flac --analyze on the song file, and got this:

[nPrev@fedora disco edits and covers]$ 

flac 'Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac' --analyze

flac 1.3.4
Copyright (C) 2000-2009  Josh Coalson, 2011-2016  Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac: *** Got error code 2:FLAC__STREAM_DECODER_ERROR_STATUS_FRAME_CRC_MISMATCH


Revivra Alias Estéphe & Vulzor - Set It Out - Mercy - 02 Mercy.flac: ERROR while decoding data
                                                                      state = FLAC__STREAM_DECODER_ABORTED

I guess we can confirm that I just happen to have a corrupted file? I have no idea how it happened though...

@uklotzde
Copy link
Contributor

Now you need to find out what caused this corruption. Mixxx could only be involved if TagLib corrupts FLAC files when writing metadata.

@nPrevail
Copy link
Author

nPrevail commented Feb 17, 2023

Now you need to find out what caused this corruption.

Got it.

Well, some good news (for now). I ran this bash script I found on github, found all the corrupted tracks, fixed all the corrupted tracks (by running the script that pretty much self-reencoded flac to flac), and it seemed to fix all the corruption!

I used this script: https://github.com/cgomesu/bash-flac-diag

Worked perfectly! I'm glad to know it can be resolved in case it happens again.

So this time around, a clean slate will help me figure out what's causing the issue.

@daschuer
Copy link
Member

Great news. Which type of issue did the script report?

The script uses flac -st in the first place:
https://github.com/cgomesu/bash-flac-diag/blob/b5a06fa03367e30c9bea929c569912693709d1e3/flac_diag.sh#L161

What would be that appropriate behavior of Mixxx that we can avoid future bug reports like this?

@daschuer
Copy link
Member

daschuer commented Feb 17, 2023

When may investigate which tests the FLAC tool does and compare it with the Mixxx implemtation.

@nPrevail
Copy link
Author

nPrevail commented Feb 18, 2023

Which type of issue did the script report?

I'll talk about what I did and the behavior I saw from the script.

The script came with four separate bash scripts: flac diagnosis for good or bad songs, flac fix, flac remove, and install (installs all related tools necessary for FLAC). I used everything except "install.sh" and "bad_flac_remover.sh"

The "flac_diag.sh" script indicated if the song file was bad or good. Within the script's directory, the script creates a folder called "log," and then the script creates two logs: one for bad, the other for good. The script also creates a directory called "errors,"and that directory created individual error logs for each bad song file.

I then ran "bad_flac_fixer.sh" to fix all the corrupted tracks mentioned from the log directory. It pretty much "self re-encoded the corrupt files as if they were new again (encoding from FLAC to FLAC, and same file source). The best part is that I didn't need to locate any directories as the script found and logged the directories of the corrupt files.

Overall, for scanning and diagnosing 4,000+ tracks, the process took about 60 minutes to analyze. I had about 70 bad FLAC files. Fixing the flac files probably took about 10 minutes. Obviously this will vary based on computer specs.

More details on how and what the script does can be found in the readme, and it's pretty straight forward.

Here are some screen shots:
Screenshot from 2023-02-17 17-14-28
Screenshot from 2023-02-17 17-14-19
Screenshot from 2023-02-17 17-11-33
Screenshot from 2023-02-17 17-11-25
Screenshot from 2023-02-17 17-22-57

What would be that appropriate behavior of Mixxx that we can avoid future bug reports like this?

It'd be neat if we could implement this type of script into Mixxx. Like a "Diagnose" and "Fix" button from the "Analyze" section in Mixxx.

Unfortunately, it seems to only currently work with FLAC files, and depending on a person's filesystem and configuration, there may be some prerequisites needed to ensure the FLAC tools work (however, the script's readme states that most modern distros should already have these FLAC tools installed by default).

@nPrevail
Copy link
Author

nPrevail commented Mar 1, 2023

Okay, I thought this issue was over, but I discovered a few more things that also conflict in Mixxx: ID3v2 tags

This error causes the same error I mentioned: incomplete waveform analysis for songs with this type of error.

As I remove the ID3v2 tags from these songs, what type of tag is recommended to be used in Mixxx?

From the same FLAC fixing script, I received the following outputs. I bolded the errors that are causing issues, and I have to fix these issues in some other method because this script can't remove ID3v2 tags:

..............
..............
Starting fixer for the bad flac files...
..............
Re-encoding file: /home/nPrev/Music/DJ (LL)/Disco/in the zone/09 Always There.flac

flac 1.3.4
Copyright (C) 2000-2009 Josh Coalson, 2011-2016 Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY. This is free software, and you are
welcome to redistribute it under certain conditions. Type `flac' for details.

ERROR: input file /home/nPrev/Music/DJ (LL)/Disco/in the zone/09 Always There.flac has an ID3v2 tag
Done. Now try playing the file again.

..............
Re-encoding file: /home/nPrev/Music/DJ (LL)/Electronic dance music/Deep house/out there/03 Glow & Glare.flac

flac 1.3.4
Copyright (C) 2000-2009 Josh Coalson, 2011-2016 Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY. This is free software, and you are
welcome to redistribute it under certain conditions. Type `flac' for details.

ERROR: input file /home/nPrev/Music/DJ (LL)/Electronic dance music/Deep house/out there/03 Glow & Glare.flac has an ID3v2 tag
Done. Now try playing the file again.
..............
..............
Re-encoding file: /home/nPrev/Music/DJ (LL)/Electronic dance music/Techno/getting started/04 The Sugarcubes - Motorcrash [Justin Robertson Mix].flac

flac 1.3.4
Copyright (C) 2000-2009 Josh Coalson, 2011-2016 Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY. This is free software, and you are
welcome to redistribute it under certain conditions. Type `flac' for details.

ERROR: input file /home/nPrev/Music/DJ (LL)/Electronic dance music/Techno/getting started/04 The Sugarcubes - Motorcrash [Justin Robertson Mix].flac has an ID3v2 tag
Done. Now try playing the file again.
..............
..............
Re-encoding file: /home/nPrev/Music/DJ (LL)/Parties/Ballroom/Disco Swing/01. Unknown - Title 1.flac

flac 1.3.4
Copyright (C) 2000-2009 Josh Coalson, 2011-2016 Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY. This is free software, and you are
welcome to redistribute it under certain conditions. Type `flac' for details.

ERROR: input file /home/nPrev/Music/DJ (LL)/Parties/Ballroom/Disco Swing/01. Unknown - Title 1.flac has an ID3v2 tag
Done. Now try playing the file again.

@uklotzde
Copy link
Contributor

uklotzde commented Mar 1, 2023

This should not be a Mixxx issue. ID3v2 tags in FLAC files are supported as a fallback by most software but not recommended.

https://xiph.org/flac/faq.html#general__tagging

Please consult the official docs when in doubt.

@nPrevail
Copy link
Author

nPrevail commented Mar 1, 2023

I ran Mixxx via terminal so I could see a live log. I reset the Mixxx file data (BPM, waveform, key, and etc.) with the ID3v2 issue, and I received this output:

`Warning [AnalyzerThread 1 #2]: SoundSourceFLAC - FLAC decoding error "STREAM_DECODER_ERROR_STATUS_LOST_SYNC" in file "/home/nPrev/Music/DJ (LL)/Disco/in the zone/09 Always There.flac"
Warning [AnalyzerThread 1 #2]: SoundSourceFLAC - FLAC decoding error "STREAM_DECODER_ERROR_STATUS_LOST_SYNC" in file "/home/nPrev/Music/DJ (LL)/Disco/in the zone/09 Always There.flac"
Warning [AnalyzerThread 1 #2]: SoundSourceFLAC - Unexpected frame index 7442432 > 7434240 while decoding FLAC file "/home/nPrev/Music/DJ (LL)/Disco/in the zone/09 Always There.flac"
Warning [AnalyzerThread 1 #2]: AudioSource - Failed to read sample frames: expected = [7434240 -> 7438336) , actual = [7434240 -> 7434240)

I know we've seen this error before, but I guess this script is unable to fix/re-encode the FLAC file because of the ID3v2 tag:
ERROR: input file /home/nPrev/Music/DJ (LL)/Disco/in the zone/09 Always There.flac **has an ID3v2 tag**

I'll read the general tagging link you sent.

@nPrevail
Copy link
Author

nPrevail commented Mar 1, 2023

The other strange thing too is that this song file 09 Always There.flac came from a compilation of other songs downloaded from Bandcamp, yet this song seems to be the only one affected in the bunch...

This song file plays fine in VLC, but it prevents Mixxx from fully scanning the waveform.

@uklotzde
Copy link
Contributor

uklotzde commented Mar 1, 2023

If the FLAC decoder fails to parse the byte stream from your files with non-standard tags this is still not a Mixxx issue. We can't do anything about it.

@nPrevail
Copy link
Author

nPrevail commented Mar 1, 2023

From what I learned, it's difficult to remove ID3v2 tags, and I did find a script to fix MP3 files (and maybe FLAC), but fortunately, I have backup copies of these songs, and these backups seem to work fine. Just wanted to share the possible solution in case folks accidentally have ID3v2 data in their FLAC files.

script: https://github.com/tillt/CleanID3

@robbert-vdh
Copy link
Contributor

Is the waveform truncated or does it not line up with the actual track (so, it ends up being stretched). I've been seeing a similar issue on the main branch where track waveforms are sometimes compressed like this: (this is deck A doubled to deck B):

image

Usually when it happens when loading a new file, it only happens the first time the file is loading. Reloading it in the same deck afterwards fixes the waveform. There's nothing in the log at the standard verbosity level, so I'll need to check if anything useful shows up in a trace log.

@nPrevail
Copy link
Author

nPrevail commented Mar 7, 2023

It's usually truncated, and also the track stops playing when it reaches the abrupt end (not playing the complete track from beginning to end, but stopping at the gap, like your Deck B's waveform).

After fixing it, the waveform still shows up truncated, but like you said, once I reset and re-analyze the song file, the waveform is analyzed correctly and is no longer truncated. HOWEVER, that only happens after I fix my track corruption, which I had a handful of corrupted FLAC files.

@uklotzde
Copy link
Contributor

uklotzde commented Mar 7, 2023

It's usually truncated, and also the track stops playing when it reaches the abrupt end (not playing the complete track from beginning to end, but stopping at the gap, like your Deck B's waveform).

This is not the case here when you look at the pictures. The waveform is just wrongly scaled, but otherwise complete on both decks. It's the same track, too.

@nPrevail
Copy link
Author

nPrevail commented Mar 7, 2023

This is not the case here when you look at the pictures.

Ah you're right! My apologies for not seeing that the first time.

From my original picture of my post, the song and waveform would instantly cut off and would stop playing. After fixing the corrupt track, it would play. After resetting and analyze the waveform, the waveform would analyze fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants