v0.1.6 (15.11.2013) - undefined index: filesize

[j-ed]

With the given snapshot the following message is displayed once the scan of the music repository has been finished:

Notice: Undefined index: filesize in /var/www/htdocs/owncloud/apps2/music/3rdparty/getID3/getid3/module.tag.apetag.php on line 24 

[MorrisJobke]

Looks like an getID3 issue. Maybe @JamesHeinrich knows more about it.

@JamesHeinrich I hope it's not importunate to call you to this issue.

[JamesHeinrich]

I'd probably need more context to figure out what's going on. From a quick glance at the getID3 code it seems possible you could get that kind of error if the audio file appears to exist but could not be opened (untested speculation). How is getID3 being called in your application? Is this error specific to one audio file in particular? If so, can you provide a link to that file?

[MorrisJobke]

@JamesHeinrich It's possible that the file isn't readable, because we experience some problems when encryption (within owncloud), which should be transparent to file reads, is enabled. But maybe the file is corrupt.

@j-ed Can you provide the file? Or test it against a the getID3 demo (there is a browse demo in the getID3 package)

[JamesHeinrich]

If the file is corrupt, I'd say it has to be corrupt at the filesystem level, as opposed to being an invalid/unsupported audio/video file: getID3 is reasonably tolerant of invalid-format files, but this error would have to occur after checking that the requested file exists, but after the getid3::openfile function tries (and apparently fails?) to open the file. I'm curious to know the results of your testing: even though I suspect that it's not really a getID3 error, perhaps I can add another error check in there somewhere to catch a situation like this where the file seems to exist but fails to open in this particular manner.

[MorrisJobke]

@JamesHeinrich I will wait for @j-ed (maybe he can send me the file and I can test and debug locally) and then maybe create a pull request on your repo to add the check. Is this ok?

[JamesHeinrich]

Please do.

[j-ed]

@kabum and @JamesHeinrich I'll try my best to identify the corrupted file, unfortunately the log file is not helpful here.

[j-ed]

@kabum I've sent you my debugging result as an email. Let me know if you need further information.

[MorrisJobke]

@j-ed I've had time to look at the file. I just uploaded it to my ownCloud and watched the log. It says

{"app":"music","message":"update - \/05-DJ Bobo-What A Feeling (Radio Version).mp3","level":0,"time":"2013-11-27T20:33:18+00:00"}
{"app":"music","message":"update - mimetype audio\/mpeg","level":0,"time":"2013-11-27T20:33:18+00:00"}
{"app":"music","message":"getID3 error occured","level":0,"time":"2013-11-27T20:34:06+00:00"}
{"app":"music","message":"getID3 error message: Could not find valid MPEG synch before end of file","level":0,"time":"2013-11-27T20:34:06+00:00"}
{"app":"music","message":"extracted metadata - artist: DJ Bobo, album: Chiuahua, title: What A Feeling (Radio Version), track#: 05, year: 2003, mimetype: audio\/mpeg, fileId: ","level":0,"time":"2013-11-27T20:34:06+00:00"}
{"app":"music","message":"addArtistIfNotExist - added - ID: 4","level":0,"time":"2013-11-27T20:34:06+00:00"}
{"app":"music","message":"addAlbumIfNotExist - added - ID: 4","level":0,"time":"2013-11-27T20:34:06+00:00"}
{"app":"music","message":"addTrackIfNotExist - added - ID: 4","level":0,"time":"2013-11-27T20:34:06+00:00"}
{"app":"music","message":"imported entities - artist: 4, album: 4, track: 4","level":0,"time":"2013-11-27T20:34:06+00:00"}

There is a getID3 error Could not find valid MPEG synch before end of file but it is able to extract the wanted metadata and doesn't claim about the undefined index.

I set the log level for my ownCloud in the admin settings to "all" so the debug messages (as posted above) appear.

It's the same for the other file. Are you able to reproduce the error?

[JamesHeinrich]

If you want to send me the file I can tell you more about how or why the file is showing up as corrupt.

[j-ed]

You have new mail.

[JamesHeinrich]

I do? I don't see anything. Please send a sample file and/or link to one to info@getid3.org

[MorrisJobke]

@j-ed Do you have encryption enabled?

[j-ed]

@kabum

1. The undefined index message was displayed on top of the screen right after the scan has been finished. It has not been written to the log file.
2. No, I've not yet enabled encryption on my
   @JamesHeinrich I used the email address given in yout GitHub profile. You should have new mail now.

[JamesHeinrich]

The provided MP3 files are plain, simple, uncorrupted files as far as I can tell. Constant bitrate, no corrupt data between ID3v2 and audio data, ID3v2 is nice and small (2kB). I don't see why getID3 would have any problem with this file.

Is getID3 operating on the full .mp3 file, or are you copying a small segment of it (e.g. over the internet) for analysis?

Is getID3 being called with a call to Analyze() or to AnalyzeString() ?
I ask because both 'filesize' and filepath/filename/filenamepath are set in openfile() which is called by Analyze but not AnalyzeString. But filesize is set in AnalyzeString, so I'm not sure if this line of thinking leads anywhere.

[benediktg]

What can I do to try to reproduce this bug?

[MorrisJobke]

What can I do to try to reproduce this bug?

I forwarded an email to you ;)

[qirtaiba]

This affects me also and I would be happy to help debug.

[paulijar]

@qirtaiba So in what situation do you get this? Is it possibly related to your other report #607?

[ionoci]

Hi, any news on this issue? I got nearly the same message in my log, combined with three other:

[PHP] Error: fread() expects parameter 1 to be resource, null given at /var/www/oc.x77.chickenkiller.com/nextcloud/apps/music/3rdparty/getID3/getid3/getid3.php#2010
[PHP] Error: ftell() expects parameter 1 to be resource, null given at /var/www/oc.x77.chickenkiller.com/nextcloud/apps/music/3rdparty/getID3/getid3/getid3.php#1976
[PHP] Error: fseek() expects parameter 1 to be resource, null given at /var/www/oc.x77.chickenkiller.com/nextcloud/apps/music/3rdparty/getID3/getid3/getid3.php#2053
[PHP] Error: Undefined index: filesize at /var/www/oc.x77.chickenkiller.com/nextcloud/apps/music/3rdparty/getID3/getid3/module.tag.apetag.php#37

[paulijar]

@ionoci Sorry for the delay. But to answer your question, there's no news or progress on this issue.

In your case, the logs indicate that opening some audio file for scanning has failed. Can you tell, which file would that be? Does the scanning break at some point, or is there a file (or many) missing from the scanned library, or shown with incomplete metadata?

I find it likely, that there might have been multiple unrelated issues through the years, which have caused this same error log "Undefined index: filesize". Some of the problems causing this kind of error might well have got fixed already and/or some new error scenarios may have been introduced by new versions of ownCloud/Nextcloud/Music/getID3.

[paulijar]

I now added checking for the file opening: If opening a file for metadata extraction fails, then an error message containing the path of the failed file is logged, and the invalid file handle is not passed on to GetID3 library.

[ionoci]

@paulijar No problem, thanks for the work you are doing.
unfortunately I can not tell which files are causing this issue, there some thousand files.
When will the release with your added check be available?

[paulijar]

@ionoci

When will the release with your added check be available?

I don't know, but there's a good chance that it will happen during this week. This all COVID-19 hassle has given me some spare time which can be invested on this project ;). There are still one or two other things which I would like to implement before making the new release.

[paulijar]

@ionoci The release v0.13.1 with the added check and logging is now available.

[ionoci]

@paulijar I tried the update v0.13.2. The error is still the same, except the line numbers. I can't see which sound files causes the error. I use the nextcloud web log, do I need to use anotherr log?

[paulijar]

@ionoci Okay, if you are still seeing those error messages [PHP] Error: fread() expects parameter 1 to be resource, null given et al, then it clearly indicates that the problem isn't where I expected it to be. But then I have no idea, what is going on.

Those error messages look like they should cause some serious issues, like scanning never finishing. Haven't you observed anything like that?

[infidelus]

I don't know if this helps but, having just set up a Nextcloud server I'm seeing the same errors:

[PHP] Error: Undefined index: filesize at /var/www/nextcloud/apps/music/3rdparty/getID3/getid3/module.tag.apetag.php#41

GET /apps/music/api/album/66/cover
from XXX.XXX.XXX.XXX by infi at 2020-03-28T12:07:24+00:00
[PHP] Error: fseek() expects parameter 1 to be resource, null given at /var/www/nextcloud/apps/music/3rdparty/getID3/getid3/getid3.php#2097

GET /apps/music/api/album/66/cover
from XXX.XXX.XXX.XXX by infi at 2020-03-28T12:07:24+00:00
[PHP] Error: ftell() expects parameter 1 to be resource, null given at /var/www/nextcloud/apps/music/3rdparty/getID3/getid3/getid3.php#2020

GET /apps/music/api/album/66/cover
from XXX.XXX.XXX.XXX by infi at 2020-03-28T12:07:24+00:00
[PHP] Error: fread() expects parameter 1 to be resource, null given at /var/www/nextcloud/apps/music/3rdparty/getID3/getid3/getid3.php#2054

GET /apps/music/api/album/66/cover
from XXX.XXX.XXX.XXX by infi at 2020-03-28T12:07:24+00:00

For info, my music is all on a read-only NFS share. It looks to have all imported from a quick glance, but I do have about 14000 songs so can't be 100% certain on that front.

[paulijar]

@infidelus Thanks for your input. Your log tells us that the problem has happened when trying to extract embedded cover image for an album. Do you see album covers missing in the Albums view?

Your log also shows the ID of the album where the problem happened; it's 66. Now, if you can run SQL commands on your database, you can find the file where the Music app tried to read the cover from:

SELECT path FROM oc_music_albums INNER JOIN oc_filecache ON cover_file_id=fileid WHERE id=66;

[infidelus]

Thanks for that.

I installed pgAdmin and ran that command and it came back with a result. I then logged into my Nextcloud instance and went to Music and found the related file, for which the (low quality) cover is displayed.

Probably not particularly relevant, but It's actually an older mp3. I've been going through and updating all the metadata for my music recently using puddletag and data from Discogs & Musicbrainz.

The file in question hasn't been updated yet but does contain enough metadata for Music to display it.

[ionoci]

@paulijar The scan works fine, the result is correct. I started a scan over all ~9000 files and it worked well. Only the error messages in the log are irritating me.

[paulijar]

@infidelus As you now have located the file related to the logged errors, could you test opening the details pane for this track on the Music app? When you do this, the metadata is extracted directly from the file, without utilizing any previously indexed data from the database. Then check the logs, if similar errors again got logged.

With this test, we get to know, if the problem happens every time that particular file is parsed, or only sporadically or depending on the situation.

[infidelus]

Yes, the errors occur again when I open the details pane.

{"reqId":"zfCDRGyJioIPuRdJJ9Qc","level":3,"time":"2020-04-01T19:25:50+01:00","remoteAddr":"XXX.XXX.XXX.XXX","user":"infi","app":"PHP","method":"GET","url":"/apps/music/api/file/1156/details","message":"fread() expects parameter 1 to be resource, null given at /var/www/nextcloud/apps/music/3rdparty/getID3/getid3/getid3.php#2054","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36","version":"18.0.3.0","id":"5e84dcb86c78c"}

Etc ...

[paulijar]

@infidelus Okay great, then we are making progress. Could you maybe share this one file with me (e.g. by email)?

[infidelus]

Email sent

[paulijar]

@infidelus Thanks for the sample file. When testing with PHP7.1 and Nextcloud 14, I saw no errors in the log when opening the details pane for this file. Then, I switched to PHP7.4 and Nextcloud 18, and now I could see the Undefined index: filesize ... error appearing to the log. I think that the PHP version is the relevant factor here, because 7.4 is more strict on the use of invalid array indices.

I still didn't see any errors about fread() or fseek(), though. I don't know, what is causing this difference.

Anyway, I managed to track down the Undefined index: filesize error within the getID3 library. The significant factor here seems to be that your sample file had lyrics3 tag set. The getID3 lib does not handle such files quite right, when the library is being used the way we use it. I need to report this to getID3 once I have time to write a coherent description about my findings.

[infidelus]

Thanks for looking. For info I'm using PHP 7.2 on Nextcloud 18 and this script that's mentioned in the Nextcloud docs.

php -v

PHP 7.2.24-0ubuntu0.18.04.3 (cli) (built: Feb 11 2020 15:55:52) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
with Zend OPcache v7.2.24-0ubuntu0.18.04.3, Copyright (c) 1999-2018, by Zend Technologies

[JamesHeinrich]

Anyway, I managed to track down the Undefined index: filesize error within the getID3 library. The significant factor here seems to be that your sample file had lyrics3 tag set. The getID3 lib does not handle such files quite right, when the library is being used the way we use it. I need to report this to getID3 once I have time to write a coherent description about my findings.

@paulijar: Please do. Your analysis along with a relevant sample file would be welcome.

[paulijar]

@infidelus @ionoci Music app v0.14.0 was just released and it contains an updated (development) version of getID3. For me, it fixes those Undefined index: filesize errors getting logged. It would be great if you could give it a try and report if you are still seeing any errors.

[infidelus]

Installed the update and opened the file that was previously a problem. No new errors showing in the logs so looks good. Thanks Pauli :)

[ionoci]

perfect! no errors at all. thanks a lot pauli. I think this really old issue may be closed now :) 👍

[paulijar]

Great, thanks for testing!