Indexer is stuck at 100.1% #245

Magneticdud · 2021-04-01T15:01:03Z

Describe the bug
I am not sure how this happened, the folder indexer processed 885 folders out of 884, so it's "stuck" at 100.1%

Then it is blocked to doing this task (which can never reach 100%)

And it can't do other jobs

I tried to stop the docker and restart it but it restarts the indexing job immediately. In the docker logs I didn't see any messages that could explain the problem.

I tried to reset the database, but then when I run the indexing again, the problem repeats. In the indexing log I see it's indexing some folders that should be ignored (.stfolder, .stversions, Picasa)

Environment:

OS: Ubuntu and docker:latest

Magneticdud · 2021-04-01T15:09:37Z

And the job can't be canceled

Magneticdud · 2021-04-01T17:21:01Z

I created another docker container with a totally different set of images and while it still shows "18 out of 17", it wasn't stuck and continued to the next step

I am even more puzzled...

bpatrik · 2021-04-01T17:26:48Z

The 18 out of 17 might be just indexing related UI issue. Can you please provide docker logs for the container? You can also set the log level to 'silly" in the config.json to force all.logs.

…

-- Sorry for being brief, sent from my phone. Magneticdud ***@***.***> ezt írta (időpont: 2021. ápr. 1., Cs 19:21):

I created another docker container with a totally different set of images and while it still shows "18 out of 17", it wasn't stuck and continued to the next step [image: image] <https://user-images.githubusercontent.com/2940598/113330637-41138680-931f-11eb-8803-f70459f18404.png> I am even more puzzled... — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#245 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABZKA5VWG54U7RVPPIGHTT3TGSTQZANCNFSM42HIICTQ> .

stumpylog · 2021-04-01T19:57:06Z

I have also begun experiencing this recently. I'm running a new index and capturing the logs. Lots of pictures, so it'll take a while.

stumpylog · 2021-04-01T20:21:02Z

I've created a gist of the log I see after triggering an index. It seems to be the ER_TRUNCATED_WRONG_VALUE_FOR_FIELD, which keeps being raised for a particular picture (02478_castleofdecay_1920x1080.jpg) over and over.

Maybe that image is what's holding up the indexing? It errors out, but isn't removed from something.

bpatrik · 2021-04-01T20:26:55Z

huh.
I think utf8_bin collation is missing for the keywords for the mysql db. I was wondering the other day, why it is not needed there (I have tests with special characters and run without an issue)

I added it, hope it fixes your issue.

stumpylog · 2021-04-01T21:12:10Z

I changed tags to :nightly-stretch and reran indexing. I still see ER_TRUNCATED_WRONG_VALUE_FOR_FIELD, but the indexing completed anyway. It did appear to only output a few times, not cyclically.

bpatrik · 2021-04-01T21:16:01Z

Pls make sure, that the version you are running was built with d8b3ff0 or later. (It takes about 40 mins to build a release and publish it on docker. )

Also if you send me some logs and corrupted images, I can take a look.

stumpylog · 2021-04-01T21:26:03Z

Ah, might not have that then. Is git revision output in the docker logs or visible in the GUI?

bpatrik · 2021-04-02T08:29:18Z

Hovering the app version in the settings prints details about the build, used commit, etc.

Magneticdud · 2021-04-02T09:18:25Z

i set to silly but i didn't notice anything strange

pigallery2    | 4/1/2021, 5:55:02 PM[SILLY] [DiskManager] scanning directory: Picasa/***Google/Picasa2/cache/feeds
pigallery2    | 4/1/2021, 5:55:02 PM[SILLY] [DiskManager] scanning directory: Picasa/***/Google/Picasa2/temp/LifescapeUpdater
pigallery2    | QueryFailedError: ER_PARSE_ERROR: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ' '', '', DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAU...' at line 1
pigallery2    |     at new QueryFailedError (/app/node_modules/typeorm/error/QueryFailedError.js:11:28)
pigallery2    |     at Query.<anonymous> (/app/node_modules/typeorm/driver/mysql/MysqlQueryRunner.js:170:45)
pigallery2    |     at Query.<anonymous> (/app/node_modules/mysql/lib/Connection.js:526:10)
pigallery2    |     at Query._callback (/app/node_modules/mysql/lib/Connection.js:488:16)
pigallery2    |     at Query.Sequence.end (/app/node_modules/mysql/lib/protocol/sequences/Sequence.js:83:24)
pigallery2    |     at Query.ErrorPacket (/app/node_modules/mysql/lib/protocol/sequences/Query.js:92:8)
pigallery2    |     at Protocol._parsePacket (/app/node_modules/mysql/lib/protocol/Protocol.js:291:23)
pigallery2    |     at Parser._parsePacket (/app/node_modules/mysql/lib/protocol/Parser.js:433:10)
pigallery2    |     at Parser.write (/app/node_modules/mysql/lib/protocol/Parser.js:43:10)
pigallery2    |     at Protocol.write (/app/node_modules/mysql/lib/protocol/Protocol.js:38:16) {
pigallery2    |   code: 'ER_PARSE_ERROR',
pigallery2    |   errno: 1064,
pigallery2    |   sqlMessage: "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ' '', '', DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAU...' at line 1",
pigallery2    |   sqlState: '42000',
pigallery2    |   index: 0,
pigallery2    |   sql: "INSERT INTO `media_entity`(`id`, `name`, `type`, `directoryId`, `metadataCaption`, `metadataCreationdate`, `metadataFilesize`, `metadataKeywords`, `metadataOrientation`, `metadataBitrate`, `metadataDuration`, `metadataSizeWidth`, `metadataSizeHeight`, `metadataCameraDataIso`, `metadataCameraDataModel`, `metadataCameraDataMake`, `metadataCameraDataFstop`, `metadataCameraDataExposure`, `metadataCameraDataFocallength`, `metadataCameraDataLens`, `metadataPositionDataCountry`, `metadataPositionDataState`, `metadataPositionDataCity`, `metadataPositionDataGPSDataLatitude`, `metadataPositionDataGPSDataLongitude`, `metadataPositionDataGPSDataAltitude`) VALUES (DEFAULT, 'Senza titolo10.jpg', 'PhotoEntity', 283, DEFAULT, 1052231094000, 931932, DEFAULT, 1, DEFAULT, DEFAULT, 1783, 1200, , '', '', DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT)",
pigallery2    |   query: 'INSERT INTO `media_entity`(`id`, `name`, `type`, `directoryId`, `metadataCaption`, `metadataCreationdate`, `metadataFilesize`, `metadataKeywords`, `metadataOrientation`, `metadataBitrate`, `metadataDuration`, `metadataSizeWidth`, `metadataSizeHeight`, `metadataCameraDataIso`, `metadataCameraDataModel`, `metadataCameraDataMake`, `metadataCameraDataFstop`, `metadataCameraDataExposure`, `metadataCameraDataFocallength`, `metadataCameraDataLens`, `metadataPositionDataCountry`, `metadataPositionDataState`, `metadataPositionDataCity`, `metadataPositionDataGPSDataLatitude`, `metadataPositionDataGPSDataLongitude`, `metadataPositionDataGPSDataAltitude`) VALUES (DEFAULT, ?, ?, ?, DEFAULT, ?, ?, DEFAULT, ?, DEFAULT, DEFAULT, ?, ?, ?, ?, ?, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT)',
pigallery2    |   parameters: [
pigallery2    |     'Senza titolo10.jpg',
pigallery2    |     'PhotoEntity',
pigallery2    |     283,
pigallery2    |     1052231094000,
pigallery2    |     931932,
pigallery2    |     1,
pigallery2    |     1783,
pigallery2    |     1200,
pigallery2    |     [],
pigallery2    |     '',
pigallery2    |     ''
pigallery2    |   ]
pigallery2    | }
pigallery2    | 4/1/2021, 5:55:08 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:55:08 PM[DEBUG] HEAD /heartbeat 200 2ms
pigallery2    | 4/1/2021, 5:55:13 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:55:18 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 7ms
pigallery2    | 4/1/2021, 5:55:24 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:55:30 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 1ms
pigallery2    | 4/1/2021, 5:55:36 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 3ms
pigallery2    | 4/1/2021, 5:55:42 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:55:48 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 3ms
pigallery2    | 4/1/2021, 5:55:49 PM[DEBUG] HEAD /heartbeat 200 2ms
pigallery2    | 4/1/2021, 5:55:55 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:56:00 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:56:06 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 3ms
pigallery2    | 4/1/2021, 5:56:12 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 3ms
pigallery2    | 4/1/2021, 5:56:18 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:56:24 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 3ms
pigallery2    | 4/1/2021, 5:56:29 PM[DEBUG] HEAD /heartbeat 200 2ms
pigallery2    | 4/1/2021, 5:56:30 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 1ms
pigallery2    | 4/1/2021, 5:56:36 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 3ms
pigallery2    | 4/1/2021, 5:56:42 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 3ms
pigallery2    | 4/1/2021, 5:56:48 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:56:54 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 1ms
pigallery2    | 4/1/2021, 5:57:00 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 3ms
pigallery2    | 4/1/2021, 5:57:06 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:57:09 PM[DEBUG] HEAD /heartbeat 200 2ms
pigallery2    | 4/1/2021, 5:57:12 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 3ms
pigallery2    | 4/1/2021, 5:57:18 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:57:24 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:57:30 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:57:36 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:57:42 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 5ms
pigallery2    | 4/1/2021, 5:57:48 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 3ms
pigallery2    | 4/1/2021, 5:57:49 PM[DEBUG] HEAD /heartbeat 200 1ms
pigallery2    | 4/1/2021, 5:57:54 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:58:00 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:58:26 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:58:29 PM[DEBUG] HEAD /heartbeat 200 1ms
pigallery2    | 4/1/2021, 5:58:31 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:58:36 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:58:41 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:58:46 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:58:51 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:58:56 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:59:01 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 3ms
pigallery2    | 4/1/2021, 5:59:06 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 3ms
pigallery2    | 4/1/2021, 5:59:09 PM[DEBUG] HEAD /heartbeat 200 3ms
pigallery2    | 4/1/2021, 5:59:11 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:59:16 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 3ms
pigallery2    | 4/1/2021, 5:59:22 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 3ms
pigallery2    | 4/1/2021, 5:59:27 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:59:33 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:59:38 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:59:44 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 3ms
pigallery2    | 4/1/2021, 5:59:49 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms
pigallery2    | 4/1/2021, 5:59:50 PM[DEBUG] HEAD /heartbeat 200 2ms
pigallery2    | 4/1/2021, 5:59:55 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 3ms
pigallery2    | 4/1/2021, 6:00:00 PM[VERBOSE] GET /api/admin/jobs/scheduled/progress 200 2ms

The error about "Senzatitolo10.jpg" is repeated another hundred times, and it's one of the first images to be scanned. The Picasa folder, that's the last one to be scanned, it's in the ignore list, should it be scanned?

By the way, I deleted that image and the picasa folder, now it gets stuck in another image

pigallery2    | QueryFailedError: ER_WARN_DATA_OUT_OF_RANGE: Out of range value for column 'metadataCreationdate' at row 1
pigallery2    |     at new QueryFailedError (/app/node_modules/typeorm/error/QueryFailedError.js:11:28)
pigallery2    |     at Query.<anonymous> (/app/node_modules/typeorm/driver/mysql/MysqlQueryRunner.js:170:45)
pigallery2    |     at Query.<anonymous> (/app/node_modules/mysql/lib/Connection.js:526:10)
pigallery2    |     at Query._callback (/app/node_modules/mysql/lib/Connection.js:488:16)
pigallery2    |     at Query.Sequence.end (/app/node_modules/mysql/lib/protocol/sequences/Sequence.js:83:24)
pigallery2    |     at Query.ErrorPacket (/app/node_modules/mysql/lib/protocol/sequences/Query.js:92:8)
pigallery2    |     at Protocol._parsePacket (/app/node_modules/mysql/lib/protocol/Protocol.js:291:23)
pigallery2    |     at Parser._parsePacket (/app/node_modules/mysql/lib/protocol/Parser.js:433:10)
pigallery2    |     at Parser.write (/app/node_modules/mysql/lib/protocol/Parser.js:43:10)
pigallery2    |     at Protocol.write (/app/node_modules/mysql/lib/protocol/Protocol.js:38:16) {
pigallery2    |   code: 'ER_WARN_DATA_OUT_OF_RANGE',
pigallery2    |   errno: 1264,
pigallery2    |   sqlMessage: "Out of range value for column 'metadataCreationdate' at row 1",
pigallery2    |   sqlState: '22003',
pigallery2    |   index: 0,
pigallery2    |   sql: "INSERT INTO `media_entity`(`id`, `name`, `type`, `directoryId`, `metadataCaption`, `metadataCreationdate`, `metadataFilesize`, `metadataKeywords`, `metadataOrientation`, `metadataBitrate`, `metadataDuration`, `metadataSizeWidth`, `metadataSizeHeight`, `metadataCameraDataIso`, `metadataCameraDataModel`, `metadataCameraDataMake`, `metadataCameraDataFstop`, `metadataCameraDataExposure`, `metadataCameraDataFocallength`, `metadataCameraDataLens`, `metadataPositionDataCountry`, `metadataPositionDataState`, `metadataPositionDataCity`, `metadataPositionDataGPSDataLatitude`, `metadataPositionDataGPSDataLongitude`, `metadataPositionDataGPSDataAltitude`) VALUES (DEFAULT, 'DSCN0447.JPG', 'PhotoEntity', 527, DEFAULT, -2211753600000, 294196, DEFAULT, 1, DEFAULT, DEFAULT, 1024, 768, 100, 'E880', 'NIKON', 3.3, 0.005737234652897304, 12.3, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT)",
pigallery2    |   query: 'INSERT INTO `media_entity`(`id`, `name`, `type`, `directoryId`, `metadataCaption`, `metadataCreationdate`, `metadataFilesize`, `metadataKeywords`, `metadataOrientation`, `metadataBitrate`, `metadataDuration`, `metadataSizeWidth`, `metadataSizeHeight`, `metadataCameraDataIso`, `metadataCameraDataModel`, `metadataCameraDataMake`, `metadataCameraDataFstop`, `metadataCameraDataExposure`, `metadataCameraDataFocallength`, `metadataCameraDataLens`, `metadataPositionDataCountry`, `metadataPositionDataState`, `metadataPositionDataCity`, `metadataPositionDataGPSDataLatitude`, `metadataPositionDataGPSDataLongitude`, `metadataPositionDataGPSDataAltitude`) VALUES (DEFAULT, ?, ?, ?, DEFAULT, ?, ?, DEFAULT, ?, DEFAULT, DEFAULT, ?, ?, ?, ?, ?, ?, ?, ?, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT)',
pigallery2    |   parameters: [
pigallery2    |     'DSCN0447.JPG',       'PhotoEntity',
pigallery2    |     527,                  -2211753600000,
pigallery2    |     294196,               1,
pigallery2    |     1024,                 768,
pigallery2    |     100,                  'E880',
pigallery2    |     'NIKON',              3.3,
pigallery2    |     0.005737234652897304, 12.3
pigallery2    |   ]
pigallery2    | }

I noticed that all the pictures in that directory have an invalid creation date in the exif set to 00/00/0000 00:00:00, maybe if the date is clearly invalid, it should be ignored and not be written in the database

I'll try to fix the exif

bpatrik · 2021-04-02T10:43:40Z

Hi,

the first issue is still the ISO is a [] that should be fixed if you pull the latest nightly.

For the second: the creation date is a negative number (the app does not support b.c. :), so ) is in an "illegal" value. Can you send me a photo that produces this error?

bpatrik · 2021-04-02T10:45:18Z

The ignore lists ignores whole folders, you cannot ignore single photos. (if the ignore does not take effect, there might be some issue with the settings.)

Magneticdud · 2021-04-02T11:02:47Z

I found out that any photos imported by the tool that was builtin in windows xp have the exif date set to an invalid value...

Why 15 years ago I was lazy and used that tool instead of taking out the compact flash???

This image has a date that is 00/00/0000 00:00:00 if read with exiftool or -0001/11/30 00:00:00 if read with verexif.com

stumpylog · 2021-04-02T15:10:11Z

I've run an index with the following revisions:

Both log an ER_TRUNCATED_WRONG_VALUE_FOR_FIELD for this photo:

There's another photo which also does it, if more data is needed

bpatrik · 2021-04-02T15:41:33Z

I made an update 3c70ce4, with that the last photo looks good.

Magneticdud · 2021-04-03T08:36:04Z

Hi, looks like all errors are gone, now I get this and it still get stuck:

pigallery2    | QueryFailedError: ER_DUP_ENTRY: Duplicate entry 'saponatoè.jpg-1746' for key 'IDX_332f316a46fcb158ac71210109'
pigallery2    |     at new QueryFailedError (/app/node_modules/typeorm/error/QueryFailedError.js:11:28)
pigallery2    |     at Query.<anonymous> (/app/node_modules/typeorm/driver/mysql/MysqlQueryRunner.js:170:45)
pigallery2    |     at Query.<anonymous> (/app/node_modules/mysql/lib/Connection.js:526:10)
pigallery2    |     at Query._callback (/app/node_modules/mysql/lib/Connection.js:488:16)
pigallery2    |     at Query.Sequence.end (/app/node_modules/mysql/lib/protocol/sequences/Sequence.js:83:24)
pigallery2    |     at Query.ErrorPacket (/app/node_modules/mysql/lib/protocol/sequences/Query.js:92:8)
pigallery2    |     at Protocol._parsePacket (/app/node_modules/mysql/lib/protocol/Protocol.js:291:23)
pigallery2    |     at Parser._parsePacket (/app/node_modules/mysql/lib/protocol/Parser.js:433:10)
pigallery2    |     at Parser.write (/app/node_modules/mysql/lib/protocol/Parser.js:43:10)
pigallery2    |     at Protocol.write (/app/node_modules/mysql/lib/protocol/Protocol.js:38:16) {
pigallery2    |   code: 'ER_DUP_ENTRY',
pigallery2    |   errno: 1062,
pigallery2    |   sqlMessage: "Duplicate entry 'scalposaponatoè.jpg-1746' for key 'IDX_332f316a46fcb158ac71210109'",
pigallery2    |   sqlState: '23000',
pigallery2    |   index: 0,
pigallery2    |   sql: "INSERT INTO `media_entity`(`id`, `name`, `type`, `directoryId`, `metadataCaption`, `metadataCreationdate`, `metadataFilesize`, `metadataKeywords`, `metadataOrientation`, `metadataBitrate`, `metadataDuration`, `metadataSizeWidth`, `metadataSizeHeight`, `metadataCameraDataIso`, `metadataCameraDataModel`, `metadataCameraDataMake`, `metadataCameraDataFstop`, `metadataCameraDataExposure`, `metadataCameraDataFocallength`, `metadataCameraDataLens`, `metadataPositionDataCountry`, `metadataPositionDataState`, `metadataPositionDataCity`, `metadataPositionDataGPSDataLatitude`, `metadataPositionDataGPSDataLongitude`, `metadataPositionDataGPSDataAltitude`) VALUES (DEFAULT, 'scalposaponatoè.jpg', 'PhotoEntity', 1746, DEFAULT, 1049510846000, 964678, DEFAULT, 1, DEFAULT, DEFAULT, 1536, 2048, 200, 'FinePix S5000 ', 'FUJIFILM', 6.3, 0.0016666666666666668, 5.7, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT)",
pigallery2    |   query: 'INSERT INTO `media_entity`(`id`, `name`, `type`, `directoryId`, `metadataCaption`, `metadataCreationdate`, `metadataFilesize`, `metadataKeywords`, `metadataOrientation`, `metadataBitrate`, `metadataDuration`, `metadataSizeWidth`, `metadataSizeHeight`, `metadataCameraDataIso`, `metadataCameraDataModel`, `metadataCameraDataMake`, `metadataCameraDataFstop`, `metadataCameraDataExposure`, `metadataCameraDataFocallength`, `metadataCameraDataLens`, `metadataPositionDataCountry`, `metadataPositionDataState`, `metadataPositionDataCity`, `metadataPositionDataGPSDataLatitude`, `metadataPositionDataGPSDataLongitude`, `metadataPositionDataGPSDataAltitude`) VALUES (DEFAULT, ?, ?, ?, DEFAULT, ?, ?, DEFAULT, ?, DEFAULT, DEFAULT, ?, ?, ?, ?, ?, ?, ?, ?, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT)',
pigallery2    |   parameters: [
pigallery2    |     'scalposaponatoè.jpg',
pigallery2    |     'PhotoEntity',
pigallery2    |     1746,
pigallery2    |     1049510846000,
pigallery2    |     964678,
pigallery2    |     1,
pigallery2    |     1536,
pigallery2    |     2048,
pigallery2    |     200,
pigallery2    |     'FinePix S5000 ',
pigallery2    |     'FUJIFILM',
pigallery2    |     6.3,
pigallery2    |     0.0016666666666666668,
pigallery2    |     5.7
pigallery2    |   ]
pigallery2    | }

I tried to reset the database but the result did not change (the entry had a different number appended to the name after the reset)

There aren't other images with this name in the directory, or other files with the same md5 hash. The image doesn't appear in the duplicates section (which is VERY useful, thanks! Noticed a lot of duplicates that I never noticed before)

Unfortunately this image contains questionable content and can't be shared

bpatrik · 2021-04-04T10:08:37Z

Unfortunately from this I cannot really figure out.
Just guessing that you might have the photo (case insensitive) in a different directory or in a directory with the same name.

You can try switching to sqlite, that usually has less issues. I'm also using that on my rpi4 with a relatively decent performance.

Magneticdud · 2021-04-04T10:46:55Z

in this case i got the image from a friend

he named all the images the same, then added a single letter at the end

like this

imagea.jpg
imageb.jpg
imagec.jpg

then because images were > 26, used accented letters èòà, and maybe I guess it could be imagee.jpg and imageè.jpg are giving the conflict. I'll properly rename them and see if this is the reason

edit: after renaming the files and removed the accented letters, the problem is gone

bpatrik · 2021-04-04T11:36:10Z

most likely that comes from the DB collocation and letters with/without accents counts to be the same when checking if the string is unique.

bpatrik · 2021-05-24T12:49:18Z

I found out that any photos imported by the tool that was builtin in windows xp have the exif date set to an invalid value...

Why 15 years ago I was lazy and used that tool instead of taking out the compact flash???

This image has a date that is 00/00/0000 00:00:00 if read with exiftool or -0001/11/30 00:00:00 if read with verexif.com

Had an other look to this photo.
We plan to switch to exifr that shows the date is Date 1899-11-29T23:00:00.000Z:

That is unfortunate, but there is no way to detect it as it is possible that there is a photo before epoch (1970). I also have some. So I will just support from now on negative timestamps.

Magneticdud changed the title ~~Job scheduler is stuck at 100.1%~~ Indexer is stuck at 100.1% Apr 1, 2021

bpatrik mentioned this issue Apr 1, 2021

Log shows SQL syntax error #244

Closed

bpatrik added a commit that referenced this issue Apr 1, 2021

Setting metadata keywords to use utf8_bin with mysql #245

d8b3ff0

bpatrik added the bug label Apr 1, 2021

bpatrik added a commit that referenced this issue Apr 2, 2021

improving MetaDataLoader to better handle edge cases #245

3c70ce4

bpatrik added this to the v1.8.5 milestone Apr 6, 2021

bpatrik closed this as completed Apr 10, 2021

bpatrik added a commit that referenced this issue May 24, 2021

Supporting date before epoch (1970) #245, #294

4664966

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Indexer is stuck at 100.1% #245

Indexer is stuck at 100.1% #245

Magneticdud commented Apr 1, 2021

Magneticdud commented Apr 1, 2021

Magneticdud commented Apr 1, 2021

bpatrik commented Apr 1, 2021 via email

stumpylog commented Apr 1, 2021

stumpylog commented Apr 1, 2021

bpatrik commented Apr 1, 2021

stumpylog commented Apr 1, 2021

bpatrik commented Apr 1, 2021

stumpylog commented Apr 1, 2021

bpatrik commented Apr 2, 2021

Magneticdud commented Apr 2, 2021

bpatrik commented Apr 2, 2021

bpatrik commented Apr 2, 2021

Magneticdud commented Apr 2, 2021

stumpylog commented Apr 2, 2021

bpatrik commented Apr 2, 2021

Magneticdud commented Apr 3, 2021

bpatrik commented Apr 4, 2021

Magneticdud commented Apr 4, 2021 •

edited

Loading

bpatrik commented Apr 4, 2021

bpatrik commented May 24, 2021 •

edited

Loading

Indexer is stuck at 100.1% #245

Indexer is stuck at 100.1% #245

Comments

Magneticdud commented Apr 1, 2021

Magneticdud commented Apr 1, 2021

Magneticdud commented Apr 1, 2021

bpatrik commented Apr 1, 2021 via email

stumpylog commented Apr 1, 2021

stumpylog commented Apr 1, 2021

bpatrik commented Apr 1, 2021

stumpylog commented Apr 1, 2021

bpatrik commented Apr 1, 2021

stumpylog commented Apr 1, 2021

bpatrik commented Apr 2, 2021

Magneticdud commented Apr 2, 2021

bpatrik commented Apr 2, 2021

bpatrik commented Apr 2, 2021

Magneticdud commented Apr 2, 2021

stumpylog commented Apr 2, 2021

bpatrik commented Apr 2, 2021

Magneticdud commented Apr 3, 2021

bpatrik commented Apr 4, 2021

Magneticdud commented Apr 4, 2021 • edited Loading

bpatrik commented Apr 4, 2021

bpatrik commented May 24, 2021 • edited Loading

Magneticdud commented Apr 4, 2021 •

edited

Loading

bpatrik commented May 24, 2021 •

edited

Loading