Use of UTC for video files

[mcwieger]

Video files do not appear to use UTC, but seem to display the GMT timezone value.

I set the following (Quicktime) tags to make sure all possible dates used for videos are correct:

[System]        FileModifyDate                  : 2023:04:21 18:53:30+02:00
[System]        FileAccessDate                  : 2023:07:27 19:58:17+02:00
[System]        FileCreateDate                  : 2023:04:21 18:53:30+02:00
[QuickTime]     CreateDate                      : 2023:04:21 18:53:30+02:00
[QuickTime]     ModifyDate                      : 2023:04:21 18:53:30+02:00
[ItemList]      ContentCreateDate               : 2023:04:21 18:53:30+02:00
[Track1]        TrackCreateDate                 : 2023:04:21 18:53:30+02:00
[Track1]        TrackModifyDate                 : 2023:04:21 18:53:30+02:00
[Track1]        MediaCreateDate                 : 2023:04:21 18:53:30+02:00
[Track1]        MediaModifyDate                 : 2023:04:21 18:53:30+02:00
[Track2]        TrackCreateDate                 : 2023:04:21 18:53:30+02:00
[Track2]        TrackModifyDate                 : 2023:04:21 18:53:30+02:00
[Track2]        MediaCreateDate                 : 2023:04:21 18:53:30+02:00
[Track2]        MediaModifyDate                 : 2023:04:21 18:53:30+02:00

All times are set to 18:53:30 with my timezone. However, date in PiGallery2 shows 16:53:30. Can UTC be implemented for video files as it was done for pictures in #480 and #469?

Environment

• OS: Synology DSM6.2
• Browser: Chrome

Used app version

• Docker Nightly Alpine v1.9.6-nightly

[bpatrik]

I think the problem lies with the metadata parsing part: The creationDate is not set correctly most likely here:

pigallery2/src/backend/model/threading/MetadataLoader.ts

Lines 20 to 93 in 3e6bb7b

	public static loadVideoMetadata(fullPath: string): Promise<VideoMetadata> {
	return new Promise<VideoMetadata>((resolve) => {
	const metadata: VideoMetadata = {
	size: {
	width: 1,
	height: 1,
	},
	bitRate: 0,
	duration: 0,
	creationDate: 0,
	fileSize: 0,
	fps: 0,
	};
	try {
	const stat = fs.statSync(fullPath);
	metadata.fileSize = stat.size;
	metadata.creationDate = stat.mtime.getTime();
	} catch (err) {
	// ignoring errors
	}
	try {
	ffmpeg(fullPath).ffprobe((err: any, data: FfprobeData) => {
	if (!!err || data === null || !data.streams[0]) {
	return resolve(metadata);
	}
	try {
	for (const stream of data.streams) {
	if (stream.width) {
	metadata.size.width = stream.width;
	metadata.size.height = stream.height;
	if (
	Utils.isInt32(parseInt('' + stream.rotation, 10)) &&
	(Math.abs(parseInt('' + stream.rotation, 10)) / 90) % 2 === 1
	) {
	// noinspection JSSuspiciousNameCombination
	metadata.size.width = stream.height;
	// noinspection JSSuspiciousNameCombination
	metadata.size.height = stream.width;
	}
	if (
	Utils.isInt32(Math.floor(parseFloat(stream.duration) * 1000))
	) {
	metadata.duration = Math.floor(
	parseFloat(stream.duration) * 1000
	);
	}
	if (Utils.isInt32(parseInt(stream.bit_rate, 10))) {
	metadata.bitRate = parseInt(stream.bit_rate, 10) || null;
	}
	if (Utils.isInt32(parseInt(stream.avg_frame_rate, 10))) {
	metadata.fps = parseInt(stream.avg_frame_rate, 10) || null;
	}
	metadata.creationDate =
	Date.parse(stream.tags.creation_time) ||
	metadata.creationDate;
	break;
	}
	}
	// eslint-disable-next-line no-empty
	} catch (err) {
	}
	metadata.creationDate = metadata.creationDate || 0;
	return resolve(metadata);
	});
	} catch (e) {
	return resolve(metadata);
	}
	});
	}

My guess would be that metadata.creationDate = stat.mtime.getTime(); is not using the right timezone maybe.

[mcwieger]

This is what Phil Harvey says: https://exiftool.org/forum/index.php?topic=15052.0

The Quicktime creation date indeed is not using the right timezone. If that field can be made to use UTC, the time will be correct.

[bpatrik]

Ahh so its Quicktime specific. Can you send me a file with what I can reproduce? Like do you have the same issue on these files (random Google search): https://file-examples.com/index.php/sample-video-files/sample-mov-files-download/

[mcwieger]

mov_sample_non-UTC.2023-01-01_12-00-00.mov

mov_sample_UTC.2023-01-01_12-00-00.mov

mp4_sample_non-UTC.2023-01-01_12-00-00.mp4

mp4_sample_UTC.2023-01-01_12-00-00.mp4

I've attached four files:

• MOV processed with EXIFTool as UTC
• MOV processed with EXIFTool as non-UTC
• MP4 processed with EXIFTool as UTC
• MP4 processed with EXIFTool as non-UTC

UTC was processed as follows (to show which fields I updated):
exiftool.exe -P -v2 "-quicktime:CreateDate<FileModifyDate" "-quicktime:ModifyDate<FileModifyDate" "-quicktime:TrackCreateDate<FileModifyDate" "-quicktime:TrackModifyDate<FileModifyDate" "-quicktime:MediaCreateDate<FileModifyDate" "-quicktime:MediaModifyDate<FileModifyDate" "-quicktime:ContentCreateDate<FileModifyDate" <filename>

Non-UTC was processed as follows (to show which fields I updated):
exiftool.exe -P -v2 "-quicktime:CreateDate<FileModifyDate" "-quicktime:ModifyDate<FileModifyDate" "-quicktime:TrackCreateDate<FileModifyDate" "-quicktime:TrackModifyDate<FileModifyDate" "-quicktime:MediaCreateDate<FileModifyDate" "-quicktime:MediaModifyDate<FileModifyDate" "-quicktime:ContentCreateDate<FileModifyDate" -api QuickTimeUTC=1 <filename>

Results in my instance:

• MOV processed with EXIFTool as UTC - incorrect (one hour early)
• MOV processed with EXIFTool as non-UTC - correct
• MP4 processed with EXIFTool as UTC - incorrect (one hour early)
• MP4 processed with EXIFTool as non-UTC - correct

[bpatrik]

a - mov_sample_non-UTC.2023-01-01_12-00-00
b - mov_sample_UTC.2023-01-01_12-00-00
c - mp4_sample_non-UTC.2023-01-01_12-00-00.mp4
d - mp4_sample_UTC.2023-01-01_12-00-00.mp4

What Windows sees:

what pigallery2 sees:

------------a.mov------------------
raw data:
{
  index: 0,
  codec_name: 'h264',
  codec_long_name: 'H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10',
  profile: 'High',
  codec_type: 'video',
  codec_time_base: '1/60',
  codec_tag_string: 'avc1',
  codec_tag: '0x31637661',
  width: 1920,
  height: 1080,
  coded_width: 1920,
  coded_height: 1088,
  has_b_frames: 2,
  sample_aspect_ratio: '1:1',
  display_aspect_ratio: '16:9',
  pix_fmt: 'yuv420p',
  level: 40,
  color_range: 'unknown',
  color_space: 'unknown',
  color_transfer: 'unknown',
  color_primaries: 'unknown',
  chroma_location: 'left',
  field_order: 'unknown',
  timecode: 'N/A',
  refs: 1,
  is_avc: 'true',
  nal_length_size: 4,
  id: 'N/A',
  r_frame_rate: '30/1',
  avg_frame_rate: '30/1',
  time_base: '1/15360',
  start_pts: 0,
  start_time: 0,
  duration_ts: 461322,
  duration: 30.033984,
  bit_rate: 447244,
  max_bit_rate: 'N/A',
  bits_per_raw_sample: 8,
  nb_frames: 901,
  nb_read_frames: 'N/A',
  nb_read_packets: 'N/A',
  tags: {
    creation_time: '2023-01-01T12:00:00.000000Z',
    language: 'eng',
    handler_name: 'DataHandler',
    encoder: 'Lavc57.16.101 libx264'
  },
  disposition: {
    default: 1,
    dub: 0,
    original: 0,
    comment: 0,
    lyrics: 0,
    karaoke: 0,
    forced: 0,
    hearing_impaired: 0,
    visual_impaired: 0,
    clean_effects: 0,
    attached_pic: 0,
    timed_thumbnails: 0
  }
}
extracted:
{
  size: { width: 1920, height: 1080 },
  bitRate: 447244,
  duration: 30033,
  creationDate: 1672574400000,
  fileSize: 2247292,
  fps: 30
}
------------b.mov------------------
raw data:
{
  index: 0,
  codec_name: 'h264',
  codec_long_name: 'H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10',
  profile: 'High',
  codec_type: 'video',
  codec_time_base: '1/60',
  codec_tag_string: 'avc1',
  codec_tag: '0x31637661',
  width: 1920,
  height: 1080,
  coded_width: 1920,
  coded_height: 1088,
  has_b_frames: 2,
  sample_aspect_ratio: '1:1',
  display_aspect_ratio: '16:9',
  pix_fmt: 'yuv420p',
  level: 40,
  color_range: 'unknown',
  color_space: 'unknown',
  color_transfer: 'unknown',
  color_primaries: 'unknown',
  chroma_location: 'left',
  field_order: 'unknown',
  timecode: 'N/A',
  refs: 1,
  is_avc: 'true',
  nal_length_size: 4,
  id: 'N/A',
  r_frame_rate: '30/1',
  avg_frame_rate: '30/1',
  time_base: '1/15360',
  start_pts: 0,
  start_time: 0,
  duration_ts: 461322,
  duration: 30.033984,
  bit_rate: 447244,
  max_bit_rate: 'N/A',
  bits_per_raw_sample: 8,
  nb_frames: 901,
  nb_read_frames: 'N/A',
  nb_read_packets: 'N/A',
  tags: {
    creation_time: '2023-01-01T11:00:00.000000Z',
    language: 'eng',
    handler_name: 'DataHandler',
    encoder: 'Lavc57.16.101 libx264'
  },
  disposition: {
    default: 1,
    dub: 0,
    original: 0,
    comment: 0,
    lyrics: 0,
    karaoke: 0,
    forced: 0,
    hearing_impaired: 0,
    visual_impaired: 0,
    clean_effects: 0,
    attached_pic: 0,
    timed_thumbnails: 0
  }
}
extracted:
{
  size: { width: 1920, height: 1080 },
  bitRate: 447244,
  duration: 30033,
  creationDate: 1672570800000,
  fileSize: 2247292,
  fps: 30
}
------------d.mp4------------------
raw data:
{
  index: 0,
  codec_name: 'h264',
  codec_long_name: 'H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10',
  profile: 'Baseline',
  codec_type: 'video',
  codec_time_base: '1/60',
  codec_tag_string: 'avc1',
  codec_tag: '0x31637661',
  width: 848,
  height: 480,
  coded_width: 848,
  coded_height: 480,
  has_b_frames: 0,
  sample_aspect_ratio: 'N/A',
  display_aspect_ratio: 'N/A',
  pix_fmt: 'yuv420p',
  level: 31,
  color_range: 'tv',
  color_space: 'bt709',
  color_transfer: 'bt709',
  color_primaries: 'bt709',
  chroma_location: 'left',
  field_order: 'unknown',
  timecode: 'N/A',
  refs: 1,
  is_avc: 'true',
  nal_length_size: 4,
  id: 'N/A',
  r_frame_rate: '30/1',
  avg_frame_rate: '30/1',
  time_base: '1/600',
  start_pts: 0,
  start_time: 0,
  duration_ts: 7740,
  duration: 12.9,
  bit_rate: 1637415,
  max_bit_rate: 'N/A',
  bits_per_raw_sample: 8,
  nb_frames: 387,
  nb_read_frames: 'N/A',
  nb_read_packets: 'N/A',
  side_data_type: 'Display Matrix',
  displaymatrix: '',
  rotation: '-90',
  tags: {
    rotate: '90',
    creation_time: '2023-01-01T11:00:00.000000Z',
    language: 'und'
  },
  disposition: {
    default: 1,
    dub: 0,
    original: 0,
    comment: 0,
    lyrics: 0,
    karaoke: 0,
    forced: 0,
    hearing_impaired: 0,
    visual_impaired: 0,
    clean_effects: 0,
    attached_pic: 0,
    timed_thumbnails: 0
  }
}
extracted:
{
  size: { width: 480, height: 848 },
  bitRate: 1637415,
  duration: 12900,
  creationDate: 1672570800000,
  fileSize: 2746160,
  fps: 30
}
------------c.mp4------------------
raw data:
{
  index: 0,
  codec_name: 'h264',
  codec_long_name: 'H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10',
  profile: 'Baseline',
  codec_type: 'video',
  codec_time_base: '1/60',
  codec_tag_string: 'avc1',
  codec_tag: '0x31637661',
  width: 848,
  height: 480,
  coded_width: 848,
  coded_height: 480,
  has_b_frames: 0,
  sample_aspect_ratio: 'N/A',
  display_aspect_ratio: 'N/A',
  pix_fmt: 'yuv420p',
  level: 31,
  color_range: 'tv',
  color_space: 'bt709',
  color_transfer: 'bt709',
  color_primaries: 'bt709',
  chroma_location: 'left',
  field_order: 'unknown',
  timecode: 'N/A',
  refs: 1,
  is_avc: 'true',
  nal_length_size: 4,
  id: 'N/A',
  r_frame_rate: '30/1',
  avg_frame_rate: '30/1',
  time_base: '1/600',
  start_pts: 0,
  start_time: 0,
  duration_ts: 7740,
  duration: 12.9,
  bit_rate: 1637415,
  max_bit_rate: 'N/A',
  bits_per_raw_sample: 8,
  nb_frames: 387,
  nb_read_frames: 'N/A',
  nb_read_packets: 'N/A',
  side_data_type: 'Display Matrix',
  displaymatrix: '',
  rotation: '-90',
  tags: {
    rotate: '90',
    creation_time: '2023-01-01T12:00:00.000000Z',
    language: 'und'
  },
  disposition: {
    default: 1,
    dub: 0,
    original: 0,
    comment: 0,
    lyrics: 0,
    karaoke: 0,
    forced: 0,
    hearing_impaired: 0,
    visual_impaired: 0,
    clean_effects: 0,
    attached_pic: 0,
    timed_thumbnails: 0
  }
}
extracted:
{
  size: { width: 480, height: 848 },
  bitRate: 1637415,
  duration: 12900,
  creationDate: 1672574400000,
  fileSize: 2746160,
  fps: 30
}

Here:

pigallery2/src/backend/model/threading/MetadataLoader.ts

Lines 20 to 93 in 3e6bb7b

	public static loadVideoMetadata(fullPath: string): Promise<VideoMetadata> {
	return new Promise<VideoMetadata>((resolve) => {
	const metadata: VideoMetadata = {
	size: {
	width: 1,
	height: 1,
	},
	bitRate: 0,
	duration: 0,
	creationDate: 0,
	fileSize: 0,
	fps: 0,
	};
	try {
	const stat = fs.statSync(fullPath);
	metadata.fileSize = stat.size;
	metadata.creationDate = stat.mtime.getTime();
	} catch (err) {
	// ignoring errors
	}
	try {
	ffmpeg(fullPath).ffprobe((err: any, data: FfprobeData) => {
	if (!!err || data === null || !data.streams[0]) {
	return resolve(metadata);
	}
	try {
	for (const stream of data.streams) {
	if (stream.width) {
	metadata.size.width = stream.width;
	metadata.size.height = stream.height;
	if (
	Utils.isInt32(parseInt('' + stream.rotation, 10)) &&
	(Math.abs(parseInt('' + stream.rotation, 10)) / 90) % 2 === 1
	) {
	// noinspection JSSuspiciousNameCombination
	metadata.size.width = stream.height;
	// noinspection JSSuspiciousNameCombination
	metadata.size.height = stream.width;
	}
	if (
	Utils.isInt32(Math.floor(parseFloat(stream.duration) * 1000))
	) {
	metadata.duration = Math.floor(
	parseFloat(stream.duration) * 1000
	);
	}
	if (Utils.isInt32(parseInt(stream.bit_rate, 10))) {
	metadata.bitRate = parseInt(stream.bit_rate, 10) || null;
	}
	if (Utils.isInt32(parseInt(stream.avg_frame_rate, 10))) {
	metadata.fps = parseInt(stream.avg_frame_rate, 10) || null;
	}
	metadata.creationDate =
	Date.parse(stream.tags.creation_time) ||
	metadata.creationDate;
	break;
	}
	}
	// eslint-disable-next-line no-empty
	} catch (err) {
	}
	metadata.creationDate = metadata.creationDate || 0;
	return resolve(metadata);
	});
	} catch (e) {
	return resolve(metadata);
	}
	});
	}

The app uses tags.creation_time to get the creation time. Based on the raw data that I get from ffprobe, it seems that the creation date does not have any time zone value.

[mcwieger]

Not sure what the call to action is.

Does it mean PG2 should be updated to reflect the timezone? Or to use a different timefield? Up to you of course.

Or should I update my date/times differently with Exiftool (I think with UTC is the best way, but I can be flexible).

[bpatrik]

I gave it some tought since my last comment. We can decide how to understand the timezoneless date communing from ffprobe. I need to check what Date.parse does, but I assume it does in local time. I think that should be rather in UTC. If I understand correctly that is basically what is this request about. The othe approach is to dedust the idea of using exiftool in the background and outsource the problem:) The problem with that the last time we tried it was extremely slowly compared to what we have. See: now #294

…

-- Sorry for being brief, sent from my phone.

On Fri, 4 Aug 2023, 09:52 mcwieger, ***@***.***> wrote: Not sure what the call to action is. Does it mean PG2 should be updated to reflect the timezone? Or to use a different timefield? Up to you of course. Or should I update my date/times differently with Exiftool (I think with UTC is the best way, but I can be flexible). — Reply to this email directly, view it on GitHub <#682 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABZKA5VNWFXHE5HQMBOB623XTSS2ZANCNFSM6AAAAAA22OLLXI> . You are receiving this because you commented.Message ID: ***@***.***>

[bpatrik]

I think #710 helps with this issue. Can you check?

I rerun my little tests with some better explanation.
Here it is:

------------a.mov------------------
--------------raw data------------
container: {
  major_brand: 'qt  ',
  minor_version: '512',
  compatible_brands: 'qt  ',
  creation_time: '2023-01-01T12:00:00.000000Z',
  encoder: 'Lavf57.19.100',
  date: '2023-01-01T12:00:00+0100'
} 
stream: {
  creation_time: '2023-01-01T12:00:00.000000Z',
  language: 'eng',
  handler_name: 'DataHandler',
  encoder: 'Lavc57.16.101 libx264'
}
------------app found this---------
extracted time: 2023-01-01T12:00:00.000Z (same as: 1672574400000 timestamp)
{
  size: { width: 1920, height: 1080 },
  bitRate: 588084,
  duration: 30033,
  creationDate: 1672574400000,
  fileSize: 2247292,
  fps: 30
}
------------b.mov------------------
--------------raw data------------
container: {
  major_brand: 'qt  ',
  minor_version: '512',
  compatible_brands: 'qt  ',
  creation_time: '2023-01-01T11:00:00.000000Z',
  encoder: 'Lavf57.19.100',
  date: '2023-01-01T12:00:00+0100'
}
stream: {
  creation_time: '2023-01-01T11:00:00.000000Z',
  language: 'eng',
  handler_name: 'DataHandler',
  encoder: 'Lavc57.16.101 libx264'
}
------------app found this---------
extracted time: 2023-01-01T11:00:00.000Z (same as: 1672570800000 timestamp)
{
  size: { width: 1920, height: 1080 },
  bitRate: 588084,
  duration: 30033,
  creationDate: 1672570800000,
  fileSize: 2247292,
  fps: 30
}
------------d.mp4------------------
--------------raw data------------
container: {
  major_brand: 'mp42',
  minor_version: '0',
  compatible_brands: 'mp42isom',
  creation_time: '2023-01-01T11:00:00.000000Z',
  date: '2023-01-01T11:00:00Z',
  'date-eng': '2023-01-01T11:00:00Z'
}
stream: {
  rotate: '90',
  creation_time: '2023-01-01T11:00:00.000000Z',
  language: 'und'
}
------------app found this---------
extracted time: 2023-01-01T11:00:00.000Z (same as: 1672570800000 timestamp)
{
  size: { width: 480, height: 848 },
  bitRate: 1695587,
  duration: 12900,
  creationDate: 1672570800000,
  fileSize: 2746160,
  fps: 30
}
------------c.mp4------------------
--------------raw data------------
container: {
  major_brand: 'mp42',
  minor_version: '0',
  compatible_brands: 'mp42isom',
  creation_time: '2023-01-01T11:00:00.000000Z',
  date: '2023-01-01T11:00:00Z',
  'date-eng': '2023-01-01T11:00:00Z'
}
stream: {
  rotate: '90',
  creation_time: '2023-01-01T12:00:00.000000Z',
  language: 'und'
}
------------app found this---------
extracted time: 2023-01-01T11:00:00.000Z (same as: 1672570800000 timestamp)
{
  size: { width: 480, height: 848 },
  bitRate: 1695587,
  duration: 12900,
  creationDate: 1672570800000,
  fileSize: 2746160,
  fps: 30
}

Based on this the app reads the best time it can read. At least I do not see any better creation date that it could read.

[mcwieger]

Appreciate the follow-up!

Not sure what/how to test. Is there a new field you're taking the date from? In that case, which field should I include in my Exiftool command so it also gets the correct UTC date?

[bpatrik]

I would recommend not changing the date at all as the app assumes that the date can be in any timezone and then display it in UTC. If you change the date and set the timezone to +0000, there is not much the app can do. It can only show that date as it shown in the previous post. Yes there is a change where the date is from. Earlier it was only from the first stream. Now it's checks both the stream and the container.

…

-- Sorry for being brief, sent from my phone.

On Thu, 14 Sept 2023, 09:54 mcwieger, ***@***.***> wrote: Appreciate the follow-up! Not sure what/how to test. Is there a new field you're taking the date from? In that case, which field should I include in my Exiftool command so it also gets the correct UTC date? — Reply to this email directly, view it on GitHub <#682 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABZKA5XYX5RDFVBUF7SZKULX2KZ3TANCNFSM6AAAAAA22OLLXI> . You are receiving this because you commented.Message ID: ***@***.***>

[bpatrik]

I will close this due to inactivity. (a bug cleanup ahead of the next release) Feel free to reopen if you think otherwise.

[gschuager]

I am facing this same issue: I have both photos and videos (mp4) taken with the same device and the sorting provided by PG2 is incorrect in the same way as described above.

It seems as if PG2 is mixing date/time from photos (with time zone info) with UTC date/time from videos (interpreting these as local)

The only way I found to make the files appear in the correct order is to update the mp4 tags with exiftool without using the QuickTimeUTC=1 flag, but this leave the files with local date/time in those tags where AFAIK they are intended to contain UTC timestamps.

(BTW, thanks very much for PiGallery2)