❮ 2024-09-06

Fun with Image-Metadata

Update 09.09.2024

I learned two more things that let me update this post:

The correct Exif Tag for specifying Timezones

I used "TimeZoneOffset" in this post, but the official Exif-Tag that got introduced in Exif 2.31 was "OffsetTime, "TimeZoneOffset" was entirely custom. I ran a script to migrate to this tag.

My camera (Fuji XT-5) does not support automatic daylight saving switching

It has a toggle button for it but you have to think about it automatically.

TZ Settings (GMT+0 is wrong here, should be +1)

Of course I did not switch the "Daylight Savings" button off in winter, so everything was an hour off (22:00 instead of 21:00) as long as official time was GMT+1 in Germany.

Under the hood, if you switch the button on, it just adds 1 hour to the DateTimeOriginal and TimeZoneIdentifier, so 21:00 GMT+1 becomes 22:00 GMT+2 in the metadata.

So I now need to remember to toggle this switch every time daylight saving time changes, and of course also change the timezone entirely if I travel.

Capture One Strips OffsetTime from Metadata

According to this Reddit Post Capture one does not support all these fancy new tags and just silently strips the OffsetTime Tag when exporting jpegs. That kinda sucks.

I never noticed that because I never mixed my photos with media from others, so nearly all my exported photos just have wrong dates.

Fixing this with setting GPS tags only works inside the interval daylight saving time was correctly set, so for me, I also had to shift the DateTimeOriginal by one hour when I took the pictures in winter.

This in combination with my inability to specify daylight saving times made things ... complicated.

To fix this issue (even the newest, 300 Euro "Pro" Version of Capture One has it, I daily drive Vers. 22) I wrote a python script that transfers the OffsetTime Tag over to the exported JPEGs, I just have to think about that every time I export new images.

Time-Travel-Timeline

A couple of years ago I was on a vacation with 4 of my friends in Scotland. We all took a lot of photos and videos with several devices, Smartphones, a GoPro, a Dashcam and an actual Camera.

Last week I tried out Immich, a media management solution in the web browser. One feature of it is a chronological timeline, so I could finally create an album and see all pictures and videos from everyone in chronological order, or at least I thought i could.

But when I was showing the album to my Grandma the experience was more like watching a movie about time traveling. Time-wise photos were scattered all over the place and going through them was a really unpleasant and confusing experience:

  • Pictures of an airport/planes
  • Pictures of us inside the plane
  • Again pictures of the airport and the plane
  • Us at the destination airport
  • Us inside the plane again

Turns out that some image-timestamps were wrong, and the devices we used behaved all differently when changing timezones. Also Immich interprets the Metadata differently based on how much of it is available, since the Exif-Format has some quirks that makes it hard to reliably determine the "correct" time.

So I wrote down everything nice to know from the journey of fixing the timeline.

My Goal

Exif

The "Exchangeable Image File Format" is a (mostly) standardized format to save metadata about an image, video or sound within the file itself inside a special data-block.

Mostly data about the camera, focal-length, camera-model, iso-speed, aperture, shutter-speed is saved, but also gps data and datetime information.

The latter has a serious limitation, inside the "DateTime", "DateTimeOriginal", and "DateTimeDigitized" Tags, only the local time is saved to the picture being taken, there is no extra field to specify the timezone, so there is no way to determine when exactly a photo got taken relative to a reference-timezone, like UTC.

Additional Tags got introduced in Exif-Version 2.31 (OffsetTime, OffsetTimeOriginal, OffsetTimeDigitized), but not every tool is able to interpret these.

Observations

Our Hardware

We used several different devices that all enriched their files with a different amount of tags, based on their capabilities (GPS, Able to Update Local Time when going to a different timezone)

Except our smartphones, no device was able to adjust to the new Timezone automatically, so only very limited information was available out of the box for Immich to create a meaningful timeline.

Immich/Exiftools Timeline-Generation

Immich uses Exiftool-Vendored to extract metadata from files. It reads it and saves in its own db, but it is able to refetch metadata from images and so called "Sidecars" at any time.

After generating the timeline for the first time, the time-offsets from the different devices looked like this:

'Delay' of an image or video on the timeline in hours

Exiftool firstly tries to align a photos DateTimeOriginal to UTC, and that is only possible if a TimeZoneOffset, or GPS Tag is present.

Of all devices that we had, only my iPhone provided everything needed out of the box, so Exiftool had to make best effort guesses for images from every other one, and because it only knew the local time, it could do nothing else than convert the local time to UTC.

That had all sorts of implications for our Timeline:

Having all there different configurations and data-quality, plus 2 timezone-switches made it really hard for me at first to get a clear picture on what happened.

But after some time, the solution was clear, I had to either provide GPS-Tags or a Timezone-Offset Tag to every Picture and Video that doesn't have the required data.

Fixes

Xiaomi

For the pictures, I went with the Timezone-Offset fix, since I had no idea where some picture were physically taken. Also Immich is able to interpret this new tag, so no worries there.

With Exiftool that was kinda easy, but the -P (preserve modification date) broke Nextcloud Sync. I used +2 for pictures taken in Germany and +1 for pictures taken in Scotland.

exiftool -P -overwrite_original -TimeZoneOffset=+1 IMG_20220925_181539.jpg

For some reason I could not update certain images:

GPS Error

But FAQ Section helped getting it to work

Fujifilm

Here I tried the "Timeshift" command to "switch" the local datetime from GMT+1 to GMT+2 for pictures taken in Germany, that command adds one hour to all present, time-related tags.

I wanted to do it that way because every other picture also has the "correct" datetime to the current timezone we were in, and so everything stays consistent.

Together with the GPS Data Immich could determine the correct position inside the timeline, no Timezone-Offset needed.

exiftool -P -overwrite_original "-AllDates+=0:00:00 01:00:00"

iPhone

No fixes needed

Videos

With video-files, Exif works a little different, as it doesn't offer the same tags as for photos, for example there is no TimezoneOffset-Tag:

➜  Smartphone exiftool -G -a -s IMG_5677.MOV | grep -i timezone
➜  Smartphone exiftool -G -a -s IMG_5677.MOV | grep -i offset
[QuickTime]     MediaDataOffset                 : 36
➜  Smartphone exiftool -time:all -G -a -s IMG_5677.MOV
[File]          FileModifyDate                  : 2022:09:21 17:54:36+02:00
[File]          FileAccessDate                  : 2022:09:21 17:54:36+02:00
[File]          FileInodeChangeDate             : 2024:09:02 00:37:09+02:00
[QuickTime]     CreateDate                      : 2022:09:21 15:54:36
[QuickTime]     ModifyDate                      : 2022:09:21 15:56:04
[QuickTime]     TrackCreateDate                 : 2022:09:21 15:54:36
[QuickTime]     TrackModifyDate                 : 2022:09:21 15:56:04
...
[QuickTime]     CreationDate                    : 2022:09:21 16:54:36+01:00
➜  Smartphone

But this tag is also not needed as there are special Quick Time Tags that are quite common due to apple, and here the QuickTime "CreationDate" Tag does in fact support a timezone identifier.

Immich uses these tags to determine an exact date:

const EXIF_DATE_TAGS: Array<keyof Tags> = [
  "SubSecDateTimeOriginal",
  "DateTimeOriginal",
  "SubSecCreateDate",
  "CreationDate",
  "CreateDate",
  "SubSecMediaCreateDate",
  "MediaCreateDate",
  "DateTimeCreated",
];

So setting one of quicktime:creationdate or quicktime:createdate to a date with timezone-information fixes also the video position in the timeline.

So I was nearly done, but I noticed something strange when using the web-metadata-editor Immich provides in its UI.

Immich Sidecar Metadata Bug

After going through every Folder and applying the required fixes, I finally had a sane, chronological Timeline, some videos had minor offsets due to a slightly incorrect time, but it was manageable.

But there was one gotcha, Immich also has an online GPS Editor:

GPS Editor

This editor does not save the GPS Position inside the image or video, but instead into a so called "Sidecar" File. That is a small companion file next to the actual image with an .xmp file extension.

So if you set the GPS inside the Web-UI and keep the files local datetime, the time inside Immich will be wrong again.

Thats because Exiftool Still only reads the Images Exif-Data (and not the sidecar files content), and can only best-guess the UTC Time again, so it writes the local time as UTC Time into its database, as there is no GPS data inside the file itself.

Then it adds the sidecars Timezone-Offset (calculated from the GPS Position) to this "best-effort-guess" time.

Example:

I circumvent this issue right now by not setting any metadata through the web-interface and save everything to the media file itself, so I am not relying on sidecar files, but lets see how they handle this issue in the future, I wrote them a message on Discord.

But finally nothing is holding me off going down the memory lane again, now even in a fancy web-ui in chronological order! ... for now.

But still Immich is one of the best image management software I've ever used and it still is in the "heavy development" phase, so such things are just expected. I really like using it and highly recommend it to anyone who wants to manage or view their photos with a web-app.

Complexity

Nevertheless setting Metadata "correctly" so that features like timelines work out of the box every time seems to be hard or even impossible. There are multiple layers that all add a bunch of complexity based on media type, application used and the devices capabilities.

With application logic on top of everything mentioned above (Sidecar-Feature of Immich), complexity just explodes even more, and with this feature being buggy right now too (at least in my impression), this added a lot more confusion to the already hard to grasp topic.

But again, I don't want to blame anyone, with all these different data-sources, layers and edge-cases there isn't an easy solution that can be implemented that lets everything "just work" straight out of the box, plus that "correct behavior" surely also differs in the eye of the beholder here.



For now my current fixes work for Immich, but who know how other apps that might emerge in the future handle things.

Nextclouds Timeline for example only uses the FileModified timestamp for ordering images and leaves other Exif-Data completely untouched.

➜  exiftool -time:all -G -a -s IMG_20220918_094820.jpg
[File]          FileModifyDate                  : 2024:09:02 01:56:22+02:00
...
[EXIF]          ModifyDate                      : 2022:09:18 09:48:20
[EXIF]          DateTimeOriginal                : 2022:09:18 09:48:20
...

But this Tag is very fragile though, because most OSes update it every time you do anything with the file, so no ordered timeline in Nextcloud for me.

Deep inside a Discord Thread or Github Issue I found a comment that stated, that Immich tries to stick as close as possible to Exiftool as "gold standard", because its really widespread, plus its easy for them to do it because then its just an api call for them.

Apart from that, of course its also dependent on how much time one is able to invest to fix incomplete or wrong metadata.

Goal Reached?

I think yes, I have a chronological timeline (for now) and learned a lot about Exiftool, found some more useful tools to Tag Metadata.

Also I now have gathered some knowledge on how to fix certain metadata-issues and what options are out there to do it properly.

Tools

I tried plenty of tools to tag photos: