Color conversion issues with ffmpeg

tl;dr Colors might off when extracting frames etc. with ffmpeg if the file is not tagged correctly. Fix it by the command mentioned at the end. A comparison between everything.

Something strange I noticed today (at the time of writing): a image I extracted with ffmpeg (command ffmpeg -i episode.mkv q-%d.png) from an episode of anime had different colors compared to when it was shown through mpv. I also tested MPC-HC, and mpv on Linux. Same results. I also tested the movie Kimetsu no Yaiba: Mugen Ressha-hen. Same thing happened.

I checked their meta data with ffprobe.

Kobayashi-san Chi no Maid Dragon S2 - 01 (1080p [6B63C756])

  • Command: ffprobe -v error -show_streams .\file.mkv
  • Output: 
    level=40
color_range=unknown
color_space=unknown
color_transfer=unknown
color_primaries=unknown

Kimetsu no Yaiba: Mugen Ressha-hen (1080p.BD.Remux.DTS-HD.MA-TTGA)

  • Command: ffprobe -v error -show_streams .\file.mkv
  • Output: 
    level=-99
color_range=pc
color_space=bt470bg
color_transfer=unknown
color_primaries=unknown

It would seem that ffmpeg defaults to BT.601 even though the content nowadays is BT.709 (HDTV, BD) or BT.2020 (UHD) [1, 2] when color_space tag is set to unknown. This can be different for SD or otherwise old content though! The movie had color_space set to bt470bg (meaning PAL BT.601). That's most likely a mistake in the release itself (the BD or the BD ripper used). It should be BT.709 as it's a HD BluRay release. Japan uses NTCS so it's even more incorrect in that sense. smpte170m would be the NTSC equivalent for BT.601.

I feel that at least the unknown tag should be handled better in ffmpeg.

This can be fixed it by specifying the color_matrix in the command, like so: ffmpeg -i INPUT.mkv -sws_flags +accurate_rnd+full_chroma_int -vf scale=in_color_matrix=bt709:out_color_matrix=bt709 q-%d.png

The -sws_flags +accurate_rnd+full_chroma_int flag might not be as necessary, but it can help with accuracy. A comparison between everything.

Sources:

A scene from the Tsukihime trailer

By Marko Leinikka