Converting mjpeg video to x264 or x265 loses color in deep blacks and bright white areas

General questions or discussion about HandBrake, Video and/or audio transcoding, trends etc.
Post Reply
daniel611
Posts: 2
Joined: Mon Jan 20, 2020 10:52 pm

Converting mjpeg video to x264 or x265 loses color in deep blacks and bright white areas

Post by daniel611 »

Description of problem or question:
I'm trying to convert some footage taken while skiing with an action cam that records 2.7k in mjpeg format (in .mov container) but after converting this in handbrake to x265 (same behaviour to x264) I notice that the very bright white areas (snow mostly) gets crushed to a light gray enough to be noticeable (the video just seems underexposed) and when viewing the videos side by side the difference is huge both in the white snow and on the black part of the skis (but blacks are very problematic here).

I've tried asking for help on reddit and the consensus seems to be that the mjpeg source has the full range of color [0-255] while the encode output has standard TV color space [16-235].

I've tryed ruling out a player problem by using 4 different apps to play this (VLC, MPC-HC, Windows media player, Films&TV) also tried disabling some hw acceleration and yuv conversions in vlc's options and nothing changed.
On Handbrake side I tried changing CRF to a lower value (tried 24, 20 and 10), encoder preset to a slower value (even placebo gives the same results), also tried the "range=full" advanced option (confirmed in mediainfo that range wasn't "Limited" anymore) but the result was the same.

Here's a 10s source sample (73MB), the converted output (11MB) and a quick side by side image of the results.


So my question is: how can I convert these videos and get consistent/correct color results? Any argument I can use? Or maybe I need to convert the video to an intermediary format/color space and then it will convert and show correctly in x265? Any help is appreciated.

Steps to reproduce the problem (If Applicable):
Load mjpeg full color range video, choose x265, crf 24, preset medium. Compare original and converted files.



HandBrake version (e.g., 1.0.0):
1.3.1



Operating system and version (e.g., Ubuntu 16.04 LTS, macOS 10.13 High Sierra, Windows 10 Creators Update):
Windows 10 (up to date)



HandBrake Activity Log ***required*** (see How-to get an activity log)

Code: Select all

HandBrake 1.3.1 (2020010400)
OS: Microsoft Windows NT 10.0.18363.0
CPU: AMD Ryzen 7 2700X Eight-Core Processor         
Ram: 16315 MB, 
GPU Information:
  NVIDIA GeForce GTX 970 - 26.21.14.3200
Screen: 1920x1080
Temp Dir: C:\Users\Daniel\AppData\Local\Temp\
Install Dir: C:\Program Files\HandBrake
Data Dir: C:\Users\Daniel\AppData\Roaming\HandBrake

-------------------------------------------


# Starting Encode ...

[01:17:07] base preset: x265 medium@crf26 [archive mid] (Modified)
[01:17:07] hb_init: starting libhb thread
[01:17:08] Starting work at: Tue Jan 21 01:17:08 2020
[01:17:08] 1 job(s) to process
[01:17:08] json job:
{
  "Audio": {
    "AudioList": [
      {
        "Bitrate": 256,
        "DRC": 0.0,
        "Encoder": "av_aac",
        "Gain": 0.0,
        "Mixdown": 4,
        "NormalizeMixLevel": false,
        "Samplerate": 0,
        "Track": 0,
        "DitherMethod": 0
      }
    ],
    "CopyMask": [
      "copy:aac",
      "copy:ac3",
      "copy:dtshd",
      "copy:dts",
      "copy:eac3",
      "copy:flac",
      "copy:mp3",
      "copy:truehd"
    ],
    "FallbackEncoder": "ac3"
  },
  "Destination": {
    "ChapterList": [],
    "ChapterMarkers": false,
    "AlignAVStart": true,
    "File": "C:\\Users\\Daniel\\Desktop\\test encode\\test .mp4",
    "Mp4Options": {
      "IpodAtom": false,
      "Mp4Optimize": false
    },
    "Mux": "mp4"
  },
  "Filters": {
    "FilterList": [
      {
        "ID": 12,
        "Settings": {
          "crop-bottom": "0",
          "crop-left": "0",
          "crop-right": "0",
          "crop-top": "0",
          "height": "1080",
          "width": "1916"
        }
      },
      {
        "ID": 6,
        "Settings": {
          "mode": "2"
        }
      }
    ]
  },
  "PAR": {
    "Num": 1,
    "Den": 1
  },
  "Metadata": {},
  "SequenceID": 0,
  "Source": {
    "Angle": 1,
    "Range": {
      "Type": "chapter",
      "Start": 1,
      "End": 1
    },
    "Title": 1,
    "Path": "C:\\Users\\Daniel\\Desktop\\test encode\\test.MOV"
  },
  "Subtitle": {
    "Search": {
      "Burn": true,
      "Default": false,
      "Enable": true,
      "Forced": true
    },
    "SubtitleList": []
  },
  "Video": {
    "Encoder": "x265",
    "Level": "auto",
    "TwoPass": false,
    "Turbo": false,
    "ColorMatrixCode": 0,
    "Options": "",
    "Preset": "medium",
    "Profile": "auto",
    "Quality": 24.0,
    "QSV": {
      "Decode": false,
      "AsyncDepth": 0
    }
  }
}
[01:17:08] CPU:
[01:17:08]  - logical processor count: 16
[01:17:08] Intel Quick Sync Video support: no
[01:17:08] hb_scan: path=C:\Users\Daniel\Desktop\test encode\test.MOV, title_index=1
udfread ERROR: ECMA 167 Volume Recognition failed
src/libbluray/disc/disc.c:323: failed opening UDF image C:\Users\Daniel\Desktop\test encode\test.MOV
src/libbluray/disc/disc.c:424: error opening file BDMV\index.bdmv
src/libbluray/disc/disc.c:424: error opening file BDMV\BACKUP\index.bdmv
src/libbluray/bluray.c:2585: nav_get_title_list(C:\Users\Daniel\Desktop\test encode\test.MOV\) failed
[01:17:08] bd: not a bd - trying as a stream/file instead
libdvdnav: Using dvdnav version 6.0.1
libdvdread: Encrypted DVD support unavailable.
libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.IFO failed
libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.BUP failed
libdvdread: Can't open file VIDEO_TS.IFO.
libdvdnav: vm: failed to read VIDEO_TS.IFO
[01:17:08] dvd: not a dvd - trying as a stream/file instead
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'C:\Users\Daniel\Desktop\test encode\test.MOV':
  Metadata:
    major_brand     : qt
    minor_version   : 512
    compatible_brands: qt
    encoder         : Lavf58.29.100
  Duration: 00:00:10.00, start: 0.000000, bitrate: 61476 kb/s
    Stream #0:0(eng): Video: mjpeg (Baseline) (jpeg / 0x6765706A), yuvj420p(pc, bt470bg/unknown/unknown), 2704x1524, 60058 kb/s, 30 fps, 30 tbr, 60k tbn, 60k tbc (default)
    Metadata:
      handler_name    : Video Handler
      encoder         : iCatch AVCC
    Stream #0:1(eng): Audio: pcm_s16le (sowt / 0x74776F73), 44100 Hz, stereo, s16, 1411 kb/s (default)
    Metadata:
      handler_name    : Audio Handler
[01:17:08] scan: decoding previews for title 1
[01:17:08] scan: audio 0x1: pcm_s16le, rate=44100Hz, bitrate=1411200 English (pcm_s16le) (2.0 ch) (1411 kbps)
[01:17:08] scan: 10 previews, 2704x1524, 30.000 fps, autocrop = 0/0/0/0, aspect 1.77:1, PAR 1:1
[01:17:08] libhb: scan thread found 1 valid title(s)
[01:17:08] Skipping subtitle scan.  No suitable subtitle tracks.
[01:17:08] Starting Task: Encoding Pass
[01:17:08] job configuration:
[01:17:08]  * source
[01:17:08]    + C:\Users\Daniel\Desktop\test encode\test.MOV
[01:17:08]    + title 1, chapter(s) 1 to 1
[01:17:08]    + container: mov,mp4,m4a,3gp,3g2,mj2
[01:17:08]    + data rate: 61476 kbps
[01:17:08]  * destination
[01:17:08]    + C:\Users\Daniel\Desktop\test encode\test .mp4
[01:17:08]    + container: MPEG-4 (libavformat)
[01:17:08]      + align initial A/V stream timestamps
[01:17:08]  * video track
[01:17:08]    + decoder: mjpeg
[01:17:08]      + bitrate 60058 kbps
[01:17:08]    + filters
[01:17:08]      + Framerate Shaper (mode=2)
[01:17:08]        + frame rate: 30.000 fps -> peak rate limited to 30.000 fps
[01:17:08]      + Crop and Scale (width=1916:height=1080:crop-top=0:crop-bottom=0:crop-left=0:crop-right=0)
[01:17:08]        + source: 2704 * 1524, crop (0/0/0/0): 2704 * 1524, scale: 1916 * 1080
[01:17:08]    + Output geometry
[01:17:08]      + storage dimensions: 1916 x 1080
[01:17:08]      + pixel aspect ratio: 1 : 1
[01:17:08]      + display dimensions: 1916 x 1080
[01:17:08]    + encoder: H.265 (libx265)
[01:17:08]      + preset:  medium
[01:17:08]      + profile: auto
[01:17:08]      + level:   auto
[01:17:08]      + quality: 24.00 (RF)
[01:17:08]      + color profile: 1-1-5
[01:17:08]  * audio track 1
[01:17:08]    + decoder: English (pcm_s16le) (2.0 ch) (1411 kbps) (track 1, id 0x1)
[01:17:08]      + bitrate: 1411 kbps, samplerate: 44100 Hz
[01:17:08]    + mixdown: Stereo
[01:17:08]    + dither: none
[01:17:08]    + encoder: AAC (libavcodec)
[01:17:08]      + bitrate: 256 kbps, samplerate: 44100 Hz
[01:17:08] sync: expecting 300 video frames
x265 [info]: HEVC encoder version 3.2.1+1-b5c86a64bbbe
x265 [info]: build info [Windows][GCC 9.2.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [info]: Main profile, Level-4 (Main tier)
x265 [info]: Thread pool created using 16 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 4 / wpp(17 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge         : hex / 57 / 2 / 3
x265 [info]: Keyframe min / max / scenecut / bias: 30 / 300 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt        : 20 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
x265 [info]: References / ref-limit  cu / depth  : 3 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 2 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress            : CRF-24.0 / 0.60
x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip signhide tmvp b-intra
x265 [info]: tools: strong-intra-smoothing lslices=6 deblock sao
[01:17:09] sync: first pts video is 0
[01:17:09] sync: Chapter 1 at frame 1 time 0
[01:17:09] sync: first pts audio 0x1 is 0
[01:17:20] reader: done. 1 scr changes
[01:17:26] work: average encoding speed for job is 21.491783 fps
[01:17:26] vfr: 300 frames output, 0 dropped and 0 duped for CFR/PFR
[01:17:26] vfr: lost time: 0 (0 frames)
[01:17:26] vfr: gained time: 0 (0 frames) (0 not accounted for)
[01:17:26] pcm_s16le-decoder done: 500 frames, 0 decoder errors
[01:17:26] mjpeg-decoder done: 300 frames, 0 decoder errors
[01:17:26] sync: got 300 frames, 300 expected
[01:17:26] sync: framerate min 30.000 fps, max 30.000 fps, avg 30.000 fps
x265 [info]: frame I:      1, Avg QP:26.44  kb/s: 32081.52
x265 [info]: frame P:     76, Avg QP:25.69  kb/s: 22638.81
x265 [info]: frame B:    223, Avg QP:29.96  kb/s: 4141.32
x265 [info]: Weighted P-Frames: Y:7.9% UV:7.9%
x265 [info]: consecutive B-frames: 3.9% 2.6% 27.3% 32.5% 33.8%
encoded 300 frames in 17.55s (17.09 fps), 8920.48 kb/s, Avg QP:28.87
[01:17:26] mux: track 0, 300 frames, 11151801 bytes, 8891.80 kbps, fifo 512
[01:17:26] mux: track 1, 431 frames, 319678 bytes, 254.89 kbps, fifo 512
[01:17:26] Finished work at: Tue Jan 21 01:17:26 2020
[01:17:26] libhb: work result = 0

# Encode Completed ...

daniel611
Posts: 2
Joined: Mon Jan 20, 2020 10:52 pm

Re: Converting mjpeg video to x264 or x265 loses color in deep blacks and bright white areas

Post by daniel611 »

I saw the github issue in one of the other threads and this sounded like my problem so I tried converting without resizing and it actually is exactly my problem (source is 2.7k, destination 1080p).

So, the immediate fix sounds pretty simple but doubles the encode time: convert 2.7k mjpeg to 2.7k x264/x265, then take the output and resize it to 1080p.

I'll probably wait for the next release with the fix though, converting twice doesn't sound good for the quality.
Post Reply