Increasing quality/size for visually difficult HD sources

[draco_myst]

Mac (not really relevant to the question)
Handbrake 1.0.2

I recently encoded a Blu Ray source file that started at 25.8GB using both the "Apple 1080p60 Surround" and the "Apple 1080p30 Surround" presets (I had been using the Apple TV 3 preset prior to this release), and in each case the file size that resulted was 3.12GB. Most similar encodes from other sources result in a file twice that size, but I figured that this one ended up small because of the low contrast of the original. The problem is that this source is a dimly lit film for much of it's length (think "Arrival" for example). Even when watching the original file, it is necessary to close the blinds in the room during the day in order to see what is going on. I would prefer a final file that will direct play on the Apple TV 4, but the encodes I got with the presets lost too much detail because of the difficult visual nature of the source. I would be fine with a file twice the size if the amount of compression could be reduced without breaking the Apple TV compatibility.

When I read the online documentation regarding presets and quality, it warned that certain settings can result in device incompatibility. What is my best bet for improving the quality, and how far is it likely to get me? (Is it worth even trying?)

Note: My final files are always m4v for use in PLEX, but I will often select mkv in Handbrake for HD sources in order to allow for DTS passthrough to be included (as I did in this case), and then I use Subler to make the final file m4v.

[rollin_eng]

Could you please post your logs, instructions can be found here:

viewtopic.php?f=6&t=31236

[draco_myst]

HandBrake Activity Log for Session: 2017-02-20 18:25:37 -0500
Handbrake Version: 1.0.2 (2017012200)
***** (2016) 30fps.mkv
Preset: Custom
[18:25:37] macgui: QueueCore scanning specifically for title: 1
[18:25:37] CPU: Intel(R) Core(TM) i7-3615QM CPU @ 2.30GHz
[18:25:37] - Intel microarchitecture Ivy Bridge
[18:25:37] - logical processor count: 8
[18:25:37] hb_scan: path=/Volumes/Plex Media 2/Encoding Files/01 Blu-Ray -> Makemkv (RAW)/*****_(2016)_t00.mkv, title_index=1
udfread ERROR: ECMA 167 Volume Recognition failed
disc.c:274: failed opening UDF image /Volumes/Plex Media 2/Encoding Files/01 Blu-Ray -> Makemkv (RAW)/*****_(2016)_t00.mkv
disc.c:352: error opening file BDMV/index.bdmv
disc.c:352: error opening file BDMV/BACKUP/index.bdmv
[18:25:37] bd: not a bd - trying as a stream/file instead
libdvdnav: Using dvdnav version 5.0.1
libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.IFO failed
libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.BUP failed
libdvdread: Can't open file VIDEO_TS.IFO.
libdvdnav: vm: failed to read VIDEO_TS.IFO
[18:25:37] dvd: not a dvd - trying as a stream/file instead
Input #0, matroska,webm, from '/Volumes/Plex Media 2/Encoding Files/01 Blu-Ray -> Makemkv (RAW)/*****_(2016)_t00.mkv':
Metadata:
title : ***** (2016)
Duration: 01:56:22.97, start: 0.000000, bitrate: N/A
Chapter #0:0: start 0.000000, end 589.755833
Metadata:
title : Chapter 01
Chapter #0:1: start 589.755833, end 1053.635917
Metadata:
title : Chapter 02
Chapter #0:2: start 1053.635917, end 1408.281875
Metadata:
title : Chapter 03
Chapter #0:3: start 1408.281875, end 2033.990292
Metadata:
title : Chapter 04
Chapter #0:4: start 2033.990292, end 2461.500708
Metadata:
title : Chapter 05
Chapter #0:5: start 2461.500708, end 3146.101292
Metadata:
title : Chapter 06
Chapter #0:6: start 3146.101292, end 3454.534417
Metadata:
title : Chapter 07
Chapter #0:7: start 3454.534417, end 3786.449333
Metadata:
title : Chapter 08
Chapter #0:8: start 3786.449333, end 4169.206708
Metadata:
title : Chapter 09
Chapter #0:9: start 4169.206708, end 4569.398167
Metadata:
title : Chapter 10
Chapter #0:10: start 4569.398167, end 5121.533083
Metadata:
title : Chapter 11
Chapter #0:11: start 5121.533083, end 5736.981250
Metadata:
title : Chapter 12
Chapter #0:12: start 5736.981250, end 6326.320000
Metadata:
title : Chapter 13
Chapter #0:13: start 6326.320000, end 6982.976000
Metadata:
title : Chapter 14
Stream #0:0(eng): Video: h264 (High)
yuv420p, 1920x1080 [PAR 1:1 DAR 16:9], PAR 1:1 DAR 16:9
23.98 fps, 1k tbn
Metadata:
BPS-eng : 27340105
DURATION-eng : 01:56:22.976000000
NUMBER_OF_FRAMES-eng: 167424
NUMBER_OF_BYTES-eng: 23864412293
SOURCE_ID-eng : 001011
_STATISTICS_WRITING_APP-eng: MakeMKV v1.10.4 darwin(x86-release)
_STATISTICS_WRITING_DATE_UTC-eng: 2017-02-15 16:33:05
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES SOURCE_ID
Stream #0:1(eng): Audio: dts (DTS)
48000 Hz, 5.1, fltp, 1536 kb/s (default)
Metadata:
title : Surround 5.1
BPS-eng : 1509000
DURATION-eng : 01:56:22.976000000
NUMBER_OF_FRAMES-eng: 654654
NUMBER_OF_BYTES-eng: 1317163848
SOURCE_ID-eng : 001100
_STATISTICS_WRITING_APP-eng: MakeMKV v1.10.4 darwin(x86-release)
_STATISTICS_WRITING_DATE_UTC-eng: 2017-02-15 16:33:05
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES SOURCE_ID
Stream #0:2(eng): Audio: ac3
48000 Hz, 5.1, fltp, 640 kb/s
Metadata:
title : Surround 5.1
BPS-eng : 640000
DURATION-eng : 01:56:22.976000000
NUMBER_OF_FRAMES-eng: 218218
NUMBER_OF_BYTES-eng: 558638080
SOURCE_ID-eng : 001103
_STATISTICS_WRITING_APP-eng: MakeMKV v1.10.4 darwin(x86-release)
_STATISTICS_WRITING_DATE_UTC-eng: 2017-02-15 16:33:05
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES SOURCE_ID
Stream #0:3(eng): Subtitle: hdmv_pgs_subtitle
Metadata:
BPS-eng : 32796
DURATION-eng : 01:48:58.469437500
NUMBER_OF_FRAMES-eng: 2444
NUMBER_OF_BYTES-eng: 26804982
SOURCE_ID-eng : 001200
_STATISTICS_WRITING_APP-eng: MakeMKV v1.10.4 darwin(x86-release)
_STATISTICS_WRITING_DATE_UTC-eng: 2017-02-15 16:33:05
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES SOURCE_ID
Stream #0:4(eng): Subtitle: hdmv_pgs_subtitle
Metadata:
BPS-eng : 36145
DURATION-eng : 01:48:58.677979166
NUMBER_OF_FRAMES-eng: 2878
NUMBER_OF_BYTES-eng: 29542793
SOURCE_ID-eng : 001201
_STATISTICS_WRITING_APP-eng: MakeMKV v1.10.4 darwin(x86-release)
_STATISTICS_WRITING_DATE_UTC-eng: 2017-02-15 16:33:05
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES SOURCE_ID
[18:25:37] scan: decoding previews for title 1
[18:25:37] scan: audio 0x1: dca, rate=48000Hz, bitrate=1536000 English (DTS) (5.1 ch)
[18:25:37] scan: audio 0x2: ac3, rate=48000Hz, bitrate=640000 English (AC3) (5.1 ch)
[18:25:40] scan: 10 previews, 1920x1080, 23.976 fps, autocrop = 138/138/0/10, aspect 16:9, PAR 1:1
[18:25:41] libhb: scan thread found 1 valid title(s)
[18:25:41] macgui: QueueCore scan done
[18:25:41] macgui: QueueCore started encoding ***** (2016) 30fps.mkv
[18:25:41] macgui: QueueCore with preset Custom
[18:25:41] 1 job(s) to process
[18:25:41] starting job
[18:25:41] decomb filter thread started for segment 0
[18:25:41] decomb filter thread started for segment 1
[18:25:41] decomb filter thread started for segment 2
[18:25:41] decomb filter thread started for segment 3
[18:25:41] decomb filter thread started for segment 4
[18:25:41] decomb filter thread started for segment 5
[18:25:41] decomb filter thread started for segment 6
[18:25:41] decomb filter thread started for segment 7
[18:25:41] decomb check thread started for segment 0
[18:25:41] decomb check thread started for segment 1
[18:25:41] decomb check thread started for segment 2
[18:25:41] decomb check thread started for segment 3
[18:25:41] decomb check thread started for segment 4
[18:25:41] decomb check thread started for segment 5
[18:25:41] decomb check thread started for segment 6
[18:25:41] decomb check thread started for segment 7
[18:25:41] mask filter thread started for segment 0
[18:25:41] mask filter thread started for segment 1
[18:25:41] mask filter thread started for segment 2
[18:25:41] mask filter thread started for segment 3
[18:25:41] mask filter thread started for segment 4
[18:25:41] mask filter thread started for segment 5
[18:25:41] mask filter thread started for segment 6
[18:25:41] mask filter thread started for segment 7
[18:25:41] mask erode thread started for segment 0
[18:25:41] mask erode thread started for segment 1
[18:25:41] mask erode thread started for segment 2
[18:25:41] mask erode thread started for segment 3
[18:25:41] mask erode thread started for segment 4
[18:25:41] mask erode thread started for segment 5
[18:25:41] mask erode thread started for segment 6
[18:25:41] mask erode thread started for segment 7
[18:25:41] mask dilate thread started for segment 0
[18:25:41] mask dilate thread started for segment 1
[18:25:41] mask dilate thread started for segment 2
[18:25:41] mask dilate thread started for segment 3
[18:25:41] mask dilate thread started for segment 4
[18:25:41] mask dilate thread started for segment 5
[18:25:41] mask dilate thread started for segment 6
[18:25:41] mask dilate thread started for segment 7
[18:25:41] yadif thread started for segment 0
[18:25:41] yadif thread started for segment 2
[18:25:41] yadif thread started for segment 1
[18:25:41] yadif thread started for segment 3
[18:25:41] yadif thread started for segment 4
[18:25:41] yadif thread started for segment 5
[18:25:41] yadif thread started for segment 6
[18:25:41] yadif thread started for segment 7
[18:25:41] job configuration:
[18:25:41] * source
[18:25:41] + /Volumes/Plex Media 2/Encoding Files/01 Blu-Ray -> Makemkv (RAW)/*****_(2016)_t00.mkv
[18:25:41] + title 1, chapter(s) 1 to 14
[18:25:41] + container: matroska,webm
[18:25:41] * destination
[18:25:41] + /Volumes/Plex Media 2/Encoding Files/02 Makemkv -> Handbrake (COMPRESSED)/***** (2016) 30fps.mkv
[18:25:41] + container: Matroska (libavformat)
[18:25:41] + chapter markers
[18:25:41] * video track
[18:25:41] + decoder: h264
[18:25:41] + filters
[18:25:41] + Comb Detect (mode=3:spatial-metric=2:motion-thresh=1:spatial-thresh=1:filter-mode=2:block-thresh=40:block-width=16:block-height=16)
[18:25:41] + Decomb (mode=39)
[18:25:41] + Framerate Shaper (mode=2:rate=27000000/900000)
[18:25:41] + frame rate: 23.976 fps -> peak rate limited to 30.000 fps
[18:25:41] + Crop and Scale (width=1920:height=800:crop-top=140:crop-bottom=140:crop-left=0:crop-right=0)
[18:25:41] + source: 1920 * 1080, crop (140/140/0/0): 1920 * 800, scale: 1920 * 800
[18:25:41] + Output geometry
[18:25:41] + storage dimensions: 1920 x 800
[18:25:41] + pixel aspect ratio: 1 : 1
[18:25:41] + display dimensions: 1920 x 800
[18:25:41] + encoder: H.264 (libx264)
[18:25:41] + preset: medium
[18:25:41] + profile: high
[18:25:41] + level: 4.0
[18:25:41] + quality: 22.00 (RF)
[18:25:41] * subtitle track 1, English (track 0, id 0x3) Picture [PGS] -> Passthrough
[18:25:41] * subtitle track 2, English (track 1, id 0x4) Picture [PGS] -> Passthrough
[18:25:41] * audio track 1
[18:25:41] + decoder: English (DTS) (5.1 ch) (track 1, id 0x1)
[18:25:41] + bitrate: 1536 kbps, samplerate: 48000 Hz
[18:25:41] + mixdown: Dolby Pro Logic II
[18:25:41] + encoder: AAC (Apple AudioToolbox)
[18:25:41] + bitrate: 160 kbps, samplerate: 48000 Hz
[18:25:41] * audio track 2
[18:25:41] + decoder: English (DTS) (5.1 ch) (track 1, id 0x1)
[18:25:41] + bitrate: 1536 kbps, samplerate: 48000 Hz
[18:25:41] + mixdown: 5.1 Channels
[18:25:41] + encoder: AC3 (libavcodec)
[18:25:41] + bitrate: 640 kbps, samplerate: 48000 Hz
[18:25:41] * audio track 3
[18:25:41] + decoder: English (DTS) (5.1 ch) (track 1, id 0x1)
[18:25:41] + bitrate: 1536 kbps, samplerate: 48000 Hz
[18:25:41] + DTS Passthru
[18:25:41] sync: expecting 167424 video frames
[18:25:41] encx264: min-keyint: 24, keyint: 240
[18:25:41] encx264: encoding at constant RF 22.000000
[18:25:41] encx264: unparsed options: vbv-bufsize=31250:level=4.0:vbv-maxrate=25000
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x264 [info]: profile High, level 4.0
[18:25:41] sync: first pts video is 0
[18:25:41] sync: “*” (1) at frame 1 time 0
[18:25:41] sync: first pts audio 0x1 is 0
[18:25:41] sync: first pts audio 0x1 is 0
[18:25:41] sync: first pts audio 0x1 is 0
[18:26:48] sync: first pts subtitle 0x4 is 8329590
[18:26:48] sync: first pts subtitle 0x3 is 8355870
[18:33:57] sync: “*” (2) at frame 14141 time 53078025
[18:39:39] sync: “*” (3) at frame 25263 time 94827232
[18:44:57] sync: “*” (4) at frame 33766 time 126745369
[18:53:47] sync: “*” (5) at frame 48770 time 183066634
[18:59:21] sync: “*” (6) at frame 59018 time 221535063
[19:08:32] sync: “*” (7) at frame 75434 time 283156624
[19:12:31] sync: “*” (8) at frame 82828 time 310911851
[19:16:59] sync: “*” (9) at frame 90787 time 340787947
[19:21:54] sync: “*” (10) at frame 99962 time 375228604
[19:27:28] sync: “*” (11) at frame 109559 time 411253343
[19:35:05] sync: “*” (12) at frame 122797 time 460945485
[19:45:01] sync: “*” (13) at frame 137552 time 516332066
[19:53:47] sync: “*” (14) at frame 151681 time 569368800
[20:03:08] reader: done. 1 scr changes
[20:03:11] work: average encoding speed for job is 28.620537 fps
[20:03:11] comb detect: heavy 780 | light 1898 | uncombed 164746 | total 167424
[20:03:11] decomb: deinterlaced 780 | blended 1898 | unfiltered 164746 | total 167424
[20:03:11] vfr: 167424 frames output, 0 dropped and 0 duped for CFR/PFR
[20:03:11] vfr: lost time: 0 (0 frames)
[20:03:11] vfr: gained time: 0 (0 frames) (0 not accounted for)
[20:03:11] dca-decoder done: 654654 frames, 0 decoder errors
[20:03:11] dca-decoder done: 654654 frames, 0 decoder errors
[20:03:11] dca-decoder done: 654654 frames, 0 decoder errors
[20:03:11] h264-decoder done: 167424 frames, 0 decoder errors
[20:03:11] sync: got 167424 frames, 167424 expected
[20:03:11] sync: framerate min 23.974 fps, max 23.981 fps, avg 23.976 fps
x264 [info]: frame I:2097 Avg QP:16.91 size: 39263
x264 [info]: frame P:59795 Avg QP:20.33 size: 8951
x264 [info]: frame B:105532 Avg QP:21.19 size: 3967
x264 [info]: consecutive B-frames: 14.8% 3.0% 1.8% 80.5%
x264 [info]: mb I I16..4: 29.7% 55.5% 14.8%
x264 [info]: mb P I16..4: 1.4% 2.9% 0.2% P16..4: 31.3% 4.3% 5.2% 0.0% 0.0% skip:54.7%
x264 [info]: mb B I16..4: 0.1% 0.2% 0.0% B16..8: 37.6% 0.8% 0.1% direct: 0.4% skip:60.9% L0:43.4% L1:55.7% BI: 0.9%
x264 [info]: 8x8 transform intra:60.9% inter:93.6%
x264 [info]: coded y,uvDC,uvAC intra: 35.5% 43.8% 11.8% inter: 6.1% 10.0% 0.1%
x264 [info]: i16 v,h,dc,p: 32% 20% 6% 42%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 23% 15% 31% 5% 6% 6% 6% 5% 4%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 37% 20% 19% 4% 6% 5% 4% 3% 2%
x264 [info]: i8c dc,h,v,p: 64% 16% 16% 3%
x264 [info]: Weighted P-Frames: Y:1.4% UV:0.4%
x264 [info]: ref P L0: 64.4% 7.3% 20.1% 8.2% 0.1%
x264 [info]: ref B L0: 87.8% 10.1% 2.1%
x264 [info]: ref B L1: 94.4% 5.6%
x264 [info]: kb/s:1187.08
[20:03:11] mux: track 0, 167424 frames, 1036100702 bytes, 1186.99 kbps, fifo 1024
[20:03:11] mux: track 1, 327328 frames, 141589934 bytes, 162.21 kbps, fifo 2048
[20:03:11] mux: track 2, 218217 frames, 558635520 bytes, 639.99 kbps, fifo 2048
[20:03:11] mux: track 3, 654654 frames, 1317163848 bytes, 1508.98 kbps, fifo 4096
[20:03:11] mux: track 4, 2444 frames, 26804982 bytes, 30.71 kbps, fifo 32
[20:03:11] mux: track 5, 2878 frames, 29542793 bytes, 33.85 kbps, fifo 32
[20:03:11] libhb: work result = 0

[mduell]

Your input is only 24fps, which is why you're not seeing any difference between the presets capped at 30 and 60 fps.

Also sounds like your TV brightness is turned up too high.

[draco_myst]

I kind of figured the frame rate was lower than the caps by the fact the the final file size was identical in each trial, but thanks for confirming that. As for the TV, It's not a case of being washed out, it's a case of taking a dark scene and removing what little contrast there was through compression resulting in a grainy appearance like you would get from an underexposed photo being compressed for the web. The TV is a professionally calibrated Pioneer plasma monitor, so I'm pretty sure it's not a case of a brightness setting. I'm just looking to reduce the amount of compression being done to preserve the image quality.

[draco_myst]

FWIW, I ran a new encode using the same base Apple 1080p30 Surround preset but I dropped the RF to 18 (from 22) and set the Encoder Preset to slow (from medium). The file size grew to 4.53GB (from 3.12GB) with a significant (to my eyes) improvement in reduced graininess, and improved contrast. I have not yet tried to play it through the Apple TV 4 (this was direct play on the Mac). Not being a Handbrake "expert", I'm not sure if this was the right approach, still hoping someone with detailed knowledge will reply.

[draco_myst]

Tried on Apple TV 4, works fine at the higher settings.

[mduell]

Increasing the quality setting, at least within sane values, is a reasonable way to improve the picture. Changing the preset won't matter much.