Best lossless settings for mp4 transformation

[lashara4]

Description of problem or question:
full backstory is available here if anyone is interested, and how I end up on forum of the handbrake
https://www.reddit.com/r/premiere/comme ... _editable/
current situation: I have 27 files that I need to convert so they would become error free and not crash premiere pro. Handbrake worked like a charm! Now I need to convert them in a way so I would not loose any quality.
https://we.tl/t-CqsrNsKu9K for example. I have tried converting video of:

Original file - 95.6 mb.
presets:
apple 4k (it leaves the video 2.7k) - 12.6mb
Vimeo 4k - 23.9
If done manual:
2704x1520, fps same as source (59.94), quality RF 22, encoder options preset: slow, variable framerate - 20.9mb.

so I guess I am loosing (or not) some video data here. What would be the best settings for lossless quality ? 27 of those files varies from 88mb to 1.2gb, so I guess I do not need any compression until prod.



HandBrake version (e.g., 1.0.0):
1.2.0

Operating system and version (e.g., Ubuntu 16.04 LTS, macOS 10.13 High Sierra, Windows 10 Creators Update):
macOS 10.14 Mojave

[rollin_eng]

Could you please post your HB logs, instructions can be found here:

https://handbrake.fr/docs/en/latest/hel ... y-log.html

[Woodstock]

If you're doing this to correct things for Premier, have you looked at the "production" presets?

[lashara4]

Could you please post your HB logs, instructions can be found here:

https://handbrake.fr/docs/en/latest/hel ... y-log.html

https://pastebin.com/bySsUjHF

[lashara4]

If you're doing this to correct things for Premier, have you looked at the "production" presets?

Hello, I've tried production presets on the file I uploaded on wetransfer (89.9mb). Production max returned 762.9mb and standard returned 328.6mb. I feel like I over-processing in this scenario as GoPro footage is 7.5MB/s, 2.7k, 59.94fps.
I used word lossless thinking that it will return same unmodified video as the footage - 89.9mb, now I understand that it really makes it lossless at 800mb. What would be the best settings for not loose any quality compared to the original gopro footage and have the size not bigger than the original?

Thank you!

[BradleyS]

Unfortunately, you can't have both. You can adjust the quality lower on the production presets, if you desire.

[lashara4]

Unfortunately, you can't have both. You can adjust the quality lower on the production presets, if you desire.

not sure what is "both" in this scenario. Can I just have the original? Conversion in handbrake returns me an error free file with which I am more than happy to work. Just trying to achieve original compression.

[BradleyS]

I was referring to size and quality. HandBrake re-encodes your video, which by nature introduces a generation of loss and inflates file size. You can accept this and work with larger files, or reduce the quality further.

The production presets are designed for exactly the purpose you're using them for, conforming your source video to a stricter standard for editing. It is generally not possible to do this reliably without re-encoding.

[lashara4]

Understood. Then if I am going to accept some loss what would be the optimal settings for output?
If I use these settings 2704x1520, fps same as source (59.94), quality RF 22, encoder options preset: slow, variable framerate I get a file around 20mbs, so I can increase the quality to grow the file size to a certain level. What should be the settings in Filters and Video tabs so I would get as close as possible to the original file size ±20% with the best quality for that file size? I uploaded the logs and the file that I am working with, I would appreciate a lot a professionals' advice.

[musicvid]

I think it is important to know that throwing more well-intentioned bits at an encode that is already optimal makes it, well, ... bigger. "Optimal" quality is casually thought of as being around 0.995 SSIM, often occurring around CRF 18-20 (no lower) at least in x264. Integrity is so good these days that "lossless" isn't that much of a consideration.
The tests I ran with high-motion ocean source, seem to bear this out. Even a bit-pig like that can only carry so much weight.
https://drive.google.com/file/d/1CTqxiw ... p=drivesdk

[lashara4]

I think it is important to know that throwing more well-intentioned bits at an encode that is already optimal makes it, well, ... bigger. "Optimal" quality is casually thought of as being around 0.995 SSIM, often occurring around CRF 18-20 (no lower) at least in x264. Integrity is so good these days that "lossless" isn't that much of a consideration.
The tests I ran with high-motion ocean source, seem to bear this out. Even a bit-pig like that can only carry so much weight.
https://drive.google.com/file/d/1CTqxiw ... p=drivesdk

thanks! What I tried now is to select preset of production max and then slide RF to 20. From the 89.9mb file I got 137mb file.I guess that will work until Cannot notice any quality loss as well.

[musicvid]

If you are that far above file size, your settings are way above optimal .
It sounds like there is not so much motion detail in your film.
You are welcome to waste all the space you want, it won't do a bit of good.
"Optimal" means enough, but not too much. Make sense?

[lashara4]

If you are that far above file size, your settings are way above optimal .
It sounds like there is not so much motion detail in your film.
You are welcome to waste all the space you want, it won't do a bit of good.
"Optimal" means enough, but not too much. Make sense?

exactly. If I am getting 150% bigger file that means it is above optimum. I uploaded the logs, so maybe you can check the source file and on that advise what options should I use. Is it only the RF I should worry about? And maybe set it to 22?
There are so many settings which I do not understand, as I am "one time" user for the purpose of fixing the corrupted video files. I see Anamorphic, Modulus, detelecine, interlace detection, encoder options, levels, presets from slow to very fast, etc. So as I do not know anything about them, and dont even imagine what role these things has to do with file size, I just selected preset production max and moved the RF bar from 0 to 20.
Maybe there is something, that enlarges file size but does not give anything to quality (as quality is limited, having the RF set to 0, gives nothing compared to 20) and I could minimise, or turn off.

[lashara4]

Also I think I made a mistake in the initial post I think it was RF18, not RF20 that put a file up to 150%.

another one, from 95.6mb I got 42.4mb with RF22:
https://pastebin.com/T6DDLdK7

and with RF 21 I got 49.6mb.
RF20 - 58.8mb
RF18 - 77.7mb

Now if I am just to play with RF I might found the "optimal" but just maybe there are some settings enabled in production max that enlarges the file but does not give any results to the footage I took.
And it looks that RF is no constant formula, one videos can get smaller in this case, and other ones might get bigger by 150%?

[BradleyS]

Remember that for editing purposes, you need additional quality beyond what you perceive to account for the generational losses created by editing intermediates and the inevitable final encode by your editing software.

[rollin_eng]

If you want the 'best' quality use RF=0, this is lossless. It will be bigger than your original but of equal quality.

If you want 'pretty close to the original' quality use RF in the 18-22 range.

If you want the best quality with a certain size use 2 pass encoding at your required bit rate.

[BradleyS]

x264 lossless at RF 0 is not compatible with Premiere. And the guidelines that apply to home archival encoding do not apply to post production intermediates.

This is why the Production presets exist, with their respective RF of 2 for Max and 10 for Standard. Something like RF 15/16 might be considered an unofficial Production Lite.

[rollin_eng]

True about Premier, I was focusing on the subject ‘Lossless’.

This guy seems to be jumping around about lossless one minute, then size dependent the next.

[lashara4]

Thank you guys, I guess I will go with RF18, because I noticed for some files it produces sometimes bigger, sometimes lower file sizes which works for me.

As for the lossless - I am new to this field, and by lossless I meant "as it is now" - I was not aware that the file can grow 10x bigger when using prod max presets with RF 2.
This is something that I do not get mathematically - if I have a faulty bucket (error playback) with 10 apples (2.7k, 59.94fps) and I need to change the bucket, I wanted to find the perfect size bucket. If I use smaller, then I will fit only 6 apples (loss in quality). If I use that big blue chemical waste bucket (RF 2) for 10 apples, it will be over-processing and definitely my apples wont grow from 10 to 200. So when I use lossless in the first place, I was thinking about a format which would be as just a perfect fit for those 10 apples that I had in the beginning.

Now I know that my logic does not work in video production, but knowing does not mean I understand. Because still, thats how my logic works when I get a file size 1500mb from 80mb it is just a 1420mb wasted on improving things that there are not any in the first place.

[BradleyS]

It's important to recognize the steps a transcoder takes between your source video and the (re-)encoded video, notably the part where each frame is decompressed into uncompressed picture planes before encoding to the final format. Basically, the original compression is not taken into account when encoding, only the decoded picture. So the bucket analogy... forget that one.

Encoding, except in the truly mathematically lossless case, introduces generational error that compounds with each subsequent encode. Think of it as a JPEG made from a JPEG made from a JPEG. Or an analogy from analog times, a tape of a tape of a tape.

Quality cannot get better, only get worse. The rate at which it gets worse depends on what was thrown away each generation. This is why people try to start with a high quality source, edit using high quality intermediates, and only for the final encode compress heavily; minimal generational loss means more detail in the end (and more latitude during editing). Good practice is to throw more bits at intermediates for editing because we want the most data possible to work with, and to offset the generational loss we expect when it's time for the final encode.

If you really want to see the effects of generational loss, using the same RF, make a video of a video of a video of a video... how much loss you perceive will be worse with higher RF (lower quality). RF 30-30-30-30 will visually degrade rapidly, while 2-2-2-2 might not at all. In both cases, the file size will inflate due to the compounding quantization error.

[lashara4]

Thank you for explaining, it makes sense now to me. I will use production max where as RF18 will be set, hopefully I will not see the difference (from 2.7k) in the final result (1080p).
I am very glad that I found handbrake and it served me in a different purpose, as I understand it is not common practice to use it in a way I am using (make error free mp4).
Also lesson learned - backup, backup, backup.