Good rule of thumb for compressing a DVD using H.265
Re: Good rule of thumb for compressing a DVD using H.265
That certainly crossed my mind, however I also wanted the capability to log into my NAS box and view the movies remotely. The data rate required is just too high for a typical 5-6GB movie.
-
rollin_eng
- Veteran User
- Posts: 4854
- Joined: Wed May 04, 2011 11:06 pm
Re: Good rule of thumb for compressing a DVD using H.265
DVD's only have a date rate of 10Mbps, this should be doable over most wifi.
Re: Good rule of thumb for compressing a DVD using H.265
I'm talking about while I'm at a friend's house or on vacation (i.e. far away from the server). In my experience there's often far too much hanging and stuttering at those file sizes. Obviously that's also a function of the local internet bandwidth (especially at a hotel).
-
rollin_eng
- Veteran User
- Posts: 4854
- Joined: Wed May 04, 2011 11:06 pm
Re: Good rule of thumb for compressing a DVD using H.265
Well now you are talking about compressing video for streaming over the internet which can have it's own issues.
What might be a good idea is to have one uncompressed copy for local viewing on your fancy 4k TV and a second highly compressed copy for remote streaming. You certainly don't want to upscale at encoding if you are planning on streaming it over the internet.
What might be a good idea is to have one uncompressed copy for local viewing on your fancy 4k TV and a second highly compressed copy for remote streaming. You certainly don't want to upscale at encoding if you are planning on streaming it over the internet.
Re: Good rule of thumb for compressing a DVD using H.265
So I've already decided not to upscale - I'm going to leave that to the TV.
This all started out with my 5x4GB NAS box (and it's duplicate hardware backup) having filled up with VOB rips that I've done over the past 25 years. Throughout that time I've repeatedly been buying bigger and bigger drives - copying my VOBs across as my collection grew, reluctant to compress anything further until the compression algorithms got to the point where file sizes could come down with only barely noticeable additional artifacts. I thought H.265 would be that savior and that's what started this thread. I've now learned that H.265 is just not effective for 480p DVDs.
(Captain's personal log: It still wigs me out that, after a quarter century, my only DVD compression options still lie with the same old antique H.264 algorithms.)
Anyway, I'm highly reluctant to spend another couple grand or so doubling my hard drive capacity (which should see me through to my expiration date) and am sorely hoping something is possible in the compression domain...
This all started out with my 5x4GB NAS box (and it's duplicate hardware backup) having filled up with VOB rips that I've done over the past 25 years. Throughout that time I've repeatedly been buying bigger and bigger drives - copying my VOBs across as my collection grew, reluctant to compress anything further until the compression algorithms got to the point where file sizes could come down with only barely noticeable additional artifacts. I thought H.265 would be that savior and that's what started this thread. I've now learned that H.265 is just not effective for 480p DVDs.
(Captain's personal log: It still wigs me out that, after a quarter century, my only DVD compression options still lie with the same old antique H.264 algorithms.)
Anyway, I'm highly reluctant to spend another couple grand or so doubling my hard drive capacity (which should see me through to my expiration date) and am sorely hoping something is possible in the compression domain...
-
rollin_eng
- Veteran User
- Posts: 4854
- Joined: Wed May 04, 2011 11:06 pm
Re: Good rule of thumb for compressing a DVD using H.265
There is nothing wrong with h265, if your hardware supports it go for it.
Your problem is that you want your DVD encode to look good on a 75" 4k TV yet be small enough to stream over the internet.
Your problem is that you want your DVD encode to look good on a 75" 4k TV yet be small enough to stream over the internet.
Re: Good rule of thumb for compressing a DVD using H.265
But I've been told a couple times already in this thread that H.265 will either make things noticeably worse for 480p content or that it's just not worth pursuing because improvements only manifest themselves for >1k/4k media. I'm confused and not sure what to make of this (no offense!!). Given that I'm in the Apple camp, my hardware should be fine for H.265.rollin_eng wrote: ↑Sun Jan 30, 2022 5:21 pm There is nothing wrong with h265, if your hardware supports it go for it.
Maybe better reworded as "look barely any worse" on a 75" 4k TV that has native upscaling capabilities.rollin_eng wrote: ↑Sun Jan 30, 2022 5:21 pm Your problem is that you want your DVD encode to look good on a 75" 4k TV yet be small enough to stream over the internet.
That being said, I get the conflict/dilemma. I'd be delighted with all file sizes compressing by only 50%. Even with that, the resultant file size would still be, from memory, bigger or similar to the HD version of the same movie you can buy online (which admittedly had better quality source material than SD from which to start the compression).
-
rollin_eng
- Veteran User
- Posts: 4854
- Joined: Wed May 04, 2011 11:06 pm
Re: Good rule of thumb for compressing a DVD using H.265
No one has said h265 would make things worse, just that it’s benefits over x264 aren’t really noticeable unless you are encoding 4K or at low bitrates.
Encoding mpeg2 video to x264/5 then upscaling to a 75” 4k is probably going to look dodgy no matter what.
Encoding mpeg2 video to x264/5 then upscaling to a 75” 4k is probably going to look dodgy no matter what.
-
rollin_eng
- Veteran User
- Posts: 4854
- Joined: Wed May 04, 2011 11:06 pm
Re: Good rule of thumb for compressing a DVD using H.265
If you remux your DVD to mkv then play that on your TV does it look good?
Re: Good rule of thumb for compressing a DVD using H.265
I will certainly try that and get back to you (although that wouldn't really solve my full NAS box issue). Possibly a silly question - if it's just a remux (i.e. no transcoding), why wouldn't it look exactly the same?
-
rollin_eng
- Veteran User
- Posts: 4854
- Joined: Wed May 04, 2011 11:06 pm
Re: Good rule of thumb for compressing a DVD using H.265
It should look exactly like the original. But does that look good on your 75” TV?
-
soulshadow
- Posts: 2
- Joined: Tue Feb 01, 2022 5:38 am
Re: Good rule of thumb for compressing a DVD using H.265
So, based on my experience ripping DVD's and BluRays and transcoding them to both h.264 and h.265 mkv's for private use on my home NAS, my advice is this:
Transcoding is all about compromises. Quality vs size vs encoding speed. What are your priorities and what works best for you?
When all else is equal, h.265 compared to h.264 provides a reduction in file size great enough that (for me) it outweighs the downside of longer encode times. I'm talking, for a typical feature length movie, 1-3 hours for h.264 with no filters and 3-5 hours for an h.264 with NLMeans filter VS. 3-5 hours for h.265 with no filters and 10-15 hours for h.265 with NLMeans filter.
HOWEVER h.265 is much more sensitive to visual noise. As the noise increases, the h.265 bitrate (and therefore the file size) also increases. It does so at a rate higher then h.264. The reason h.265 isn't used as much for older SD quality video (In my opinion) has more to do with the higher level of noise present in the source media. The effects can easily be seen by running multiple test encodes on the same source media using h.264 and h.265 with the only change being the use of Denoise NLMeans presets. What this boils down to is that if you have noisy video, as many DVD's will, then you'd need to employ denoise in your encoding process so see any benefit of h.265 over h.264. Even with h.264 you can often see multiple gigabytes in file size reduction by adding the NLMeans UltraLight preset to you encode settings.
As an example of real data that might pertain to your DVD situation, a significant amount of extras on BluRay Discs (BD) for movies released prior to about 2008 are still in SD (Standard Definition, aka 480i/p). When transcoding non-interlaced video using h.265 RF 18, Preset: Slow, Tune: Film, Profile: High, Level: 4.1 with denoise filter NLMeans preset Medium the resulting files were about 50-90% smaller then the originals (anywhere from 1..5-5mbit). Personally, unlike the extras, for the feature films I use the above settings with RF16 instead of RF18 and the lowest preset for NLMeans that meets my size/average bitrate target.
I would encourage anyone to take some time and do careful testing to determine the settings that make sense for YOUR needs. My needs prioritized quality over file size, up to a limit of about 8mbit or 60mb/minute. For me that results in a 2 hour movie being reduced from 30Gb on the bluray to about 7Gb as either an h.264 RF18 mkv or an h.265 RF16 mkv (I do both because of some legacy device support, though h.265 is my preferred format until/unless av1 becomes viable)
Here's my settings, which can be used as a starting point for your own testing. They were developed to meet my needs (as noted in the previous paragraph) through testing on bluray source material, however at 1/3 - 1/4 the pixels they should scale nicely to DVD source material as well:
First off, content is cropped only to remove black bars. No upscaling is done and aspect ratios and animorphic content is preserved.
For feature movies and TV series I use:
h.264: RF 18 Constant Quality, Preset: Slow, Tune: Film or Animation, Profile: High, Level: 4.1.
h.265: RF 16 Constant Quality, Preset: Slow, Tune: None or Animation, Profile: Main10, Level: 5.1
NLMeans filter used from off up to preset Medium as needed to help bring destination file within my 8mbit or 60mb/min target for FHD (Full HD, aka 1080p). I've found no benefit to transcodes from DVD/BD by using anything other than the Denoise Tune "Film". There may be uses for other tunes, but no compelling data in my testing.
For DVD and BD extras (which are less important to me) I use:
h.265 RF 18 Constant Quality, Preset: Slow, Tune: Film or Animation, Profile: Main10, Level: 5.1 - Denoise NLMeans, Medium - Film
As for audio, that's it's own topic. For testing though I'd just pick 1 setup and stick with it.
I chose h.265 10-bit because it should produce better results with gradients and reduced banding. As such I've seen no issues there, combined with my other settings, including denoise. Maybe it's a fluke, maybe not. Ultimately my testing found no downsides to using h.265 10-bit vs 8-bit for encoding, including file size and encode time. Also, all the players I use that support h.265 also support h.265 10-bit. So, why not.
As for the Profile/Level combinations, those are based on industry standards and represent a maximum limit that most hardware (TV's, DVD/BluRay players) is built to handle.
**** Interlaced Content ****
Deinterlaced content tends to only see a 30-50% reduction compared to the original file size. This could be due to my deinterlace filter choices or just as a result of the deinterlacing process. I don't know. Any interlaced content is deinterlaced using decomb with default preset (for some reason, other settings either result in larger file sizes or _extremely_ long encode times, this combination is the best I've found, so far). *Note: I leave Interlace Detection off unless I know the content has at least some interlacing. I use ffmpeg directly to scan the sources and then I evaluate the last 3 lines of the results. From the command line on linux I run: However if you routinely deal with interlaced content, as would be the case with DVDs, then leaving the interlace detection on might just be the easier solution.
-------------
I initially wrote about 20 long paragraphs going into detail about my encode settings and testing, but started fresh on this "short" post instead. I may still post it as a new topic in the future as I found some interesting patterns in my testing.
Transcoding is all about compromises. Quality vs size vs encoding speed. What are your priorities and what works best for you?
When all else is equal, h.265 compared to h.264 provides a reduction in file size great enough that (for me) it outweighs the downside of longer encode times. I'm talking, for a typical feature length movie, 1-3 hours for h.264 with no filters and 3-5 hours for an h.264 with NLMeans filter VS. 3-5 hours for h.265 with no filters and 10-15 hours for h.265 with NLMeans filter.
HOWEVER h.265 is much more sensitive to visual noise. As the noise increases, the h.265 bitrate (and therefore the file size) also increases. It does so at a rate higher then h.264. The reason h.265 isn't used as much for older SD quality video (In my opinion) has more to do with the higher level of noise present in the source media. The effects can easily be seen by running multiple test encodes on the same source media using h.264 and h.265 with the only change being the use of Denoise NLMeans presets. What this boils down to is that if you have noisy video, as many DVD's will, then you'd need to employ denoise in your encoding process so see any benefit of h.265 over h.264. Even with h.264 you can often see multiple gigabytes in file size reduction by adding the NLMeans UltraLight preset to you encode settings.
As an example of real data that might pertain to your DVD situation, a significant amount of extras on BluRay Discs (BD) for movies released prior to about 2008 are still in SD (Standard Definition, aka 480i/p). When transcoding non-interlaced video using h.265 RF 18, Preset: Slow, Tune: Film, Profile: High, Level: 4.1 with denoise filter NLMeans preset Medium the resulting files were about 50-90% smaller then the originals (anywhere from 1..5-5mbit). Personally, unlike the extras, for the feature films I use the above settings with RF16 instead of RF18 and the lowest preset for NLMeans that meets my size/average bitrate target.
I would encourage anyone to take some time and do careful testing to determine the settings that make sense for YOUR needs. My needs prioritized quality over file size, up to a limit of about 8mbit or 60mb/minute. For me that results in a 2 hour movie being reduced from 30Gb on the bluray to about 7Gb as either an h.264 RF18 mkv or an h.265 RF16 mkv (I do both because of some legacy device support, though h.265 is my preferred format until/unless av1 becomes viable)
Here's my settings, which can be used as a starting point for your own testing. They were developed to meet my needs (as noted in the previous paragraph) through testing on bluray source material, however at 1/3 - 1/4 the pixels they should scale nicely to DVD source material as well:
First off, content is cropped only to remove black bars. No upscaling is done and aspect ratios and animorphic content is preserved.
For feature movies and TV series I use:
h.264: RF 18 Constant Quality, Preset: Slow, Tune: Film or Animation, Profile: High, Level: 4.1.
h.265: RF 16 Constant Quality, Preset: Slow, Tune: None or Animation, Profile: Main10, Level: 5.1
NLMeans filter used from off up to preset Medium as needed to help bring destination file within my 8mbit or 60mb/min target for FHD (Full HD, aka 1080p). I've found no benefit to transcodes from DVD/BD by using anything other than the Denoise Tune "Film". There may be uses for other tunes, but no compelling data in my testing.
For DVD and BD extras (which are less important to me) I use:
h.265 RF 18 Constant Quality, Preset: Slow, Tune: Film or Animation, Profile: Main10, Level: 5.1 - Denoise NLMeans, Medium - Film
As for audio, that's it's own topic. For testing though I'd just pick 1 setup and stick with it.
I chose h.265 10-bit because it should produce better results with gradients and reduced banding. As such I've seen no issues there, combined with my other settings, including denoise. Maybe it's a fluke, maybe not. Ultimately my testing found no downsides to using h.265 10-bit vs 8-bit for encoding, including file size and encode time. Also, all the players I use that support h.265 also support h.265 10-bit. So, why not.
As for the Profile/Level combinations, those are based on industry standards and represent a maximum limit that most hardware (TV's, DVD/BluRay players) is built to handle.
**** Interlaced Content ****
Deinterlaced content tends to only see a 30-50% reduction compared to the original file size. This could be due to my deinterlace filter choices or just as a result of the deinterlacing process. I don't know. Any interlaced content is deinterlaced using decomb with default preset (for some reason, other settings either result in larger file sizes or _extremely_ long encode times, this combination is the best I've found, so far). *Note: I leave Interlace Detection off unless I know the content has at least some interlacing. I use ffmpeg directly to scan the sources and then I evaluate the last 3 lines of the results. From the command line on linux I run:
Code: Select all
ffmpeg -filter:v idet -frames:v 1440 -an -f rawvideo -v /dev/null -i <filename> 2>&1 | tail -n3-------------
I initially wrote about 20 long paragraphs going into detail about my encode settings and testing, but started fresh on this "short" post instead. I may still post it as a new topic in the future as I found some interesting patterns in my testing.
Re: Good rule of thumb for compressing a DVD using H.265
I did the VOB->MKV conversion and it looks just like the original VOB file, as expected. It looks fine on my 75" TV (given that it's a DVD) - my concern all along has been how the subsequent compression looks. I've been playing with HQ and SHQ 480p H.264 compressions and am still disappointed by color banding in several movies (even with SHQ! and whatever interpolating my fancy TV does). On a whim, I took my HQ preset and changed the encoder from H.264 to H.265 (leaving everything else the same) and re-ran the compression. Based on earlier comments, I wasn't expecting to see anything of consequence but it definitely made a totally worthwhile improvement the color banding effects (that I could live with), and as a bonus the file size was also smaller! Obviously the transcoding time was longer, but I don't mind that. Anyway, now I'm not so sure about all the H.265 downplay for DVDs that I was hearing.rollin_eng wrote: ↑Sun Jan 30, 2022 6:59 pm It should look exactly like the original. But does that look good on your 75” TV?
I'm currently going through several DVDs and extracting some problematic scenes that highlight H.264 color banding in an attempt to prove if H.265 is indeed useful or not for DVDs. I'll keep you posted when I get more results.
Re: Good rule of thumb for compressing a DVD using H.265
soulshadow: Many thanks for your great post. I'm currently working on my H.265 experiments, and will digest your comments after that. In the meantime, I've also discovered another promising site that talks about NOT using constant quality encoding that I also want to explore. Scroll down to the "Explanation" heading in https://github.com/donmelton/video_transcoding
-
rollin_eng
- Veteran User
- Posts: 4854
- Joined: Wed May 04, 2011 11:06 pm
Re: Good rule of thumb for compressing a DVD using H.265
I think the moral here is: Try things and see what works for YOU.
Glad you have found what you need (hopefully).
Glad you have found what you need (hopefully).
-
rollin_eng
- Veteran User
- Posts: 4854
- Joined: Wed May 04, 2011 11:06 pm
Re: Good rule of thumb for compressing a DVD using H.265
I believe the donmelton tool is designed to stop bitrates getting to high or low, but give it a try and find out.
Re: Good rule of thumb for compressing a DVD using H.265
Finally, here are my results (in a 280MB download):
https://www.dropbox.com/s/fqu6dfxm2juv3 ... g.zip?dl=0
The contents of the "Read Me.txt" file in the download is pasted below:
I'm trying to compress my 20TB collection of ripped, but uncompressed, DVD movies (i.e. this is all focused on SD video, not HD video via Blu-rays). DVDs are particularly susceptible to further compression artifacts because they're already starting from a place of high compression (unlike Blu-rays). I had previously already gone through this compression process (which took 2-3 months) only to find out that, amongst other things, many of the compressed movies had awful video banding typically in dimly lit or highly monochromatic scenes. Banding is by far the worst and most noticeable compression artifact (for me). Next would be the jittery effects you get when zooming in/out on scenes with lots of edges (e.g. skyscrapers with many windows, or the long wide steps at city hall etc.). My goal was to do a great job on the compressed video quality, sacrificing compression time as needed. I'm hopeful I can just end up with a one size fits all HandBrake setting, but who knows...
As advised by the HandBrake forum "experts", because I'm dealing with standard definition 480p DVDs, there is simply no point using any HandBrake preset exceeding 480p. I therefore started with the standard presets "HQ 480p30 Surround" and "Super HQ 480p30 Surround" as my baselines. I was also told that, for DVDs, the newer H.265 compression algorithm simply wasn't worth the effort because it's effects would only be noticeable at either really low bit rates or for 4K (or better) video. However, I also wanted to prove that for myself while doing my experiments.
Here are the three compression settings I used for HandBrake (each targeted to make an MKV file - via the Summary tab):
"SHQ H264" files come from the standard preset "Super HQ 480p30 Surround"
"HQ H264" files come from the standard preset "HQ 480p30 Surround"
"HQ H265" files come from the above "HQ H264" setting, but with the video encoder subsequently changed to H.265
Notes:
1. In each case I also modified the preset for the video framerate to be "Same as Source" (because I hear that's just better/safer).
2. I did not do a "SHQ H265" version because the compression time, for the corresponding movie itself, was exponentially bigger than for the "HQ H265" version (I saw numbers like 8 hours or more!). The compression time for "HQ H265" was still quite acceptable - it did take longer than the rest, but it also yielded the smallest file size.
3. I confess to not fully understanding all the HandBrake settings (which is why I started with the recommended presets). I'm unclear if the H.264 preset parameters (e.g. filter choices, RF quality value) either are valid or carry the same weight/meaning when the H.265 encoder is subsequently selected.
Included with this Read Me file are several "problematic" DVD video clips that readily produce banding when compressed, even though the are acceptable to view in their RAW DVD state. The compression used for each clip is obvious from its file name. Most people will probably only look at a few examples and move on, so given that I at least recommend focusing on:
Moonlight (lighting) <76 seconds>
Mist, The (fog, grainy) <27 seconds>
Lawrence of Arabia (direct sun) <10 seconds>
Moonlight (table leg jitter) <8 seconds - this one is to highlight some "jitter">
Please first watch the original DVD clip, then "HQ H264", then "SHQ H264" and finally "HQ H265".
So here are my thoughts/conclusions based on viewing the results on my 75" Samsung QLED TV:
- Absolutely, positively, no question about it: "HQ H265" does the best job on reducing banding effects. This is counter to what I was told in the forums :-/. It also produces the smallest file sizes - except for only the "Moonlight (table leg jitter)" clip.
- I didn't particularly focus on other video artifacts but, aside from banding, I found that a movie compressed with "HQ H265" otherwise compared similarly with the other two compression types.
- I don't know enough about the H.264 and H.265 HandBrake setting to say that there may exist some even higher quality H.264 setting that takes the same time and yields the same quality as the H.265 setting I used. That being said, I doubt it would end up with the same small file size.
Based on the above, I find myself wishing that the HandBrake team could make some optimized H.265 specific presets (because those guys know much more about tuning all the H.265 settings) so people like me could just use them.
Anyway, this has been a chunk of work for me that I hope will be useful to others.
https://www.dropbox.com/s/fqu6dfxm2juv3 ... g.zip?dl=0
The contents of the "Read Me.txt" file in the download is pasted below:
I'm trying to compress my 20TB collection of ripped, but uncompressed, DVD movies (i.e. this is all focused on SD video, not HD video via Blu-rays). DVDs are particularly susceptible to further compression artifacts because they're already starting from a place of high compression (unlike Blu-rays). I had previously already gone through this compression process (which took 2-3 months) only to find out that, amongst other things, many of the compressed movies had awful video banding typically in dimly lit or highly monochromatic scenes. Banding is by far the worst and most noticeable compression artifact (for me). Next would be the jittery effects you get when zooming in/out on scenes with lots of edges (e.g. skyscrapers with many windows, or the long wide steps at city hall etc.). My goal was to do a great job on the compressed video quality, sacrificing compression time as needed. I'm hopeful I can just end up with a one size fits all HandBrake setting, but who knows...
As advised by the HandBrake forum "experts", because I'm dealing with standard definition 480p DVDs, there is simply no point using any HandBrake preset exceeding 480p. I therefore started with the standard presets "HQ 480p30 Surround" and "Super HQ 480p30 Surround" as my baselines. I was also told that, for DVDs, the newer H.265 compression algorithm simply wasn't worth the effort because it's effects would only be noticeable at either really low bit rates or for 4K (or better) video. However, I also wanted to prove that for myself while doing my experiments.
Here are the three compression settings I used for HandBrake (each targeted to make an MKV file - via the Summary tab):
"SHQ H264" files come from the standard preset "Super HQ 480p30 Surround"
"HQ H264" files come from the standard preset "HQ 480p30 Surround"
"HQ H265" files come from the above "HQ H264" setting, but with the video encoder subsequently changed to H.265
Notes:
1. In each case I also modified the preset for the video framerate to be "Same as Source" (because I hear that's just better/safer).
2. I did not do a "SHQ H265" version because the compression time, for the corresponding movie itself, was exponentially bigger than for the "HQ H265" version (I saw numbers like 8 hours or more!). The compression time for "HQ H265" was still quite acceptable - it did take longer than the rest, but it also yielded the smallest file size.
3. I confess to not fully understanding all the HandBrake settings (which is why I started with the recommended presets). I'm unclear if the H.264 preset parameters (e.g. filter choices, RF quality value) either are valid or carry the same weight/meaning when the H.265 encoder is subsequently selected.
Included with this Read Me file are several "problematic" DVD video clips that readily produce banding when compressed, even though the are acceptable to view in their RAW DVD state. The compression used for each clip is obvious from its file name. Most people will probably only look at a few examples and move on, so given that I at least recommend focusing on:
Moonlight (lighting) <76 seconds>
Mist, The (fog, grainy) <27 seconds>
Lawrence of Arabia (direct sun) <10 seconds>
Moonlight (table leg jitter) <8 seconds - this one is to highlight some "jitter">
Please first watch the original DVD clip, then "HQ H264", then "SHQ H264" and finally "HQ H265".
So here are my thoughts/conclusions based on viewing the results on my 75" Samsung QLED TV:
- Absolutely, positively, no question about it: "HQ H265" does the best job on reducing banding effects. This is counter to what I was told in the forums :-/. It also produces the smallest file sizes - except for only the "Moonlight (table leg jitter)" clip.
- I didn't particularly focus on other video artifacts but, aside from banding, I found that a movie compressed with "HQ H265" otherwise compared similarly with the other two compression types.
- I don't know enough about the H.264 and H.265 HandBrake setting to say that there may exist some even higher quality H.264 setting that takes the same time and yields the same quality as the H.265 setting I used. That being said, I doubt it would end up with the same small file size.
Based on the above, I find myself wishing that the HandBrake team could make some optimized H.265 specific presets (because those guys know much more about tuning all the H.265 settings) so people like me could just use them.
Anyway, this has been a chunk of work for me that I hope will be useful to others.
Re: Good rule of thumb for compressing a DVD using H.265
The encoder presets are a great way to tune x265 to your speed/efficiency preference.
Re: Good rule of thumb for compressing a DVD using H.265
Just thought I'd also clarify something (that might be obvious to many) regarding my various compression test clips. If you view them on your computer screen you'll be hard pressed to see any video banding compression artifact differences. However, they are painfully obvious when magnified onto a big screen TV - especially when viewed in a dark room i.e. pretty much the very conditions in place for all of my movie watching at home.
Re: Good rule of thumb for compressing a DVD using H.265
soulshadow: I'm interested in understanding your testing a little more, even including your longer version of testing out encoder settings. I get that it might be a little overwhelming top post, but I'd be happy to receive it privately.
I thought I'd ask about your compression times. You say "for a typical feature length movie, 1-3 hours for h.264 with no filters and 3-5 hours for an h.264 with NLMeans filter VS. 3-5 hours for h.265 with no filters and 10-15 hours for h.265 with NLMeans filter." Are you referring to a Blu-ray compression or a DVD compression? I only have DVD source material but, while I certainly share your quality over compression time ideals, those compression times exceed my thresholds. I'd possibly die of old age before my entire library is compressed
. My upper threshold for compression time would be around the duration of the movie itself.
Given the lengthy compression times, did you base your evaluations on full movies or did you have favorite "pathological" clips to speed up the process?
I do have several more questions, but thought I'd just start with this.
I thought I'd ask about your compression times. You say "for a typical feature length movie, 1-3 hours for h.264 with no filters and 3-5 hours for an h.264 with NLMeans filter VS. 3-5 hours for h.265 with no filters and 10-15 hours for h.265 with NLMeans filter." Are you referring to a Blu-ray compression or a DVD compression? I only have DVD source material but, while I certainly share your quality over compression time ideals, those compression times exceed my thresholds. I'd possibly die of old age before my entire library is compressed
. My upper threshold for compression time would be around the duration of the movie itself.Given the lengthy compression times, did you base your evaluations on full movies or did you have favorite "pathological" clips to speed up the process?
I do have several more questions, but thought I'd just start with this.
-
soulshadow
- Posts: 2
- Joined: Tue Feb 01, 2022 5:38 am
Re: Good rule of thumb for compressing a DVD using H.265
Sorry for the delay, just saw the reply. So, when I first started copying my disc collection to my computer hard drives I spent over 6 months of free time researching and experimenting. While already knowledgeable about computers and programming I learned a lot of new things. So, before I directly answer your questions there's a few things that I learned which I should briefly mention:
Encoding:
First, don't confuse encoding and compression. Technically compression is a form of lossless encoding, but for video the terms refer to different parts of the process by which the visual data is stored. Yes, different encoding schemes do employ different methods and levels of compression, but that is not the important difference. The big difference is the way the visual data is represented in the storage file. Encoding is a big topic in it's own right, though at a high level it's fairly easy to understand. Simply put, instead of storing information about the exact pixels you see, each encoding scheme has a way of storing information about common patterns of pixels. Think of it as storing a description of a picture instead of the actual picture. How it does this is the biggest difference between codecs. Newer schemes usually produce better visual accuracy while also reducing the size of the descriptions. The trade off is higher complexity in the encoding process which is manifest in longer processing times and/or higher computational complexity. If you want to learn more try web searching for information on video encoding, especially h.264 and h.265. A good place to start is by learning about Huffman coding (try the "computerphile" channel on youtube for some great videos including Huffman coding and their playlist "How JPEG works" which is similar enough to how mpeg and it's descendants (h.264/h.265) work).
Also, be aware that we are using Handbrake to transcode video files. Transcoding is the process of taking an already encoded video, decoding it then encoding it again but in a different way, usually with a different algorithm (or codec). It's a small distinction and the terms encoding and transcoding are often used interchangeably (including by myself). But it is important to know the difference because video encoding is a lossy process and each algorithm discards different information.
Video resolutions:
For those that don't know, video resolution is typically referred to by the vertical size of the picture, IE: 480, 720, 1080, 2160. It works good for TV's and monitors to understand what they are capable of displaying. However when talking about video files using those numbers can be misleading or just confusing to some. This is because those standard screen sizes don't always correspond to the size of the video as it's stored or displayed. Most notably if you crop unneeded black bars off the edges of movies that were shot in an aspect ratio other than 16:9 or 4:3 then the resulting video file will have at least one size that differs from the norm.
Instead of using the vertical numbers to identify video content I encode, I use the highest classification that the video would fit into natively (this is probably the same logic other people also use, but best not to make assumptions):
When it comes to consumer video you'll mainly see SD and FHD content with some occasional HD content. Though anything that exceeds SD formats is considered and referred to as HD. Which just makes it all the more confusing.
There are some additional variations and inconsistencies between marketing terms and different manufacturers, but they can be mostly ignored.
Lastly, for those dealing with DVD content you should also read up on Anamorphic video and understand what's going on and why. It's too much to try to sum up here. Personally, if the source video is anamorphic I preserve this and it's sizes/aspect ratios in the output file. I'll crop if needed and let handbrake do the math to preserve the aspect ratio without resizing the video.
Speed:
So there are, of course, a lot of variables that go into the over-all encode time. First and foremost is your computer setup. To crunch those numbers during encoding takes a lot of work. However any modern (3-4 years old or newer) 6-8 core Intel or AMD chip should yield roughly the same result. I currently have 3 laptops encoding nearly 24/7 for the past 4 months. One Intel core I7 from 2020. One AMD Ryzen 5 and one Ryzen 7 from 2021. They are not high end gaming machines though not strictly budget equipment either. The Intel encodes slightly faster then the others, though the difference between all 3 is so minute that it's very hard to observe. (The Ryzen's run MUCH cooler which means less electricity is used and they also cost less to buy) Older/slower or budget options may run a little slower. I only mention hardware because it will be a baseline factor no matter what encoder or settings are used. So, aside from hardware constraints here's the biggest factors in encode time that I've found:
OK, with that background info out of the way lets dive into your questions:
Here are links to some presets I created:
One final note for this post, and I think others have already mentioned this. Even without transcoding, the visual quality of SD content will suffer when viewed on a large modern TV such as yours. While most of the better manufacturers have hardware and software built into their TV's to "upscale" lower resolution content, it's still about 1/4th of the visual information that is present in FHD content. It will never look as good as any HD content.
Maybe in the next week or so I'll pause my current BD queue and do a few DVD's to compare.
Encoding:
First, don't confuse encoding and compression. Technically compression is a form of lossless encoding, but for video the terms refer to different parts of the process by which the visual data is stored. Yes, different encoding schemes do employ different methods and levels of compression, but that is not the important difference. The big difference is the way the visual data is represented in the storage file. Encoding is a big topic in it's own right, though at a high level it's fairly easy to understand. Simply put, instead of storing information about the exact pixels you see, each encoding scheme has a way of storing information about common patterns of pixels. Think of it as storing a description of a picture instead of the actual picture. How it does this is the biggest difference between codecs. Newer schemes usually produce better visual accuracy while also reducing the size of the descriptions. The trade off is higher complexity in the encoding process which is manifest in longer processing times and/or higher computational complexity. If you want to learn more try web searching for information on video encoding, especially h.264 and h.265. A good place to start is by learning about Huffman coding (try the "computerphile" channel on youtube for some great videos including Huffman coding and their playlist "How JPEG works" which is similar enough to how mpeg and it's descendants (h.264/h.265) work).
Also, be aware that we are using Handbrake to transcode video files. Transcoding is the process of taking an already encoded video, decoding it then encoding it again but in a different way, usually with a different algorithm (or codec). It's a small distinction and the terms encoding and transcoding are often used interchangeably (including by myself). But it is important to know the difference because video encoding is a lossy process and each algorithm discards different information.
Video resolutions:
For those that don't know, video resolution is typically referred to by the vertical size of the picture, IE: 480, 720, 1080, 2160. It works good for TV's and monitors to understand what they are capable of displaying. However when talking about video files using those numbers can be misleading or just confusing to some. This is because those standard screen sizes don't always correspond to the size of the video as it's stored or displayed. Most notably if you crop unneeded black bars off the edges of movies that were shot in an aspect ratio other than 16:9 or 4:3 then the resulting video file will have at least one size that differs from the norm.
Instead of using the vertical numbers to identify video content I encode, I use the highest classification that the video would fit into natively (this is probably the same logic other people also use, but best not to make assumptions):
- SD (Standard Definition) = 640 x 480 (4:3 aspect) Most DVD content** (There are some variations in dimensions, especially with NTSC and PAL video)
- HD (High Definition) = 1280 x 720 (16:9 aspect) anything that exceeds SD formats is considered HD)
- FHD (Full High Definition) = 1920 x 1080 (16:9 aspect) Most BluRay content
- QHD (Quad High Definition) = 2560 x 1440 (16:9 aspect)
- UHD/4k (Ultra High Definition) = 3840 x 2160 (16:9 aspect)
When it comes to consumer video you'll mainly see SD and FHD content with some occasional HD content. Though anything that exceeds SD formats is considered and referred to as HD. Which just makes it all the more confusing.
There are some additional variations and inconsistencies between marketing terms and different manufacturers, but they can be mostly ignored.
Lastly, for those dealing with DVD content you should also read up on Anamorphic video and understand what's going on and why. It's too much to try to sum up here. Personally, if the source video is anamorphic I preserve this and it's sizes/aspect ratios in the output file. I'll crop if needed and let handbrake do the math to preserve the aspect ratio without resizing the video.
Speed:
So there are, of course, a lot of variables that go into the over-all encode time. First and foremost is your computer setup. To crunch those numbers during encoding takes a lot of work. However any modern (3-4 years old or newer) 6-8 core Intel or AMD chip should yield roughly the same result. I currently have 3 laptops encoding nearly 24/7 for the past 4 months. One Intel core I7 from 2020. One AMD Ryzen 5 and one Ryzen 7 from 2021. They are not high end gaming machines though not strictly budget equipment either. The Intel encodes slightly faster then the others, though the difference between all 3 is so minute that it's very hard to observe. (The Ryzen's run MUCH cooler which means less electricity is used and they also cost less to buy) Older/slower or budget options may run a little slower. I only mention hardware because it will be a baseline factor no matter what encoder or settings are used. So, aside from hardware constraints here's the biggest factors in encode time that I've found:
- Resolution. Most of my experience so far has been with bluray/FHD* video. From what I've seen though, the lower the resolution the shorter the encode times.
- The "Preset" slider for h.264 and h.265. Set to "Medium" you will usually process around 1-2 minutes of FHD video per minute of encode time.
- Deinterlace filter. I store all my video files as progressive, but I'll run deinterlace only when needed. Even the interlace detection will slow down you encode times.
- Denoise filter. This is where I personally see a lot of my encoding time going. However is is also one of the best ways to reduce file size and h.265 is much more sensitive to noise then h.264.
OK, with that background info out of the way lets dive into your questions:
I've run testing on both video clips and full length FHD movies. At first I was anxious for encoding to be done fast so I could see the results. But these days most of the time my computers are running unattended while I do other things and only require a little time to create the queue files with handbrake. Admittedly having multiple computers working on multiple files really helped with testing, but when it comes to the day to day process of converting my library it's mostly set it up and check in periodically to monitor progress, create new queue files and clean up the old files after the transcodes are done.Given the lengthy compression times, did you base your evaluations on full movies or did you have favorite "pathological" clips to speed up the process
First, you're probably referring to encoding and not compression (see section above). Second, the actual difference between bluray and dvd is quite literally the storage capacity. Other than that (and the related low level implementation details that we don't care about) the only other differences between bluray and dvd content is the codecs used and the resolution of the video files. Encoding times are directly related to the amount of information that needs to be encoded and the amount of calculation needed to achieve the chosen quality of the resulting video. To answer what I think was the underlying question, 95% of my experience so far has been transcoding FHD video ripped from bluray sources. Also note that those times you quoted are with an encoder preset of "Slow", not to be confused with handbrake's own presets.Are you referring to a Blu-ray compression or a DVD compression?
Here are links to some presets I created:
- http://soulshadow.net/hb/Handbrake_Pres ... C_mkv.json
- http://soulshadow.net/hb/Handbrake_Pres ... C_mkv.json
- I am not an expert. I'm sure there are even better settings to use than what I have in those presets. Although those presets meet my needs, I'm always open to any tips others might have.
- 10-bit h265 should help reduce the "banding" effects compared to h265 8-bit, however the lower the encoder quality the more likely you'll get this effect.
- I like using RF16 for h265 and RF18 for h264 with the encoder preset "Slow". Higher RF #'s mean smaller files but worse quality. Faster encoder presets save time at the enxpense of reducing quality and increasing filesize. The gains from Slower/Slowest/Placebo aren't really worth the extra time invovled.
- I never go below RF15 as there's no perceivable difference in visual quality but file size goes up a lot.
- If the default settings with no denoise filter produce a file that's too big (which is about 90% of the time) then i add denoise NLMeans starting at Denoise Preset "Ultralight" and increasing it to "Light" then up to "Medium" until I'm at or below my target. (8 mbps, or 60mb per minute for FHD content -- this equates to about 2mbps, or 15mb per minute for SD content though I'm more flexible with SD due to the smaller files sizes). I've also found that the NLMeans Denoise Tune of "Film" seems to produce the best overall results and almost never change it. I only use the Denoise Preset "Strong" for the worst source files I come across since it's smoothing effects start to really become noticeable for FHD content, especially in faces.
- If the default RF settings with denoise up to Medium aren't enough I'll usually start to raise the RF in 1.0 increments until I get it where I want. Typically the default RF along with denoise at "Light" is enough for most bluray rips, but older content or really high motion movies might need the RF adjusted. It's rare that I need to go above h264 RF20 or h265 RF18 along with NLMeans "Medium" to achieve my target file sizes, even with the worst noisy, high motion source material (like the bluray of the original Resident Evil).
- Adjust audio and subtitle settings to your needs/preferences and always check the cropping because handbrakes auto detection isn't that good. (sorry devs!)
One final note for this post, and I think others have already mentioned this. Even without transcoding, the visual quality of SD content will suffer when viewed on a large modern TV such as yours. While most of the better manufacturers have hardware and software built into their TV's to "upscale" lower resolution content, it's still about 1/4th of the visual information that is present in FHD content. It will never look as good as any HD content.
Maybe in the next week or so I'll pause my current BD queue and do a few DVD's to compare.