How does x264 compare with HEVC (h.265)?

Update: I stopped using x265 because my Plex server is running off of an old Core 2 Quad Q9450 and it just can’t keep up decoding x265. I’m back to h264, speed: Slower, and RF20. Also, I now have lots of space with my new NAS so I’m not as worried about filesize anymore. Maybe someday I’ll move back to x265.

I have been toying around with HEVC (x265) for the past couple days and I’m quite impressed. Billed as having the same quality H.264 with half the bitrate, I was a bit skeptical at that claim, but I’m pleased to say it’s true.

I use a Plex server to serve up my media to a Roku so my goal is simple for this project: I’m looking for the easiest method of getting near original quality in the smallest filesize with reasonable encode times. I’m not one to fiddle with settings for each movie. I’m looking for reasonable catch all settings that gets me the quality I want that’s easy to use.

I have been ripping my dvds with AnyDVD for a while now and I’m not going back. It is a fantastic little program and was lucky enough to get my lifetime license before they went to a subscription model and upped the prices. I’ve also been encoding with Handbrake and I’m going to continue using it. It’s easy to use and well known.

I was excited to the QuickSync when I rebuild my rig a few months ago. By setting the QuickSync settings to maximum quality and just letting Handbrake do it’s thing I was getting good quality encodes at an incredible 400 frames per second. (for dvds not blu-rays) The results were 3GB encodes. Those aren’t terrible filesizes. They are smaller than the 8GB needed for the original dvd rip; however, I wasn’t pleased with it. I know it could be better. I’ve seen 1-2GB encodes that look good and my plex server is only sporting a 3TB drive for storage. If I can stretch that out by cutting 3GB encodes in half, that will double my movie capacity. It also makes backups much easier. When all my friends spend bookoo bucks on raid or ZFS setups, I’m content with mirrored 3TB drives.

Here are some filesize samples from chapter eight of Tucker and Dale vs Evil (blu-ray): pic

The Handbrake defaults for x265 are Very Fast and RF20. Already I see that x265 on Very Fast is eeking ahead of x264 on Slow.

Now, this doesn’t tell the quality of the encode. I used the Handbrake x265 defaults as a starting point: Very Fast and RF20. I opened up each sample and jumped around to different frames and noted the quality. I looked mostly for actor’s faces. I find that to be the easiest when when I’m judging quality. The x265 defaults look great to me, so the slower speeds with the same RF number looked just as good. Plus, it looks like x265 is delivering on the promise same quality at half the bitrate. If I want smaller filesize with similar quality, turn the speed down.

(Some of my latest tests are indicating that x265 Slow is too slow for me. I was getting 4fps during an encode and it would have taken 6+ hours.)

When I first tried the PSNR tune, I thought it was a must have. It dropped the filesize significantly with similar quality. However, I was using an older source for my initial testing (The 5 Deadly Venoms blu-ray release) and the quality loss was unnoticeable during playback. I later found that on a new source like Tucker and Dale vs Evil the quality was noticeably degraded. I’m thinking I will leave PSNR off for my rips, especially with newer sources.

I don’t follow too closely to all the forums on x265, so I’m not sure if I’m missing any easy/must-do tricks. I want something easy and bulletproof, so my final choice for ripping and encoding will be Handbrake for it’s ease of use, and x265, Medium, RF20. That seems to get me original quality (or at least my eyes can’t tell the difference) with the smallest filesize without taking forever.