2010-07-18

Sony HX5 - Video Encoding

2010-7-19 15.21
Why would you want to re-compress you video files?
Well, I don´t know about you, but with the quality the HX5 gives me in videos, I find myself using the video function more and more. And that shows on my disc. Where before a DVD was enough to archive several years of picture memories including some clips, now a DVD would be filled in a matter of weeks.
So I need a way of keeping the videos I want but if I could get them smaller without loss of visible quality that would be perfect.

After spending several days work to try all kinds of programs and fiddle with the settings to get exactly the results the way I wanted it it is time to share that information with you. In one sentence: You can save 50 to 75% of space and keep a good quality.
 
If you have lots of movement over most of the screen, then re-compression might not save you much. This 3 min clip "Fiesta del Carmen Gran Canaria 2010 - Sony HX5.MP4" in the original is 150 Mb re-compressed with RF:25 (see below) brought the file down to 132 Mb. In a case like this you already got the best compression / quality ratio in the original file.


If you have a mostly static scene like on this 32 second clip a small grasshopper drinking, you can archive a compression from the original 24 Mb to 3.3 Mb at setting RF:25 what is quite a lot for nearly the same quality. This clip was taken to test if the HX5 macro sweet spot from 2.4 x works also in video mode. And yes it does very well, our grasshopper here is roughly 1 inch = 2,5 cm long. Focus was set to spot.




Now, how is it possible to compress one file so much and another nearly not at all?
Coming from over 10 years of getting video clips to the computer starting with VHS-C, the Snappy and Real Media with QVGA in the last Millenium we have come a long way and the two pass encoding in AVI is so yesterday.

From the HandBrake wiki:
With the average bitrate or target size methods, you control the size of the output file but give up control over the video's quality.
Constant quality mode does the opposite; you specify a quality level and HandBrake adjusts the bitrate (that is, the size) to meet it.
Because it takes a different bitrate to reach a given quality level for any given part of any movie, the output size is unpredictable and varies from source to source. 

I will show you how you can easy decide what quality YOU like and how you can shrink all your video files quite a bit and always have the same quality.
Before we finally start, here a sample from a HD encoding with the old 2 pass method. If you have extremely fast flashing of pictures during a scene, two pass just can not handle the fast changing amount of information.
The result is a quality like this:

  In constant quality mode this part of the movie would be as sharp and without noticeable compression artifacts. And this is why you should spread the word about HandBrake and the gorgeous constant quality mode you can use to compress your video files.

...

To start, first let´s take a short trip to France.
Go visit the makers of Handbrake and get a free copy. You can have it for: Mac OS, Windows, Ubuntu and Fedora 12
If you want to have the newest version from Handbrake, check out the nightly builds.
Install and run it. Drag and drop a video file into it.

Beginner
SETTINGS
:
Video Filters - If you recorded in AVCHD set Decomb to Default. If you recorded in MP4, skip this step.
Video - Now comes the decision how good the quality YOU are desire has to be. The lowest acceptable level for me is RF 32 @ Constant Quality, RF 28 is a good balance for size and quality.
Audio - no change
Presets - Regular Normal
Destination - choose where you want to have the re encoded video
press the green Start button

Advanced
SETTINGS
:
Picture - if you want to have your video in 4:3 set Cropping to Custom and 160 for left and right
Video Filters - If you recorded in AVCHD set Decomb to Default. If you recorded in MP4, skip this step.
Video - Now comes the decision how good the quality YOU are desire has to be. The lowest acceptable level for me is RF 32 @ Constant Quality, RF 28 is a good balance for size and quality. For a nice quality RF 25. If you go up to RF 20 and above,  you will have nearly the same size than the original.
Audio - no change
Presets - Regular High Profile This encodes slower but more precise, the file gets a bit bigger also.
Advanced - for even more precise encoding copy and paste in the bottom box:
ref=6:mixed-refs=1:bframes=3:b-pyramid=1:weightb=1:subq=7:trellis=2:analyse=all:8x8dct=1:no-fast-pskip=1:no-dct-decimate=1:me=umh:merange=64:deblock=-2,-1:direct=auto
Destination - choose where you want to have the re encoded video
press the green Start button

Under the Hood
If you want to do some testing for your self, cut 10 seconds of your favorite video. Take a part where you have a steady  object and a part where there is lots of movement. If you do not have that, just grab something quick with your camera. Why only 10 sec? Because it takes a lot of time to encode if you want to test things for yourself.
Here are some of my results:
Since in some parts of our globe a breast and especial an uncovered nipple is a taboo I decided to crop the screen grabs to keep you poor folks sane.
Here is what to look out for:
The compression artifacts will appear first on the painters upper arm, also look at the mesh of her top for compression smearing. The face and the eyes of our model and the contour of the face to the water are an indicator for the sharpness, or loss of.

Original file 720p - 16 sec - 12 Mb

H.264 codec

RF:20 - Normal - 13 Mb - excellent


RF:23 - Normal - 9 Mb - excellent


RF:25.5 - Normal - 6 Mb - excellent


RF:28 - High Profile - 4.8 Mb - good


RF:28 - Normal - 4.4 Mb - good


RF:30.5 - High Profile - 3.3 Mb - good


RF:30.5 - Normal - 2.9 Mb - OK


RF:33 - Normal - 1.9 Mb - starting to loose detail and compression artifacts visible


RF:35.5 - Normal - 1.4 Mb -  bad



RF:32 -Normal - 2.3 Mb - starting to loose detail


RF:32 - High Profile - 2.7 Mb - OK


RF:32 - High Profile extreme setting, see above- 2.9 Mb - OK



MPEG-4 codec
QP:8 - Normal - 8.2 Mb


QP:10 - Normal - 6.2 Mb


QP:12 - Normal - 4.9 Mb


-----------------------

I did some more testing with several blue ray disks and moved the RF (Rate Factor) to 25.
RF 28 did show some slight problems with compression artifacts in dark areas RF 25 has less.

The Die Hard 132 minutes long movie
with RF 28 @ 482 Kbps  encoded and Dolby Pro Logic II @ 160 Kbps resulted for the 720p version in a tiny 611 Mb file. Acceptable quality and a 2 hour 720p movie on a CD size - I am impressed.


the back of the dark suit is where the compression artifacts did show a bit


in this bright scene every thing looks good to my eye

with RF 25 @ 750 Kbps encoded and Dolby Pro Logic II @ 160 Kbps resulted for the 720p version in a still tiny 865 Mb file.

a bit less visible



with RF 20 @ 2046 Kbps encoded and Dolby Pro Logic II @ 160 Kbps resulted for the 720p version in a 2040 Mb file.


still not gone completely





Overview Compression, Size and Time
Audio 2=Dolby II @ 160 - 6=6 Channel @ 384 - 6p=6 Channel AC3 Passthru
nt = not tested

the first 60 seconds from Serenity @ 720
RF - MB - sec - Audio - Profile - Comments
20 - 25.1 - nt - 6+6p - high
20 - 23.5 - 195 - 2+6p - high
20 - 21.8 - 195 - 6 - high
20 - 20.2 - nt - 2 - high

20 - 19.5 - 105 - 2 - normal

21 - 17.4 - nt - 2 - high

22 - 14.9 - nt - 2 - high

23 - 14.4 - 177 - 6 - high
23 - 13.9 - 158 - 6 - normal
23 - 12.8 - nt - 2 - high

23 - 12.2 - 95 - 2 - normal

24 - 11.0 - nt - 2 - high

25 - 12.8 - 164 - 2+6p - high
25 - 11.1 - 164 - 6 - high
25 - 10.8 - 94 - 6 - normal
25 - 9.5 - nt - 2 - high

25 - 9.1 - 90 - 2 - normal

26 - 8.3 - nt - 2 - high

27 - 7.3 - nt - 2 - high

28 - 6.5 - nt - 2 - high

29 - 5.9 - nt - 2 - high

30 - 5.3 - nt - 2 - high


the first 60 seconds from the 6th day @ 1080
the 1080 screen shots are cropped because Blogger has a maximum image size of 1600

RF - MB - sec - Audio - Profile - Comments
23 - 15.3 - 285 - 6 - high - 1080

23 - 12.0 - 140 - 6 - normal - 1080

23 - 8.4 - 170 - 6 - high - resized to 720

23 - 6.1 - 81 - 2 - normal - resized to 720

With this work I have done, it should be pretty easy now for you to have an idea what compression to choose.
----

If you want to know more, I started a new blog where I will go into this topic a bit deeper over time as I discover new and better ways to re-compress.

small files - good quality - no problems playing

video codecs and compressions

6 comments:

  1. Great Tips!
    For some reason, when i played my raw videos from my camera on my laptop, the video plays very slowly. Very frame by frameish. Also, i noticed theres alot of noise compared to the videos ive seen on your blog and youtube. Are there anyways to fix this?

    ReplyDelete
  2. did you miss Tips & Tricks?

    ReplyDelete
  3. Hey,
    Yes i did try the media player you recommended, however, the video and its audio is still very choppy. It plays perfectly on the camera. Maybe i should try watching it on a differnt media player. Any other suggestions?

    ReplyDelete
  4. Joe:You have covered many interesting topics with the camera. My main problem with the camera is still picture smearing of detail, in such things as birds,grass,tree bark,and even water color effects on water. It seems that the camera
    believes sometimes that certain objects are noise and applies excessive NR. As you know there is no way to turn NR down or off. To me the camera mostly takes good pictures at full zoom up to about 10 feet, then excessive NR may happen for unknown reasons at least with my camera. From reading other forums many are returning the camera for this reason.
    Any thoughts.
    Old Mike

    ReplyDelete
  5. @Xopher
    your computer is not powerful enough to play the original files then.
    you need a dedicated graphics card.
    what ar your specs?
    simple solution, re-encode as I showed above and check @ what encoding setting RF you computer has trouble.

    ReplyDelete
  6. @Old Mike
    the secret lies in post processing.
    you need to sharpen the image
    my HX1 does not have more detail, things in the background gets smeared.
    that is why the NEX-7 might tempt me it Sony improves drastically the lenses.

    and mike, this is a video thread, next time please post under the right topic, so other people can benefit from the answers too.

    ReplyDelete

Do not use any more, please use our forum!

Note: Only a member of this blog may post a comment.