Editing with MP4 video
If you shoot MP4 video and then use that MP4 file in your favourite editing package, it’s very likely your entire workflow is slow and clunky. Your software might be slow to respond, the audio might be jerky or not match at all. There’s a much better way to help you edit these files which also greatly reduces the time it takes you to edit and render out your final videos. Read on!
MP4 is a codec. Codec simply means “Compress-Decompress“, or “Code-Decode“. In other words it’s a specific way in which your video files are converted from video into computer data, and back again. There are many different codecs for both video and audio, such as AVI, MP2, MP3, MP4, H263, H264, WAV, WMV, MPG, to name but a few. Over the years, lots of companies have created more efficient ways of encoding ( and compressing) video, which helps to keep a good amount of detail in your videos, but takes up less space in storage.
Here’s an example of space-saving compression;
Frame one is blue. Frame two is blue. Frame three is blue. Frame four is blue. Frame five is blue. Frame six is blue. Frame seven is blue. Frame eight is blue. Frame nine is blue. Frame ten is blue. Frame eleven is blue. Frame twelve is blue. Frame thirteen is blue. Frame fourteen is blue. Frame fifteen is blue. Frame sixteen is blue. Frame seventeen is blue. Frame eighteen is blue. Frame nineteen is blue. Frame twenty is blue. Frame twenty one is blue. Frame twenty two is blue. Frame twenty three is blue. Frame twenty four is blue. Frame twenty five is blue.
All 25 frames are blue.
The example above describes exactly the same thing. Obviously this is an incredibly simplified example of compression, but that’s what it is! The first paragraph takes up lots of space to say one thing, whereas the second paragraph says the same thing using mush less information, and therefore much less space.
MP4 is more specifically a playback Codec. It is designed to allow lots of devices such as your TV, phone, tablet, etc, to play the video back “as is”. But if you shoot lots of MP4 video footage and then later want to edit that footage together, it can be tricky and cause lots of problems (and stress!). Let me explain why;
It comes down to keyframes (or intra frames), which is a unique way in which Codecs save space when storing video. A keyframe is simply one complete frame, full of video data. Like a full resolution photograph. And these keyframes contain a HUGE amount of data. But MP4 video doesn’t record keyframes for every frame of video. Instead it only records ONE keyframe for every second (on average). To save precious space, the Codec simply stores data on what has changed between those keyframes.
MP4 video compression is therefore described as a “lossy compression“, because in reality you are losing information and detail within the video. A video encoded with Lossless Compression would look visually perfect, but would take up lots more space. Lossy Compression stores less data, and can be bad quality, but takes up less space.
Imagine ten seconds of video footage. In the world of european TV, there are 25 frames shown on-screen to every second. Like a flip book. Therefore that 10 second clip contains 250 frames. But in MP4 video, it doesn’t store all 250 frames. As mentioned previously, MP4 only stores (in most cases) ONE full frame of information for every second. So a ten second video clip only actually contains TEN full screen, full data, captures of your video. For ten seconds that’s ten frames, out of 250? So really only 1/25th of video data is recorded!
With me so far? So the first frame of every second exists, yet theoretically all the other frames don’t. This is important for editing, because you want to be frame-accurate. If you want to cut your video at exactly “frame 27”, and then have the next frame of your video be ten seconds later, at “frame 277”, well both of those frames don’t exist. When you click on “Frame 27” your Video Editing Software will have to look back at the first keyframe for that second, read the data of what has changed up to that point and show you an interpretation of “Frame 27”. It then needs to do the same thing for “Frame 277”. So something as simple as clicking on a frame that doesn’t exist means a delay to you as your computer and software churn through all the data to interpret and show you that frame. Imagine the processing power it therefore takes if you click on your clip, and then skip back a few frames. Click, click, click. It takes the computer a few seconds to work everything out.
Even doing something simple like a one second fade between two MP4 video clips means your computer and editing package have to interpret those twenty-five frames (most of which don’t exist) as well as apply the processing to fade between each one. That’s a lot of processing for something very simple. No wonder the fans inside your computer are spinning at top speed! Editing several minutes of MP4 video like this could take hours, days, weeks!
As a side-note, occasionally you can see your computer trying to interpret data in-between keyframes if you scrub through a video clip and it tries to play it from a point where no keyframe exists. The video goes blocky, and parts of the video move around in a strange way. Sometimes there’s a green flash, and then BANG, the video clip hits a keyframe and the whole video is back in sync.
So what can we do to reduce all this processing power and delays..?
Convert MP4 video from a playback Codec to an edit-friendly Codec. And an edit-friendly Codec I recommend is “ProRes 4:2:2“.
At the very beginning, before you even open your video editing package, you should convert all your MP4 videos into ProRes 4:2:2 video clips.
There are lots of different types of software for both PC and Mac which allow you to convert those MP4 videos, and I highly recommend free software called MPEGStreamClip. This software will read your MP4 video file, and convert them to ProRes (it even has a Batch function, so you can drop an entire folder, and it’ll convert them all in one go). You can download MPEGStreamClip here. It can convert to many other codecs too, but stick to ProRes for editing. If ProRes isn’t available (it can depend on what software you have installed on your computer) then try Apple Intermediate Codec, or AIC.
Your newly converted “ProRes” codec video clips will visually be exactly the same video clips, except every single frame will be full of data. Effectively, every frame is a keyframe. So when you click on “frame 27”, the data is already there and the image appears, instead of the computer processing all the data to work out what “frame 27” might look like.
By doing this conversion at the very start of your workflow, you will see a noticable improvement when you come to edit. Your software will be more responsive, your computer will be doing less processing, and you’ll save time overall.
Please note that converting video takes time. But it’s still better to convert everything at the beginning before you start your edit process. You can batch-convert all your clips whilst you sleep, or take a shower, have lunch, etc, instead of having those irritating delays during the edit.
In a nutshell it’s the same for audio. MP3 audio saves a vast amount of space by not recording the frequencies that the human ear can’t detect. Anything above 22KHz just isn’t there. Although this is great for storage, it’s horrendous for audiophiles! Classical music which contains a combination of high frequencies is therefore stripped out. And although we might not hear those high frequencies, it has an affect on the other notes which we can hear.
Update: 2nd Jan 2016: Interesting Reddit post showing GIFs gone wrong. This shows the same principal of keyframing and how it can catastrophically and hiliariously wrong: