If you shoot MP4 video and then use that MP4 file in your favourite editing package, it’s very likely your entire workflow is slow and clunky. Your software might be slow to respond, the audio might be jerky or not match at all. There’s a much better way to help you edit these files which also greatly reduces the time it takes you to edit and render out your final videos. Read on!
MP4 is a codec. Codec simply means “Compress-Decompress“, or “Code-Decode“. In other words it’s a specific way in which your video files are converted from video into computer data, and back again. There are many different codecs for both video and audio, such as AVI, MP2, MP3, MP4, H263, H264, WAV, WMV, MPG, to name but a few. Over the years, lots of companies have created more efficient ways of encoding ( and compressing) video, which helps to keep a good amount of detail in your videos, but takes up less space in storage.
Here’s an example of space-saving compression;
Frame one is blue. Frame two is blue. Frame three is blue. Frame four is blue. Frame five is blue. Frame six is blue. Frame seven is blue. Frame eight is blue. Frame nine is blue. Frame ten is blue. Frame eleven is blue. Frame twelve is blue. Frame thirteen is blue. Frame fourteen is blue. Frame fifteen is blue. Frame sixteen is blue. Frame seventeen is blue. Frame eighteen is blue. Frame nineteen is blue. Frame twenty is blue. Frame twenty one is blue. Frame twenty two is blue. Frame twenty three is blue. Frame twenty four is blue. Frame twenty five is blue.
All 25 frames are blue.
The example above describes exactly the same thing. Obviously this is an incredibly simplified example of compression, but that’s what it is! The first paragraph takes up lots of space to say one thing, whereas the second paragraph says the same thing using mush less information, and therefore much less space.
MP4 is more specifically a playback Codec. It is designed to allow lots of devices such as your TV, phone, tablet, etc, to play the video back “as is”. But if you shoot lots of MP4 video footage and then later want to edit that footage together, it can be tricky and cause lots of problems (and stress!). Let me explain why;
It comes down to keyframes (or intra frames), which is a unique way in which Codecs save space when storing video. A keyframe is simply one complete frame, full of video data. Like a full resolution photograph. And these keyframes contain a HUGE amount of data. But MP4 video doesn’t record keyframes for every frame of video. Instead it only records ONE keyframe for every second (on average). To save precious space, the Codec simply stores data on what has changed between those keyframes.
MP4 video compression is therefore described as a “lossy compression“, because in reality you are losing information and detail within the video. A video encoded with Lossless Compression would look visually perfect, but would take up lots more space. Lossy Compression stores less data, and can be bad quality, but takes up less space.
Imagine ten seconds of video footage. In the world of european TV, there are 25 frames shown on-screen to every second. Like a flip book. Therefore that 10 second clip contains 250 frames. But in MP4 video, it doesn’t store all 250 frames. As mentioned previously, MP4 only stores (in most cases) ONE full frame of information for every second. So a ten second video clip only actually contains TEN full screen, full data, captures of your video. For ten seconds that’s ten frames, out of 250? So really only 1/25th of video data is recorded!
With me so far? So the first frame of every second exists, yet theoretically all the other frames don’t. This is important for editing, because you want to be frame-accurate. If you want to cut your video at exactly “frame 27”, and then have the next frame of your video be ten seconds later, at “frame 277”, well both of those frames don’t exist. When you click on “Frame 27” your Video Editing Software will have to look back at the first keyframe for that second, read the data of what has changed up to that point and show you an interpretation of “Frame 27”. It then needs to do the same thing for “Frame 277”. So something as simple as clicking on a frame that doesn’t exist means a delay to you as your computer and software churn through all the data to interpret and show you that frame. Imagine the processing power it therefore takes if you click on your clip, and then skip back a few frames. Click, click, click. It takes the computer a few seconds to work everything out.
Even doing something simple like a one second fade between two MP4 video clips means your computer and editing package have to interpret those twenty-five frames (most of which don’t exist) as well as apply the processing to fade between each one. That’s a lot of processing for something very simple. No wonder the fans inside your computer are spinning at top speed! Editing several minutes of MP4 video like this could take hours, days, weeks!
As a side-note, occasionally you can see your computer trying to interpret data in-between keyframes if you scrub through a video clip and it tries to play it from a point where no keyframe exists. The video goes blocky, and parts of the video move around in a strange way. Sometimes there’s a green flash, and then BANG, the video clip hits a keyframe and the whole video is back in sync.
So what can we do to reduce all this processing power and delays..?
Convert MP4 video from a playback Codec to an edit-friendly Codec. And an edit-friendly Codec I recommend is “ProRes 4:2:2“.
At the very beginning, before you even open your video editing package, you should convert all your MP4 videos into ProRes 4:2:2 video clips.
There are lots of different types of software for both PC and Mac which allow you to convert those MP4 videos, and I highly recommend free software called MPEGStreamClip. This software will read your MP4 video file, and convert them to ProRes (it even has a Batch function, so you can drop an entire folder, and it’ll convert them all in one go). You can download MPEGStreamClip here. It can convert to many other codecs too, but stick to ProRes for editing. If ProRes isn’t available (it can depend on what software you have installed on your computer) then try Apple Intermediate Codec, or AIC.
Your newly converted “ProRes” codec video clips will visually be exactly the same video clips, except every single frame will be full of data. Effectively, every frame is a keyframe. So when you click on “frame 27”, the data is already there and the image appears, instead of the computer processing all the data to work out what “frame 27” might look like.
By doing this conversion at the very start of your workflow, you will see a noticable improvement when you come to edit. Your software will be more responsive, your computer will be doing less processing, and you’ll save time overall.
Please note that converting video takes time. But it’s still better to convert everything at the beginning before you start your edit process. You can batch-convert all your clips whilst you sleep, or take a shower, have lunch, etc, instead of having those irritating delays during the edit.
In a nutshell it’s the same for audio. MP3 audio saves a vast amount of space by not recording the frequencies that the human ear can’t detect. Anything above 22KHz just isn’t there. Although this is great for storage, it’s horrendous for audiophiles! Classical music which contains a combination of high frequencies is therefore stripped out. And although we might not hear those high frequencies, it has an affect on the other notes which we can hear.
Update: 2nd Jan 2016: Interesting Reddit post showing GIFs gone wrong. This shows the same principal of keyframing and how it can catastrophically and hiliariously wrong:
We all know the meaning of life is 42. But what’s the purpose? This is the question suffered by Qohen Leth in Terry Gilliam’s The Zero Theorem.
This week, thanks to Den Of Geek, I was invited to a preview screening of the movie at the headquarters of Sony Europe in London. This is the second time Den Of Geek have offered me a free ticket, having invited me to watch The LEGO Movie a few weeks back at the headquarters of Warner Brothers. Before I go any further I should point out that this review is my own, and is not a request by Den Of Geek. Any opinions are my own and should not be linked with the opinions of Den Of Geek, Sony Europe or any other affiliate.
I would also like to mention the mind blowing bonus that the audience received; before the movie started our host asked us what we’d do if Terry Gilliam were to walk through the screening doors and say hello. Half of me was disappointed that such a carrot would be dangled in front us. And yet, like rubbernecking witnesses at a car crash, one by one people in front of me turned and gawked open-mouthed as the man himself bounded down the side aisle toward the screen. Mr Gilliam was pleasant enough to talk for a few minutes about the production of the film, the friendships with the cast and his perpetual want for being alone!
So thank you Terry Gilliam for adding that extra brilliance to what was simply a free ticket to a preview screening!
Now, not everyone gets the visions that Terry Gilliam puts on-screen. Mostly outlandish. Definitely bizarre. Some within the grasp of steampunk. All of it very clever. Thankfully having spent a childhood watching “Time Bandits” over and over, and falling into philosophical discussions at 4am over the meaning of “Twelve Monkeys”, I was prepared to accept what was coming.
I wasn’t disappointed.
Stylistically, there is no mistake this is a Terry Gilliam film. Nobody else can portray a story like he can; clunky ridiculous fashion, day-glo neon, impressionable Governments, OTT commercialism (I particularly liked the advert on the building that tracked alongside the main character as he walked down the street, whilst all the time referring to everyone as “madam”).
Practical visuals aside, the concept delves deeper into human nature (as does every Terry Gilliam film). Almost to the point of forensics. Life as we know it blown into ridiculous proportions, yet seemingly accepted as the norm by those not-so-far-future characters. People go to work, they consume and partake in the controlled world around them. They accept the systems in place, playing along like sheep. All the while the main character simply wants to know the purpose of life. With the whirlwind of other peoples lives around him that he blatantly wants no part of, he’s forced to follow along and step out into the big wide world (which he keeps securely hidden behind the several locks on his front door). With no option but to accept fate, we’re taken on a Gilliam-style roller coaster with an inevitable end.
Gilliam said before the screening that the film was meant to be set in the not-so-distant future, but by the time it came to Production and release the concept had slipped into what it’s actually like today.
It won’t be long before it’s a period drama!
And I totally get what he means. In one scene, a party is taking place with lots of people in attendance. But everyone’s wearing headphones and listening to their own music, staring at the computer tablet in front of them, and hardly interacting with anyone else in the room. And if they are interacting, it’s by video stream to other people at the party. From an outward point of view everyone is enjoying themselves at this party, yet everyone is actually only enjoying their own closed-off personal experience as opposed to the group experience.
The cast is great. Christopher Waltz as the troubled Qohen Leth, Melanie Thierry (you’ll seek out more of her films, I guarantee!) as the gorgeous Bainsley, David Thewlis (who played King Einon in DragonHeart!) as supervisor Joby, Matt Damon with his bit-part as Management, and the unforgettable Tilda Swinton as the fantastic Doctor Shrink-Rom.
If you’re a Terry Gilliam fan, you are in for one hell of a ride. If you’re not then sit back and absorb the double-meanings, enjoy the subtle hints of oppression, compare your own life and wellbeing to those of the main characters.
I guarantee every single person will relate to one of the characters. Whether you’re the drone in the fast-lane taking everything for granted, the adolescent kid who knows more about the world than their older siblings, the camouflaged management in control of every situation, the worker in diluted fear of that management, or Qohen who’s misused talent draws him fatefully toward his quest to find his answer.
The film will no doubt create debate. Its impressions and meanings can be interpreted a million different ways, leaving the viewer to ponder on just what happened exactly. With this in mind, I foresee The Zero Theorem becoming a typical Gilliam classic.
Condensing each frame of a video into a single line and stacking them together. This is what Video Barcoding is. By taking more than one image and merging them together, it can create fascinating views into the colour schemes and lighting. I’m gone through my collection of favourite Film and TV series and started to create a Video Barcode project.
Here’s my first two attempts at Video Barcoding. The first is episode 1 of series 1 along the top, merged with the last episode of the last series of Breaking Bad (with a kick-ass watermark to boot!). The second is Terminator along the top, merged with Terminator 2 Judgment Day along the bottom.
See if you can work out the scenes. In the T2 barcode there’s a pretty obvious red line near the start for the biker bar termovision scene, a clear blue line for the Pescadero State Hospital scenes, the Cyberdyne assault and the ultimate bright orange molten steel pit;
Summary: A blog entry regarding support in bringing change to the VFX industry, to help shed light on the unsustainable financial structures that VFX companies work in.
Note: If you don’t watch all of the documentary below, “Being sharked” is the term used by Bill Westenhofer (Visual Effects Supervisor at Rhythm And Hues) to describe the moment at the BAFTA ceremonies where the award-winners go over their allotted time, cause the orchestra to play the Jaws theme as a warning, and ultimately ends with the stage microphones being cut.
I grew up with the hunger for Visual Effects. It was a strange passion to have as a 10-year-old, but it drove me to the right places. At 18 years old I left my friends and family and moved 500 miles to get a job in VFX. Although I didn’t end up doing the job 10-year-old me thought I really wanted (seriously, at school I was telling my Careers Advisor I wanted to be “A Compositor“. They had no idea what I was talking about), that decision still crafted the career I have now, as a freelancer behind-the-scenes in the Film and TV industry.
Back in the day, as a kid, when I’d research how movies were made I’d come across companies and individuals whom I knew were key to the industry. Their names would come up time and time again. Twenty five years later some of those names still come up, but unfortunately some don’t. And it wasn’t till I found out about this finance issue that I understood what was happening;
If a VFX House bids to do 500 shots for a feature film at a fixed price, but then the storyboards and script changes cause a series of dramatic edits to be made (several months later, and after the agreed “fixed bid” payment), the VFX House is still expected to create those “final” 500 shots no matter how far over the Production may have run and how much work they may have already done. The simple fact of spending $1 million per month on freelancers for ten months on those shots but then being asked to work for another 3 months without any change to the “fixed bid” payment clearly means that companies might not make any profit at all. Yet that’s how the industry works!
Rhythm And Hues is a VFX House well-known around the world. There’s no doubt you’ll have seen the work created by their teams of incredible freelance artists in the hundreds of movies they’ve been part of (Lord Of The Rings, The Green Mile, Babe, X-Men, The Hunger Games, Men In Black, Life Of Pi, and over 140 others). They are big hitters in the VFX industry. Yet in 2013, 11 days after winning the BAFTA for Best Visual Effects for Life Of Pi, the company filed for bankruptcy…
They’re not the first company to suffer at the hands of what’s now understood to be quite an unfair industry, but they’re the first one to document their financial demise in such a way that the message has been globally recognised.
Much more eloquently than I can write, the financial impact comes across in the Rhythm And Hues documentary “Life After Pi”:
I love my job. I love the people I work with, I love the studios I get to work in, and I love the things I get to do, being part of something that brings me such joy. If that was to be taken away, I’d be devastated. Understanding how much they love their jobs too, it’s therefore difficult to watch the documentary and not put myself in their shoes.
There’s something fundamentally understood between humans when one of us looks upon another at a point when they are at their most vulnerable, most honest and open. Seeing John Hughes, one of the founders of Rhythm And Hues, 20 min 7 sec in to their story is one of those moments…