If you shoot MP4 video and then use that MP4 file in your favourite editing package, it’s very likely your entire workflow is slow and clunky. Your software might be slow to respond, the audio might be jerky or not match at all. There’s a much better way to help you edit these files which also greatly reduces the time it takes you to edit and render out your final videos. Read on!
MP4 is a codec. Codec simply means “Compress-Decompress“, or “Code-Decode“. In other words it’s a specific way in which your video files are converted from video into computer data, and back again. There are many different codecs for both video and audio, such as AVI, MP2, MP3, MP4, H263, H264, WAV, WMV, MPG, to name but a few. Over the years, lots of companies have created more efficient ways of encoding ( and compressing) video, which helps to keep a good amount of detail in your videos, but takes up less space in storage.
Here’s an example of space-saving compression;
Frame one is blue. Frame two is blue. Frame three is blue. Frame four is blue. Frame five is blue. Frame six is blue. Frame seven is blue. Frame eight is blue. Frame nine is blue. Frame ten is blue. Frame eleven is blue. Frame twelve is blue. Frame thirteen is blue. Frame fourteen is blue. Frame fifteen is blue. Frame sixteen is blue. Frame seventeen is blue. Frame eighteen is blue. Frame nineteen is blue. Frame twenty is blue. Frame twenty one is blue. Frame twenty two is blue. Frame twenty three is blue. Frame twenty four is blue. Frame twenty five is blue.
All 25 frames are blue.
The example above describes exactly the same thing. Obviously this is an incredibly simplified example of compression, but that’s what it is! The first paragraph takes up lots of space to say one thing, whereas the second paragraph says the same thing using mush less information, and therefore much less space.
MP4 is more specifically a playback Codec. It is designed to allow lots of devices such as your TV, phone, tablet, etc, to play the video back “as is”. But if you shoot lots of MP4 video footage and then later want to edit that footage together, it can be tricky and cause lots of problems (and stress!). Let me explain why;
It comes down to keyframes (or intra frames), which is a unique way in which Codecs save space when storing video. A keyframe is simply one complete frame, full of video data. Like a full resolution photograph. And these keyframes contain a HUGE amount of data. But MP4 video doesn’t record keyframes for every frame of video. Instead it only records ONE keyframe for every second (on average). To save precious space, the Codec simply stores data on what has changed between those keyframes.
MP4 video compression is therefore described as a “lossy compression“, because in reality you are losing information and detail within the video. A video encoded with Lossless Compression would look visually perfect, but would take up lots more space. Lossy Compression stores less data, and can be bad quality, but takes up less space.
Imagine ten seconds of video footage. In the world of european TV, there are 25 frames shown on-screen to every second. Like a flip book. Therefore that 10 second clip contains 250 frames. But in MP4 video, it doesn’t store all 250 frames. As mentioned previously, MP4 only stores (in most cases) ONE full frame of information for every second. So a ten second video clip only actually contains TEN full screen, full data, captures of your video. For ten seconds that’s ten frames, out of 250? So really only 1/25th of video data is recorded!
With me so far? So the first frame of every second exists, yet theoretically all the other frames don’t. This is important for editing, because you want to be frame-accurate. If you want to cut your video at exactly “frame 27”, and then have the next frame of your video be ten seconds later, at “frame 277”, well both of those frames don’t exist. When you click on “Frame 27” your Video Editing Software will have to look back at the first keyframe for that second, read the data of what has changed up to that point and show you an interpretation of “Frame 27”. It then needs to do the same thing for “Frame 277”. So something as simple as clicking on a frame that doesn’t exist means a delay to you as your computer and software churn through all the data to interpret and show you that frame. Imagine the processing power it therefore takes if you click on your clip, and then skip back a few frames. Click, click, click. It takes the computer a few seconds to work everything out.
Even doing something simple like a one second fade between two MP4 video clips means your computer and editing package have to interpret those twenty-five frames (most of which don’t exist) as well as apply the processing to fade between each one. That’s a lot of processing for something very simple. No wonder the fans inside your computer are spinning at top speed! Editing several minutes of MP4 video like this could take hours, days, weeks!
As a side-note, occasionally you can see your computer trying to interpret data in-between keyframes if you scrub through a video clip and it tries to play it from a point where no keyframe exists. The video goes blocky, and parts of the video move around in a strange way. Sometimes there’s a green flash, and then BANG, the video clip hits a keyframe and the whole video is back in sync.
So what can we do to reduce all this processing power and delays..?
Convert MP4 video from a playback Codec to an edit-friendly Codec. And an edit-friendly Codec I recommend is “ProRes 4:2:2“.
At the very beginning, before you even open your video editing package, you should convert all your MP4 videos into ProRes 4:2:2 video clips.
There are lots of different types of software for both PC and Mac which allow you to convert those MP4 videos, and I highly recommend free software called MPEGStreamClip. This software will read your MP4 video file, and convert them to ProRes (it even has a Batch function, so you can drop an entire folder, and it’ll convert them all in one go). You can download MPEGStreamClip here. It can convert to many other codecs too, but stick to ProRes for editing. If ProRes isn’t available (it can depend on what software you have installed on your computer) then try Apple Intermediate Codec, or AIC.
Your newly converted “ProRes” codec video clips will visually be exactly the same video clips, except every single frame will be full of data. Effectively, every frame is a keyframe. So when you click on “frame 27”, the data is already there and the image appears, instead of the computer processing all the data to work out what “frame 27” might look like.
By doing this conversion at the very start of your workflow, you will see a noticable improvement when you come to edit. Your software will be more responsive, your computer will be doing less processing, and you’ll save time overall.
Please note that converting video takes time. But it’s still better to convert everything at the beginning before you start your edit process. You can batch-convert all your clips whilst you sleep, or take a shower, have lunch, etc, instead of having those irritating delays during the edit.
In a nutshell it’s the same for audio. MP3 audio saves a vast amount of space by not recording the frequencies that the human ear can’t detect. Anything above 22KHz just isn’t there. Although this is great for storage, it’s horrendous for audiophiles! Classical music which contains a combination of high frequencies is therefore stripped out. And although we might not hear those high frequencies, it has an affect on the other notes which we can hear.
Update: 2nd Jan 2016: Interesting Reddit post showing GIFs gone wrong. This shows the same principal of keyframing and how it can catastrophically and hiliariously wrong:
Vasquez lives – and the charismatic actress being the smart-gun toting marine is armed against extraterrestrial trouble.
Somewhere, beneath the makeup, sweat, ferocity and courage that made up Private Vasquez, the rippling heroine of Aliens, there’s a soft-spoken, freckled, 5’2″ woman named Jenette Goldstein.
Goldstein, to her credit as a performer, shares precious little with her scene-stealing character; in fact, had the film been a true representation of the 26-year-old actress, it might have been titled Beverly Hills Marine. Yes, far from the gritty deprivation of Aliens, the actress was raised in that much publicised community. Despite growing yo so near the glamour of Hollywood, the trappings of celebrity remain foreign to Goldstein. She enjoys the anonymity, and she’s still a bit amused by the flood of fan mail she received – ranging from the US Marine Corps to often offers of marriage (to late, but more…
View original post 2,115 more words
This post is dedicated to Ken Melville, a co-founder of Digital Pictures and writer-designer of the legendary FMV game Sewer Shark. Melville passed away in February 2014, aged 65.
There’s a collaboration between the Film and TV world when it comes to the Gaming industry. Production companies shoot real actors on real sets, and programmers incorporate these clips into computer games. But there was a short time when entire games were interactive Full Motion Video, and not just clips.
The first FMV games were actually born in the 80’s, inspired by an interactive play that was shown in Hollywood, California. The play, called Tamara, had parallel storylines acted out at the same time across multiple rooms. The audience were encouraged to physically move around all three stories of the building, to watch and observe the nine different storylines from different angles, playing out in thirteen different rooms. The understanding of the storyline was dependant on what the audience saw. Time and time again people came back (at $80 per ticket!) in order to fully piece together the story.
In 1986 Hasbro were developing their own VHS-based games console called Project NEMO (“Never Even Mentioned Outside”), later renamed Control-Vision. The console was to be billed as a cheaper alternative to rivals Nintendo and Sega. A groundbreaking demo game with parallel storylines was created in order to further push the development of the console.
Produced by Robert Fulop and directed by James Riley, a four minute Full Motion Video Game demo called Scene Of The Crime was made. Shot in a Hillsborough house over a single weekend, the demo shadowed the concept of the Tamara play, with parallel storylines and multiple rooms to view. It was a groundbreaking idea in the realms of gaming. Hasbro seemed happy with the (rather adult orientated) demo and ordered the go-ahead for full titles;
This is great. Now go and make it for kids.
A video clip of the Hasbro executives watching the Project NEMO demo of Scene Of The Crime was a hidden Easter Egg on one of their future games releases. You can see it below;
Three months later, in 1987, with a budget of $1’000’000 and a 250-page script, the first FMV Game was put into production. Shot over a three week period on a soundstage at GMT Studios, California, the game Night Trap (eBay link here) became a reality. The cast and crew shot their scenes, and one by one each scene was reset and shot again with a different outcome. The variation in outcomes would eventually allow the player to interact and choose storylines to follow.
As a member of the Sega Control Attack Team (SCAT), you have control over the CCTV system and a series of traps within a house. The house owners are hosting a second sleepover for their daughter. Mysteriously the girls from the first sleepover disappeared. Amd its your job to monitor the house, find out what happened, and protect the girls.
Night Trap became world-renowned politically, albeit for the wrong reasons. Video game violence was brought to light in Congress as a legitimate debate. But Night Trap was tarnished as a violent and bloody game unfit for children. Although Night Trap was designed predominantly as a fun vampire game, and showed no violence or blood at all.
Nevertheless, the debate over game violence caused serious damage to the games industry. And Night Trap is the reason there are now Ratings on all video games. The game was later censored and re-released.
Off the back of Night Trap came another FMV Game called Sewer Shark (eBay link here), directed by John Dykstra (Visual Effects operator on Batman Forever, Stuart Little, Spiderman and X-Men First Class). The game would allow the player to became a pilot in the sewer tunnels, shooting “ratigators” and interacting with their co-pilot “Ghost”, on a mission to reach Solar City.
Ignition on three… Ready? THREE!
Shortly after completing the shoot for Sewer Shark, Hasbro stopped developing Project NEMO. The entire idea was shelved, and the rights to the two games sold off. Tom Zito, who later created Digital Pictures, bought the rights to the two games.
It appears only one thing killed Project NEMO; the rising popularity of Personal Computers at the time meant the cost of VRAM hardware chips needed for the console went through the roof, as retailers bought out entire batches of stock in advance of manufacturing powerful PC’s. The games console would no longer be a cheap alternative to Nintendo or Sega. Cutting losses, Hasbro pulled the plug entirely.
It wasn’t till 1992 when Sega released their Mega CD games console that Night Trap and Sewer Shark would see the light of day. Digital Pictures used the archived 1987 footage to finally port the games from Project NEMO to Mega CD.
With the money made off the back of bundling Sewer Shark with the the purchase of a Mega CD console, Digital Pictures created their “second generation” game, Ground Zero: Texas (eBay link here). With a budget of $2’000’000, the game used a full Hollywood crew to shoot the 110 minutes of footage required for the game. The production of FMV Games appeared to be ramping up.
However, the overall cost to hire a crew and effectively shoot a Hollywood blockbuster became one of the underlying reasons for the rapid demise in FMV Game production. Not to mention the lack of divergence in the games, with minimal level of choice leading to the same outcomes. Quality was also an issue;
All our video had to be tortured, kicking and screaming, into the most horrifying, blurry, reduced-colour-palette mess imaginable in the Mega CD. I shudder to think about it.
Ken Melville – Digital Pictures co-founder
With its high production value, Double Switch had the potential to be a blockbuster game. Starring Corey Haim (Lost Boys), Deborah Harry (Blondie), R Lee Ermey (Full Metal Jacket), the player switched between six cameras in an apartment block to trap “mobsters” and “hitmen” as a plot unfolded around the apartment tenants and a mysterious egyptian statue.
Fun Fact: The Thugs and Intruders in Double Switch were mostly stunt men and women, who went on to do stunts in movies such as Mortal Kombat, Con Air, Timecop, Face/Off, Strange Days, Spiderman, Avatar, Vanilla Sky and Pirate Of The Carribean.
Unfortunately by the time these games were released, the want for FMV games had already declined. One other Digital Pictures FMV game was eventually scrapped; Citizen X. With the still-bubbling political rage started in Congress over the contents of Night Trap, and the unfathomable high cost of shooting video, the production of all FMV Games came to an unglorified end.
However. With todays technology, a revival of FMV games wouldn’t be a problem. Perhaps with the help of Kickstarter or IndieGoGo we could re-ignite the love once had for FMV games.
Come back, Digital Pictures. We miss you!
Footnote: This entire blog came about because I was searching the internet for information regarding the production of Double Switch, after I got hooked playing it again. I recently discovered I was able to play Night Trap on my Sony PSP thanks to a clever emulator called PicoDrive! It’s still one of my favourite games.
We all know the meaning of life is 42. But what’s the purpose? This is the question suffered by Qohen Leth in Terry Gilliam’s The Zero Theorem.
This week, thanks to Den Of Geek, I was invited to a preview screening of the movie at the headquarters of Sony Europe in London. This is the second time Den Of Geek have offered me a free ticket, having invited me to watch The LEGO Movie a few weeks back at the headquarters of Warner Brothers. Before I go any further I should point out that this review is my own, and is not a request by Den Of Geek. Any opinions are my own and should not be linked with the opinions of Den Of Geek, Sony Europe or any other affiliate.
I would also like to mention the mind blowing bonus that the audience received; before the movie started our host asked us what we’d do if Terry Gilliam were to walk through the screening doors and say hello. Half of me was disappointed that such a carrot would be dangled in front us. And yet, like rubbernecking witnesses at a car crash, one by one people in front of me turned and gawked open-mouthed as the man himself bounded down the side aisle toward the screen. Mr Gilliam was pleasant enough to talk for a few minutes about the production of the film, the friendships with the cast and his perpetual want for being alone!
So thank you Terry Gilliam for adding that extra brilliance to what was simply a free ticket to a preview screening!
Now, not everyone gets the visions that Terry Gilliam puts on-screen. Mostly outlandish. Definitely bizarre. Some within the grasp of steampunk. All of it very clever. Thankfully having spent a childhood watching “Time Bandits” over and over, and falling into philosophical discussions at 4am over the meaning of “Twelve Monkeys”, I was prepared to accept what was coming.
I wasn’t disappointed.
Stylistically, there is no mistake this is a Terry Gilliam film. Nobody else can portray a story like he can; clunky ridiculous fashion, day-glo neon, impressionable Governments, OTT commercialism (I particularly liked the advert on the building that tracked alongside the main character as he walked down the street, whilst all the time referring to everyone as “madam”).
Practical visuals aside, the concept delves deeper into human nature (as does every Terry Gilliam film). Almost to the point of forensics. Life as we know it blown into ridiculous proportions, yet seemingly accepted as the norm by those not-so-far-future characters. People go to work, they consume and partake in the controlled world around them. They accept the systems in place, playing along like sheep. All the while the main character simply wants to know the purpose of life. With the whirlwind of other peoples lives around him that he blatantly wants no part of, he’s forced to follow along and step out into the big wide world (which he keeps securely hidden behind the several locks on his front door). With no option but to accept fate, we’re taken on a Gilliam-style roller coaster with an inevitable end.
Gilliam said before the screening that the film was meant to be set in the not-so-distant future, but by the time it came to Production and release the concept had slipped into what it’s actually like today.
It won’t be long before it’s a period drama!
And I totally get what he means. In one scene, a party is taking place with lots of people in attendance. But everyone’s wearing headphones and listening to their own music, staring at the computer tablet in front of them, and hardly interacting with anyone else in the room. And if they are interacting, it’s by video stream to other people at the party. From an outward point of view everyone is enjoying themselves at this party, yet everyone is actually only enjoying their own closed-off personal experience as opposed to the group experience.
The cast is great. Christopher Waltz as the troubled Qohen Leth, Melanie Thierry (you’ll seek out more of her films, I guarantee!) as the gorgeous Bainsley, David Thewlis (who played King Einon in DragonHeart!) as supervisor Joby, Matt Damon with his bit-part as Management, and the unforgettable Tilda Swinton as the fantastic Doctor Shrink-Rom.
If you’re a Terry Gilliam fan, you are in for one hell of a ride. If you’re not then sit back and absorb the double-meanings, enjoy the subtle hints of oppression, compare your own life and wellbeing to those of the main characters.
I guarantee every single person will relate to one of the characters. Whether you’re the drone in the fast-lane taking everything for granted, the adolescent kid who knows more about the world than their older siblings, the camouflaged management in control of every situation, the worker in diluted fear of that management, or Qohen who’s misused talent draws him fatefully toward his quest to find his answer.
The film will no doubt create debate. Its impressions and meanings can be interpreted a million different ways, leaving the viewer to ponder on just what happened exactly. With this in mind, I foresee The Zero Theorem becoming a typical Gilliam classic.