Just Adam 360 Experience | Mettle SkyBox Suite
Steven Emerson (www.stevenemerson.co.uk) developed a terrific 360/VR workflow that incorporates traditional 3D methods (Maxon Cinema 4D), and combines them with Adobe Premiere Pro and After Effects with Mettle Skybox Suite of 360/VR production tools . He carefully documented the whole process, from shooting to post, which we are sharing below. Thanks Steven, for using our products!
The Just Adam 360 Experience is a music video that doesn’t try to hide the 360 production process, but makes it part of the concept. The viewer watches the band walk into a studio set up with green screens, lights and cameras. As the music video plays, we gradually see the green screen and all traces of the studio disappear, while the musicians perform. The scene around the musicians changes to 3D generated images, combined with traditional VFX generated in Adobe After Effects.
Just Adam 360 Experience
Steven Emerson explains the workflow:
We were lucky to be able to hire a location that suited multiple greenscreens and multiple softbox lighting. Even the standard roof lighting in our studio location allowed for a more evenly lit chrom-key result. Lots of takes were filmed and all the usual 360 video shooting practices were carried out.
The Just Adam 360 Shoot
The Just Adam 360 Stitch
The workflow for the Just Adam 360 Experience, like many other 360 projects, all started in Autopano Video Pro.
Once the video was stitched and exported with Cineform in its 2:1 equirectangular format it was imported both into Adobe Premiere Pro and Adobe After Effects simultaneously. Premiere Pro was used for fluid playback and to drop markers to the beat of the recorded song. This is always a must when working on music videos. While After Effects is where the real work for the project would be done.
The 2:1 equirectangular image was treated like any other piece of video footage for masking and keying out the greenscreen parts, with the intention of using these alternative footage versions for transitions later. Mettles Skybox Extractor was used on a medium quality setting to create a skybox conversion to allow for some editing to start.
The first thing was to create a PreComp with the back-plate solely to fix the nadir and remove the visible tripod. This was achieved similar to previous workflows that would use a single frame export in Photoshop, paint the tripod and reimport back into After Effects for masking. The only difficulty that I have with this is that a single, frozen frame has no video noise and would stand out in the final output. Instead I used a piece of looped footage from a separate empty room take. Mettle Skybox Converter was used to Tilt the PreComp contents -90 degrees and allow for similar workflow only this time with a looped video clip.
The Mettle Skybox Boxmap Rig was copied in principle within Maxon Cinema 4D. Six cameras could be exported and integrated into separate Mettle Skybox Outputs to be composited as normal within Adobe After Effects.
Other Mettle effects such as the Skybox 360 Post FX Glow, Sharpen and DeNoiser were used to help with the 360 workflow.
Q&A With Steven Emerson
What was the goal of this project?
The goal of this project was to use 3D applications with 360 video. Previous examples show plugins such as Video Copilots Element 3D being used to add OBJ models into After Effects with Mettles Skybox Extractor. This a fantastic method for adding 3D to 360 video although my interest was trying to come up with some sort of pipeline or workflow to allow traditional methods with 3D applications such as Maxon Cinema 4D. The Mettle Skybox suite was the tool of choice!
What were the challenges that you faced and overcame?
Render time! 4000+ frames, x6 cameras, at 2024×2048 pixels was heavy demand for any 3D animated project. There was a balance between render time and output quality decided but the saviour of the project was a render engine from OTOY called OctaneRender. OctaneRender can use multiple NVidia GPUs with parallel processing to greatly speed up 3D rendering times. In this projects case, 3x NVidia GeForce GTX 780Tis were used to power-through the rendering.
At the start, keeping a high video quality throughout the project while footage would be rendered at multiple stages was a concern. GoPros Cineform codec was just and although it has some frame size and ratio limitations, it was easily manageable and allowed for a close to lossless workflow.
A final challenge worth mentioning was the masking and frame by frame keying of the shiny, reflective drum kit and cymbals. It’s well known a larger room is required to minimise reflections such as these, but it was a conscious effort to take time and properly mask the drum kit and every cymbal hit by hand. Shout to the Brograph Podcast for getting me through the monotony.
How did you get into VR production?
I tried an Oculus Rift Development Kit 2 with some video games around a year ago and thought VR could be an incredible format. At around the right time, camera rigs and software became more available and affordable for me to apply my own knowledge and skillset to 360 video. Interest and knowledge of the format grew and before jumping in and making 360 video myself, I created a explainer video to help everyone understand what the format is and how it works. See the animated explainer video below.
What is 360 Video? – Steven Emerson
How did you find out about mettle plugins?
From a nice simple Google search! At the time of researching and finding out what plugin or software solutions were available for helping with a 360 video workflow; Mettle were one of the top results. This was only emphasised and affirmed by Mettles success and continued development as I got closer to post-production stage of the project.