How to turn tricky-to-scan historic treasures into photorealistic video game models

An actor wearing a medieval knight’s armor while being 3D scanned with Artec Leo

When it comes to turning real-world people, objects, and environments into lifelike TV, film, and video game models, photogrammetry remains the industry standard.

Often deployed in the form of camera-filled booths, this process of creating models from high-resolution overlapping photos, allows physical items to be accurately recreated in the virtual realm. However, 3D scanning is quickly emerging as a faster, more flexible alternative, with which props and character models can accurately be digitized in seconds.

These benefits aren’t theoretical either, the technology is taking off in-industry. To date, Artec 3D scanning has helped create everything from high-resolution military video game models for online FPS World War 3, to frightfully realistic VFX for horror hit Sleepy Hollow.

Given the efficacy 3D scanning is already showing as a cutting-edge CGI tool, many will be asking: how straightforward is adopting the technology? And how can scans be turned into visual effects? Thankfully, Artec expert Mikhail Shumikhin is on-hand to talk you through the process step-by-step, and explain how to optimize results.

Digitizing a knight in shining armor

Wherever you look in gaming, whether it be Dark Souls, Dragon Age, or a plethora of other big titles, knight’s armor has become a common sight – it’s a costume trend developers won’t seem to let go of.

That’s why Shumikhin chose armor for his 3D scanning demo. In borrowing it from the Medieval Kotor living history museum, where conservators and actors keep the country’s history alive by restoring and performing in period costume, he wanted to show how 3D scanning can pick up even the most difficult surfaces through an industry-relevant use case.

A knight being painstakingly digitized by a traditional high-res camera via a series of photos – as would be necessary to carry out photogrammetry

Once he’d identified a costume to capture and convinced a colleague to act as a character model, Shumikhin began 3D scanning them using Artec Leo. With a 35 million points/s scan speed, Leo can digitize entire objects and people in seconds. This, in addition to the device’s wireless nature made digitization a rapid, straightforward process.

To account for micro-movements when digitizing such actors, Shumikhin advises dividing the scan into sections. In his armor project, he says this gave him “more control over quality,” and allowed him to avoid “artifacts caused by slight movements.”

Powered by AI, HD Mode – a feature open to the users of Leo and the lower-cost, time-tested Artec Eva – also helped maximize resolution and minimize data lost to shininess.

“HD reconstruction is an extremely useful tool,” explained Shumikhin. “This allows you to increase the amount of data you can accumulate, gathering 2, 4, or even 8 times more. With Leo, it’s also possible to pause scanning, move to the other side of the object, and continue.”

“The biggest advantage of our 3D scanners is their mobility and quality of texture and geometry capture. They can easily be transported anywhere in the world to capture objects in high resolution.”

Mesh-tidying on Artec Studio

In the process of tidying and preparing scans for export, Shumikhin’s efforts were boosted by the advanced tools included in Artec Studio. In particular, he says the simplicity of the scan capture and data processing software’s global registration algorithm makes it a user-friendly, quick way of correcting errors and creating the highest-quality mesh from captured frames.

Some of the armor’s shiny surfaces captured in Shumikhin’s 3D model

When it comes to optimizing resolution during scan fusion, Shumikhin has a top tip for those seeking to replicate his workflow. For best results, he suggests setting their fusion resolution to 0.5 mm, not necessarily the full 0.1mm Eva and Leo are capable of.

Likewise, Shumikhin adds that Artec Studio’s hole-filling tool is extremely effective at closing any mesh gaps, where reflections have prevented full data capture, and he didn’t use photo-texturing either – a feature that can boost resolution even further.

“Artec Studio has a very cool algorithm for photo-texturing and registration,” said Shumikhin. “With this, you can achieve extremely detailed models. In this project, I captured such high-quality textures with Leo that I didn’t need to use it, but photo-texturing would come in very handy for many others.”

Going from mesh to model

After tidying up his mesh, Shumikhin sent it to a handful of third-party programs for texture baking, rendering, and tackling another common CGI modeling issue: symmetry. He calls achieving this “a fundamental rule” for making models video game-ready.

As it’s pretty much impossible to get actors to stand exactly symmetrically, he says meshes need to be evened out digitally via editing software. To achieve this, Shumikhin recommends exporting meshes to platforms like ZBrush, where their left and right sides can be mirrored and used as ‘guides’ for making them more symmetrical.

Shumikhin then exported his model to Maya for the creation of low-polygon geometry and to take advantage of the software’s UV layout tools (for texture mapping), before sending it to Marmoset Toolbag and Substance 3D Painter.

The Montenegrin knight’s armor model being rendered and prepared for animation

With Toolbag, Shumikhin later rendered his model and perfected its lighting, before baking in textures. However, while the standard texture map baking setup is sufficient for most models, he advises reducing them to 4K beforehand to minimize file size. After texture cleaning in Substance for “stylization,” he sent the model back to Maya for the addition of realistic hair to his character via the software’s Ornatrix plugin.

Finally, once he was finished texturing, it was simply a case of using Maya’s auto skeleton creation and ‘skinning’ tools to prepare it for animation or rigging. As there’s “such a huge community of people working on Maya,” he adds that it’s the perfect place to “find answers to common questions and get advice” on rigging and other parts of the process.

3D scanning: the future of CGI?

Having successfully turned an actor and medieval suit of armor into a fully animated, costumed character, Shumikhin believes his 3D scanning process can now be built upon and utilized by others – not just those in the CGI industry either.

Shumikhin’s final knight’s armor model (with sword and shield models designed earlier)

While a scanning expert, Shumikhin says his role “doesn’t require an extensive set of skills” around video game modeling. Instead, he argues that his “inner enthusiasm and desire to develop” requires him to “make great discoveries and persevere in 3D.”

As such, he now encourages other aspiring 3D artists to follow in his footsteps and “look for lessons and new tricks to apply,” while using scanning to develop models of their own.

“Everything we see in the modern world can be recreated as a 3D model. In this way, 3D scanners represent a bridge between worlds – the real and the virtual,” concluded Shumikhin. ”With game studios, movie companies, and the VR industry continuing to need high-quality VR content, this can be provided by Artec 3D scanning and Artec Studio.”

“3D scanning has already found its niche quite comfortably: the creation of highly realistic props, environments, armor, vehicles, characters, and actors.”