NC

The TECH TRACK series sheds light on the developer roles of the projects currently under development by NC. In the TECH TRACK series, we will meet individuals in developer roles that effectively showcase the latest direction and characteristics of the game industry. We will listen to their roles and visions. First up is Shin Joongmok who produces digital humans for Project M.

The digital humans defined in Project M refer to the recreation of actual actors or models within the game’s universe and its settings. As Project M aims to achieve AAA console game quality, we strive to create characters that are as realistic as possible, paying attention to even intricate details, such as facial expressions, skin texture, hair, body types, and outfits.

The "realistic visual" that we pursue does not refer to the detailed reproductions of pores or blackheads. We continue to contemplate and make attempts to determine the level of expression and detail necessary to create characters that are both realistic and appealing, in order to find the ideal type of digital human for Project M. We also strive to identify the visuals that will enable players to immerse themselves in the game. Based on these experiences, our team is conducting various research and development (R&D) efforts to accumulate production know-how.

Out of a total of six years of experience, I have spent five years specifically focused on digital human creation. Currently, I create digital human characters and related assets in the Project M development division. Simply put, I am responsible for creating realistic appearances among the various elements that constitute a digital human.

I concentrate particularly on implementing hair that reacts sensitively to the character's movements, ensuring a seamless and natural appearance. Utilizing the Unreal Groom Plugin, we are now able to partially incorporate strands of hair in-game. This allows us to analyze photos of actual styled hair and optimize it to fit the game. Furthermore, we complete the final style and look by adjusting the physics value of the simulated hair when applying animations.

I also focus on finding the points where the shapes and movements of the body can seamlessly blend into the game during the creation process. In accordance with the universe set in Korea as the background, body shapes are produced, taking into account the physique of Koreans. Additionally, scanning technology is utilized to achieve a detailed range of elements that are difficult to express solely through imagination. Scanning technology is widely used to depict realistic appearances, not only for the body but also for various elements such as facial expressions, outfits, and other items.

As the game is set in the present day, numerous everyday outfits and items make frequent appearances. In such cases, the character team autonomously plans the coordination of outfits and proceeds with modeling by scanning necessary assets or using simulation-based outfit production techniques.

Casting an actor that will serve as a reference to the 3D character is required before creating a digital human. Sometimes the character team participates in the casting and auditioning process because the characters are created based on these actors. The character team verifies through videos and auditions whether the actor can be represented in a natural form when recreated in 3D, if the forms and combinations of their facial expressions, which serve as materials for FACS¹ Facial Rigging², are natural and stable, and if there are no abnormalities in the alignment of the jaw. Afterward, the team creates a "FACS Analysis & Review" document, their internal evaluation criteria, to assess whether the candidate actor is suitable to be recreated as their game character, and casts actors based on this document.

Various episodes have occurred during the actor's audition process. One time, we needed to cast a middle-aged actor, and even after a long search, it wasn’t easy to find the right candidate. And then, coincidentally, we discovered that our team leader has a background in acting. The acting skills of an actor, who serves as a reference, are also important factors behind the realistic expressions of a digital human. In the end, the team leader personally donated his face to the project and was created as an NPC in the game. While working on character development, the team leader of the character team ended up becoming a real character himself. It makes me laugh whenever I see the character in the game.

The first company that I worked for was a small start-up that developed a battle royale game. I learned the fundamental skills as a modeler in an environment where I had to create numerous assets within a short amount of time. It was at the second company that I worked for where I started digital human creation. It may have been a small company, but it was equipped with a digital human creation environment and development know-how. That was where I learned about the overall character creation pipeline. I experienced BlendShapeBuild for facial rigging creation and had the opportunity to learn the concepts of deep learning input and output through Analysing and Retargeting. Additionally, I underwent an advanced process of creating cards, grooming hair, and designing outfits with a realistic look for digital humans.

The skills required for creating digital humans are essentially similar to those needed as a character modeler. In order to create the required modeling and hair for the project, it is necessary to have basic proficiency in using DCC tools such as Maya or 3ds Max. You also need an understanding of engines such as Real-Time and Unreal Engine, and the ability to create optimized resources and PBR textures based on that knowledge is required. Skills in utilizing Wrap 3D to process scan data or edit topology, as well as techniques in shaping expressions or forms with BlendShape, are additionally required. These aspects can be largely acquired through practical experience in the field.

However, before all these practical skills, the important competency lies in the meticulousness and perseverance to add detail upon detail until the very end. There is no objective standard for "completion" when it comes to creating characters. Hence, in order to approach perfection where there is no right answer, the attitude of embracing various attempts and iterations must be the foundation.

The expectations of players who engage with AAA games have significantly increased. Having a sophisticated universe and narrative is a fundamental requirement. In addition, it is essential to have compelling characters capable of natural facial expressions and dialogue, enabling players to immerse themselves in the game. In Project M, characters play a central role in the game by closely interacting with the players.

Many games utilizing digital humans are already being developed overseas. In particular, digital human utilization has become a prevailing trend in overseas AAA games. The Last of Us Part II, Uncharted 4: A Thief's End, the God of War series, Hellblade, Horizon Zero Dawn series, and many other games are utilizing digital humans as characters. Thanks to this trend, scanning techniques, and AI technology have been rapidly advancing, allowing for more realistic character creation and enhanced interactive experiences. NC also actively participates in the utilization of digital humans, collaborating with leading photo scan systems and AI technologies.

The character team also focuses on R&D to enhance the overall pipeline of creating digital humans while working on development.T he most impressive outcome is the solution that integrates the Combination System with the FACS Facial Rigging. It was an R&D effort to enable the natural transition of intermediate facial expressions in the process of combining two or more expressions within the rig. When combining facial expressions that share a similar range and expressions that move in different directions, if the value exceeds a certain threshold, excessive double transforms occur. To prevent this, we aimed to reproduce stable combinations of facial expressions by adjusting them using Combinations and Correctives. In order to develop a proprietary rigging tool that can achieve this, we collaborated with the Rigging team and successfully created a FACS Facial Rigging tool capable of incorporating the facial expression combinations produced from scanning.

Another significant outcome was the Analyze & Retargeting R&D. Due to the extensive workload involved in creating rigging for every character, we aimed to establish a single, standardized rig to be universally used for NPC character rigging. For this purpose, we carefully selected models and conducted R&D specifically for NPC Seed Rigging. During this process, we were able to confirm the significant impact of realistically reproducing expressions, which serve as inputs for deep learning, on the quality of the primary animation. We also have plans to support the Animation team by creating profiles of major facial expressions, which will serve as inputs for deep learning.

Moving forward, Project M aims to go beyond game development and strive to leap forward from the industry's digital human standards through continuous R&D.

An environment where developers can solely focus on development is probably the ideal development environment that every developer dreams of. NC has organizations dedicated to the professional development of assets and systems required for development. As we have a continuous interest, particularly in real-time digital human creation technologies such as AI, scanning, motion capture, and digital doubles, these dedicated organizations are actively developing related infrastructure and technologies. The Character team is advancing R&D by actively leveraging these technological infrastructure and foundational technologies.

Another notable example is the Scanning Studio. The Scanning team provides significant support for the preliminary work of FACS Facial Rigging and animations. By internally processing high-quality scanning data without the need for external handling, we can reduce the burden of resource management and focus more on development tasks. There are not many companies that possess an internal scanning studio with the level of technology and talent that NC has.

There are times when you may feel the limitations of expression when trying to reproduce an imaginary object with graphics. At such times, it is necessary to secure good scanning sources and new infrastructure. In this regard, the Character team receives support to produce high-quality results by utilizing the LightCage, a 3D scanning device, for capturing high-quality face scans, as well as scanning booths for capturing costumes, bodies, and other elements.

Our Character Team is dedicated to each and every task at hand. We have zero compromise, especially when it comes to quality.We are sincere in our desire for the game to progress and develop.There are times when team members become so obsessed with quality that they can make things difficult for each other. At times, there can be intense debates over a mere difference of 1-2 mm.I believe that such obsessions not only contribute to the personal growth of individuals but also have a positive impact on the outputs created by the team. I believe that the passion of each individual to develop their favorite aspects of the game is what enabled us to achieve so much in a short period of time.

Because the field of digital human is still a relatively new profession, the scope and definition of its work are not yet firmly established. We are gradually establishing these aspects as we progress with the project and enhance our work. As we need to explore new territories, I can personally feel that the Character team of Project M is at the forefront of change. We have high expectations that we will continue to grow based on our unwavering dedication to quality.