• 한국어
    • 日本語
    • 中文-繁體

    2022.08.24 NC Leadership

    Dreaming of Digital Actors that Break the Boundaries between Reality and Imagination, Byung-gun Jung

    NCSOFT has newly established its Digital Actor Division and appointed Byung-gun Jung as the Head of the division. Byung-gun Jung has participated in creating Korea's very first virtual influencer ‘Rozy’, and worked as a technical director at global companies such as Disney animation studio, Weta Digital, Digital Domain and Blizzard. This interview introduces the future of digital actors that Byung-gun will draw at NC. The interview will also look into the dreams and goals that Byung-gun wishes to achieve at NC after his long career as a visual artist in global companies.

    Byung-gun Jung

    Head of Digital Actor Division, NCSOFT

    Byung-gun is a digital actor expert who has overseen the production of virtual influencers like Lizzie Yeo, Ho•GON•HEIL and Ryu ID as well as Korea's first virtual influencer, ‘Rozy’, at Sidus Studio X. He has also been involved in the production of Hollywood films such as <Avatar>, <Interstellar>, <Guardians of Galaxy> and <Thor> as a technical director. After being appointed as the head of Digital Actor Division at NCSOFT in July 2022, he is now responsible for strengthening NC’s visual R&D capabilities.

    Bringing Imagination to Reality

    You have built careers as a technical director at global companies including Disney and Weta. It’s interesting that you started your career in computer graphics(CG) as an Arts major.

    When I was in college, I bought Apple’s Macintosh as my first computer under the influence of my senior. At that time, I came across graphic apps such as Photoshop and Illustrator at a Mac user group. I felt interest in computer graphics where dynamic and new technologies keep emerging rather than static arts. Such interest naturally led me to start my career in the CG field.

    There are various fields in computer graphics. Please tell us about the work that you did specifically.

    My first career started out as a texture artist in the film industry. I had steadily built experiences in the texture field and expanded my work area into look development at Disney. Thereafter I built another career as a lighting artist as well as texture artist at Weta. Texture, look development and lighting are closely related each other in VFX(Visual Effects). I believe the reason I could produce better results than others was because I was able to carry out all three fields.

    The concept of look development is new to us. Please explain what it is.

    Look development is the 3D implementation of 2D concept created by an art director or concept designer. It can be defined as a process of making the final image that is visually shown when making a video. Look development is interesting in the way that it is created from the final results which are visible.

    What was your most memorable project?

    The times when I took part in the look development/lighting works for film <Real Steel> is most memorable. Since I extensively experienced overall VFX processes after building a career as a visual artist for about 10 years, I was able to work on the project with confidence that the results would come out well.

    You must have felt the development of technology while building your career as an artist for a long time. How is the visual R&D going in the actual industry?

    Visual R&D is carried out according to the needs of production and is largely divided into long-term R&D to enhance the overall efficiency of production and short-term R&D for individual projects. For example, the ‘USD(Universal Scene Description)’ format developed by PIXAR for easy and efficient asset management is part of a long-term R&D.

    In terms of short-term R&D for individual projects, <Tangled> which I took part in its pre-production can be an example. Although there were many technical challenges that needed to be solved in <Tangled> such as implementing the character’s 20-meter-long hair and simulating of multi-layered medieval clothes, it was impossible to visually implement them with the tools of that time. Accordingly, our production team back then solved such challenges by adding functions required in production to several tools like x-gen.

    You must have realized the importance of visual R&D at the forefront of the industry.

    If one succeeded in realizing a non-implementable technology through visual R&D, this could be a way for the studio to hold an exclusive position in the VFX industry. 10 to 15 years ago, it was not easy to realistically implement water, fire, hair and skin; thus, some studios that were capable of doing so such as ILM or Weta could enjoy a monopoly position.

    However, as the market has grown to some degree, using commercialized tools instead of developing one’s own custom tools may be efficient in terms of development and more advantageous in terms of human resource management. If you need to carry out small-scale R&D due to difficulties affording a larger one, a good alternative could be to select such commercially available tools and establish a process for what type of technology is needed and how we can use it.

    Creating a Digital Actor that is Beyond Natural but Splendid

    Now we would like to move on to talking about NC. What made you decide to join this company?

    I sympathized with the digital actor roadmap that NC was envisioning. I made the decision to join the team since I thought NC is a company that is equipped with the manpower and resources that could help realize the vision I was dreaming.

    You oversaw the production of virtual influencers well-known to the public such as Lizzie Yeo and Ho•GON•HEIL as well as Korea's first virtual influencer, ‘Rozy’. How did you gain interest in the production of ‘virtual human’?  

    When creating a *digital double in the film VFX, I took part in the production with the best artists in the field for over several months to a year, but I barely managed to create a virtual human that just got out of awkwardness. It is that hard to embody a sophisticated and realistic virtual human that is indistinguishable from humans. But as an artist, I truly wanted to challenge this field.

    *Digital double: Technology to overlay a virtual character's face on a human face after analyzing the face of real human through 3D modeling

    You will be leading the ‘Digital Actor Division’ that is newly established by NC. What will be your work and goal there?

    Our division is currently making digital actor implementable in 3D environment through collaboration with many internal divisions including the AI Center, Visual Center and Art Lab. Our division has a simple goal which is ‘Creating a digital actor that is more natural, greater and better than anyone else’.

    What are the strengths that NC has in creating a digital actor?

    There are many talented developers and artists at NC. We also have 3D scanning and motion capture system, as well as visual R&D capabilities for the implementation of realistic and sophisticated digital actors. Another advantage is that we can combine the digital actors created with various games and research results of NC’s AI Center and utilize them. NC’s technology and games will be the stage for our digital actors.

    Then what are the difficulties during the production process?

    The most difficult part is to create a natural ‘Facial Animation’. The faces of humans actually contain a lot of ‘data’. By looking at someone’s face, you can tell how that person is feeling, whether the person has a good feeling toward me, and whether that person is lying, etc. As we have repeatedly trained to read other’s face and expression since childhood, we can notice the unnatural movement of face more quickly than body movement. In fact, many of the times that we think a virtual human is awkward or unlike a real human, is often attributed to unnatural facial animation.

    What effort is NC is making to embody natural facial animation?

    In order to obtain high-quality motion capture data, we are planning to case actors with good facial expression acting, or to elaborately improve the *rigging, which is the skeleton of facial animation. We will also be looking at omnidirectional ways to automate facial capture to the level of 50-60% of the final output which includes applying AI technology such as deep learning.

    *Rigging: Building the digital skeleton to embody the movement of character in 3D computer animation

    In our previous interview, we met CRO Jehee Lee to talk about interactive digital humans. It seems important to implement digital actors that can interact with the players. What is the roadmap that NC is drawing specifically in this aspect?

    In line with what CRO Jehee Lee said, our final goal is to implement a digital actor that can naturally talk and interact with humans by combining AI technology. However, this is a goal shared by many other global technology companies and some are already making some progress. As NC has just taken its first step, we will be steadily moving forward toward our goal to create a digital actor that can truly interact with humans, starting with what we can implement at the current phase such as digital humans.

    At the current stage, we have a plan to launch NC's own influencer that supplements the shortcomings of existing virtual influencers. Then we will begin utilizing digital actor technology in earnest in NC’s games. Digital actors may directly appear as the main/supporting roles in a game, or we could utilize the skills accumulated through creating digital actors in the game development process. This is an advantage that NC holds having our own IP.

    *Reference: Interview with CRO Jehee Lee (Link)

    What changes can players notice when digital actor technologies are applied to games?

    In the end, immersing players into the game is important. As we are capable of implementing game characters that are more realistic based on digital actor technology, players will be able to experience greater immersion and enjoyment in the game.

    Drawing Upon an Environment that Helps Artists Grow

    There must have been concerns as well along your career as an artist. What do you think needs to be in place for artists to be able to develop their competencies?

    An environment needs to be in place where good projects and talented artists can come together. Looking back upon the path I took as an artist, the times when I grew the most was when I finished a difficult project that back then I thought was impossible to implement with the given time and technology. I was also able to grow significantly when I was part of an organization that had talented colleagues, outstanding supervisors and art directors. The condition for growth is sufficient at NC in that there are many outstanding artists. The education programs provided at NC are also strong even when compared with other global leading studios.

    The visuals of videos and games are created through the efforts of various members. What do you think is important to improve the quality of productions in the collaboration process?

    First, it is important to deliver accurate directions and references to team members, and to allocate works in a way that enables them to fully utilize their abilities. We also have to make a strong plan within limited resources. When we produce outputs on a long-term basis like a movie or a game, we produce about 1~2 ideal looks or visual assets that we want to finally embody in the pre-production stage. After that, the most efficient way in terms of time and budget is to optimize and simplify the process, then apply it to the production on a large scale.

    As a project leader, your role seems to be very important.

    A leader must never go into a panic. When the leader loses his/her composure, it also affects the team. This is why I believe a leader should keep calm and solve the problems no matter what difficult situation comes.

    I also believe in the power of support and encouragement. I have seen many cases where good results were achieved when the team is appreciated for what they did well. I wish to be a leader who gives full support and encouragement at NC.

    Lastly, what are the goals you want to achieve at NC?

    For a long time until now, I have been working on deriving the final output in the field of videos, such as movies, animations, and advertisements. Now, my goal is to become a leader in the industry by producing the best output in games and next-generation platforms with the talented teams at NC.

    * The above presents the personal opinions of the interviewee and not the official opinions of NCSOFT.