In a blog post on April 13, Unity announced that Unity 2021 LTS is available to download and use, drawing the attention of a large number of game developers. In parallel with the rise of the metaverse, the use of digital humans such as "Liu Yexi" and "AYAYI" is also increasing rapidly in China, making it a highly attractive market for investors.

At the moment, there is not yet a unified standard for the workflow of digital human creation, as each company has its own procedure. In spite of this, Unity Greater China became the first company to disclose their digital human production process last year. Furthermore, the company has acquired Weta Digital and Ziva Dynamics, two of the largest firms in visual effects and real-time character creation, making Unity a powerhouse of real-time machine learning and character creation technology.

In March, Unity released a demo titled "Enemies-real-time cinematic teaser," showcasing the breakthroughs in digital human technologies. The cross-platform gaming engine's latest matrix of graphic capabilities for creating digital humans is demonstrated in this trailer, including new hair simulation, coloring, and updated eyeball and skin simulation technologies. These solutions are rendered at 4K resolution in real-time, with the capillaries of the character's skin clearly visible and her hair moving as she performs. Both the high-definition picture quality and the realistic effect are stunning.

In this article, we invited Mr. Yang Dong (Richard Yang), Unity Technical Director (Greater China), to share his thoughts and insights on gaming engines, digital humans, and the metaverse.

?

Q: Mr. Yang, could you please introduce yourself to our readers a bit?

A: Currently, I serve as the Technical Director of Unity Greater China, and I am responsible for technical communication and project management.

I have been joining Unity since March 2016 and am now in my seventh year. At that time, I was the only evangelist on the China team, and part of my role was to promote Unity in the country. The year 2016 was known as the year of virtual reality, and Unity was one of the most widely used engines for developing VR experiences. At present, my primary responsibility is leading the Unity Greater China evangelism team that promotes various technologies. I could be called a "coding veteran," and I have previously developed video games and web servers. To this day, I have not forgotten my coding skills.

I am also in charge of some project management activities for Unity Greater China since Unity now has an extensive non-game business.

Q: We have noticed that Unity's latest version, version 2021 LTS, is now available for download and use. What new features have been included in this version? How will it change developers' experience?

A: The latest version of Unity is Unity 2021 LTS, which is intended for the newly established project as this version is more stable with fewer bugs and no added features.

In fact, the current version is quite mixed. The advent of a new version will bring updated features and improvements to the existing one.

(The new feature of URP-Deferred Rendering)

We have set three critical directions for this latest version. First of all, there is an update to HDRP, which includes cloud rendering, ray tracing, and other updated features.

Meanwhile, a new feature in this release is the Universal Render Pipeline (URP). Despite URP's availability for more than two years, the pipeline still needs improvements compared to the earliest built-in rendering pipeline.

Another direction is DOTS, the Data-Oriented Tech Stack. It should be noted that this is DOTS version 0.5, which is not considered a full release. Yet 2021 LTS's UI is significantly improved over the previous version. Later this year, we'll release version 1.0 based on that. DOTS is like a module within the Unity editor that you can load as required and offload when you don't, as the DOTS module is not built-in.

Some of our users are interested in working on the latest features. Therefore, Unity will also release a TECH stream beta version to meet the needs of our users alongside the LTS one. Iterations will continue to be quicker in this version, and more features will be included.

Q: Unity's roadmap for 2022 is about to be released. What are the specific plans for technology?

A: The current LTS version is the foundation for the 2022 version, with four key components. A high-quality rendering will include more features, such as hair and fabric renderings.

We will also launch DOTS version 1.0 shortly. This will enable data-oriented programming in the official product, allowing for more efficient multi-threading and better memory management to handle large numbers of objects, props, etc.

In addition, ray tracing will be officially introduced in the 2022 preview version. Currently, ray tracing relies on NVIDIA's RTX series of graphics cards for hardware support. As NVIDIA's 40 series graphics cards could come out in 2022, it could increase the popularity of ray tracing.

Besides, Unity has a complete suite of multiplayer online game systems—NetCode, which is currently in development. The popularity of multiplayer online games, which is also true for most metaverse applications, calls for underlying capabilities to support them.

However, developing multiplayer modules and functions can be laborious, which is beyond the capabilities of small and medium-sized studios. Thus, we are developing this NetCode system, which consists of some general networking services, and turning them into cloud services that require no development or maintenance. At present, we are working on the domestic implementation for Chinese developers.

Q: In addition to providing solutions for games, in what other areas does Unity currently focus? Where do you see more development opportunities?

A: At the moment, games still make up the biggest part of Unity's revenue and have a special place in our business. In addition, China's automotive, construction, and smart manufacturing industries show plenty of potential, and Unity is active in these markets as well.

Several of Unity's projects, such as Unity's collaboration with Haier and Hong Kong Airport, are related to the digital twin in the construction and smart manufacturing sectors. For instance, Haier's smart manufacturing project used Unity to build a one-to-one digital twin pipeline. With data connected, workers can operate the virtual production line while controlling the physical one remotely—this dramatically improves production efficiency. For Hong Kong Airport, Unity provided an equivalent solution. Unity simulated its new terminal building and kept it synchronized with its digital twin.

Many 3D rendering animations in the automotive industry are based on Unity, including Oshan Z6. Large screens will play a more important role in electric vehicles than in traditional gas-powered cars. The engine can also render content in real-time to interact with in real-time, which makes 3D rendering one of the most remarkable ways to do it right now. A model of the car is always displayed on the screen in these types of vehicles. This model can be interacted with as well. You can use it to implement some functions, such as opening the windows in the car and listening to music.

Furthermore, the integration of 3D maps is a viral trend for most major providers. In some cases, 3D map vendors collect outside data, such as when you drive on a viaduct, the map app can automatically switch to a 3D model for better visualization. This function is more easily accomplished with electric cars equipped with large displays since the necessary data will be obtained from the map on the vendor's server, and the rendering engine will generate a 3D model in real-time.

On top of that, there is also the intelligent assistant. Not only do vendors think about voice services, but most are considering digital human applications. In a voice-driven digital human, the user will be able to receive immediate feedback. Having information passed to you through a persona will be more vivid than using traditional voice services.

In addition to this, game-playing is also an important scenario. For example, you could play the game while parking your car. The system can function like a gigantic game console that can create a high level of immersion. A Bluetooth or even USB joystick can be connected to it.

Q: As the competition between gaming engines becomes fiercer, how has Unity differentiated itself from its competitors? What are the plans for Unity?

A: Gaming engines compete for platform support, meaning which engine supports the most significant number of platforms. We all possess a variety of electronic devices, including Macs, Linux, and other operating systems on our computers; Android and iOS phones; PlayStations, Xboxes, Switches, and various types of STBs. As a result, if your engine is not comprehensive enough to provide sufficient support , or it does not have content creation tools, it will fall behind in this battle.

For the moment, Unity offers a wide selection of platforms with substantial support and a highly developed toolchain. In this way, cross-platform games like "Genshin Impact" can be played on a cell phone, a PlayStation, a PC, and even a cloud gaming platform with the same account. This implies all its data is synced.

Furthermore, various devices are becoming increasingly influential as time passes. Therefore, people are more interested in seeing how your engine can assist developers in creating higher-quality content quickly. High-quality images can be defined in two ways, including stylized high-quality images and realistic high-quality images. Unity and its main competitors are on par in these respects. Furthermore, Unity has performed better in cross-platform capabilities, workflow, ease of use, and market share.

Q: Recent trends in virtual digital humans have gained momentum, and Unity has made quite a splash in this field. Since digital humans are still relatively expensive to create, how can this process be improved to reduce costs and improve efficiency? If so, what challenges will it face?

A: When I started working on graphic rendering tools over a decade ago, there were already many people creating digital humans, but in those days, they were all rendered offline, with tiny animation, and the price of production and rendering was extremely high. Nowadays, the development of real-time rendering technology has made it possible for high-quality digital images of humans to be produced. Unity can render digital humans at a rate of at least 30 frames per second.

There are, however, three significant difficulties with the current production. First, we have an image of a digital human, which is called a model. However, in reality, it has to be generated by traditional modeling methods.

Another example would be digital human accessories, such as hair and fabric, which can be rendered digitally through real-time rendering but are still somewhat difficult to reproduce. As shown in the video, we used Maya's X-Gen data to generate the guideline data for the hair, and then Unity used its hair system to create the hair according to the guidelines. However, the entire process still needs to be done by professionals, which is a much more challenging task. In the ideal scenario, we would develop a human head model without hair, then import it into the engine, and then comb the hair in the engine and adjust its color and dynamics.

Additionally, the fabric can be simulated and rendered. Modeling it out is not a difficult task, but making it move realistically is more challenging. Looking at "The Heretic" and "Enemies" demos, these two digital humans are using Marvelous Designer, a Korean company's software simulation, after exporting the cached fabric vertex movement data and then playing using Unity. What is played is an animated still. If you bind the clothing in a skeletal fashion, one cannot drive it in real-time and the type of clothing will be limited. To represent the fitting clothes, this can only be done by modeling. It is therefore complicated to simulate realistic, real-time interactive fabrics. However, Unity will also be able to come up with corresponding solutions for the problems of hair and fabric.

(Digital Human-"The Heretic")

In addition, facial expressions are one of the most challenging things to replicate. Ziva Dynamics was acquired by Unity this spring, and its technology offers a solution to the low quality of traditional face animation by using artificial intelligence to train heads and faces in such a way as to produce a realistic-looking model. Ultimately, a fully fledged digital human can be created in the engine. In another year or two, perhaps, this task will be completed, and you will be able to control the expressions, movements, and nearly all of the renderings. It is also a point that is challenging for the industry to overcome in the near future.

Q: The creation of immersive environments is not enough to achieve a true metaverse. The user must also experience the experience with low latency. What has Unity done to make this possible?

A: It is related to the aforementioned NetCode Multiplayer Suite, as building multiplayer apps or games is difficult. In addition to the cost of getting a server, there are not many off-the-shelf options. Tech giants can develop good multiplayer games because they have a large team, but their tools are not publicly available. Multiplayer, for ordinary users, includes the ability to manage the server remotely, program the server remotely, and ensure a proper client connection. NetCode and DOTS combined by Unity now help lower the threshold.

Q: As an IT veteran, what advice would you give to developers or managers in the game industry?

A: As a technician, you must specialize in a particular field, but you must also develop your vision, expand your thinking, and read books in other areas. You may not have much knowledge about art, but you can communicate with colleagues from the arts community to take advantage of their skills. By doing this, you expand your horizons.

Having only your own interests in mind makes the journey more difficult. Although we may not be as physically capable as younger individuals, our experience and wisdom compensate for our inferior physical abilities. These experiences can be achieved not only by working all day long but also by engaging in activities that interest you. To be promoted in a company, you must demonstrate your technical ability, communication skills, and coordination capabilities. It is not individuals but the whole group that will determine the quality of a project. When the overall product experience is poor, it makes no difference if the individual modules are well written. Therefore, a holistic approach is essential. Being involved in a field to which you are unaccustomed can be challenging. You could hit a wall, but then you will learn and grow.

About the Guest

Yang Dong (Richard Yang) currently works as Unity Technical Director (Greater China) and is responsible for the China Evangelism team and Product Management team. He has extensive experience developing games and interactive content and focuses on spreading technologies to Unity community developers.