Hwangnyongsa, a Buddhist temple located in the city of Gyeongju, South Korea, is one of the representative temples of the Silla era, along with the Bulguksa temple. The entire building, however, was destroyed due to an unfortunate event, leaving only the site of Hwangnyongsa today. The South Korean National Research Institute of Cultural Heritage regretted the loss of the magnificent relics of Silla and, after much deliberation, decided to restore Hwangnyongsa. Since restoring cultural heritage with augmented reality (AR) is not a simple task, it was essential to collaborate with VIRNECT, a company that specializes in virtual reality (VR) and AR content. After much historical research and study, the middle gate and southern corridor of Hwangnyongsa were restored with AR. This AR content was recently recognized for its quality, receiving the 2021 Korean Made With Unity (MWU) Best Visualization Award. To hear more about the restoration process of this cultural heritage, we sat down with Project Manager Choi Joo-gon and Developer Kang Hyun-seok of VIRNECT.
When Choi received the offer from the National Research Institute of Cultural Heritage to collaborate on restoring this cultural site, he was delighted. It was because he couldn't hide his excitement for new challenges and anticipation for differentiated results. Choi thought AR should be developed as content that connects the past, present, and future, rather than remaining a future-oriented technology, so the cultural heritage restoration project was very attractive to him.
At the time, the project was still an internal project of the National Research Institute of Cultural Heritage. The initial purpose of it was to visualize the academic and research results of the researchers, but – with the active participation of Choi and Kang – it was able to expand into content that could be serviced to the public. The AR implementation of the middle gate and the southern corridor of Hwangnyongsa went beyond historical research and succeeded in appearing more realistic than expected.
“We always welcome a new challenge. We don't need to set limits because we have the technology secured.” – Choi Joo-gon, project manager at VIRNECT
The middle gate and the southern corridor of Hwangnyongsa, which were reborn as XR experience content, implemented the interior of a one-story single floor and the interior of a two-story middle floor in actual size. Users can walk around the site, enjoy taking pictures all around the building, and feel the seasonal changes depending on the time of their visit. In particular, the interior of the middle floor is designed so that users can feel as if they are actually on the second floor. For this reason, it received high public and media praise during its one-year demonstration period. But, the process was not simple. As the content was restoration of cultural heritage, historical research was the most important issue. However, due to the lack of data on Hwangnyongsa, the team had to analyze and study various materials at the same time. They received basic data from the institute, but also had to actively seek advice from professors. Choi and Kang repeatedly experienced trial and error while trying to restore Hwangnyongsa from its existing floor plan into 3D.
“Rafters, lattice window, stylobate, roof ridge ornament… The words were so unfamiliar. We needed some time to get used to them; thus, lots of trial and error was inevitable.” – Choi Joo-gon
Part of the reason for this was that whenever they tried 3D modeling, they would find structural defects – such as gaps and heights that couldn't be seen in the 2D floor plans. Lots of time was consumed with the discovery of each defect as they were forced to research, modify, and implement, all while looking for other case studies. They also faced a physical limitation on conducting face-to-face meetings. Instead, they used Unity to create a 3D model viewer, and received and confirmed feedback in real-time. This helped them communicate efficiently and save time, as well as reduced the range of errors.
Developer Kang Hyun-seok started the interview saying that this was the first time he'd walked so much while developing. The essential part of creating AR content is how real it feels for the users. So, to make a realistic implementation, reducing the error range between the actual location and the location within the content was important.
“The middle gate itself was huge, but if you include the southern corridor, it's nearly 270 meters. How did we solve the error range? We did the legwork and went to the site ourselves.” – Kang Hyun-seok, developer at VIRNECT
Kang thought of ways to reduce the error range of 270 meters from the middle gate to the southern corridor. Since 270 meters is a great distance, it cannot be solved with simply a high-performance tablet or AR core. He also tried with GPS, but it had a large margin of error as well, so he proceeded by placing a lot of markers in between. Furthermore, Kang performed numerous tests to determine optimal intervals between markers with the legwork. If a building of 270 meters tilts even a little, tens of meters will consequently tilt as well, so every effort was made to make an accurate measurement. Another aspect that makes for a realistic experience is the replication of light and shadows. By matching the shadow direction of the real environment with that of the digital object, he was able to enhance the realistic immersion.
Kang said that using Unity is the optimal choice, not just for one or two reasons, but because Unity has shown cutting-edge technology in every way. Moreover, Unity provided services on various platforms, including mobile. In addition, of all options Kang investigated, only Unity had the technology to use a single user interface (UI) to support various tablets with different resolutions. It is the function called preset provided by Unity. It operates by loading the UI information that is stored in the preset whenever the resolution changes, so that it can operate in a single UI.
In addition, as this project is an experience, Unity was actively used to develop settings that provided enjoyment for its users. Taking a picture with Hwangnyongsa as the background can be said to be the representative function. When the 3D was displayed over the taken picture, the 3D object would appear over the camera background, thus creating a problem where people were covered up. Here, they used Unity's occlusion technique to make the 3D object stay behind the people. Thanks to this technique, people were able to take pictures with Hwangnyongsa in the background.
“In the end, it’s the technique that completes the immersion. We found no alternative except for Unity to implement the technique.” – Kang Hyun-seok
Both Choi and Kang said that the key of developing AR content is in interaction that allows people to communicate with one another. The reason why metaverse is attracting attention is because of the various interactions possible within it. During the interview, VIRNECT revealed that it plans to expand its project to focus on creating meaning between the vigorous communication between people. VIRNECT's work in creating a communication-filled AR is something to look forward to.
Want to hear more about this project? Check out the video interview on the Unity Korea YouTube channel.