Pixels to Pavement: Video Game Engines Can Help the U.S. Develop Smarter Cities
By: Gabrielle Roberts
Development has begun on an €8 billion smart city in Greece. Already, there are concerns about whether Europe’s newest and biggest megacity is even feasible.1 Doubts are not unfounded. Smart city initiatives worldwide have faced roadblocks including deficiencies in financing, coordination, trust in digital infrastructure, and resident recruitment.8 The United States, in particular, faces challenges in smart city development due to a lack of federal, centralized support.14 While the U.S. is a world leader in most forms of technological innovation, it trails behind China, Singapore, South Korea, and the UAE’s ambitious smart city initiatives.
If implemented well, smart cities can provide residents a better quality of life, improve sustainability on a large scale, and equip cities with the resilience and resources to deal with future crises such as natural disasters or disease outbreaks. While countries such as China and the UAE have developed brand-new cities with smart components integrated every step of the way, it is unrealistic for the U.S. to build smart cities from the ground up. Instead, policymakers and urban planners should focus on making existing cities “smarter.”9 In doing so, we don’t have to reinvent the wheel completely. Technology that can help us transform pre-existing infrastructure into smart, sustainable cities of the future already exists in an unexpected form: video game engines.
Video game engines, like Unreal Engine or Unity, are the future of sustainable and tech-savvy urban development. Game engines are software frameworks that provide developers with tools (such as physics, advanced graphics, and AI systems) to create their games. These tools also allow us to create digital twins of physical cities and simulate how new technologies (such as optimized traffic management systems, smart power grids, autonomous vehicles, and 5G networks) would behave in the real world.4
While not yet ubiquitous, digital twins are fast becoming an essential component of smart city development. Prominent industry software includes Microsoft Azure Digital Twins, OpenCities Planner, and 3DEXPERIENCE. These programs are separate from game engine technology, and their primary purpose is the development of digital twins. In contrast, Unreal Engine and Unity are powerful software development environments initially designed for games and have only recently been used in new ways. Despite their recent introduction, game engines have already proven their worth in smart city development.
Shanghai, Adelaide, and Dubai all have digital twins created using video game engines.3,16,19 Helsinki created a digital twin in Unreal Engine and has prioritized using the technology for climate change planning.18 Liverpool's digital twin, also created in Unreal, was used to simulate the city’s rollout of 5G.4 Video game engines for smart city development are an incredibly powerful but underutilized tool that policymakers and urban planners should take better advantage of.
One of the biggest benefits of digital twins is the ability to test new construction or installations before investing significant time and money.14 City planners and tech companies can simulate their ideas within the digital city, allowing them to make informed decisions faster and be more targeted with their investments. Digital twins can also help predict how cities will react to climate change or natural disasters, helping governments plan more effectively. Game engines can accomplish all of this with the advantage of high-quality, realistic graphics. Current physical infrastructure can be 3D imaged (using LiDAR or Gaussian splats), brought into the game engine, and remodeled to adapt new smart technology or more sustainable building practices. These changes can then be accurately represented to urban planners within the game engine environment.
Simulating scenarios in game engines can also help lessen privacy concerns. Smart cities rely on an Internet of Things (IoT) system where sensors track metrics of traffic flows, energy usage, waste, crime, and other essential urban functions. Data from these sensors is incredibly valuable, but many people are resistant to companies collecting such granular information about their daily lives13. Using game engine technology and artificial intelligence (AI), we can feed sensor data into a digital twin, simulate behavior, and generate new synthetic data from the simulated results. This allows the digital twins to train on new data sets while alleviating the need to stream real-time data. It also protects confidential information from bad actors in the event they can access these twins. While more research is needed to test the use of synthetic data with digital twins, evidence supports that synthetic data can provide comparable results to real-world data to train AI systems. Another possibility Unity raises is the ability to make the NPC (non-player characters) representative of real-life people to anonymize data2.
However, the most important advantage of using game engines to aid in smart city development targets the area of concern most overlooked: collaboration. Smart city initiatives in the past have lacked a holistic strategy and were hampered by fragmented leadership and communication between city agencies and tech companies.13 A digital twin that can visualize and encompass data from many different sectors encourages cross-collaboration and could even shed light on how organizations can come together to innovate. A virtual representation of the city and its data ensures everyone is on the same page and no one is siloed. Game engines are designed to create visually appealing end products (games) with user interfaces that are easy to interact with, making them the perfect medium to foster collaboration.
Game engine technology provides an internal ecosystem of tools, including accurate 3D modeling and visualization of cities, the ability to apply physics simulations (i.e., natural disasters) to digital twins, the power to harness AI, and the means to create visually friendly user interfaces for policymakers less familiar with technology to interact with and understand data collected from IoT sensors.
Of course, technology engineered to develop video games is not the perfect solution for managing data for massive urban ecosystems. Game engines are not currently equipped to handle, safely store, and add to such large amounts of data, and there are no methods of standardization or regulation yet. Digital equity and inclusion could be difficult to maintain as smart cities develop. They will likely further exacerbate the digital divide as certain communities may not have the digital literacy or financial means to engage with “smart” resources.12 Communities that do not have access to devices that connect residents to IoT sensors could be forgotten in this emerging digital landscape.
In the same vein, data collected to train digital twins and help city planners make informed decisions could skew efforts toward communities that are able to be more technologically engaged or can afford more smart devices. Training AI to produce new data on skewed information would only deepen bias. Policymakers and city planners are also wary of adopting video game technology as it requires an initial buy-in of talent and financing. Game engines such as Unreal can be intimidating even for game developers and artists, let alone people working in government. It will take time to create digital twins — and for leaders to get accustomed to the technology.
Creating the right tool sets for policymakers or urban planners to interact with this technology and writing regulations around them will not happen overnight. While not yet a perfect solution to issues of accessibility and fairness, video game engines and digital twins offer a promising approach for transforming pre-existing cities into smart, resilient urban ecosystems. Countries like the United States have developed a “watch-and-wait” strategy in relation to smart cities precisely because confronting the legal, ethical, and technological implications is too overwhelming.7 Yet these issues only clarify the importance of conducting research and development on smart cities now. It will take time to understand how best to use game engine technology, determine how to foster digital inclusion, manage the data sets required to run smart cities, and create protections around data privacy. Watching and waiting while some countries, such as China, test their own smart city initiatives is not a viable strategy as the data each country has access to will differ.5
By leveraging the powerful visual and technical capabilities of video game engines, the U.S. can transform existing infrastructure and create smarter, more resilient cities without building from the ground up. City planners and policymakers should embrace technology that can help foster the next phase of sustainable, efficient, and equitable urban modernization.
End Notes
- Cameron, Hugh. 2024. “Construction Underway on €8 Billion ‘Smart City.’” Newsweek. June 27, 2024. https://www.newsweek.com/ellinikon-new-8-billion-futuristic-city-greece-1918305.
- D’Anastasio, Cecilia. 2022. “Gaming Giant Unity Wants to Digitally Clone the World.” Wired. January 18, 2022. https://www.wired.com/story/gaming-giant-unity-wants-to-digitally-clone-the-world/.
- “Digital Twin for Dubai with 2D, 3D Maps of All Assets Planned.” n.d. Gulfnews.com. https://gulfnews.com/uae/digital-twin-for-dubai-with-2d-3d-maps-of-all-assets-planned-1.72185064.
- Engineering.com, and Epic Games. 2019. “Using Real-Time Engines to Create Digital Twins and Smart Cities.” Engineering.com. engineering.com. https://image.engineering.com/239575/articles/Nov+2019/Epic+Games+Smart+Cities.pdf.
- Ismagilova, Elvira, Laurie Hughes, Nripendra P. Rana, and Yogesh K. Dwivedi. 2020. “Security, Privacy and Risks within Smart Cities: Literature Review and Development of a Smart City Interaction Framework.” Information Systems Frontiers 24 (July). https://doi.org/10.1007/s10796-020-10044-1.
- Kite-Powell, Jennifer. 2022. “Can Game Engines Help Digital Twins Visualize Data Better?” Forbes. June 22, 2022. https://www.forbes.com/sites/jenniferhicks/2022/06/22/can-game-engines-help-digital-twins-visualize-data-better/.
- Knowledge at Wharton Staff. 2019. “What’s Fueling the Smart City Backlash?” Knowledge at Wharton. September 24, 2019. https://knowledge.wharton.upenn.edu/article/whats-behind-backlash-smart-cities/.
- Martínez Euklidiadas, Marcos. 2020. “Smart Cities That Failed Along the Way.” Tomorrow City. Fira de Barcelona. November 25, 2020. https://www.tomorrow.city/smart-cities-that-failed-along-the-way/.
- Meddeb, Riad, and Calum Handforth. 2022. “We Need Smarter Cities, Not ‘Smart Cities.’” MIT Technology Review. June 27, 2022. https://www.technologyreview.com/2022/06/27/1053896/we-need-smarter-cities/.
- National Academies of Sciences, Engineering, and Medicine. 2023. Opportunities and Challenges for Digital Twins in Engineering: Proceedings of a Workshop—in Brief. PubMed. Washington (DC): National Academies Press (US). https://www.ncbi.nlm.nih.gov/books/NBK594823/.
- Robbins, Jim. 2021. “Why the Luster on Once-Vaunted ‘Smart Cities’ Is Fading.” Yale E360. December 1, 2021. https://e360.yale.edu/features/why-the-luster-is-fading-on-once-vaunted-smart-cities.
- Roth, Zoe, and Melissa Incera. 2024. “The Rise of AI-Powered Smart Cities | S&P Global.” S&P Global. May 18, 2024. https://www.spglobal.com/en/research-insights/special-reports/ai-smart-cities.
- Russo, Massimo, and Tian Feng. 2020. “The Risks and Rewards of Data Sharing for Smart Cities.” BCG Global. August 5, 2020. https://www.bcg.com/publications/2020/smart-cities-need-to-understand-the-risks-and-rewards-of-data-sharing-part-3.
- “Smart Cities Technologies: Driving Economic Growth and Community Resilience.” n.d. Federation of American Scientists. https://fas.org/publication/smart-cities-technologies-driving-economic-growth-and-community-resilience/.
- “Twin Cities: Harnessing Gaming Technology for Immersive Digital Twins.” 2024. CIBSE Journal. February 2024. https://www.cibsejournal.com/technical/twin-cities-harnessing-gaming-technology-for-immersive-digital-twins/#:~:text=Game%20engines%20enable%20Buro%20Happold.
- Twinmotion. 2024. Review of A Virtual Clone of Adelaide for Digital Twin and Smart City Applications. https://www.twinmotion.com/en-US/spotlights/a-virtual-clone-of-adelaide-for-digital-twin-and-smart-city-applications.
- Unreal Engine. 2021. “Digital Twins: Building Cities of the Future | the Pulse | Unreal Engine.” YouTube. May 6, 2021. https://www.youtube.com/watch?v=i12kObBpz-E&t=1200s.
- “Virtual Cities for Very Real Benefits: From Local Digital Twins to the Cityverse - Stardust.” 2023. Stardustproject.eu. 2023. https://stardustproject.eu/news/virtual-cities-for-very-real-benefits-from-local-digital-twins-to-the-cityverse/#:~:text=At%20first%2C%20the%20city%20used.
- Weir-McCall, David. 2020. “51World Creates Digital Twin of the Entire City of Shanghai.” Unreal Engine. September 15, 2020. https://www.unrealengine.com/en-US/spotlights/51world-creates-digital-twin-of-the-entire-city-of-shanghai.
About the Author
Gabrielle Roberts is an Adjunct Professor at the USC School of Cinematic Arts, where she teaches an advanced graduate course in virtual production (a cutting-edge field that combines video game technology with filmmaking). She pioneered the virtual human industry at the Sequoia-backed startup Brud (which developed Miquela – named by Time as one of the 25 most influential people on the internet), worked on Netflix's Avatar: the Last Airbender, and currently works as a Virtual Production & Visualization Lead for an upcoming feature from a major film franchise. Gabrielle studied Film & Television Production and Computer Science at USC and received a Master of Management Science in Global Affairs from Tsinghua University as a Schwarzman Scholar.