Design of Holographic Display Systems based on Artificial Intelligence
Suyeon Choi, Postdoctoral Scholar, Stanford University
Abstract: Emerging spatial computing systems are widely believed to be next-generation computing platforms, offering transformative experiences across various domains such as teleconferencing, education, entertainment, industry, and basic vision research, among others. To deliver perceptually realistic and visually comfortable experiences, optical interfaces, such as imagers or displays, must be compact, adaptable, seamlessly integrated, and capable of efficiently encoding and robustly reconstructing visual information. However, most of these parameters are not achieved by existing optical devices, mainly limited by the high-dimensional, low-latency nature of the visual signals and the sensitive, complex nature of human sensory systems that must be fully satisfied. In this talk, I will describe how computational optical systems can provide solutions to these interfaces for augmented and virtual reality (AR/VR) applications. First, I will introduce an artificial intelligence (AI)-based algorithmic framework that comprises several parts, including a novel camera-in-the-loop optimization strategy that allows us to either optimize a hologram directly or train a parameterized model of the optical wave propagation and a neural network architecture that represents the first computer-generated holography (CGH) algorithm capable of generating full-color high-quality holographic images at 1080p resolution in real-time. Efficiently and accurately modeling arbitrary optical systems, these frameworks overcome long-standing challenges in holography, such as low image quality and prohibitive computational cost. Next, I will present computational design methodologies powered by my framework, which enable full-color, high-quality 3D images within glasses form factors for AR/VR displays. I will also discuss how this advancement paves the way for studying 3D human perception, leveraging a holographic perceptual testbed. Finally, I will discuss future directions that I’m excited to explore.
Bio: Suyeon Choi is a final-year Ph.D. student at Stanford University, advised by Professor Gordon Wetzstein. His research centers on developing computational optical systems at the intersection of graphics, computational optics, artificial intelligence, and applied vision science. He received a Meta Research PhD Fellowship, a SPIE Optics and Photonics Education Scholarship, a Kwanjeong Scholarship, a Korean Government Scholarship, and a Frontiers of Science Award. He was also a finalist for the NVIDIA Graduate Fellowship, a recipient of the Presidential Science Scholarship for undergraduate studies. Previously, he received his Master's and Bachelor’s degrees in Electrical Engineering from Stanford University and Seoul National University, respectively.
His research was demonstrated at SIGGRAPH 2023 Emerging Technologies.
Published works:
Time-multiplexed Neural Holography: A flexible framework for holographic near-eye displays with fast heavily-quantized spatial light modulators,Choi, Gopakumar et al., SIGGRAPH 2022
Holographic glasses for virtual reality, Kim et al., SIGGRAPH 2022
Holographic Parallax Improves 3D Perceptual Realism, Kim, Nam, Choi et al., SIGGRAPH 2024
Full-colour 3D holographic augmented-reality displays with metasurface waveguides, Gopakumar, Lee et al., Nature (2024)
https://choisuyeon.github.io/
https://scholar.google.com/citations?...
https://www.meetup.com/sv-siggraph/ev...
Chapters (work-in-progress)
0:00 Chapter Intros
5:53 Speaker Intro
6:51 Presentation
8:59 Goals: Perceptual realism, Visual comfort
10:21 Display Systems
13:51 Current Near-eye Displays
14:54 Design
15:30 Computational Near-eye Display Systems
16:07 Focus cues
17:04 Technology Trade space
19:52 Holographic displays
20:46 Fundamental principle of digital holograms
22:36 Holographic Display Systems: 1. Advanced optical design
23:47 AI-based: 2. Modeling and learning optical systems
24:13 3. AI-based, human-centric CGH rendering
24:48 Advanced Optical Design: Squeezing the form factor of AR/VR displays
25:03 Holographic Near-eye Displays
26:14 Why so bulky?
27:04 Squeezing the form factor of VR/AR displays
28:19 of OST-AR displays
29:12 Limited image quality using naïve CGH algorithms
29:55 Challenges of Experimental Holography
31:59 Model mismatch
33:18 AI-based Holographic Display Systems
34:11 Q: What encoder?
38:32 Challenges of Experimental Holography
39:54 Camera-in-the-loop (CITL) Optimization
45:20 Parameterized wave propagation model
52:32 Physics-based Wave Propagation Model
1:04:15 Experimentally captured results using different CGH
1:07:31 Recap & Open questions
1:09:08 Acknowledgments
1:09:32 Q&A
1:09:47 Q: Alpha/transparent channel?
1:11:10 Q: Foveated rendering to improve performance?
1:11:35 Q: When in a pair of Ray-Bans?
1:12:22 Q: Next steps for you?
1:13:34 Q: Other benefits from this?
Смотрите видео Design of Holographic Display Systems based on Artificial Intelligence онлайн без регистрации, длительностью часов минут секунд в хорошем качестве. Это видео добавил пользователь Silicon Valley ACM SIGGRAPH 01 Январь 1970, не забудьте поделиться им ссылкой с друзьями и знакомыми, на нашем сайте его посмотрели 50 раз и оно понравилось людям.