Nerf2vec: Deep Learning on Neural Radiance Fields

Virtualization of 3D world remains a challenge, as a standardized technique has yet to emerge. Neural Radiance Fields (NeRFs), a recent and promising approach, have attracted a lot of excitement due to their speed and quality reconstruction capabilities. Thus, NeRFs are poised to shape the future of 3D world modeling. However, a question arises about the potential use of NeRFs as input and output data for other algorithms due to their neural network nature. Additionally, since they have been introduced recently, there is no publicly available large dataset of NeRFs suitable for training deep learning models. Hence, in the initial phase of this thesis, a new and rigorously organized dataset of NeRFs was assembled. This dataset served as the bedrock upon which the subsequent research was built. Besides, it brings considerable value to the wider research community, paving the way for future advancements in the field. The following stride involved the development of "nerf2vec", a new framework designed to learn embeddings that serve as compressed representations of input NeRFs. This endeavor highlighted the capacity of these embeddings to faithfully represent the underlying NeRFs, maintaining high reconstruction quality. Moreover, this work showcased the direct applicability of these embeddings within deep neural architectures, tackling tasks like classification, retrieval, embeddings interpolation, and adversarial generation. It achieved noteworthy results comparable to state-of-the-art methods, while optimizing resource usage and eliminating the need for costly machinery. To the best of our knowledge, this is the first work that introduces these approaches for NeRFs and makes a significant contribution to the adoption of NeRFs as a standard way to represent 3D scenes.