Alignment of Implicit Neural Representations of 3D Shapes via Permutation Symmetries

Multi-Layer Perceptrons (MLPs) have long been known to enjoy permutation symmetries, namely the fact that activations of intermediate layers can be swapped without changing the underlying function. Research works inspired by this property have been recently building up to a conjecture stating that any two Stochastic Gradient Descent (SGD) solutions, up to a permutation, lie in the same low-loss region of the parameter space, i.e. they enjoy Linear Mode Connectivity (LMC). Concurrently, Implicit Neural Representations (INRs) have been emerging as a new unifying paradigm to store and represent various signals of interest (such as images, audio, and 3D surfaces), which has sparked a novel interest in machine learning architectures designed to work on neural networks themselves by processing their weights only. The recently proposed inr2vec framework is one such architecture, and has been shown to be able to embed input INRs of 3D shapes into latent codes that can be effectively fed into standard deep learning pipelines. A key requirement for the convergence of inr2vec is that input INRs share the same random initialization, which has been pointed out as a potential limitation to its applicability. Hence, the motivation behind our work is to eventually allow inr2vec to work with arbitrary in-the-wild INRs, regardless of their initialization. As a first attempt to tackle this complex task, we apply and thoroughly analyze recent combinatorial methods which lay their foundations on the aforementioned conjecture by computing a weight permutation that aims at aligning two INRs of the same 3D surface. This approach, although promising in some of its results, is ultimately shown to be unable to move INRs into the same basin of the loss landscape. Finally, we present preliminary results of a novel deep learning methodology devised to achieve LMC by directly learning new weights for one of the two models.