Growing 3D Artefacts and Functional Machines with Neural Cellular Automata

Neural Cellular Automata (NCAs) have been proven effective in simulating morphogenetic processes, the continuous
construction of complex structures from very few starting
cells. Recent developments in NCAs lie in the 2D domain,
namely reconstructing target images from a single pixel or
infinitely growing 2D textures. In this work, we propose
an extension of NCAs to 3D, utilizing 3D convolutions in
the proposed neural network architecture. Minecraft is selected as the environment for our automaton since it allows the generation of both static structures and moving machines. We show that despite their simplicity, NCAs are capable of growing complex entities such as castles, apartment
blocks, and trees, some of which are composed of over 3,000
blocks. Additionally, when trained for regeneration, the system is able to regrow parts of simple functional machines,
significantly expanding the capabilities of simulated morphogenetic systems. The code for the experiment in this paper
can be found at: https://github.com/real-itu/3d-artefacts-nca.