The behaviour of many real-world systems is affected by physical processes at multiple length and time scales. These processes typically exhibit heterogeneous behaviour, i.e. the laws that govern each scale are drastically different. The archetypical example of such systems is fluids, in which macroscopic changes in density is caused by atomistic interactions at the smallest levels. One key difficulty in simulating such systems is how to account for the microscopic behaviour, without fully resolving the simulation domain. The Hetrogeneous Multiscale Method is a model framework to this end, and works by replacing the microscopic dynamics with a closure model, that eliminates the need for finer discretisations. Still, evaluating this closure model requires costly computations that involve simulation and averaging over microscopic quantities which can be prohibitively expensive in many applications. Our aim is to replace these closure models with learned neural surrogates, that account for microscopic behaviour without the need for costly micro-scale simulations. A key problem is to encorporate adequate domain-specific knowledge about the model into the neural architecture. Such knowledge can stabilize the learning process, make a learned surrogate more robust, and improve its generalization properties.