Neural ordinary differential equations (NODEs) mark a significant advance in geometric deep learning, the pursuit of incorporating non-Euclidean structures in machine learning using geometrical principles. NODEs describe the dynamics of information propagating through recurrent neural networks in the limit of infinite depth using ordinary differential equations. These models offer several beneficial properties, e.g. bijectivity, scalability and universality, and have been used successfully primarily to construct continuous normalizing flows (CNFs) to model probability densities in different geometries. An important aspect of geometric deep learning is the ability to accommodate symmetries of the data, or equivalently, the space where the NODE is defined, by constructing models that are manifestly equivariant with respect to the action of Lie groups of symmetry transformations. However, the dynamical systems in NODE models are constrained in that the dimension of their state vectors is fixed, which hinders the efficient exploration of diverse data representations. For example, a powerful architectural element in feed-forward machine learning models is the encoder-decoder that extracts a lower-dimensional latent representation from which the original data can be efficiently reconstructed. The dimensional bottleneck is used to extract the features of information-dense representations and is used, e.g., in ubiquitous models like autoencoders and sequence-to-sequence prediction. The fact that similar architectural constraints cannot be efficiently enforced in NODEs is a significant limitation. A related problem is the inability of NODEs to model applications in which the dimensionality of the state space changes dynamically. Important examples include quantum mechanical systems interacting with classical external fields, where quantisation effects cause freeze-out of degrees of freedom. The state space in the high energy limit of these systems is typically infinite-dimensional, which precludes embedding it in some ambient space. Currently, there is no way to incorporate such variable dimension dynamics into the existing manifold NODE framework due to the intrinsic nature of the dimension of manifolds. To remedy the limitations in the current framework, we propose to extend NODEs from manifolds to M-polyfolds, which are generalisations of manifolds modelled on Banach spaces rather than Euclidean spaces. The number of local coordinates in M-polyfold theory is allowed to vary smoothly through the introduction of the scale structures, or sc-structures. The construction of sc-differential equations in this framework will allow us to conceptualize sc-NODEs; a novel class of machine learning models based on ODEs on M-polyfolds, which not only allow the state dimension of the dynamical system in NODEs to vary during evolution, but also innovatively enable the local dimension to be learned during training rather than predetermined. The project goal is to develop a comprehensive geometric framework for equivariant NODEs on M-polyfolds to accommodate variable dimension dynamics in geometric deep learning.