Stochastic computational methods are used abundantly in modern deep learning applications. Such methods include a wide range of methods ranging from stochastic gradient methods for optimization, used in the training of deep neural networks, to deep generative models used to generate synthetic samples of complex data and overcoming rare-event slowdown in physical systems. The performance of such methods relies critically on the algorithmic design and in this project we aim to develop rigorous performance analysis based on the mathematical frameworks of large deviations and stochastic approximation. Such rigorous performance analysis provides insights on the most important parameters for speeding up the algorithms and how to tune hyperparameters. For example, large deviations for stochastic approximations provides valuable insights on how to tune learning rates in stochastic gradient methods and by studying Lyapunov functions associated with limit ODEs of stochastic approximations improvements of the convergence of extended ensemble algorithms and adapted MCMC algorithms can be obtained. Moreover, deep neural networks can be used to approximate solutions and subsolutions of PDEs associated with large deviations rate functions and can thereby be used to design stochastic simulation algorithms for rare events. The proposed project relates well to the theme of generative models as it develops a mathematical foundation for design and analysis of generative models for computation.