One of the core tools of the structure from motion process is the so called bundle adjustment step. From a rough initial guess of camera positions and orientations as well as 3D point estimates the goal is to refine parameters to create an accurate map of the 3D environment. Classical modelbased formulations based on optimization methods such as Levenberg-Marquardt work well but are computationally expensive. Recent learningbased methods using feed-forwards networks provide geomentrically reasonable solutions, but they fail to achieve the same accuracy that classical methods do. In this project we are aiming to understand this phenomenon better and consequently to develop neural architechtures that can achieve the same accuracy as classical approaches.