This project is part of ongoing research to develop deep neural network model designs to simulate activity-based travel demand. This approach seeks to enhance the predictive capabilities of traditional linear utility models traditionally used in travel demand forecasting. A previous baseline neural network model evaluates 22,009 trip options across various activities, destinations, and transportation modes. One of its unique features dynamically adjusts the attractiveness of each option by interconnecting all evaluations. The model's effectiveness is demonstrated using data from a Stockholm travel survey. In this project, additional designs based on this baseline model will be explored, including the use of Long-Short Term Memory (LSTM) and Transformers to capture correlations in the decision-making process across different time steps of the day.