Through reinforcement learning, agents in multi-agent settings have been demonstrated to derive efficient communications protocols in simple settings. However, many challenges still remain including the ability to derive more complex recursive numeral systems, similar to that of English or Swedish. We plan to investigate a neuro-symbolic agent architecture where we anticipate an opportunity to enable agents to reflect on their current protocols and to explore revisions which may enable the introduction of systematic abstractions.