The goal of this project is to study the modes of reasoning that text classification systems learn
from training data, and the interaction between properties of the training data and properties of the trained classifier.
The Recognizing Textual Entailment (RTE) task has been a staple in natural language
processing for over twenty years. The RTE entailment relation is not formally defined, and its
interpretation has been contested. In this project, we propose different formal readings of the
RTE entailment relation, based on its description in the RTE task guidelines. We perform a
comprehensive analysis of the meta-inferential properties of these relations and compare them
to other well-defined entailment relations. We use this analysis as grounds for experimentally
investigating to which degree the entailment relation encoded by the RTE dataset matches the
proposed formal readings of the relation. Lastly, we perform a comparison with our previous
study of the Stanford Natural Language Inference dataset, essentially striving for a better
understanding of the different aspects of entailment these datasets seem to capture.
We use datasets from the RTE 3–5 challenges, that are annotated according to a three-way
classification scheme. To be able to analyse the meta-inferential relations and conduct a
comparison with previously studied entailment relations, we need to generate new RTE-style
sentence pairs that form implicational chains in order to be able to test transitivity and similar
properties of the learned entailment relation. Such data is not available in the original RTE datasets.
The results of this project will be presented as an invited contribution to the CLASP Concluding
Conference (https://gu-clasp.github.io/concluding-conference/; October 5–6 2026) with an
extended version published in a special issue of the NEALT Proceedings series (final version due March 15 2027).