Test oracle generation aims to determine whether a program behaves correctly. Many existing methods generate assertions, but the generated assertions may have weak bug-detection ability.
The previous SEERLL project studied a discriminative approach to this problem. The model receives a method under test and a test body with the oracle information removed. It predicts whether the test should pass or fail and provides an explanation for the decision.
I fine-tuned CodeBERT, CodeT5 with 220M and 770M parameters, and Qwen2.5. The experiments used Defects4J, Methods2Test, and GrowingBugs. Compared with SEER, the current model achieves better overall accuracy and better results for failing tests. It also provides clearer explanations. A paper based on this work is now being revised after peer review.
The continuation will first complete the remaining ablation experiments and error analysis. It will then use the discriminative oracle to improve generative oracle methods. A generative model can produce several candidate assertions, but many of them may fail to detect real bugs. The discriminative model will score and rank these candidates. Candidates that are more likely to expose faulty behavior will be tested first.
The goal is to improve the bug detection rate of generated test oracles. The project will also study whether the ranking method works across different models, datasets, and Java projects.