[5] A test-suite – the input/output pairs specify the functionality of the program, possibly captured in assertions can be used as a test oracle to drive the search.
For example, there are targeted bug-fixing techniques validating that the patched program can no longer trigger overflow errors in the same execution path.
[5] The effectiveness of generate-and-validate techniques remains controversial, because they typically do not provide patch correctness guarantees.
[5][13] Recent approaches use more fine-grained operators at the abstract syntax tree level to generate more diverse set of candidate patches.
Angelix[17] introduced the concept of angelic forest in order to deal with multiline patches.
[7] One example of such techniques learns from past successful patches from human developers collected from open source repositories in GitHub and SourceForge.
Example approaches include mining patches from donor applications[10] or from QA web sites.
When new bugs get detected, Getafix applies its previously learnt patterns to produce candidate fixes and ranks them within seconds.
The search space of automatic bug fixing is composed of all edits that can be possibly made to a program.
Sometimes, in test-suite based program repair, tools generate patches that pass the test suite, yet are actually incorrect, this is known as the "overfitting" problem.
[34] Alternatively, Tian et al. propose heuristic approaches to assess patch correctness.
To minimize the human labelling effort, an automatic test oracle can be trained that gradually learns to automatically classify test cases as passing or failing and only engages the bug-reporting user for uncertain cases.
State-of-the-art systems address this problem by assuming that a small modification is enough for fixing a bug, resulting in a search space reduction.
The limitation of approaches based on symbolic analysis[16][17] is that real world programs are often converted to intractably large formulas especially for modifying statements with side effects.