Human-in-the-loop

[4] In machine learning, HITL is used in the sense of humans aiding the computer in making the correct decisions in building a model.

[4] HITL improves machine learning over random sampling by selecting the most critical data needed to refine the model.

Utilizing HITL allows participants to interact with realistic models and attempt to perform as they would in an actual scenario.

A real-world example of HITL simulation as an evaluation tool is its usage by the Federal Aviation Administration (FAA) to allow air traffic controllers to test new automation procedures by directing the activities of simulated air traffic while monitoring the effect of the newly implemented procedures.

Intelligent systems can only go so far in certain circumstances to automate a process; only humans in the simulation can accurately judge the final design.