AI Test Generation Quiz

Answers and explanations appear as you take the quiz.

1. What is the 'oracle problem' in AI test generation?
   • Knowing the correct expected output for a given input
   • Generating enough random inputs to run the program
   • Connecting the test runner to a database
   • Choosing which programming language to test in
2. What does a 'characterization test' lock in?
   • The fastest possible runtime
   • The current behavior of the code as a regression baseline
   • The original author's coding style
   • A formal mathematical proof of correctness
3. What programming language did the TestPilot study implement and evaluate its LLM test-generation approach for?
   • Rust
   • Fortran
   • JavaScript
   • SQL only
4. A key risk of accepting AI-generated tests without review is that they may:
   • Run too slowly to be useful
   • Delete the source code automatically
   • Only work on weekends
   • Assert and freeze existing bugs as if they were correct
5. How do LLM-based test assistants like Copilot typically produce a test?
   • By predicting test code from the function and surrounding context
   • By formally proving the code is correct
   • By measuring CPU temperature
   • By recording the developer's screen
6. Coverage-guided fuzzing steers random inputs using:
   • User star ratings
   • Code instrumentation that measures which branches were hit
   • The phase of the moon
   • Only the file name
7. What did the TestPilot evaluation report as its median statement and branch coverage?
   • 100% statement and branch coverage on every package
   • 0% statement coverage and 52.8% branch coverage
   • 70.2% statement coverage and 52.8% branch coverage across its studied packages
   • The paper did not measure code coverage
8. What does mutation testing assess about a test suite?
   • Whether every test uses the same name
   • Whether a characterization test preserves all current behavior
   • Whether the fuzzer can reach a branch without running tests
   • Whether tests detect small, deliberate changes made to the program