गाइड से जुड़ी प्रश्नोत्तरी · मध्यम स्तर
बेंचमार्क संदूषण प्रश्नोत्तरी
परीक्षण करता है कि बेंचमार्क आइटम प्रशिक्षण डेटा में कैसे लीक होते हैं, एन-ग्राम ओवरलैप और मिन-के% प्रोब जैसे पता लगाने के तरीके, और कैनरी और टाइम स्प्लिट जैसे शमन।
प्रश्न 1 का 8
बेंचमार्क संदूषण क्या है?
प्रत्येक प्रश्न का पूर्वावलोकन करें8 प्रश्न
जैसे ही आप प्रश्नोत्तरी लेते हैं उत्तर और स्पष्टीकरण दिखाई देते हैं।
- बेंचमार्क संदूषण क्या है?
- When a benchmark contains incorrect answer keys
- When a model is tested on too few questions
- When benchmark test items or their answers appear in a model's training data
- When two benchmarks measure the same skill
- बिग-बेंच कार्यों में शामिल कैनरी स्ट्रिंग का उद्देश्य क्या है?
- To let data teams filter those documents from training data and test whether a model saw them
- To encrypt the answers
- To mark which tasks are the hardest
- To identify which model generated each answer
- GSM1k अध्ययन ने क्या सुझाव दिया?
- All models scored higher on the new problems
- GSM8K has no errors
- Contamination cannot be measured without training data
- Some models scored noticeably lower on fresh, matched problems than on GSM8K
- GPT-3 पेपर में कौन सा ओवरलैप विश्लेषण चला?
- Embedding similarity on full documents
- 13-gram overlap between benchmarks and training data
- Human review of every training document
- Min-K% Prob on each benchmark
- मिन-के% प्रोब का मूल विचार क्या है?
- Removing the least common k percent of words from training data
- Training a second model to classify contaminated text
- Text the model has seen tends to have few very low-probability tokens
- Measuring the length of the model's answers
- How does a time-based split, as in LiveCodeBench, help detect contamination?
- It compares performance on problems published before and after the model's training cutoff
- It measures how long the model takes to answer
- It splits the model into older and newer layers
- It only counts problems answered within a time limit
- Why does finding no exact overlap not prove a benchmark is clean?
- Exact matching is too slow to run
- Benchmarks change their answers over time
- Models never memorize exact text
- Paraphrased or translated copies can slip past string matching
- In n-gram decontamination, what happens if n is chosen too small?
- Near-copies with small edits are missed
- Many false positives from common phrases are flagged
- The filter stops working entirely
- Only answers, not questions, are matched