Bài kiểm tra có liên kết hướng dẫn · Trung bình Cấp độ

Bài kiểm tra chống trùng lặp tập dữ liệu và phát hiện gần trùng lặp

So sánh sự trùng lặp dữ liệu dựa trên sự tương đồng và chính xác, ngăn chặn sự chồng chéo trong quá trình kiểm tra và kiểm tra sự đánh đổi ngưỡng và rủi ro đa dạng.

Đường dẫn hướng dẫn liên quanChống trùng lặp tập dữ liệu gần trùng lặp
Câu hỏi 1 của 8

What does a cryptographic hash detect most directly?

Xem trước mọi câu hỏi8 câu hỏi

Câu trả lời và giải thích xuất hiện khi bạn làm bài kiểm tra.

  1. What does a cryptographic hash detect most directly?
    • Exact matching bytes, or exact normalized content when normalization is defined
    • Every paraphrase automatically
    • All examples with the same meaning
    • The label quality of a dataset
  2. What kind of similarity can MinHash over text shingles help identify?
    • Any two sentences with opposite meanings
    • Image color differences only
    • Lexical near-duplicates with overlapping sets of text features
    • Whether a dataset is unbiased
  3. Why cluster near-duplicates before making train-test partitions?
    • To guarantee every label is balanced
    • To keep related examples from leaking across partitions
    • To remove the need for a test set
    • To make all examples byte-identical
  4. What did the cited 2021 language-model deduplication study report?
    • Deduplication always improves every metric on every corpus
    • Every benchmark was fully deduplicated
    • Exact hashes identify all paraphrases
    • Deduplication reduced memorized text emission in its studied setting
  5. Why can a global similarity threshold be harmful?
    • It may remove meaningful variants or disproportionately filter rare language examples
    • It always makes models more fair
    • It identifies only exact matches
    • It guarantees no train-test overlap
  6. What should be reviewed before choosing a similarity threshold?
    • Only processing speed
    • The file extension
    • Whether the score is an integer
    • Precision and recall on manually reviewed example pairs
  7. What should an audit record after deduplication?
    • Only the final number of rows
    • A model’s final accuracy only
    • The algorithm, normalization, threshold, source, and retention decisions
    • Nothing if rows were deleted
  8. Does high embedding similarity prove two examples are interchangeable?
    • Yes, for every label and task
    • No; similarity is a review signal and task-specific meaning may differ
    • Yes, if both vectors are normalized
    • No, because embeddings cannot compare text