Guide-linked quiz · Medium Level

Entropy and Information Gain Quiz

Checks binary entropy, weighted child impurity, a numerical information-gain example, greedy splits, overfitting, and limits of the metric.

Related guide pathsEntropy And Information Gain
Question 1 of 8

Using base-two logarithms, what is the entropy of a node with equally many yes and no labels?

Preview every question8 questions

Answers and explanations appear as you take the quiz.

  1. Using base-two logarithms, what is the entropy of a node with equally many yes and no labels?
    • A value determined only by the feature scale
    • Zero bits because the classes cancel
    • Two bits because there are two labels
    • One bit
  2. A child node contains only one class. What is its label entropy?
    • It equals the parent entropy
    • Zero, because its class label is pure
    • It grows with the feature's units
    • One bit for every example in the node
  3. Why are child entropies weighted by child size when computing information gain?
    • To give every child equal influence regardless of sample count
    • To use a child's entropy itself as its numerical weight
    • Each child should contribute in proportion to the examples it contains
    • To sum child impurities without normalizing for parent size
  4. A balanced binary parent has one bit of entropy. Two equal-sized children each have about 0.811 bits. What is the split's gain?
    • About 0.189 bits, the parent minus weighted children
    • One bit, because the parent is balanced
    • About 1.811 bits, the sum of parent and child entropy
    • About 0.811 bits, the average child entropy
  5. At one node, what does an entropy-based classification tree seek among candidate splits?
    • The feature with the largest raw numeric values
    • A large local reduction in weighted label impurity
    • The split with the most children regardless of labels
    • The feature with the strongest causal effect
  6. Why does the largest training-set information gain at a node not guarantee the best deployed tree?
    • The choice is local and may overfit small training groups
    • Information gain is always negative for useful splits
    • All candidate features necessarily yield identical gain
    • A high-gain split proves future outcomes are deterministic
  7. A split sharply separates historical labels using a location proxy. What does high information gain fail to establish?
    • That the training labels were divided into children
    • That the children have a weighted entropy
    • That the split reduced the observed label mixture
    • That the feature is causal, appropriate, or fair to use
  8. If a team switches from base-two logs to another log base for every candidate at the same node, what happens?
    • The training labels change
    • Only pure-node entropy becomes positive
    • Gain values rescale, but the candidate ranking stays the same
    • Information gain stops requiring child-size weights

Keep testing yourself

More quizzes picked for your level.

View all quizzes