Module · Decision Trees✅Knowledge checkCheck: measuring a mixed pile5 questions · Pass at 60%1. Gini impurity was defined without a formula, as a game. Which game?Split the pile in two and see which half is biggerDraw one item, put it back, draw another — how often do the two disagree?Count how many items are in the minorityAsk every yes/no question and keep the best2. Ten deals split into four with impurity 0 and six with impurity 0.444. The weighted result is 0.267 and the plain average is 0.222. Why is the plain average wrong?It lets four spotless deals count as much as six messy onesIt uses the wrong number of decimal placesAverages cannot be used on probabilitiesIt should have been the average of the squares3. A pile has three possible outcomes rather than two, and scores 0.6. Is that good?No — 0.6 is above the worst case of 0.5Yes — anything under 1 is acceptableIt cannot be said from the number alone: with three outcomes the worst case is 0.667Yes — impurity always falls as outcomes are added4. Sixteen equally likely animals have an entropy of 4 bits, and the best possible questioning strategy averaged exactly 4 questions. What does the 4 mean?The number of columns in the dataA floor: no strategy can name the animal in fewer questions on averageThe depth of the tree that was actually builtA target the tree failed to reach5. Skewing the game — one animal picked half the time — dropped entropy from 4 to 2.95. What does that say about a dataset where 99% of rows share one outcome?There is very little to tell anybody, so there is very little for a model to earnThe model will find it easier and perform betterEntropy is the wrong measure for imbalanced dataThe rare outcome should be deletedCheck answers← Back to Decision TreesNext topic →
