CS 411 Fall 2025  >  Outline for November 19, 2025

---

CS 411 Fall 2025
Outline
for November 19, 2025

Outline

Decision Trees [L 11.2]

• Definition
  • Now we look closer at the idea of an information-theoretic lower-bound argument.
  • A decision tree is a binary tree in which:
    • Each internal node (non-leaf) represents a 2-way decision to be made, i.e., a yes/no question. One subtree represents a “yes” answer; the other represents a “no” answer.
    • Each leaf represents a possible outcome.
  • Procedure: start at the root, working down, asking questions and moving to the left or right child according to the answers, ending at a leaf.
  • Note: we consider only binary decision trees. But decision trees with more than 2 possible answers to each question are possible.
• Bounds
  • If tree has height \(h\) and \(L\) leaves, then \(2^h \ge L\). Since \(h\) is an integer, we have \(h\ge\lceil\log_2 L\rceil\).
  • So the worst-case number of questions required is at least \(\lceil\log_2 L\rceil\), where \(L\) is the number of possible outcomes. his is the information-theoretic lower bound.
• Examples
  • Comparison Sorting. There are \(n!\) possible orderings of an \(n\)-item list. And \(\lceil\log_2 (n!)\rceil\) is \(\Omega(n\log n)\). Thus, the worst case for any comparison sort uses \(\Omega(n\log n)\) comparisons.
  • Insertion Sort of a \(3\)-item list. Max number of comparisons is \(3\). Since there are \(3!=6\) possible orderings of a \(3\)-item list, the worst case requires \(\lceil\log_2 6\rceil = 3\) comparisons. Conclusion: For a \(3\)-item list, we cannot improve on the worst case of Insertion Sort.
  • A Huffman Tree is a decision tree that minimizes the average number of questions required, given the probability of each outcome.