CS 411 Fall 2025  >  Outline for November 17, 2025

---

CS 411 Fall 2025
Outline
for November 17, 2025

Outline

Limitations of Algorithmic Power [L Ch 11 intro]

• Done covering algorithmic strategies. Now we discuss what algorithms cannot do.
• When we say something is impossible, we are not saying merely that we do not yet know how to do it; we are saying we will never know how to do it.
• Saying that a computation with a certain complexity is possible, is a statement about one algorithm. Saying such a computation is impossible, is a statement about all algorithms. This requires more precision and care to be sure we are correct.
• Later, in chapter 12, we will discuss how we deal with the fact that algorithms have limitations.

Lower-Bound Arguments [L 11.1]

• Lower Bounds
  • A lower bound is a statement that some quantity is at least a certain value. For example, \(x\ge 5\) is a lower bound on \(x\).
  • A bound is tight if equality is possible.
    • So if \(x\) is always at least \(6\), then the above bound is not tight. If \(x\) can be \(5\), then it is tight.
  • We study lower bounds on the number of basic operations that must be performed in order to solve some problem.
  • When discussing lower bounds, we often use big-omega notation: \(\Omega(\dots)\).
  • Examples
    • We know that comparison sorting a list of \(n\) items must, in the worst case, do \(\Omega(n\log n)\) comparisons. This bound is tight, since Merge Sort does \(\Theta(n\log n)\) comparisons.
    • We know that multiplication of two \(n\times n\) matrices requires \(\Omega(n^2)\) arithmetic operations. It is not known whether this is tight. Best known [J. Alman et al 2024]: approximately \(\Theta(n^{2.371339})\).
• Trivial Arguments
  • We can argue based on the number of output or input operations required.
  • Examples
    • Printing the integers from \(1\) to \(n\) requires \(\Omega(n)\) print-a-number operations.
    • Finding the sum of \(n\) arbitrary integers requires \(\Omega(n)\) read-a-number operations.
  • It is not always necessary to read every input item (think about Binary Search). For the sum problem, we know that we need to read all \(n\) numbers because changing any one of them can change our result.
• Information-Theoretic Arguments
  • We must gather enough information to be able to decide between all possible outcomes.
  • Example
    • When comparison sorting, if we do \(k\) comparisons, then we can decide between at most \(2^k\) possible outcomes. There are \(n!\) possible orderings of \(n\) items. So we must do enough comparisons that \(2^k\ge n!\). Applying Stirling’s Formula, we see that \(k\) is \(\Omega(n\log n)\).
• Adversary Arguments
  • We want to solve a problem quickly. The adversary (sometimes called the devil) wants to slow us down. We give the adversary the power to change things about the input in the middle of execution of the algorithm, as long as this does not change any information we already know.
    • This is not really cheating, since, whatever the adversary decides, it might have been that way to begin with.
    • Adversary arguments are useful ways to reason about the worst case.
  • Example
    • We wish to determine whether a graph is connected by asking questions of the form: are vertices \(a\), \(b\) joined by an edge? The adversary answers “no” unless this would let us conclude that the graph is not connected; then the adversary answers “yes”. Conclusion: In the worst case, we must ask about all possible edges. So we ask \(\Omega(n^2)\) questions, for a graph with \(n\) vertices.
• Problem Reduction
  • If we can solve problem A using an algorithm that solves problem B, then a lower bound for A gives us a lower bound for B.
  • Example. We can sort a list of \(n\) items by doing \(n\) calls to selection, one for each position in the sorted list. Since comparison sorting uses \(\Omega(n\log n)\) comparisons, we conclude that comparison selection uses \(\Omega(\log n)\) comparisons. (This is not a terribly good example; we look more at problem-reduction arguments in a few days.)