Stacks

Week 5, Monday

February 2, 2026

A Big Question about Sorting

Can We Do Better?

We’ve seen:

• Insertion sort: \(O(n^2)\) (PEX1, HW1)
• Selection sort: \(O(n^2)\)
• Merge sort: \(O(n \log n)\)

Question: Is \(O(n \log n)\) the best we can do?

Or is there some clever algorithm that sorts in \(O(n)\)?

The Surprising Answer

Theorem: Any comparison-based sorting algorithm requires \(\Omega(n \log n)\) comparisons in the worst case.

This means:

• Merge sort is optimal
• No comparison sort can do better
• \(O(n \log n)\) is a fundamental barrier

“This is one of the most beautiful results in CS: we can prove a lower bound on ALL possible algorithms, not just the ones we know about.”

Decision Trees

The Key Idea

Every comparison-based sort makes decisions by comparing pairs of elements:

if arr[i] < arr[j]:
    ...
else:
    ...

We can model the algorithm as a binary tree of decisions.

Example: Sorting 3 Elements

Suppose we want to sort \([v_1, v_2, v_3]\).

What the Tree Represents

• Internal nodes: Comparisons
• Edges: Answers (yes/no, or \(<\) / \(\geq\))
• Leaves: sorted ordering of the elements
• Height: running time of the algorithm

Note:

• There must be a leaf (endpoint) for each ordering of the elements
• the tree itself is an encoding of an algorithm

Counting Leaves

How many different rearrangements are possible for an \(n\) element list?

Each ordering needs its own leaf (otherwise we’d give the same answer for different inputs).

The decision tree must have at least \(n!\) leaves!

A Question for Later

Whose answer we need now…

A decision tree of height \(h\) has at most _______ leaves?

The Lower Bound Proof

Connecting Height to Comparisons

Recall: The height of the decision tree is the maximum number of comparisons in the worst case.

If the tree has height \(h\), then:

• At most \(2^h\) leaves
• We need at least \(n!\) leaves

Therefore: \(2^h \geq n!\)

Taking logarithms: \(h \geq \log_2(n!)\)

But How Big is \(\log(n!)\)?

Claim: \(\log(n!) = \Omega(n\log n)\)

Proof: \[ \begin{align} \log(n!) &= \log n + \log(n-1) + \cdots + \log 2 + \log 1 \\ &\geq \log n + \log(n-1) + \cdots + \log\left(\tfrac{n}{2}\right) & \text{(drop small half)} \\ &\geq \tfrac{n}{2} \cdot \log\left(\tfrac{n}{2}\right) & \text{(each term } \geq \log(n/2)\text{)} \\ &= \tfrac{n}{2}\log n - \tfrac{n}{2} \\ &= \tfrac{n}{4}\log n + \tfrac{n}{4}\log n - \tfrac{n}{2} \\ &\geq \tfrac{n}{4}\log n & \text{(when } \log n \geq 2 \text{, i.e., } n \geq 4\text{)} \\ &= \Omega(n \log n) \quad \blacksquare \end{align} \]

The Result

Observation: Any comparison-based sorting algorithm requires \(\Omega(n \log n)\) comparisons in the worst case.

Justification:

1. Model the algorithm as a decision tree
2. The tree needs at least \(n!\) leaves
3. A tree with \(\geq n!\) leaves has height \(\geq \log_2(n!)\)
4. \(\log_2(n!) = \Theta(n \log n)\)
5. Therefore, worst-case comparisons \(\Omega(n \log n)\) ∎

“This proof is remarkable: we didn’t analyze any specific algorithm. We proved a limit on ALL possible comparison-based sorting algorithms!”

What This Means

Algorithm	Worst Case	Optimal?
Merge Sort	\(O(n \log n)\)	Yes!
Heap Sort	\(O(n \log n)\)	Yes!
Quick Sort	\(O(n^2)\)	No (but avg is optimal)
Insertion Sort	\(O(n^2)\)	No
Selection Sort	\(O(n^2)\)	No

Merge sort achieves the theoretical limit.

Can We Beat \(O(n \log n)\)?

The lower bound applies to comparison-based sorts.

If we know something about the input, we can do better:

Algorithm	Constraint	Time
Counting Sort	Integers in \([0, k]\)	\(O(n + k)\)
Radix Sort	Fixed-width integers	\(O(d \cdot n)\)

In practice: Just use Python’s sorted() (Timsort, \(O(n \log n)\)).

Summary

• Labs this week will close up sorting.
• Sorting has been a good playground for examples.

Ordered Search

The Maze

You wake up in a maze. Every room has a number. Some rooms have multiple exits.

Question: How do you explore systematically to find the way out?

https://www.youtube.com/watch?v=8qrZ1clEp-Y

we don’t have yarn!

What Did You Notice?

Write down everything you noticed:

Bag (random):

• Chaotic, unpredictable order
• Hard to reason about
• Different every time

Stack (LIFO):

• Goes deep first
• Backtracks when stuck
• Consistent, predictable
• tells us something aobut shape of the maze.

Queue (FIFO): - explores at same depth before moving on. - consistent, predictable

Abstract Data Type (ADT) Stack

Programmatic realization of a ________________.

 

Stack functions:

• Insert: push(item)
• Remove: pop() \(\rightarrow\) item

List functions:

• Insert: append(item)
• Remove: pop() \(\rightarrow\) item

Tuesday Video:

“Dynamic Array Diversion”