The Hierarchy of Speed

Week 2, Monday

January 12, 2026

Announcements

• Labs begin this week!
  • Monday 1-3pm or Tuesday 4-6pm

Warm-up: Code Analysis

Rules

For \(T(n)\), the running time of an algorithm on input of size \(n\)…

1. Constants don’t matter: \(5n = O(n)\)
2. Lower terms don’t matter: \(n^2 + n = O(n^2)\)
3. Log base doesn’t matter: \(log_2 n = O(log_{10}n)\)

“What’s the intuition for #3?”

All logs differ by a constant factor. log₂n = log₁₀n / log₁₀2

Example 1

0	1	2	3	4	5	6	7	8	9	10	11
3	7	2	9	1	8	4	6	5	2	7	3

• Python questions?
• What’s the worst-case running time?

One pass through the array. \(\Theta(n)\), tight.

“We’ll see this again later with pandas.”

Example 2

For \(n = 6\):

• What are the values of i and j for the 17th element counted?

• What are the values of i and j for the 32nd element counted?

• What’s the worst-case running time?

Let them trace: i=2, j=4 for 17th. i=5, j=1 for 32nd.

“Predict for n = 8?” (64)

Double n → QUADRUPLE the work. \(\Theta(n^2)\), tight.

Example 3

For \(n = 6\):

• What are the values of i and j for the 4th element counted?

• What are the values of i and j for the 12th element counted?

• What’s the worst-case running time?

i=2: j=0,1 (2 elements). i=3: j=0,1,2 (3 elements). So 4th is i=3, j=0.

\(0 + 1 + 2 + \ldots + (n-1) = n(n-1)/2\)

Still \(\Theta(n^2)\), tight! The triangle is half the square.

From Last Week

The Two Pillars

1. Correctness — Does it work?
2. Efficiency — How fast?

“Last week: efficiency. This week: we add correctness.”

The Analysis Recipe

1. Establish the case — usually worst case
2. Analyze the function — count operations for that case
3. State the bound:
   • \(\Theta\) (or “tight \(O\)”) if you know it’s exact
   • \(O\) if you’re only claiming an upper bound
   • \(\Omega\) if you’re only claiming a lower bound

“When we say ‘binary search is \(O(\log n)\)’ we mean it’s tight — \(\Theta(\log n)\). We’re not hedging.”

“First: which case? Then: what’s the function? Then: is it tight or just a bound?”

The Hierarchy

Big-\(O\) is a Set

\(O(g(n))\) is the set of all functions \(f(n)\) satisfying the definition.

\[O(n) = \{f(n) : \exists c > 0, n_0 \geq 0 \text{ such that } f(n) \leq c \cdot n \text{ for all } n \geq n_0\}\]

“From discrete math: Big-O defines a SET of functions.”

The Ladder

\[O(1) \subset O(\log n) \subset O(n) \subset O(n \log n) \subset O(n^2) \subset O(2^n)\]

Each class contains all the ones to its left.

“Every \(O(1)\) function is also \(O(n)\), also \(O(n^2)\), etc.”

A Common Abuse of Notation

We write: \(\quad 4n + 2 = O(n)\)

We mean: \(\quad 4n + 2 \in O(n)\)

“You’ll see this everywhere. Just remember: we’re really talking about set membership.”

True or False?

• \(3n + 2 \in O(n)\)
• \(3n + 2 \in O(n^2)\)
• \(n^2 \in O(n)\)
• \(5 \in O(1)\)
• \(5 \in O(n)\)

T, T, F, T, T

“If \(f \in O(g)\) and \(O(g) \subset O(h)\), then \(f \in O(h)\).”

“Let’s see what these mean at scale.”