Loop Invariants

Week 2, Tuesday (Video)

January 13, 2026

From Monday

We now have two pillars:

1. Efficiency — How fast? (Big-\(O\))
2. Correctness — Does it work?

What Big-\(O\) Actually Tells Us

Consider \(T(n) = 3n^2 + 5n + 7\). What happens when \(n\) doubles?

The ratio approaches 4. Always. For ANY quadratic.

“The constant 3, the +5n, the +7 — none of them matter. Doubling n → quadrupling T(n). That’s what \(O(n^2)\) means.”

The Doubling Test

Class	When \(n\) doubles, time…
\(O(1)\)	stays the same
\(O(\log n)\)	increases by a constant
\(O(n)\)	doubles
\(O(n \log n)\)	slightly more than doubles
\(O(n^2)\)	quadruples
\(O(2^n)\)	squares (!!)

“This is what Big-O tells us: the SHAPE of growth, not the exact numbers.”

The Practical Classes

\(O(1)\): Constant

arr[0]
d[key]
stack.pop()

“No matter how big the input, same time.”

Dictionary lookup, array access, stack operations.

\(O(\log n)\): Logarithmic

def binary_search(arr, target):
    lo, hi = 0, len(arr) - 1
    while lo <= hi:
        mid = (lo + hi) // 2
        if arr[mid] == target:
            return mid
        elif arr[mid] < target:
            lo = mid + 1
        else:
            hi = mid - 1
    return -1

“Halving at each step. \(\log_2(1 \text{ billion}) \approx 30\).”

“Worst case: target not in array. We halve until the range is empty → \(\Theta(\log n)\).”

We’ll prove this correct on Wednesday.

\(O(n)\): Linear

def find_max(arr):
    max_val = arr[0]
    for x in arr:
        if x > max_val:
            max_val = x
    return max_val

“Must look at every element once.”

“Worst case = every case: we always scan the whole array. \(\Theta(n)\), tight.”

\(O(n \log n)\): Linearithmic

sorted(arr)          # Python's built-in
df.sort_values()     # pandas

“The sweet spot for sorting. We’ll see why in Week 6.”

\(O(n^2)\): Quadratic

def has_duplicate(arr):
    for i in range(len(arr)):
        for j in range(i + 1, len(arr)):
            if arr[i] == arr[j]:
                return True
    return False

why not j in range(n)? “All pairs. Gets painful quickly.”

“Worst case: no duplicates → check all \(n(n-1)/2\) pairs. \(\Theta(n^2)\), tight.”

\(O(2^n)\): Exponential

def all_subsets(arr):
    if not arr:
        return [[]]
    rest = all_subsets(arr[1:])
    return rest + [[arr[0]] + s for s in rest]

“Doubles with each element.”

n=50: more operations than atoms in a grain of sand. n=100: more operations than atoms in the observable universe.

At 1 billion ops/sec, n=100 would take \(10^{21}\) years — heat death of universe.

“Often intractable. We’ll see NP-completeness in Week 12.”

Racing Algorithms

The Setup

“Same problem, two approaches. Let’s race them.”

The Race

“We’re testing worst case: no duplicates, so both algorithms scan everything.”

“What do you notice?”

\(\Theta(n^2)\): 4× slower when n doubles — that’s quadratic! \(\Theta(n)\): barely changes — that’s linear!

“The set gives \(O(1)\) lookup — transforms \(\Theta(n^2)\) into \(\Theta(n)\).”

What Happened

Worst case: \(\Theta(n^2) \to \Theta(n)\)

The set provides \(O(1)\) lookup.

“This is the power of data structures. We’ll learn how sets work in Week 5.”

“Both bounds are tight. We’re not hedging — we analyzed the worst case and know exactly how it scales.”

When Does It Matter?

The Crossover

“At small n, constants dominate. The ‘\(O(1)\)’ algorithm with 10ms overhead is slower!”

Live demo: add larger n values (10000, 100000, 1000000) to show the crossover where linear catches up and passes.

The Lesson

Small data: write clear code.

Big data: Big-O is destiny.

Galactic Pizza

Two Ways to Compute the Mean

“Both are \(O(n)\). Why the difference?”

Same Big-\(O\), Different Speed

Both touch every row: \(O(n)\)

But:

• Vectorized: tiny constants (compiled C)
• Python loop: large constants (interpreter overhead)

“Big-O tells you SCALING, not SPEED.”

“In pandas: prefer vectorized operations. But that’s about constants, not complexity class.”