ExamplesBy LevelBy TopicLearning Paths

Huffman Coding

TreesGreedy Algorithms

Tutorial

The Problem

Given a list of symbols and their frequencies, build a Huffman tree and derive an optimal prefix-free binary code for each symbol.

🎯 Learning Outcomes

  • • Modelling recursive ADTs in Rust with enum + Box
  • • Using BinaryHeap plus a wrapper type to get a min-priority queue
  • • Why HTree deliberately isn't Ord (no total ordering on trees) and how
  • to introduce a deterministic tiebreak

  • • Translating an OCaml sort/merge loop into an iterator-friendly Rust version
  • 🦀 The Rust Way

    Mirror the ADT with an enum whose recursive arms are boxed. For the priority queue, wrap each subtree in a HeapEntry that flips the default max-heap into a min-heap. A monotonically-increasing tiebreak counter keeps pops deterministic when frequencies collide, which the OCaml version got for free from list order. Code extraction is a standard in-place DFS that pushes/pops '0' / '1' on a shared String buffer.

    Code Example

    pub fn build_tree_heap(freqs: &[(char, u32)]) -> Option<HTree> {
        if freqs.is_empty() { return None; }
        let mut heap: BinaryHeap<HeapEntry> = freqs.iter().enumerate()
            .map(|(i, &(c, f))| HeapEntry { freq: f, tiebreak: i, tree: HTree::Leaf(c, f) })
            .collect();
        let mut tiebreak = freqs.len();
        while heap.len() > 1 {
            let a = heap.pop()?;
            let b = heap.pop()?;
            let total = a.freq + b.freq;
            let merged = HTree::Node(Box::new(a.tree), Box::new(b.tree), total);
            heap.push(HeapEntry { freq: total, tiebreak, tree: merged });
            tiebreak += 1;
        }
        heap.pop().map(|e| e.tree)
    }

    Key Differences

  • Recursive data: OCaml allows direct Node of htree * htree * int; Rust
  • needs Box<HTree> to give the enum a known size.

  • Priority queue: OCaml re-sorts a list each round (O(n² log n)); Rust's
  • BinaryHeap gives O(n log n) with the standard lazy-rebalance idiom.

  • Ordering: OCaml's compare orders any two values structurally; Rust
  • requires you to spell out what "smaller" means for trees — we only compare frequencies, so we keep Ord on a wrapper, not on HTree itself.

  • String building: OCaml uses persistent prefix ^ "0" concatenation;
  • Rust pushes/pops a single String to avoid O(n²) allocation during the walk.

    OCaml Approach

    OCaml defines type htree = Leaf of char * int | Node of htree * htree * int and keeps a list sorted by frequency. Each step: take the two smallest, merge them into a Node whose frequency is the sum, re-sort, repeat until one tree remains. Matches naturally on | [t] -> t | a :: b :: rest -> ....

    Full Source

    #![allow(dead_code)]
    //! Huffman coding tree construction and prefix-code generation.
    //!
    //! Two build strategies are provided:
    //! * [`build_tree_heap`] — idiomatic Rust using a min-heap ([`BinaryHeap`]
    //!   with [`Reverse`]).  O(n log n), no repeated scanning.
    //! * [`build_tree_sorted`] — mirrors the OCaml original's sort/merge loop
    //!   for easy side-by-side comparison.
    
    use std::cmp::Ordering;
    use std::collections::BinaryHeap;
    
    /// A Huffman tree: either a leaf carrying a character + frequency, or an
    /// internal node with cached total frequency and two children.
    #[derive(Debug, Clone, PartialEq, Eq)]
    pub enum HTree {
        Leaf(char, u32),
        Node(Box<HTree>, Box<HTree>, u32),
    }
    
    impl HTree {
        /// Total frequency at this subtree.  O(1) because internal nodes cache it.
        pub fn freq(&self) -> u32 {
            match self {
                HTree::Leaf(_, f) | HTree::Node(_, _, f) => *f,
            }
        }
    }
    
    // Wrapper that makes a [`BinaryHeap`] behave as a min-heap keyed on
    // `(freq, tiebreak)`.  `HTree` itself intentionally does not implement `Ord`
    // because there is no meaningful total ordering on trees.
    struct HeapEntry {
        freq: u32,
        tiebreak: usize,
        tree: HTree,
    }
    
    impl PartialEq for HeapEntry {
        fn eq(&self, other: &Self) -> bool {
            (self.freq, self.tiebreak) == (other.freq, other.tiebreak)
        }
    }
    impl Eq for HeapEntry {}
    impl Ord for HeapEntry {
        // Flipped comparison — BinaryHeap is a max-heap, so reversing here gives
        // a min-heap by frequency, with `tiebreak` breaking ties deterministically.
        fn cmp(&self, other: &Self) -> Ordering {
            other
                .freq
                .cmp(&self.freq)
                .then_with(|| other.tiebreak.cmp(&self.tiebreak))
        }
    }
    impl PartialOrd for HeapEntry {
        fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
            Some(self.cmp(other))
        }
    }
    
    // --- Approach 1: idiomatic Rust using a min-heap -----------------------------
    
    /// Build a Huffman tree with a [`BinaryHeap`] acting as a min-priority queue.
    /// O(n log n).  Returns `None` for empty input.
    pub fn build_tree_heap(freqs: &[(char, u32)]) -> Option<HTree> {
        if freqs.is_empty() {
            return None;
        }
        let mut heap: BinaryHeap<HeapEntry> = freqs
            .iter()
            .enumerate()
            .map(|(i, &(c, f))| HeapEntry {
                freq: f,
                tiebreak: i,
                tree: HTree::Leaf(c, f),
            })
            .collect();
        let mut tiebreak = freqs.len();
        while heap.len() > 1 {
            let a = heap.pop()?;
            let b = heap.pop()?;
            let total = a.freq + b.freq;
            let merged = HTree::Node(Box::new(a.tree), Box::new(b.tree), total);
            heap.push(HeapEntry {
                freq: total,
                tiebreak,
                tree: merged,
            });
            tiebreak += 1;
        }
        heap.pop().map(|e| e.tree)
    }
    
    // --- Approach 2: sort/merge loop (OCaml-parallel) ----------------------------
    
    /// Build a Huffman tree by re-sorting the working list on every merge.
    /// A direct translation of the OCaml reference implementation; clearer for
    /// teaching, O(n² log n) instead of O(n log n).
    pub fn build_tree_sorted(freqs: &[(char, u32)]) -> Option<HTree> {
        if freqs.is_empty() {
            return None;
        }
        let mut trees: Vec<HTree> = freqs.iter().map(|&(c, f)| HTree::Leaf(c, f)).collect();
        trees.sort_by_key(HTree::freq);
        while trees.len() > 1 {
            let a = trees.remove(0);
            let b = trees.remove(0);
            let total = a.freq() + b.freq();
            let merged = HTree::Node(Box::new(a), Box::new(b), total);
            let pos = trees
                .iter()
                .position(|t| t.freq() > total)
                .unwrap_or(trees.len());
            trees.insert(pos, merged);
        }
        trees.into_iter().next()
    }
    
    // --- Code extraction ---------------------------------------------------------
    
    /// Walk the tree and emit `(char, bitstring)` pairs, one per leaf.  Going left
    /// appends `0`, going right appends `1`.  A single-leaf tree yields an empty
    /// code — matching the OCaml behaviour.
    pub fn codes(tree: &HTree) -> Vec<(char, String)> {
        fn go(tree: &HTree, prefix: &mut String, out: &mut Vec<(char, String)>) {
            match tree {
                HTree::Leaf(c, _) => out.push((*c, prefix.clone())),
                HTree::Node(l, r, _) => {
                    prefix.push('0');
                    go(l, prefix, out);
                    prefix.pop();
                    prefix.push('1');
                    go(r, prefix, out);
                    prefix.pop();
                }
            }
        }
        let mut prefix = String::new();
        let mut out = Vec::new();
        go(tree, &mut prefix, &mut out);
        out
    }
    
    #[cfg(test)]
    mod tests {
        use super::*;
    
        fn sample() -> Vec<(char, u32)> {
            vec![
                ('a', 5),
                ('b', 9),
                ('c', 12),
                ('d', 13),
                ('e', 16),
                ('f', 45),
            ]
        }
    
        #[test]
        fn empty_input_returns_none() {
            assert!(build_tree_heap(&[]).is_none());
            assert!(build_tree_sorted(&[]).is_none());
        }
    
        #[test]
        fn single_leaf_is_returned_as_is() {
            let t = build_tree_heap(&[('z', 7)]).expect("one leaf");
            assert_eq!(t, HTree::Leaf('z', 7));
            let pairs = codes(&t);
            assert_eq!(pairs, vec![('z', String::new())]);
        }
    
        #[test]
        fn root_frequency_is_total_of_inputs() {
            let total: u32 = sample().iter().map(|&(_, f)| f).sum();
            let t = build_tree_heap(&sample()).expect("non-empty");
            assert_eq!(t.freq(), total);
        }
    
        #[test]
        fn codes_are_prefix_free_and_complete() {
            let t = build_tree_heap(&sample()).expect("non-empty");
            let pairs = codes(&t);
            assert_eq!(pairs.len(), 6);
            // prefix-free: no code is a prefix of another
            for (i, (_, a)) in pairs.iter().enumerate() {
                for (j, (_, b)) in pairs.iter().enumerate() {
                    if i != j {
                        assert!(!b.starts_with(a), "{a} is a prefix of {b}");
                    }
                }
            }
        }
    
        #[test]
        fn most_frequent_symbol_gets_shortest_code() {
            let t = build_tree_heap(&sample()).expect("non-empty");
            let pairs = codes(&t);
            let shortest = pairs.iter().min_by_key(|(_, s)| s.len()).unwrap();
            assert_eq!(shortest.0, 'f');
            assert_eq!(shortest.1.len(), 1);
        }
    
        #[test]
        fn heap_and_sort_builds_produce_same_code_lengths() {
            // Exact tree shape can differ on ties, but the multiset of code
            // lengths is a Huffman invariant.
            let a = codes(&build_tree_heap(&sample()).unwrap());
            let b = codes(&build_tree_sorted(&sample()).unwrap());
            let mut la: Vec<usize> = a.iter().map(|(_, s)| s.len()).collect();
            let mut lb: Vec<usize> = b.iter().map(|(_, s)| s.len()).collect();
            la.sort_unstable();
            lb.sort_unstable();
            assert_eq!(la, lb);
        }
    }
    ✓ Tests Rust test suite
    #[cfg(test)]
    mod tests {
        use super::*;
    
        fn sample() -> Vec<(char, u32)> {
            vec![
                ('a', 5),
                ('b', 9),
                ('c', 12),
                ('d', 13),
                ('e', 16),
                ('f', 45),
            ]
        }
    
        #[test]
        fn empty_input_returns_none() {
            assert!(build_tree_heap(&[]).is_none());
            assert!(build_tree_sorted(&[]).is_none());
        }
    
        #[test]
        fn single_leaf_is_returned_as_is() {
            let t = build_tree_heap(&[('z', 7)]).expect("one leaf");
            assert_eq!(t, HTree::Leaf('z', 7));
            let pairs = codes(&t);
            assert_eq!(pairs, vec![('z', String::new())]);
        }
    
        #[test]
        fn root_frequency_is_total_of_inputs() {
            let total: u32 = sample().iter().map(|&(_, f)| f).sum();
            let t = build_tree_heap(&sample()).expect("non-empty");
            assert_eq!(t.freq(), total);
        }
    
        #[test]
        fn codes_are_prefix_free_and_complete() {
            let t = build_tree_heap(&sample()).expect("non-empty");
            let pairs = codes(&t);
            assert_eq!(pairs.len(), 6);
            // prefix-free: no code is a prefix of another
            for (i, (_, a)) in pairs.iter().enumerate() {
                for (j, (_, b)) in pairs.iter().enumerate() {
                    if i != j {
                        assert!(!b.starts_with(a), "{a} is a prefix of {b}");
                    }
                }
            }
        }
    
        #[test]
        fn most_frequent_symbol_gets_shortest_code() {
            let t = build_tree_heap(&sample()).expect("non-empty");
            let pairs = codes(&t);
            let shortest = pairs.iter().min_by_key(|(_, s)| s.len()).unwrap();
            assert_eq!(shortest.0, 'f');
            assert_eq!(shortest.1.len(), 1);
        }
    
        #[test]
        fn heap_and_sort_builds_produce_same_code_lengths() {
            // Exact tree shape can differ on ties, but the multiset of code
            // lengths is a Huffman invariant.
            let a = codes(&build_tree_heap(&sample()).unwrap());
            let b = codes(&build_tree_sorted(&sample()).unwrap());
            let mut la: Vec<usize> = a.iter().map(|(_, s)| s.len()).collect();
            let mut lb: Vec<usize> = b.iter().map(|(_, s)| s.len()).collect();
            la.sort_unstable();
            lb.sort_unstable();
            assert_eq!(la, lb);
        }
    }

    Deep Comparison

    OCaml vs Rust: Huffman Coding

    Side-by-Side Code

    OCaml

    type htree = Leaf of char * int | Node of htree * htree * int
    
    let freq = function Leaf (_,f) -> f | Node (_,_,f) -> f
    
    let build_tree freqs =
      let trees =
        List.map (fun (c,f) -> Leaf (c,f)) freqs
        |> List.sort (fun a b -> compare (freq a) (freq b))
      in
      let rec go = function
        | [t] -> t
        | a :: b :: rest ->
            let merged = Node (a, b, freq a + freq b) in
            go (List.sort (fun a b -> compare (freq a) (freq b)) (merged :: rest))
        | [] -> failwith "empty"
      in go trees
    
    let rec codes prefix = function
      | Leaf (c, _) -> [(c, prefix)]
      | Node (l, r, _) -> codes (prefix ^ "0") l @ codes (prefix ^ "1") r
    

    Rust (idiomatic — min-heap)

    pub fn build_tree_heap(freqs: &[(char, u32)]) -> Option<HTree> {
        if freqs.is_empty() { return None; }
        let mut heap: BinaryHeap<HeapEntry> = freqs.iter().enumerate()
            .map(|(i, &(c, f))| HeapEntry { freq: f, tiebreak: i, tree: HTree::Leaf(c, f) })
            .collect();
        let mut tiebreak = freqs.len();
        while heap.len() > 1 {
            let a = heap.pop()?;
            let b = heap.pop()?;
            let total = a.freq + b.freq;
            let merged = HTree::Node(Box::new(a.tree), Box::new(b.tree), total);
            heap.push(HeapEntry { freq: total, tiebreak, tree: merged });
            tiebreak += 1;
        }
        heap.pop().map(|e| e.tree)
    }
    

    Rust (functional — sort/merge, OCaml-parallel)

    pub fn build_tree_sorted(freqs: &[(char, u32)]) -> Option<HTree> {
        if freqs.is_empty() { return None; }
        let mut trees: Vec<HTree> =
            freqs.iter().map(|&(c,f)| HTree::Leaf(c,f)).collect();
        trees.sort_by_key(HTree::freq);
        while trees.len() > 1 {
            let a = trees.remove(0);
            let b = trees.remove(0);
            let total = a.freq() + b.freq();
            let merged = HTree::Node(Box::new(a), Box::new(b), total);
            let pos = trees.iter().position(|t| t.freq() > total).unwrap_or(trees.len());
            trees.insert(pos, merged);
        }
        trees.into_iter().next()
    }
    

    Type Signatures

    ConceptOCamlRust
    Tree typetype htree = Leaf .. | Node ..pub enum HTree { Leaf(..), Node(..) }
    Recursive childhtree (direct)Box<HTree> (sized requirement)
    Build signature(char * int) list -> htreefn build_tree(&[(char,u32)]) -> Option<HTree>
    Codes signaturestring -> htree -> (char*string) listfn codes(&HTree) -> Vec<(char,String)>
    Failure on emptyfailwith "empty"Option::None

    Key Insights

  • Boxing is mandatory for recursive enums in Rust. Without Box, the
  • compiler can't compute a fixed size for HTree and rejects the type.

  • **BinaryHeap is max-heap; wrap to reverse.** We implement Ord on a
  • HeapEntry wrapper so flipping cmp order is local to the PQ use-site rather than polluting HTree with an orientation-specific Ord impl.

  • No free structural ordering on trees. OCaml's compare sorts any two
  • values; Rust refuses to derive Ord for HTree because Box<HTree> doesn't have a meaningful comparison beyond "compare the frequencies."

  • String accumulation differs. OCaml's immutable prefix ^ "0" creates a
  • fresh string per step; the Rust DFS pushes a single byte onto a shared String buffer and pops it on the way up — linear total work.

  • Determinism needs a tiebreak. When two frequencies are equal, OCaml's
  • stable sort preserves insertion order; Rust's BinaryHeap is unstable, so a monotonically-increasing tiebreak field is added to recover reproducible code assignments across runs.

    When to Use Each Style

    **Use idiomatic Rust (build_tree_heap) when:** encoding real data and you care about throughput — it's O(n log n) and streams input into the heap naturally.

    **Use recursive Rust (build_tree_sorted) when:** teaching the algorithm or comparing against the OCaml reference — the sort/merge loop reads the same way and makes the invariant ("smallest two merge next") blindingly obvious.

    Open Source Repos