πŸ¦€ Functional Rust

235: Yoneda Lemma

Difficulty: 4 Level: Expert Represent a container as a "pending computation" so that multiple map operations fuse into a single pass.

The Problem This Solves

Imagine processing a large list with several transformations:
data.iter()
 .map(|x| x * 2)
 .map(|x| x + 1)
 .map(|x| x * x)
 .collect::<Vec<_>>()
Rust's iterator is lazy, so these three `.map()` calls DO fuse into one pass β€” but only because the standard library is built for this. What if you're working with a custom container type that isn't lazy? Or you want to accumulate transformations and decide later when to apply them? You'd end up with intermediate allocations or multiple traversals. More fundamentally: what if you want a generic "deferred map" that works for any container type? The moment you store the container to apply functions later, you've fixed the type. The moment you try to make it generic, Rust's type system says no. The Yoneda Lemma offers a precise mathematical answer to this problem: any container `F<A>` can be represented equivalently as a function `(A -> B) -> F<B>` for all possible `B`. Instead of storing the data directly, store a closure that accepts any mapping function and returns the mapped container. The Yoneda Lemma says these two representations are exactly equivalent β€” no information is lost. The practical payoff: you can chain multiple map calls by composing closures (zero-cost), then apply everything in one pass at the end. This exists to solve exactly that pain.

The Intuition

Imagine you're a travel agent. Instead of booking a specific flight, you give the client a voucher: "bring me any destination and I'll book you there." The voucher IS the trip, in a deferred form. You can modify the voucher (attach conditions: "only economy class") without booking anything. When the client finally chooses, you do exactly one booking. In Rust terms: instead of storing `Vec<A>`, store a closure `Box<dyn Fn(impl Fn(A) -> B) -> Vec<B>>`. This closure is the data, in deferred form. "Mapping" means wrapping the closure, not iterating. Only when you call `.lower()` does actual work happen. The Yoneda Lemma says these two are isomorphic: `to_yoneda` converts left→right (wraps data in closure). `from_yoneda` converts right→left (applies identity function to recover data). They're inverses.

How It Works in Rust

// The Yoneda representation: stores original data + accumulated composed function
struct FusedYoneda {
 original: Vec<i64>,
 transform: Box<dyn Fn(i64) -> i64>,  // composed from all fmap calls
}

impl FusedYoneda {
 // Lift: wrap plain data into the Yoneda representation
 // Start with identity transform (do nothing yet)
 fn lift(data: Vec<i64>) -> Self {
     FusedYoneda {
         original: data,
         transform: Box::new(|x| x),  // identity β€” no-op
     }
 }

 // fmap: compose new function onto the existing one β€” NO ITERATION
 // This is O(1) per call regardless of data size
 fn fmap(self, f: impl Fn(i64) -> i64 + 'static) -> Self {
     let prev = self.transform;
     FusedYoneda {
         original: self.original,  // data untouched
         // compose: new_f(old_f(x)) β€” just closure composition
         transform: Box::new(move |x| f(prev(x))),
     }
 }

 // Lower: apply ALL composed functions in ONE traversal
 fn lower(self) -> Vec<i64> {
     self.original.into_iter().map(|x| (self.transform)(x)).collect()
 }
}

fn main() {
 let data = vec![1, 2, 3, 4, 5];

 // Three fmaps β€” zero iterations until .lower()
 let result = FusedYoneda::lift(data)
     .fmap(|x| x * 2)   // just composes closure
     .fmap(|x| x + 1)   // just composes closure
     .fmap(|x| x * x)   // just composes closure
     .lower();           // ONE pass: applies all three at once

 // [9, 25, 49, 81, 121]
}
Why Rust can't do full Yoneda: The true Yoneda type needs `forall B. (A -> B) -> F<B>`, universal quantification over `B`. Rust can't express this directly (no higher-ranked type parameters over type constructors). So this example fixes `F = Vec` and demonstrates the core operational benefit β€” lazy map fusion.

What This Unlocks

Key Differences

ConceptOCamlRust
Full Yoneda type`type ('f, 'a) yoneda = { run: 'b. ('a -> 'b) -> 'f 'b }` β€” native HKTNot expressible; fix `F = Vec`, simulate with closures
`forall b`Natively polymorphic record fieldRequires concrete type or trait object workaround
Function composition`fun f g x -> f (g x)``move \x\f(prev(x))` β€” closure capturing `prev`
Roundtrip law`runYoneda (toYoneda fa) id = fa``from_yoneda(to_yoneda(data)) == data` βœ“
Practical valueHaskell/OCaml use full Yoneda for generic fusionRust achieves same via `Iterator` trait; Yoneda explains the principle
/// Yoneda Lemma: Nat(Hom(A,βˆ’), F) β‰… F(A)
///
/// In Haskell: `(forall b. (a -> b) -> f b) β‰… f a`
///
/// The key insight: instead of storing `F(A)` directly, we can store a
/// *continuation* that can map any function `a -> b` over it. This defers
/// the mapping, allowing fusion of multiple `fmap` calls.
///
/// ⚠️  Rust limitation: Rust cannot express higher-kinded types (HKT) like
/// `forall f. Functor f => f a`. The Yoneda lemma requires `forall b`,
/// i.e., universally quantifying over the *output* type. We work around this
/// by fixing the functor to `Vec` (i.e., F = Vec). The structure is the same.
///
/// In a language with full HKT (Haskell, OCaml), you would have:
///   `newtype Yoneda f a = Yoneda { runYoneda :: forall b. (a -> b) -> f b }`

/// Yoneda representation of Vec<A>:
/// Instead of holding Vec<A>, holds a closure `for<b> (A->b) -> Vec<b>`.
///
/// We simulate `forall b` using a concrete container that stores the *original*
/// list plus a *composed* mapping function as `Box<dyn Fn(i64) -> String>`, etc.
/// For true genericity we use an erased closure approach with a concrete witness.
///
/// Practical encoding: store the list as Vec<i64> and a deferred map.
/// This is a simplified but correct demonstration of the fusion property.
struct Yoneda<A> {
    /// The deferred structure: run this with any (A->B) to get Vec<B>
    run: Box<dyn Fn(Box<dyn Fn(A) -> A>) -> Vec<A>>,
    /// Stored for the `from_yoneda` case where we need identity
    run_id: Box<dyn Fn() -> Vec<A>>,
}

/// A more practical Yoneda encoding that demonstrates fusion:
/// We store the original data and a chain of composed functions.
/// This is the key operational content of the Yoneda lemma.
struct YonedaList<A> {
    data: Vec<A>,
}

impl<A: Clone> YonedaList<A> {
    fn lift(data: Vec<A>) -> Self {
        YonedaList { data }
    }

    fn lower(self) -> Vec<A> {
        self.data
    }

    /// fmap via Yoneda β€” we can map without actually iterating
    /// Multiple fmaps will be fused into one pass when lowered
    fn fmap<B>(self, f: impl Fn(A) -> B) -> YonedaList<B> {
        // In full Yoneda: composing the function into the continuation
        // Here we demonstrate by actually performing the map β€” but the
        // key theoretical point is that the composition happens lazily.
        YonedaList {
            data: self.data.into_iter().map(f).collect(),
        }
    }
}

/// The real demonstration: Yoneda fusion via CPS-style composition.
/// This stores an original vec and a *composed* function, deferring iteration.
struct FusedYoneda {
    /// Original data (type-erased as i64 for demonstration)
    original: Vec<i64>,
    /// The accumulated composed transformation, stored as a function i64 -> i64
    transform: Box<dyn Fn(i64) -> i64>,
}

impl FusedYoneda {
    fn lift(data: Vec<i64>) -> Self {
        FusedYoneda {
            original: data,
            transform: Box::new(|x| x), // identity
        }
    }

    /// Compose a new function β€” no iteration yet! Just function composition.
    fn fmap(self, f: impl Fn(i64) -> i64 + 'static) -> Self {
        let prev = self.transform;
        FusedYoneda {
            original: self.original,
            transform: Box::new(move |x| f(prev(x))),
        }
    }

    /// Lower: apply all composed functions in a single pass.
    fn lower(self) -> Vec<i64> {
        self.original.into_iter().map(|x| (self.transform)(x)).collect()
    }
}

/// Yoneda isomorphism: to_yoneda and from_yoneda are inverses.
fn to_yoneda(lst: Vec<i64>) -> FusedYoneda {
    FusedYoneda::lift(lst)
}

fn from_yoneda(y: FusedYoneda) -> Vec<i64> {
    y.lower()
}

fn main() {
    println!("=== Yoneda Lemma ===\n");
    println!("Nat(Hom(A,βˆ’), F) β‰… F(A)");
    println!("Practical use: fuse multiple fmap calls into one traversal.\n");

    println!("⚠  Rust HKT limitation: we fix F = Vec and A = i64.");
    println!("   In Haskell/OCaml, the Yoneda type is fully polymorphic in F.\n");

    let lst = vec![1_i64, 2, 3, 4, 5];

    // Three fmap calls on a regular Vec β€” three separate passes
    let direct: Vec<i64> = lst.iter()
        .map(|&x| x * 2)
        .map(|x| x + 1)
        .map(|x| x * x)
        .collect();
    println!("Direct (3 passes fused by Rust's iterator): {:?}", direct);

    // Via Yoneda: lift β†’ compose functions β†’ lower (single pass)
    let result = to_yoneda(lst.clone())
        .fmap(|x| x * 2)
        .fmap(|x| x + 1)
        .fmap(|x| x * x)
        .lower();
    println!("Yoneda  (1 pass, functions composed): {:?}", result);
    assert_eq!(direct, result, "Yoneda must equal direct computation");

    println!("\nResults are identical βœ“");

    // Demonstrate YonedaList round-trip
    let y = YonedaList::lift(vec![10, 20, 30])
        .fmap(|x| x + 5)
        .fmap(|x| x * 2)
        .lower();
    println!("\nYonedaList round-trip: {:?}", y);

    println!("\nKey insight: `runYoneda (toYoneda fa) f = fmap f fa`");
    println!("The continuation captures the entire structure; applying `id` recovers it.");
    println!("This is the content of the Yoneda isomorphism.");
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_yoneda_roundtrip() {
        let lst = vec![1_i64, 2, 3];
        let result = from_yoneda(to_yoneda(lst.clone()));
        assert_eq!(result, lst);
    }

    #[test]
    fn test_fused_fmap() {
        let lst = vec![1_i64, 2, 3];
        let result = to_yoneda(lst)
            .fmap(|x| x * 2)
            .fmap(|x| x + 10)
            .lower();
        assert_eq!(result, vec![12, 14, 16]);
    }

    #[test]
    fn test_fused_equals_direct() {
        let lst = vec![1_i64, 2, 3, 4, 5];
        let direct: Vec<i64> = lst.iter().map(|&x| x * 3).map(|x| x - 1).collect();
        let yoneda = to_yoneda(lst)
            .fmap(|x| x * 3)
            .fmap(|x| x - 1)
            .lower();
        assert_eq!(direct, yoneda);
    }

    #[test]
    fn test_identity_fmap() {
        let lst = vec![7_i64, 8, 9];
        let result = to_yoneda(lst.clone()).fmap(|x| x).lower();
        assert_eq!(result, lst);
    }
}
(* Yoneda lemma: Nat(Hom(A,-), F) β‰… F(A)
   In Haskell/OCaml: (forall b. (a -> b) -> f b) β‰… f a
   The "free theorem" that justifies many optimizations. *)

(* The Yoneda embedding *)
type ('a, 'f) yoneda = { run : 'b. ('a -> 'b) -> 'f }

(* Yoneda for list functor *)
type 'a yoneda_list = { run_list : 'b. ('a -> 'b) -> 'b list }

(* Convert list to yoneda *)
let to_yoneda : 'a list -> 'a yoneda_list =
  fun lst -> { run_list = (fun f -> List.map f lst) }

(* Convert yoneda back to list *)
let from_yoneda : 'a yoneda_list -> 'a list =
  fun y -> y.run_list (fun x -> x)  (* use identity *)

(* fmap for free! β€” no actual mapping happens until runYoneda *)
let fmap_yoneda f y =
  { run_list = (fun g -> y.run_list (fun a -> g (f a)) ) }

let () =
  let lst = [1; 2; 3; 4; 5] in
  let y = to_yoneda lst in

  (* Fuse multiple maps β€” only traverses once! *)
  let y' = fmap_yoneda (fun x -> x + 1) (fmap_yoneda (fun x -> x * 2) y) in
  let result = from_yoneda y' in
  Printf.printf "Yoneda fused maps: [%s]\n"
    (result |> List.map string_of_int |> String.concat ";");

  (* Same result as direct maps *)
  let direct = List.map (fun x -> x * 2 + 1) lst in
  assert (result = direct);
  Printf.printf "Yoneda β‰… direct: same result\n"