๐Ÿฆ€ Functional Rust

236: Coyoneda

Difficulty: 4 Level: Expert Make any type constructor act like a functor for free, with automatic map fusion as a bonus.

The Problem This Solves

Suppose you have a custom type โ€” say a priority queue, a graph node, or a database cursor โ€” that doesn't implement `map`. You can't call `.map()` on it. You have to manually unpack it, transform the values, and repack. That's annoying, but manageable. Now suppose you need to apply several transformations in sequence. Each one unpacks and repacks. Worse, if the container is expensive to traverse (a lazy I/O stream, a tree with millions of nodes), you've paid the traversal cost multiple times for what is conceptually one operation. The deeper problem: you'd like to write generic code that works with "any container that can be mapped" โ€” but your type doesn't have a `Functor` instance. You can't use any of the generic infrastructure. Coyoneda solves both problems at once. It wraps any type constructor (whether or not it has `map`) and gives you a `fmap` that defers all work. You compose as many maps as you want โ€” zero traversals. Only when you call `lower()` does the container get traversed exactly once, with all transformations fused into a single function. This exists to solve exactly that pain.

The Intuition

Think of Coyoneda as a "pending receipt" for a series of operations on a container. When you buy something and request a refund, the store doesn't immediately restock the item and re-issue currency. They give you a receipt that says "you are owed X." You can endorse that receipt (add conditions), give it to someone else โ€” all without touching the actual inventory. `Coyoneda::lift(data)` creates the receipt. Each `.fmap(f)` adds an endorsement (composes `f` into a pending function). `.lower()` is when you cash it in โ€” the container is traversed once with the fully composed function applied. The "free functor" part: even if your original container type has no `map`, Coyoneda wraps it and provides one. You get functor behavior for free, just by wrapping in Coyoneda.
Original data:  [1, 2, 3, 4, 5]
                  โ†“ lift
Coyoneda:       (data=[1..5], transform=identity)
                  โ†“ .fmap(|x| x*2)      โ€” no traversal yet
Coyoneda:       (data=[1..5], transform=|x| x*2)
                  โ†“ .fmap(|x| x+1)      โ€” no traversal yet
Coyoneda:       (data=[1..5], transform=|x| x*2+1)
                  โ†“ .lower()            โ€” ONE traversal
Result:         [3, 5, 7, 9, 11]

How It Works in Rust

// Coyoneda wraps any Vec<A> and holds the pending transformation
pub struct Coyoneda<A> {
 // A boxed closure: when called, applies ALL pending fmaps and returns Vec<A>
 lower_fn: Box<dyn FnOnce() -> Vec<A>>,
}

impl<A: 'static> Coyoneda<A> {
 // Lift: wrap plain data โ€” transformation is identity (do nothing yet)
 pub fn lift(data: Vec<A>) -> Self {
     Coyoneda {
         lower_fn: Box::new(move || data),  // closure owns the data
     }
 }

 // fmap: compose new function AROUND the existing closure โ€” no traversal
 // The key: this is O(1), regardless of how large the data is
 pub fn fmap<B: 'static>(self, f: impl Fn(A) -> B + 'static) -> Coyoneda<B> {
     Coyoneda {
         lower_fn: Box::new(move || {
             let data = (self.lower_fn)();       // get data from inner closure
             data.into_iter().map(f).collect()   // apply f
         }),
         // NOTE: the inner closure isn't called yet โ€” this is still deferred
     }
 }

 // Lower: execute everything โ€” the one and only traversal
 pub fn lower(self) -> Vec<A> {
     (self.lower_fn)()
 }
}

fn main() {
 // Three fmaps โ€” but data is only traversed ONCE at .lower()
 let result = Coyoneda::lift(vec![1_i32, 2, 3, 4, 5])
     .fmap(|x| x * 2)          // pending
     .fmap(|x| x + 1)          // pending
     .fmap(|x| x.to_string())  // pending โ€” note: type changes from i32 to String
     .lower();                  // executed: ["3", "5", "7", "9", "11"]
}
The "explicit" version in the code makes the structure even clearer โ€” it stores the original data and a composed `i32 -> A` function as separate fields, so you can see exactly what's accumulated before `.lower()` runs. Rust limitation: The true Coyoneda type is `โˆƒB. (B -> A, F<B>)` โ€” an existential type hiding the intermediate type `B`. Rust doesn't have native existential types, so we use trait objects (`Box<dyn Fn>`) to erase the intermediate type. This works correctly but loses some type information.

What This Unlocks

Key Differences

ConceptOCamlRust
Full Coyoneda`type ('f,'a) coyoneda = Coyoneda: ('b -> 'a) * 'f 'b -> ('f,'a) coyoneda` โ€” native existentialSimulated via `Box<dyn FnOnce() -> Vec<A>>`
Type `F`Polymorphic over any functor `F`Fixed to `Vec` (HKT limitation)
Existential `โˆƒB`Native GADT encodingType-erased via trait objects
`fmap` costO(1) โ€” composes functionsO(1) โ€” wraps closure
`lower` costO(n) โ€” one traversalO(n) โ€” one traversal
Use in ecosystemHaskell: `Data.Functor.Coyoneda`Rust iterator adapters use same principle
/// Coyoneda: The Free Functor.
///
/// Coyoneda f a = โˆƒb. (b -> a, f b)
///
/// Any type constructor `f` becomes a functor "for free" via Coyoneda,
/// even if `f` itself has no Functor instance. This is the *left* Kan
/// extension of `f` along the identity functor.
///
/// Key property: fmap via Coyoneda composes functions *without* running them.
/// Only `lower` (the "run") actually traverses the structure. This gives
/// automatic fusion: N fmaps = 1 traversal.
///
/// โš   Rust HKT limitation: we can't write `Coyoneda<F, A>` for arbitrary `F`.
/// We fix F = Vec and use a trait object for the erased intermediate type `b`.

/// Coyoneda of Vec: holds a Vec<B> (for some hidden B) and a function B->A.
/// The existential `โˆƒb` is encoded via a trait object + boxed closure.
pub struct Coyoneda<A> {
    /// The internal implementation that, when called, applies the accumulated
    /// function to the original data and returns Vec<A>.
    lower_fn: Box<dyn FnOnce() -> Vec<A>>,
}

impl<A: 'static> Coyoneda<A> {
    /// Lift a Vec<A> into Coyoneda โ€” the identity transformation.
    /// Corresponds to: `lift xs = Coyoneda id xs`
    pub fn lift(data: Vec<A>) -> Self {
        Coyoneda {
            lower_fn: Box::new(move || data),
        }
    }

    /// fmap: compose a new function WITHOUT executing it.
    /// Corresponds to: `fmap f (Coyoneda g xs) = Coyoneda (f . g) xs`
    pub fn fmap<B: 'static>(self, f: impl Fn(A) -> B + 'static) -> Coyoneda<B> {
        Coyoneda {
            lower_fn: Box::new(move || {
                let data = (self.lower_fn)();
                data.into_iter().map(f).collect()
            }),
        }
    }

    /// Lower: execute the accumulated transformation (single traversal).
    pub fn lower(self) -> Vec<A> {
        (self.lower_fn)()
    }
}

/// Alternative encoding that makes the deferred composition more explicit.
/// Stores (original_data, composed_function) as separate fields.
pub struct CoyonedaExplicit<A> {
    // We store the original data as Vec<i32> (a concrete "hidden" type B=i32)
    // and a composed function i32 -> A.
    // In full Coyoneda, B would be existentially hidden.
    original: Vec<i32>,
    transform: Box<dyn Fn(i32) -> A>,
}

impl<A: 'static> CoyonedaExplicit<A> {
    pub fn lift(data: Vec<i32>) -> CoyonedaExplicit<i32>
    where
        i32: 'static,
    {
        CoyonedaExplicit {
            original: data,
            transform: Box::new(|x| x),
        }
    }
}

impl<A: 'static> CoyonedaExplicit<A> {
    /// fmap: just composes โ€” O(1) per fmap call, not O(n)
    pub fn fmap<B: 'static>(self, f: impl Fn(A) -> B + 'static) -> CoyonedaExplicit<B> {
        let prev = self.transform;
        CoyonedaExplicit {
            original: self.original,
            transform: Box::new(move |x| f(prev(x))),
        }
    }

    /// Lower: single pass applies ALL composed functions
    pub fn lower(self) -> Vec<A> {
        self.original.into_iter().map(|x| (self.transform)(x)).collect()
    }
}

fn main() {
    println!("=== Coyoneda: The Free Functor ===\n");
    println!("Coyoneda f a = โˆƒb. (b -> a, f b)");
    println!("Any type constructor becomes a functor for free.\n");

    println!("โš   Rust HKT limitation: F is fixed to Vec; b is type-erased.");
    println!("   In OCaml/Haskell, Coyoneda works for ANY type constructor F.\n");

    let data = vec![1_i32, 2, 3, 4, 5];

    // Via Coyoneda: lift, chain fmaps, lower
    let result = Coyoneda::lift(data.clone())
        .fmap(|x| x * 2)
        .fmap(|x| x + 1)
        .fmap(|x| x.to_string())
        .lower();

    println!("Coyoneda fused maps: {:?}", result);

    // Direct computation for comparison
    let direct: Vec<String> = data.iter()
        .map(|&x| (x * 2 + 1).to_string())
        .collect();
    println!("Direct:              {:?}", direct);
    assert_eq!(result, direct, "Coyoneda must equal direct");
    println!("Results identical โœ“\n");

    // Explicit Coyoneda
    let result2 = CoyonedaExplicit::<i32>::lift(vec![10, 20, 30])
        .fmap(|x: i32| x + 5)
        .fmap(|x: i32| x * 2)
        .fmap(|x: i32| format!("val={}", x))
        .lower();
    println!("CoyonedaExplicit: {:?}", result2);

    println!();
    println!("Key property: N calls to fmap = 1 traversal when lowered.");
    println!("The functions compose: fmap f (fmap g x) = fmap (f.g) x");
    println!("This is the Coyoneda lemma: Nat(Coyoneda F, G) โ‰… Nat(F, G)");
    println!("(natural transformations out of Coyoneda = natural transformations out of F)");
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_lift_lower_roundtrip() {
        let data = vec![1, 2, 3, 4, 5];
        let result = Coyoneda::lift(data.clone()).lower();
        assert_eq!(result, data);
    }

    #[test]
    fn test_fmap_fusion() {
        let result = Coyoneda::lift(vec![1_i32, 2, 3])
            .fmap(|x| x * 10)
            .fmap(|x| x + 1)
            .lower();
        assert_eq!(result, vec![11, 21, 31]);
    }

    #[test]
    fn test_type_change() {
        let result = Coyoneda::lift(vec![1_i32, 2, 3])
            .fmap(|x| x.to_string())
            .lower();
        assert_eq!(result, vec!["1", "2", "3"]);
    }

    #[test]
    fn test_explicit_fmap() {
        let result = CoyonedaExplicit::lift(vec![1, 2, 3])
            .fmap(|x: i32| x * 2)
            .lower();
        assert_eq!(result, vec![2, 4, 6]);
    }

    #[test]
    fn test_fused_equals_composed() {
        let data = vec![1_i32, 2, 3, 4];
        let fused = Coyoneda::lift(data.clone())
            .fmap(|x| x * 3)
            .fmap(|x| x - 1)
            .lower();
        let direct: Vec<i32> = data.iter().map(|&x| x * 3 - 1).collect();
        assert_eq!(fused, direct);
    }
}
(* Coyoneda: the free functor. Wraps a value and defers the mapping.
   Coyoneda f a = exists b. f b * (b -> a)
   Any type constructor becomes a functor for free via Coyoneda. *)

(* Existential Coyoneda *)
type 'f coyoneda_inj = {
  inject: 'b 'a. ('b -> 'a) -> 'b -> 'f
}

(* Using GADT to pack existential *)
type _ coyoneda =
  | Coyoneda : ('b -> 'a) * 'b list -> 'a coyoneda

(* Constructor *)
let lift lst = Coyoneda ((fun x -> x), lst)

(* fmap: compose the function, don't traverse yet *)
let fmap f (Coyoneda (g, lst)) = Coyoneda ((fun x -> f (g x)), lst)

(* Lower: apply the accumulated function *)
let lower (Coyoneda (f, lst)) = List.map f lst

let () =
  let colist = lift [1; 2; 3; 4; 5] in

  (* Fuse three maps โ€” they compose, only one traversal at lower *)
  let result =
    colist
    |> fmap (fun x -> x * 2)
    |> fmap (fun x -> x + 1)
    |> fmap string_of_int
    |> lower
  in
  Printf.printf "Coyoneda fused: [%s]\n" (String.concat ";" result);

  (* No functor constraint needed โ€” works for any type constructor *)
  Printf.printf "Coyoneda: deferred functor mapping\n"