236: Coyoneda
Difficulty: 4 Level: Expert Make any type constructor act like a functor for free, with automatic map fusion as a bonus.The Problem This Solves
Suppose you have a custom type โ say a priority queue, a graph node, or a database cursor โ that doesn't implement `map`. You can't call `.map()` on it. You have to manually unpack it, transform the values, and repack. That's annoying, but manageable. Now suppose you need to apply several transformations in sequence. Each one unpacks and repacks. Worse, if the container is expensive to traverse (a lazy I/O stream, a tree with millions of nodes), you've paid the traversal cost multiple times for what is conceptually one operation. The deeper problem: you'd like to write generic code that works with "any container that can be mapped" โ but your type doesn't have a `Functor` instance. You can't use any of the generic infrastructure. Coyoneda solves both problems at once. It wraps any type constructor (whether or not it has `map`) and gives you a `fmap` that defers all work. You compose as many maps as you want โ zero traversals. Only when you call `lower()` does the container get traversed exactly once, with all transformations fused into a single function. This exists to solve exactly that pain.The Intuition
Think of Coyoneda as a "pending receipt" for a series of operations on a container. When you buy something and request a refund, the store doesn't immediately restock the item and re-issue currency. They give you a receipt that says "you are owed X." You can endorse that receipt (add conditions), give it to someone else โ all without touching the actual inventory. `Coyoneda::lift(data)` creates the receipt. Each `.fmap(f)` adds an endorsement (composes `f` into a pending function). `.lower()` is when you cash it in โ the container is traversed once with the fully composed function applied. The "free functor" part: even if your original container type has no `map`, Coyoneda wraps it and provides one. You get functor behavior for free, just by wrapping in Coyoneda.Original data: [1, 2, 3, 4, 5]
โ lift
Coyoneda: (data=[1..5], transform=identity)
โ .fmap(|x| x*2) โ no traversal yet
Coyoneda: (data=[1..5], transform=|x| x*2)
โ .fmap(|x| x+1) โ no traversal yet
Coyoneda: (data=[1..5], transform=|x| x*2+1)
โ .lower() โ ONE traversal
Result: [3, 5, 7, 9, 11]
How It Works in Rust
// Coyoneda wraps any Vec<A> and holds the pending transformation
pub struct Coyoneda<A> {
// A boxed closure: when called, applies ALL pending fmaps and returns Vec<A>
lower_fn: Box<dyn FnOnce() -> Vec<A>>,
}
impl<A: 'static> Coyoneda<A> {
// Lift: wrap plain data โ transformation is identity (do nothing yet)
pub fn lift(data: Vec<A>) -> Self {
Coyoneda {
lower_fn: Box::new(move || data), // closure owns the data
}
}
// fmap: compose new function AROUND the existing closure โ no traversal
// The key: this is O(1), regardless of how large the data is
pub fn fmap<B: 'static>(self, f: impl Fn(A) -> B + 'static) -> Coyoneda<B> {
Coyoneda {
lower_fn: Box::new(move || {
let data = (self.lower_fn)(); // get data from inner closure
data.into_iter().map(f).collect() // apply f
}),
// NOTE: the inner closure isn't called yet โ this is still deferred
}
}
// Lower: execute everything โ the one and only traversal
pub fn lower(self) -> Vec<A> {
(self.lower_fn)()
}
}
fn main() {
// Three fmaps โ but data is only traversed ONCE at .lower()
let result = Coyoneda::lift(vec![1_i32, 2, 3, 4, 5])
.fmap(|x| x * 2) // pending
.fmap(|x| x + 1) // pending
.fmap(|x| x.to_string()) // pending โ note: type changes from i32 to String
.lower(); // executed: ["3", "5", "7", "9", "11"]
}
The "explicit" version in the code makes the structure even clearer โ it stores the original data and a composed `i32 -> A` function as separate fields, so you can see exactly what's accumulated before `.lower()` runs.
Rust limitation: The true Coyoneda type is `โB. (B -> A, F<B>)` โ an existential type hiding the intermediate type `B`. Rust doesn't have native existential types, so we use trait objects (`Box<dyn Fn>`) to erase the intermediate type. This works correctly but loses some type information.
What This Unlocks
- Lazy transformation pipelines: Build a sequence of transforms on expensive containers (files, network streams, trees) and pay the traversal cost exactly once.
- Generic functor infrastructure: Wrap any type in `Coyoneda` to use it with code that requires `fmap` โ even types you don't control.
- Understanding Rust's iterator fusion: `Iterator` chains in Rust work by exactly this principle โ `.map().map().filter()` fuses into one loop because each adapter wraps the previous in a closure.
Key Differences
| Concept | OCaml | Rust |
|---|---|---|
| Full Coyoneda | `type ('f,'a) coyoneda = Coyoneda: ('b -> 'a) * 'f 'b -> ('f,'a) coyoneda` โ native existential | Simulated via `Box<dyn FnOnce() -> Vec<A>>` |
| Type `F` | Polymorphic over any functor `F` | Fixed to `Vec` (HKT limitation) |
| Existential `โB` | Native GADT encoding | Type-erased via trait objects |
| `fmap` cost | O(1) โ composes functions | O(1) โ wraps closure |
| `lower` cost | O(n) โ one traversal | O(n) โ one traversal |
| Use in ecosystem | Haskell: `Data.Functor.Coyoneda` | Rust iterator adapters use same principle |
/// Coyoneda: The Free Functor.
///
/// Coyoneda f a = โb. (b -> a, f b)
///
/// Any type constructor `f` becomes a functor "for free" via Coyoneda,
/// even if `f` itself has no Functor instance. This is the *left* Kan
/// extension of `f` along the identity functor.
///
/// Key property: fmap via Coyoneda composes functions *without* running them.
/// Only `lower` (the "run") actually traverses the structure. This gives
/// automatic fusion: N fmaps = 1 traversal.
///
/// โ Rust HKT limitation: we can't write `Coyoneda<F, A>` for arbitrary `F`.
/// We fix F = Vec and use a trait object for the erased intermediate type `b`.
/// Coyoneda of Vec: holds a Vec<B> (for some hidden B) and a function B->A.
/// The existential `โb` is encoded via a trait object + boxed closure.
pub struct Coyoneda<A> {
/// The internal implementation that, when called, applies the accumulated
/// function to the original data and returns Vec<A>.
lower_fn: Box<dyn FnOnce() -> Vec<A>>,
}
impl<A: 'static> Coyoneda<A> {
/// Lift a Vec<A> into Coyoneda โ the identity transformation.
/// Corresponds to: `lift xs = Coyoneda id xs`
pub fn lift(data: Vec<A>) -> Self {
Coyoneda {
lower_fn: Box::new(move || data),
}
}
/// fmap: compose a new function WITHOUT executing it.
/// Corresponds to: `fmap f (Coyoneda g xs) = Coyoneda (f . g) xs`
pub fn fmap<B: 'static>(self, f: impl Fn(A) -> B + 'static) -> Coyoneda<B> {
Coyoneda {
lower_fn: Box::new(move || {
let data = (self.lower_fn)();
data.into_iter().map(f).collect()
}),
}
}
/// Lower: execute the accumulated transformation (single traversal).
pub fn lower(self) -> Vec<A> {
(self.lower_fn)()
}
}
/// Alternative encoding that makes the deferred composition more explicit.
/// Stores (original_data, composed_function) as separate fields.
pub struct CoyonedaExplicit<A> {
// We store the original data as Vec<i32> (a concrete "hidden" type B=i32)
// and a composed function i32 -> A.
// In full Coyoneda, B would be existentially hidden.
original: Vec<i32>,
transform: Box<dyn Fn(i32) -> A>,
}
impl<A: 'static> CoyonedaExplicit<A> {
pub fn lift(data: Vec<i32>) -> CoyonedaExplicit<i32>
where
i32: 'static,
{
CoyonedaExplicit {
original: data,
transform: Box::new(|x| x),
}
}
}
impl<A: 'static> CoyonedaExplicit<A> {
/// fmap: just composes โ O(1) per fmap call, not O(n)
pub fn fmap<B: 'static>(self, f: impl Fn(A) -> B + 'static) -> CoyonedaExplicit<B> {
let prev = self.transform;
CoyonedaExplicit {
original: self.original,
transform: Box::new(move |x| f(prev(x))),
}
}
/// Lower: single pass applies ALL composed functions
pub fn lower(self) -> Vec<A> {
self.original.into_iter().map(|x| (self.transform)(x)).collect()
}
}
fn main() {
println!("=== Coyoneda: The Free Functor ===\n");
println!("Coyoneda f a = โb. (b -> a, f b)");
println!("Any type constructor becomes a functor for free.\n");
println!("โ Rust HKT limitation: F is fixed to Vec; b is type-erased.");
println!(" In OCaml/Haskell, Coyoneda works for ANY type constructor F.\n");
let data = vec![1_i32, 2, 3, 4, 5];
// Via Coyoneda: lift, chain fmaps, lower
let result = Coyoneda::lift(data.clone())
.fmap(|x| x * 2)
.fmap(|x| x + 1)
.fmap(|x| x.to_string())
.lower();
println!("Coyoneda fused maps: {:?}", result);
// Direct computation for comparison
let direct: Vec<String> = data.iter()
.map(|&x| (x * 2 + 1).to_string())
.collect();
println!("Direct: {:?}", direct);
assert_eq!(result, direct, "Coyoneda must equal direct");
println!("Results identical โ\n");
// Explicit Coyoneda
let result2 = CoyonedaExplicit::<i32>::lift(vec![10, 20, 30])
.fmap(|x: i32| x + 5)
.fmap(|x: i32| x * 2)
.fmap(|x: i32| format!("val={}", x))
.lower();
println!("CoyonedaExplicit: {:?}", result2);
println!();
println!("Key property: N calls to fmap = 1 traversal when lowered.");
println!("The functions compose: fmap f (fmap g x) = fmap (f.g) x");
println!("This is the Coyoneda lemma: Nat(Coyoneda F, G) โ
Nat(F, G)");
println!("(natural transformations out of Coyoneda = natural transformations out of F)");
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_lift_lower_roundtrip() {
let data = vec![1, 2, 3, 4, 5];
let result = Coyoneda::lift(data.clone()).lower();
assert_eq!(result, data);
}
#[test]
fn test_fmap_fusion() {
let result = Coyoneda::lift(vec![1_i32, 2, 3])
.fmap(|x| x * 10)
.fmap(|x| x + 1)
.lower();
assert_eq!(result, vec![11, 21, 31]);
}
#[test]
fn test_type_change() {
let result = Coyoneda::lift(vec![1_i32, 2, 3])
.fmap(|x| x.to_string())
.lower();
assert_eq!(result, vec!["1", "2", "3"]);
}
#[test]
fn test_explicit_fmap() {
let result = CoyonedaExplicit::lift(vec![1, 2, 3])
.fmap(|x: i32| x * 2)
.lower();
assert_eq!(result, vec![2, 4, 6]);
}
#[test]
fn test_fused_equals_composed() {
let data = vec![1_i32, 2, 3, 4];
let fused = Coyoneda::lift(data.clone())
.fmap(|x| x * 3)
.fmap(|x| x - 1)
.lower();
let direct: Vec<i32> = data.iter().map(|&x| x * 3 - 1).collect();
assert_eq!(fused, direct);
}
}
(* Coyoneda: the free functor. Wraps a value and defers the mapping.
Coyoneda f a = exists b. f b * (b -> a)
Any type constructor becomes a functor for free via Coyoneda. *)
(* Existential Coyoneda *)
type 'f coyoneda_inj = {
inject: 'b 'a. ('b -> 'a) -> 'b -> 'f
}
(* Using GADT to pack existential *)
type _ coyoneda =
| Coyoneda : ('b -> 'a) * 'b list -> 'a coyoneda
(* Constructor *)
let lift lst = Coyoneda ((fun x -> x), lst)
(* fmap: compose the function, don't traverse yet *)
let fmap f (Coyoneda (g, lst)) = Coyoneda ((fun x -> f (g x)), lst)
(* Lower: apply the accumulated function *)
let lower (Coyoneda (f, lst)) = List.map f lst
let () =
let colist = lift [1; 2; 3; 4; 5] in
(* Fuse three maps โ they compose, only one traversal at lower *)
let result =
colist
|> fmap (fun x -> x * 2)
|> fmap (fun x -> x + 1)
|> fmap string_of_int
|> lower
in
Printf.printf "Coyoneda fused: [%s]\n" (String.concat ";" result);
(* No functor constraint needed โ works for any type constructor *)
Printf.printf "Coyoneda: deferred functor mapping\n"