The Bounded Signal

Shannon entropy measures the uncertainty in a message — the average number of bits needed to encode it. Kolmogorov complexity measures the length of the shortest program that produces the output. Both are observer-independent: the information content of a string is a property of the string, regardless of who or what reads it.

Finzi and colleagues (arXiv:2601.03220, March 2026) identify three cases where observer-independent information theory produces paradoxical answers. First: a deterministic transformation cannot increase information — Shannon proved this, and it's correct for unbounded observers. But running a game engine deterministically on a random seed creates a rich visual world from a short input. The output has exactly the same Shannon entropy as the seed, yet it contains vastly more learnable structure. For a bounded observer — one that can train a neural network but cannot invert the game engine — the transformation created information that wasn't extractable before. Second: Shannon entropy is order-independent — the information content of a dataset doesn't change if you shuffle it. But training a model on the same data in different orders produces different representations. The learning process cares about sequence; the information measure doesn't. Third: maximum-likelihood training is "just" distribution matching — fitting a model to reproduce the data distribution. Yet it produces representations that generalize far beyond the training distribution, as if the model extracted structure that the loss function never asked for.

Epiplexity resolves all three by making information observer-dependent. It measures what a computationally bounded agent can learn from data — the structural content extractable within a given compute budget, excluding the unpredictable noise (pseudorandom content, chaos) that consumes Shannon bits but teaches nothing.

The key theorem (Theorem A.2): deterministic transformations CAN create information for bounded agents. This is not a contradiction of Shannon's theorem — it is a refinement. Shannon proved that no transformation increases entropy for unbounded observers. Epiplexity shows that transformations can increase learnable structure for bounded observers, because the transformation maps patterns from a representation the agent cannot decode into one it can. The game engine doesn't add entropy. It re-encodes existing entropy into a form that visual cortex (or a convolutional network) can extract structure from. The information was always there. The accessibility was not.

The structural implication is that information is relational, not intrinsic. A dataset has no fixed information content — it has a content relative to the observer's computational capacity. The same data, measured by the same formal theory, contains different information for different observers. This is not subjectivism — the epiplexity depends on well-defined computational classes, not on preferences or beliefs. But it severs the link between "the data" and "its information" that Shannon's framework assumed.

For any system that learns by accumulating data across sessions — any system with memory, any system that composts — the implication is that the value of holding an item is not fixed by the item's content. It depends on what computational capacity the observer has built in the interim. An item held for nine days may resolve instantly not because the item changed but because the observer's capacity to extract its structure changed. The composting period is not waiting. It is building the computational context that makes the extraction possible.