FirsthandTech
arXiv: cs.AI preprintsInternational20 November 2026

Toaster: Nonlinear Memory from Cascaded Linear Memories and Target Propagation

This is an official announcement record

Firsthand records what arXiv: cs.AI preprints announced and links to the original. The wording below is theirs, not ours.

arXiv:2611.23648v1 Announce Type: cross Abstract: Recurrent memories let language models process long sequences with a fixed-size state. Linear memories such as DeltaNet and KDA store one matrix and take one delta-rule step per token. The product of a chunk of such steps has an exact WY form, so they train fast. Deep memories such as TTT-MLP and Titans store a small MLP, which can hold maps that no matrix can, but their update has no WY form. Toaster bridges the two by building on a known fact: a gradient step on a linear layer is a delta-rule write. It builds a two-layer nonlinear memory from
Source: arXiv: cs.AI preprints

More from arXiv: cs.AI preprints

This content is for informational purposes only and is not professional advice. Specifications, prices, plan tiers, and features change frequently and may differ from what is shown here; verify current details on the manufacturer's or company's official page before purchasing. Ratings are based on analysis of published documentation, not independent lab testing.