Personal Tensor Memory
Article Sidebar
Main Article Content
Abstract
Large language models (LLMs) excel at general knowledge but struggle when they must remember the preferences, profile facts, and long-term context of a specific user—especially on constrained devices. We introduce Personal Tensor Memory (PTM), a privacy-preserving add-on that assigns every user a fixed-shape matrix, which the frozen backbone can query through one additional attention head. A nightly routine— Hebbian add + decay, norm clipping, slot merge/evict, and occasional orthogonal rotation—re‑organises information inside that matrix without changing its shape or touching billions of backbone weights. On synthetic concept‑drift streams and anonymised personal‑assistant logs, PTM matches kNN‑LM perplexity while needing only 5 % of its context window, and surpasses rank‑8 LoRA under few‑shot data—all using < 8 MB per user and < 1 s daily CPU on a smartphone.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
How to Cite
References
Hu, E. et al. “LoRA: Low‑Rank Adaptation of Large Language Models.” ICLR 2022. DOI: http://doi.org/10.48550/arXiv.2106.09685
Khandelwal, U. et al. “Generalization through Memorization: Nearest Neighbor Language Models.” ICLR 2020.
DOI: http://doi.org/10.48550/arXiv.1911.00172
Guo, D. et al. “GraphCodeBERT: Pre‑Training Code Representations with Data Flow.” ICLR 2021. DOI: http://doi.org/10.48550/arXiv.2009.08366
Zaken, E. B. et al. “BitFit: Simple Parameter‑Efficient Fine‑Tuning for Transformer‑Based Masked Language‑Models.” arXiv 2021.
DOI: http://doi.org/10.48550/arXiv.2106.10199
Munkhdalai, T., Trischler, A. “Meta Networks.” ICML 2017. DOI: http://doi.org/10.48550/arXiv.1703.00837
Ghorpade, M. et al. “Efficient Low‑Rank Adaptation via Randomized SVD.” arXiv 2023 . DOI: http://doi.org/10.48550/arXiv.2306.06029
Chu, X. & Zaniolo, C. “Selective and Efficient Reservoir Sampling for Data Streams.” IEEE TKDE 2020.
DOI: http://doi.org/10.1109/TKDE.2020.2988027
Ramasesh, V. et al. “An Embedding View of Continual Learning.” NeurIPS 2021. DOI: http://doi.org/10.48550/arXiv.2102.06253