About Me
Hi!
I’m Mark, a Master’s student in Language Science and Technology at Saarland University.
I work as a research assistant at Language, Computation and Cognition Lab. We use methods ranging from Formal Language Theory to Mechanistic Interpretability to look into the black boxes of modern LLMs.
My primary research interest is building the high-level intuition of LLMs, as well as applying that intuition to get actionable insights. I believe that by combining theoretical and empirical methods, we can bit-by-bit construct a Theory of Everything for LLMs, uncovering the bounds of learnability, role of different layers, dynamics of information flow, and other phenomena. As training new models becomes increasingly expensive, this research will only grow in significance, potentially saving billions in compute costs.
News
August 2024 — Honored and excited to receive ACL 2024 best paper award for "Why are Sensitive Functions Hard for Transformers?"!
May 2024 — Our paper "Why are Sensitive Functions Hard for Transformers?" was accepted to ACL 2024! My talk about it at FLaNN seminar: link. Twitter thread: link.
April 2024 — Had a great time at ALPS 2024 winter school! Magnificent mountains, great lectures, and insightful discussions.
February 2024 — A new preprint is out! We find a theoretical explanation for why Transformers struggle to learn sensitive functions such as Parity and show empirical evidence supporting our reasoning. arXiv link
November 2023 — I am joining Department of Language Science and Technology at Saarland University as a research assistant! I will be working on LLM interpretability under the supervision of Michael Hahn.
October 2023 — I have arrived to Saarbrücken to start my Master's studies at Saarland University. Extremely happy to finally be here!
June 2023 — Today I have successfully defended my Bachelor Thesis! Now I officially hold a B.S. in Computer Science.
May 2023 — I took part in EACL 2023, located in scenic Dubrovnik, Croatia. A big thanks to all the organizers for their excellent work!
January 2023 — Excited to share that our paper "Vote’n’Rank: Revision of Benchmarking with Social Choice Theory" got accepted to EACL 2023!
November 2022 — This month, I have started working as a Large Language Model Developer at Yandex.