About Me
Hi!
I’m Mark, a Master’s student in Language Science and Technology at Saarland University.
I work as a research assistant at Language, Computation and Cognition Lab. We use methods ranging from Formal Language Theory to Mechanistic Interpretability to look into the black boxes of modern LLMs.
My primary research interest is understanding the principles governing the behavior and performance limits of Deep Learning models, especially LLMs. What capabilities can they learn from data, and how does that happen? What inductive biases are introduced by specific architectural modifications? I believe that by combining theoretical and empirical methods, we can answer these questions, which is both interesting by itself and useful to improve our models.
News
February 2025 — In our new preprint, we study how long chains of thought need to be for transformers to solve different algorithmic problems.
August 2024 — Honored and excited to receive ACL 2024 best paper award for "Why are Sensitive Functions Hard for Transformers?"!
May 2024 — Our paper "Why are Sensitive Functions Hard for Transformers?" was accepted to ACL 2024! My talk about it at FLaNN seminar: link. Twitter thread: link.
April 2024 — Had a great time at ALPS 2024 winter school! Magnificent mountains, great lectures, and insightful discussions.
February 2024 — A new preprint is out! We find a theoretical explanation for why Transformers struggle to learn sensitive functions such as Parity and show empirical evidence supporting our reasoning. arXiv link
November 2023 — I am joining Department of Language Science and Technology at Saarland University as a research assistant! I will be working on LLM interpretability under the supervision of Michael Hahn.
October 2023 — I have arrived to Saarbrücken to start my Master's studies at Saarland University. Extremely happy to finally be here!
June 2023 — Today I have successfully defended my Bachelor Thesis! Now I officially hold a B.S. in Computer Science.
May 2023 — I took part in EACL 2023, located in scenic Dubrovnik, Croatia. A big thanks to all the organizers for their excellent work!
January 2023 — Excited to share that our paper "Vote’n’Rank: Revision of Benchmarking with Social Choice Theory" got accepted to EACL 2023!
November 2022 — This month, I have started working as a Large Language Model Developer at Yandex.