A new preprint is out! We find a theoretical explanation for why Transformers struggle to learn sensitive functions such as Parity and show empirical evidence supporting our reasoning. <a href="https://arxiv.org/pdf/2402.09963.pdf">arXiv link</a>