Publications

On the Representational Capacity of Neural Language Models with Chain-of-Thought Reasoning

Franz Nowak*, Anej Svete*, Alexandra Butoi, and Ryan Cotterell. 2024. On the Representational Capacity of Neural Language Models with Chain-of-Thought Reasoning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Bangkok, Thailand.

What Languages are Easy to Language-Model? A Perspective from Learning Probabilistic Regular Languages

Nadav Borenstein, Anej Svete, Robin Shing Moon Chan, Josef Valvoda, Franz Nowak, Isabelle Augenstein, Eleanor Chodroff, and Ryan Cotterell. 2024. What languages are easy to language-model? a perspective from learning probabilistic regular languages. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Bangkok, Thailand.

Lower Bounds on the Expressivity of Recurrent Neural Language Models

This paper shows how probabilistic finite state machines can be encoded in RNN language models.

Anej Svete*, Franz Nowak*, Anisha Mohamed Sahabdeen, and Ryan Cotterell. 2024. Lower bounds on the expressivity of recurrent neural language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics, Mexico City, Mexico.

On the Representational Capacity of Recurrent Neural Language Models

An investigation into the formal languages and types of distributions that can be expressed by language models based on RNNs under different assumptions.

Franz Nowak, Anej Svete, Li Du, and Ryan Cotterell. 2023. On the Representational Capacity of Recurrent Neural Language Models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), Singapore.

A Fast Algorithm for Computing Prefix Probabilities

This paper proposes a speedup of a classic algorithm for computing prefix probabilities under a probabilistic context-free grammar (PCFG).

Franz Nowak and Ryan Cotterell. 2023. A Fast Algorithm for Computing Prefix Probabilities. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), Toronto, Canada.