Publications

This page contains all my publications; for more details, see my Google Scholar profile.

Published Papers

Joseph L. Watson , David Juergens , Nathaniel R. Bennett , Brian L. Trippe , Jason Yim , Helen E. Eisenach , Woody Ahern , Andrew J. Borst , Robert J. Ragotte , Lukas F. Milles , Basile I. M. Wicky , Nikita Hanikel , Samuel J. Pellock , Alexis Courbet , William Sheffler , Jue Wang , Preetham Venkatesh , Isaac Sappington , Susana Vázquez Torres , Anna Lauko , Valentin De Bortoli , Emile Mathieu , Sergey Ovchinnikov , Regina Barzilay , Tommi S. Jaakkola , Frank DiMaio , Minkyung Baek , David Baker , De Novo Design of Protein Structure and Function with RFdiffusion, Nature, 1–3, Jul. 2023.
```
@article{watson2023novo,
  title = {De Novo Design of Protein Structure and Function with {{RFdiffusion}}},
  author = {Watson, Joseph L. and Juergens, David and Bennett, Nathaniel R. and Trippe, Brian L. and Yim, Jason and Eisenach, Helen E. and Ahern, Woody and Borst, Andrew J. and Ragotte, Robert J. and Milles, Lukas F. and Wicky, Basile I. M. and Hanikel, Nikita and Pellock, Samuel J. and Courbet, Alexis and Sheffler, William and Wang, Jue and Venkatesh, Preetham and Sappington, Isaac and Torres, Susana V{\'a}zquez and Lauko, Anna and De Bortoli, Valentin and Mathieu, Emile and Ovchinnikov, Sergey and Barzilay, Regina and Jaakkola, Tommi S. and DiMaio, Frank and Baek, Minkyung and Baker, David},
  year = {2023},
  month = jul,
  journal = {Nature},
  pages = {1--3},
  publisher = {{Nature Publishing Group}},
  issn = {1476-4687},
  doi = {10.1038/s41586-023-06415-8},
  copyright = {2023 The Author(s), under exclusive licence to Springer Nature Limited},
  langid = {english}
}
```

Emile Mathieu , Vincent Dutordoir , Michael J. Hutchinson , Valentin De Bortoli , Yee Whye Teh , Richard E. Turner , Geometric Neural Diffusion Processes, no. arXiv:2307.05431. arXiv, Jul-2023.

@misc{mathieu2023Geometric,
  title = {Geometric {{Neural Diffusion Processes}}},
  author = {Mathieu, Emile and Dutordoir, Vincent and Hutchinson, Michael J. and De Bortoli, Valentin and Teh, Yee Whye and Turner, Richard E.},
  year = {2023},
  month = jul,
  number = {arXiv:2307.05431},
  primaryclass = {cs, stat},
  publisher = {{arXiv}},
  doi = {10.48550/arXiv.2307.05431}
}

Nic Fishman , Leo Klarner , Emile Mathieu , Michael Hutchinson , Valentin de Bortoli , Metropolis Sampling for Constrained Diffusion Models, no. arXiv:2307.05439. arXiv, Jul-2023.

@misc{fishman2023Metropolis,
  title = {Metropolis {{Sampling}} for {{Constrained Diffusion Models}}},
  author = {Fishman, Nic and Klarner, Leo and Mathieu, Emile and Hutchinson, Michael and {de Bortoli}, Valentin},
  year = {2023},
  month = jul,
  number = {arXiv:2307.05439},
  primaryclass = {cs},
  publisher = {{arXiv}},
  doi = {10.48550/arXiv.2307.05439}
}

Jason Yim , Brian L. Trippe , Valentin De Bortoli , Emile Mathieu , Arnaud Doucet , Regina Barzilay , Tommi Jaakkola , SE(3) Diffusion Model with Application to Protein Backbone Generation, in Proceedings of the 40th International Conference on Machine Learning, 2023.
```
@inproceedings{yim2023SE,
  title = {{{SE}}(3) Diffusion Model with Application to Protein Backbone Generation},
  author = {Yim, Jason and Trippe, Brian L. and De Bortoli, Valentin and Mathieu, Emile and Doucet, Arnaud and Barzilay, Regina and Jaakkola, Tommi},
  year = {2023},
  booktitle = {Proceedings of the 40th International Conference on Machine Learning},
  langid = {english}
}
```

Nic Fishman , Leo Klarner , Valentin De Bortoli , Emile Mathieu , Michael Hutchinson , Diffusion Models for Constrained Domains, Transactions on Machine Learning Research, 2023.

@article{fishman2023Diffusion,
  title = {Diffusion Models for Constrained Domains},
  author = {Fishman, Nic and Klarner, Leo and De Bortoli, Valentin and Mathieu, Emile and Hutchinson, Michael},
  journal = {Transactions on Machine Learning Research},
  year = {2023},
  note = {Under review}
}

Ning Miao , Tom Rainforth , Emile Mathieu , Yann Dubois , Yee Whye Teh , Adam Foster , Hyunjik Kim , Learning Instance-Specific Augmentations by Capturing Local Invariances, in Proceedings of the 40th International Conference on Machine Learning, 2023.
```
@inproceedings{miao2023Learning,
  title = {Learning {{Instance-Specific Augmentations}} by {{Capturing Local Invariances}}},
  author = {Miao, Ning and Rainforth, Tom and Mathieu, Emile and Dubois, Yann and Teh, Yee Whye and Foster, Adam and Kim, Hyunjik},
  booktitle = {Proceedings of the 40th International Conference on Machine Learning},
  year = {2023},
  langid = {english}
}
```

Angus Phillips , Thomas Seror , Michael Hutchinson , Valentin De Bortoli , Arnaud Doucet , Emile Mathieu , Spectral Diffusion Processes, no. arXiv:2209.14125. arXiv, Nov-2022.

@misc{phillips2022Spectral,
  title = {Spectral {{Diffusion Processes}}},
  author = {Phillips, Angus and Seror, Thomas and Hutchinson, Michael and De Bortoli, Valentin and Doucet, Arnaud and Mathieu, Emile},
  year = {2022},
  month = nov,
  number = {arXiv:2209.14125},
  primaryclass = {cs, stat},
  publisher = {{arXiv}}
}

Valentin De Bortoli , Emile Mathieu , Michael John Hutchinson , James Thornton , Yee Whye Teh , Arnaud Doucet , Riemannian Score-Based Generative Modelling, in Advances in Neural Information Processing Systems, 2022.

@inproceedings{debortoli2022Riemannian,
  title = {Riemannian Score-Based Generative Modelling},
  author = {Bortoli, Valentin De and Mathieu, Emile and Hutchinson, Michael John and Thornton, James and Teh, Yee Whye and Doucet, Arnaud},
  booktitle = {Advances in Neural Information Processing Systems},
  editor = {Oh, Alice H. and Agarwal, Alekh and Belgrave, Danielle and Cho, Kyunghyun},
  year = {2022},
  outstanding = {https://blog.neurips.cc/2022/11/21/announcing-the-neurips-2022-awards/}
}

Ning Miao , Emile Mathieu , Siddharth N , Yee Whye Teh , Tom Rainforth , On Incorporating Inductive Biases into VAEs, in Tenth International Conference on Learning Representations, 2022.
```
@inproceedings{miao2022incorporating,
  title = {On Incorporating Inductive Biases into {{VAEs}}},
  booktitle = {Tenth International Conference on Learning Representations},
  author = {Miao, Ning and Mathieu, Emile and N, Siddharth and Teh, Yee Whye and Rainforth, Tom},
  year = {2022}
}
```
James Thornton , Michael Hutchinson , Emile Mathieu , Valentin De Bortoli , Yee Whye Teh , Arnaud Doucet , Riemannian Diffusion Schrödinger Bridge, in ICML Workshop on Continuous Time Methods for Machine Learning, 2022.
```
@inproceedings{thornton2022riemannian,
  title = {Riemannian Diffusion Schr\"odinger Bridge},
  booktitle = {ICML Workshop on Continuous Time Methods for Machine Learning},
  author = {Thornton, James and Hutchinson, Michael and Mathieu, Emile and De Bortoli, Valentin and Teh, Yee Whye and Doucet, Arnaud},
  year = {2022}
}
```

Rodrigo Oyanedel , Stefan Gelcich , Emile Mathieu , E. J. Milner-Gulland , A Dynamic Simulation Model to Support Reduction in Illegal Trade within Legal Wildlife Markets, Conservation Biology, Aug. 2021.

@article{oyanedel2021dynamic,
  title = {A Dynamic Simulation Model to Support Reduction in Illegal Trade within Legal Wildlife Markets},
  author = {Oyanedel, Rodrigo and Gelcich, Stefan and Mathieu, Emile and {Milner-Gulland}, E. J.},
  year = {2021},
  month = aug,
  journal = {Conservation Biology},
  publisher = {{John Wiley \& Sons, Ltd}},
  issn = {0888-8892},
  doi = {10.1111/cobi.13814}
}

Emile Mathieu , Adam Foster , Yee Whye Teh , On Contrastive Representations of Stochastic Processes, in Advances in Neural Information Processing Systems 33, 2021.
Learning representations of stochastic processes is an emerging problem in machine learning with applications from meta-learning to physical object models to time series. Typical methods rely on exact reconstruction of observations, but this approach breaks down as observations become high-dimensional or noise distributions become complex. To address this, we propose a unifying framework for learning contrastive representations of stochastic processes (CReSP) that does away with exact reconstruction. We dissect potential use cases for stochastic process representations, and propose methods that accommodate each. Empirically, we show that our methods are effective for learning representations of periodic functions, 3D objects and dynamical processes. Our methods tolerate noisy high-dimensional observations better than traditional approaches, and the learned representations transfer to a range of downstream tasks.
```
@inproceedings{mathieu2021Contrastive,
  title = {On Contrastive Representations of Stochastic Processes},
  author = {Mathieu, Emile and Foster, Adam and Teh, Yee Whye},
  booktitle = {Advances in Neural Information Processing Systems 33},
  year = {2021},
  publisher = {Curran Associates, Inc.}
}
```
Emile Mathieu , Maximilian Nickel , Riemannian Continuous Normalizing Flows, in Advances in Neural Information Processing Systems 33, 2020, 2503–2515.
Normalizing flows have shown great promise for modelling flexible probability distributions in a computationally tractable way. However, whilst data is often naturally described on Riemannian manifolds such as spheres, torii, and hyperbolic spaces, most normalizing flows implicitly assume a flat geometry, making them either misspecified or ill-suited in these situations. To overcome this problem, we introduce Riemannian continuous normalizing flows, a model which admits the parametrization of flexible probability measures on smooth manifolds by defining flows as the solution to ordinary differential equations. We show that this approach can lead to substantial improvements on both synthetic and real-world data when compared to standard flows or previously introduced projected flows.
```
@inproceedings{mathieu2019Riemannian,
  title = {Riemannian Continuous Normalizing Flows},
  author = {Mathieu, Emile and Nickel, Maximilian},
  booktitle = {Advances in Neural Information Processing Systems 33},
  year = {2020},
  pages = {2503--2515},
  publisher = {Curran Associates, Inc.}
}
```
Emile Mathieu , Tom Rainforth , N Siddharth , Yee Whye Teh , Disentangling Disentanglement in Variational Autoencoders, https://icml.cc/media/Slides/icml/2019/halla(12-11-00)-12-11-35-4811-disentangling_d.pdf, in Proceedings of the 36th International Conference on Machine Learning, Long Beach, California, USA, 2019, vol. 97, 4402–4412.
We develop a generalisation of disentanglement in VAEs—decomposition of the latent representation—characterising it as the fulfilment of two factors: a) the latent encodings of the data having an appropriate level of overlap, and b) the aggregate encoding of the data conforming to a desired structure, represented through the prior. Decomposition permits disentanglement, i.e. explicit independence between latents, as a special case, but also allows for a much richer class of properties to be imposed on the learnt representation, such as sparsity, clustering, independent subspaces, or even intricate hierarchical dependency relationships. We show that the β-VAE varies from the standard VAE predominantly in its control of latent overlap and that for the standard choice of an isotropic Gaussian prior, its objective is invariant to rotations of the latent representation. Viewed from the decomposition perspective, breaking this invariance with simple manipulations of the prior can yield better disentanglement with little or no detriment to reconstructions. We further demonstrate how other choices of prior can assist in producing different decompositions and introduce an alternative training objective that allows the control of both decomposition factors in a principled manner.
```
@inproceedings{mathieu2019Disentaling,
  title = {Disentangling Disentanglement in Variational Autoencoders},
  author = {Mathieu, Emile and Rainforth, Tom and Siddharth, N and Teh, Yee Whye},
  booktitle = {Proceedings of the 36th International Conference on Machine Learning},
  pages = {4402--4412},
  year = {2019},
  volume = {97},
  series = {Proceedings of Machine Learning Research},
  address = {Long Beach, California, USA},
  month = {09--15 Jun},
  publisher = {PMLR},
  oral = {https://icml.cc/media/Slides/icml/2019/halla(12-11-00)-12-11-35-4811-disentangling_d.pdf}
}
```
Emile Mathieu , Charline Le Lan , Chris J. Maddison , Ryota Tomioka , Yee Whye Teh , Continuous Hierarchical Representations with Poincaré Variational Auto-Encoders, in Advances in Neural Information Processing Systems 32, 2019, 12565–12576.
The Variational Auto-Encoder (VAE) is a popular method for learning a generative model and embeddings of the data. Many real datasets are hierarchically structured. However, traditional VAEs map data in a Euclidean latent space which cannot efficiently embed tree-like structures. Hyperbolic spaces with negative curvature can. We therefore endow VAEs with a Poincaré ball model of hyperbolic geometry as a latent space and rigorously derive the necessary methods to work with two main Gaussian generalisations on that space. We empirically show better generalisation to unseen data than the Euclidean counterpart, and can qualitatively and quantitatively better recover hierarchical structures.
```
@inproceedings{mathieu2019Continuous,
  title = {Continuous Hierarchical Representations with Poincar\'{e} Variational Auto-Encoders},
  author = {Mathieu, Emile and Le Lan, Charline and Maddison, Chris J. and Tomioka, Ryota and Teh, Yee Whye},
  booktitle = {Advances in Neural Information Processing Systems 32},
  pages = {12565--12576},
  year = {2019},
  publisher = {Curran Associates, Inc.}
}
```
Benjamin Bloem-Reddy , Adam Foster , Emile Mathieu , Yee Whye Teh , Sampling and Inference for Beta Neutral-to-the-Left Models of Sparse Networks, https://www.youtube.com/watch?v=0PlIFXBpIgU, in Conference on Uncertainty in Artificial Intelligence, 2018.
Empirical evidence suggests that heavy-tailed degree distributions occurring in many real networks are well-approximated by power laws with exponents η that may take values either less than and greater than two. Models based on various forms of exchangeability are able to capture power laws with η<2, and admit tractable inference algorithms; we draw on previous results to show that η>2 cannot be generated by the forms of exchangeability used in existing random graph models. Preferential attachment models generate power law exponents greater than two, but have been of limited use as statistical models due to the inherent difficulty of performing inference in non-exchangeable models. Motivated by this gap, we design and implement inference algorithms for a recently proposed class of models that generates ηof all possible values. We show that although they are not exchangeable, these models have probabilistic structure amenable to inference. Our methods make a large class of previously intractable models useful for statistical inference.
```
@inproceedings{BloemReddy2018Sampling,
  author = {Bloem-Reddy, Benjamin and Foster, Adam and Mathieu, Emile and Teh, Yee Whye},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  title = {Sampling and Inference for Beta Neutral-to-the-Left Models of Sparse Networks},
  month = aug,
  year = {2018},
  oral = {https://www.youtube.com/watch?v=0PlIFXBpIgU}
}
```
Benjamin Bloem-Reddy , Emile Mathieu , Adam Foster , Tom Rainforth , Hong Ge , María Lomelí , Zoubin Ghahramani , Yee Whye Teh , Sampling and inference for discrete random probability measures in probabilistic programs, in NIPS Workshop on Advances in Approximate Bayesian Inference, 2017.
We consider the problem of sampling a sequence from a discrete random probability measure (RPM) with countable support, under (probabilistic) constraints of finite memory and computation. A canonical example is sampling from the Dirichlet Process, which can be accomplished using its stick-breaking representation and lazy initialization of its atoms. We show that efficiently lazy initialization is possible if and only if a size-biased representation of the discrete RPM is used. For models constructed from such discrete RPMs, we consider the implications for generic particle-based inference methods in probabilistic programming systems. To demonstrate, we implement SMC for Normalized Inverse Gaussian Process mixture models in Turing.
```
@inproceedings{bloemreddy2017Sampling,
  title = {Sampling and inference for discrete random probability measures in probabilistic programs},
  author = {Bloem-Reddy, Benjamin and Mathieu, Emile and Foster, Adam and Rainforth, Tom and Ge, Hong and Lomelí, María and Ghahramani, Zoubin and Teh, Yee Whye},
  journal = {NIPS Workshop on Advances in Approximate Bayesian Inference},
  year = {2017}
}
```