Pinned

1. attention_with_linear_biases attention_with_linear_biases Public

   Code for the ALiBi method for transformer language models (ICLR 2022)

   Python 467 35

2. self-ask self-ask Public

   Code and data for "Measuring and Narrowing the Compositionality Gap in Language Models"

   Jupyter Notebook 278 30

3. shortformer shortformer Public

   Code for the Shortformer model, from the ACL 2021 paper by Ofir Press, Noah A. Smith and Mike Lewis.

   Python 146 8

4. YouMayNotNeedAttention YouMayNotNeedAttention Public

   Code for the Eager Translation Model from the paper You May Not Need Attention

   Python 292 29

5. sandwich_transformer sandwich_transformer Public

   This repository contains the code for running the character-level Sandwich Transformers from our ACL 2020 paper on Improving Transformer Models by Reordering their Sublayers.

   Python 55 2

6. UsingTheOutputEmbedding UsingTheOutputEmbedding Public

   Code for the EACL paper "Using the Output Embedding to Improve Language Models" by Ofir Press and Lior Wolf

   Lua 44 7