ASAPP - Figure 2: Dev BPC on Enwik8 dataset vs GPU hours used for training. The SRU++ and Transformer-XL model both have 41-42M parameters and are trained with fp32 precision and comparable settings (such as learning rate).
{}