1 pointsby mysteryboxabout 2 years ago

2 comments

mysteryboxabout 2 years ago

I found this interesting link in paper, so this can be used as ready code for RMT Transformer<p><a href="https://github.com/booydar/t5-experiments/tree/scaling-report/rmt_utils">https://github.com/booydar/t5-experiments/tree/scaling-repor...</a>

mysteryboxabout 2 years ago

Finetuning:<p><a href="https://github.com/booydar/t5-experiments/blob/scaling-report/run_finetuning_babilong_rmt.py">https://github.com/booydar/t5-experiments/blob/scaling-repor...</a>

Code for Scaling Transformer to 1M tokens and beyond with RMT (arxiv.org)

2 comments

Code for Scaling Transformer to 1M tokens and beyond with RMT (arxiv.org)

2 comments