Generative self-supervised model for robotics data.
Highlights:
- Transformer model for autoregressive prediction of states and actions over time to implicitly encode dynamics and behaviors for a particular robot
- The representation learned can be fine-tuned to distinct tasks (navigation, mapping, localization) with minimal data
- The base transformer model serves as a generative robotics model, similarly to GPT-3, and can be prompted to result in different robot behaviors<p>Paper: <a href="https://arxiv.org/abs/2209.11133;" rel="nofollow">https://arxiv.org/abs/2209.11133;</a>
Video: <a href="https://youtu.be/mNQvQu_atuw;" rel="nofollow">https://youtu.be/mNQvQu_atuw;</a>
Code and data: <a href="https://github.com/microsoft/PACT" rel="nofollow">https://github.com/microsoft/PACT</a>