As part of my master's thesis on optimizing LLMs for code completion in OpenAPI format, I implemented a script for fine-tuning Code Llama for Copilot-style code completion. Surprisingly, I was not able to find a similar implementation before since most works focus on instruction fine-tuning for chat-like interaction.<p>Related publication: <a href="https://arxiv.org/abs/2405.15729" rel="nofollow">https://arxiv.org/abs/2405.15729</a>
Code Llama fill-in-the-middle fine-tuning: <a href="https://github.com/BohdanPetryshyn/code-llama-fim-fine-tuning">https://github.com/BohdanPetryshyn/code-llama-fim-fine-tunin...</a>
OpenAPI completion benchmark: <a href="https://github.com/BohdanPetryshyn/openapi-completion-benchmark">https://github.com/BohdanPetryshyn/openapi-completion-benchm...</a>