This is clever but seems to work through brute force and my guess is that results should be accompanied by some kind of confidence measure. You're forcing the model to choose a path it wasn't particularly interested in going down.<p>In the ideal case, such as when you ask for a json array, the first "[" token you enforce will have a relatively high probability and forcing it to go down that path will give you good results.<p>In the dangerous case the model doesn't have a good structure-compliant completion to your prompt and the regex you supply forces the model into a path of extremely low probability and you get trash results.
Is similar to this work on constrained text generation:<p><a href="https://github.com/hellisotherpeople/constrained-text-generation-studio">https://github.com/hellisotherpeople/constrained-text-genera...</a>
This seems extremely similar to <a href="https://github.com/newhouseb/clownfish">https://github.com/newhouseb/clownfish</a>
How’s this compared to ReLM:
<a href="https://arxiv.org/abs/2211.15458" rel="nofollow">https://arxiv.org/abs/2211.15458</a>