Salesforce releases language model bigger than GPT-2 large

I am working on a guide (should be released tomorrow) to easily get it up and running for personal use. Here&#x27;s my Twitter thread of current experiments with the model: <a href="https:&#x2F;&#x2F;twitter.com&#x2F;minimaxir&#x2F;status&#x2F;1173081315177975810" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;minimaxir&#x2F;status&#x2F;1173081315177975810</a><p>I recommend reading the linked paper in the repo as it gives decent examples&#x2F;instructions on how to use the model. Although the size and architecture is comparable to GPT-2, the emphasis on conditional generation differentiates it.

Wow, that&#x27;s some license addendum:<p>&gt; This software should not be used to promote or profit from:<p>&gt; violence, hate, and division,<p>&gt; environmental destruction,<p>&gt; abuse of human rights, or<p>&gt; the destruction of people&#x27;s physical and mental health.

Anyone have a real-world use case for something like this? I must admit I&#x27;m having trouble thinking of any that aren&#x27;t essentially deceptive. Because in my little biased world, I have no need of &quot;text&quot; per se, and what value any text has to me is closely linked to the fact that it came from a human.

From the blog post: &quot;Beyond the technical work to develop this model, we’ve also taken several steps to anticipate and mitigate malicious use cases where possible.&quot;<p>From the preprint, this seems to be doing some review before release and having a code of conduct in the GitHub repo.

The unicorn prompt is the new text generator lorem ipsum

It was trained on 140GB of text on 256 TPUs for 2 weeks, the model being made of 48 transformer layers. I&#x27;m wondering when we will see a model trained on 1TB or 10TB of text.

Could someone provide a high level summary of what this is for a technical person not conversant with the field?

&gt; Advertisement<p>Yeap, This one is indistinguishable from reality

Are there any hardware reqs to work with this?

Open AI did the right thing by not releasing their model; it&#x27;s disappointing that researchers are so callous about the potential effects of their research in the name of progress.