Just like various chat bot incarnations, it's reasonably OK at generating generic sentences without real understanding. I tried several attempts with this leading fragment: "Is borrow checking unique to Rust? Probably not. But". The results are not pretty (the best one is as follows). Not impressed at all with this giant model :|<p>"Is borrow checking unique to Rust? Probably not. But the Rust library is written by and for Rust programmers, so if you're looking for a library that's designed for the language you already know, this might be your first choice.<p>The library is still in a fairly early stage. It does not support the standard libraries.<p>If you're interested in reading about how the library works, the docs are up on GitHub for reference!<p>How can I get this for my project?<p>Get it here: <a href="https://github.com/paulp/rustbinder" rel="nofollow">https://github.com/paulp/rustbinder</a><p>I've just read your blog post and want to use it, what can I do?<p>If you're looking for a project to use this library for, check out the docs for the "RustBinder project".<p>What are the current limitations and how will future changes affect this library?<p>This is still a work in progress. The library currently does not support the standard Rust library. There are a few work in progress"
At a credibility score of 6.91/10, many people will rightly judge that the full GPT-2 model will remain insufficient for malicious use in creating fake news.<p>However, even the smaller models are already good enough for spamming/trolling/astroturfing. It doesn’t take a Shakespearean soliloquy to convince people of a point. Just enough of a flood of short 1-3 sentence pro/con comments on a forum can drastically affect the perceived public opinion of an issue. Those comments can then spur real people to reply, which could result in an ultimately organic but directed propaganda vector. Propaganda directors will carefully craft something for people to look at, and the GPT-2 bots will move people’s eyes in that direction.<p>You can see the same happen on r/subsimulatorgpt2, where the longer titles and prompts and replies eventually sprawl into incoherence, but the shorter sentences from the finetuned bots in the comments section are effectively indistinguishable from the kinds of short comments you would find on their respective subreddits.<p>Or in other words, the malicious uses for GPT-2 won’t be a tidal wave, but a flash flood.
Wow, some samples are frighteningly good. I was impressed by previous models and I don't know if I'm just lucky this time, but... wow. Can anybody who is not into climbing even tell this is all fake?<p><i>Jain Kim is an experienced climber.</i><p>In 2006, she became the first woman from Korea to climb all five 8,000 meters (24,064 ft) peaks in the Swiss alpine ski run Alps in 24 hours. In 2009, she made history again by setting the record for the fastest time to climb an 8,000 meter peak with a team from China and South Korea.<p>She made the first ascent of 8,832-meter K2 in China, the second highest mountain in the world, in 2009 and the third highest mountain in Europe. She also is the first female Korean to summit a world-class peak.<p>During her two years as a mountaineering professor at Sogang University in Korea, she established two new routes in the Yalu River area. The first of these routes is a 3,547-meter peak named K2 on Mount Long in China. Her second route is on the same mountain, called the Lomonosov Ridge, at 3,632 meters.
We (Hugging Face) added it to Write With Transformers if you want to try the text generation capabilities of the model: <a href="https://transformer.huggingface.co/doc/gpt2-xl" rel="nofollow">https://transformer.huggingface.co/doc/gpt2-xl</a>
Prompt: "Real things don't exist unconditionally and things that exist unconditionally are not real. However the reality has an essense. It is"<p>Response: "an actual thing, and it is not the thing to which we attach meaning. It is not real because it is not a thing. And therefore, it does not possess the qualities that are inherent in all real things."<p>Just wow. Sure, there are a few logical mistakes here, but this response serves as a good prompt for my bio-GPT. In other words, we usually need some starting points or hints for analysis and discovering these hints is non trivial because whatever we can think of is not very new to us. This GPT just gave me an answer that smells like a serious wisdom and I'll surely dig in that direction to see if this idea has any substance.<p>Edit: what's happening here is while I can't ask this model to give me a short and conscise summary on a topic, I can still interrogate this model and find out what it's seen in the training set. I can't possibly read all these books in the training set, but now I can rapidly navigate in the multidimensional meaning space: I tell it where to start and it says what it seems in close proximity to my prompt. This is a breakthru.
> (CTEC) found that extremist groups can use GPT-2 for misuse, specifically by fine-tuning GPT-2 models on four ideological positions: white supremacy, Marxism, jihadist Islamism, and anarchism. CTEC demonstrated that it’s possible to create models that can generate synthetic propaganda for these ideologies<p>I wonder how they tested that
Prompt: The coming global recession is a real possibility and"<p>Response: "The coming global recession is a real possibility and the Fed is playing games, creating artificial market conditions to make a recovery seem possible in the short-term. The Fed has an option to change its monetary policies but it will not make the problem go away, so it is in their best interest to pretend it won't happen."<p>Change and to however and you'll get another stereotype opinion. It really just composes pieces of texts it's seen around the prompt, but it does this really well.<p>Most of the news agencies can now fire most of their monkey typewriters: this GPT will outperform them on every metric.
Omfg can we stop making these things bigger PLEASE<p>Like, who cares??<p>* What I mean is, text gen models are big enough. We need controllable text generation; like, so it can talk about a specific THING sensibly. Rather than spew statistically plausible nonsense.
Sampling realistic text from large pretrained models is non-trivial. I came across this paper in one of ACL 2019 workshops:<p><a href="https://arxiv.org/pdf/1904.09751.pdf" rel="nofollow">https://arxiv.org/pdf/1904.09751.pdf</a>
Tangent rant.<p>I'm skimming over some of the code at <a href="https://github.com/openai/gpt-2/blob/master/src/model.py" rel="nofollow">https://github.com/openai/gpt-2/blob/master/src/model.py</a> and I can't help but feel frustrated at how unreadable this stuff is.<p>1. Why is it acceptable to have single-letter variable names everywhere?<p>2. There's little to almost no documentation in the code itself. It's unclear what the parameters of any given function mean.<p>3. There are magic constants everywhere.<p>4. Function names are so terse... ("gelu", "attn")