I find it really hard to see how productive a collective pause and 'think' about something so inherently unpredictable will be.<p>> "implement a set of shared safety protocols for advanced AI design and development that are rigorously audited and overseen by independent outside experts"<p>Who deserves to be called an expert on this? Feels like Economics or something, where you have camps of thought advocating for themselves but little way of knowing who's right. Best to break things and develop antibodies whilst the stakes are still low.
So Musk was outed from OpenAI and is now salty? Or are they trying to build a competitor and are falling behind and want OpenAI to get a break so they can catch up? In either way, the ethics talk is total bullshit.
People are really freaking out about AI aren’t they?<p>Why bother? It’s moving super fast, just wait and see what happens.<p>And even if you could control or regulate it, exactly how would you do that? What would you be regulating/controlling? How would you define it?<p>And why would you want to anyway? The party has just started, if you think the revolution has arrived, your completely wrong - this is just the beginning - the most amazing stuff is yet to come.<p>These people begging for the pace to slow, it’s analogous to the newspapers and music companies wanting the internet to slow down as they were being rapidly involuntarily made redundant.
The letter: <a href="https://futureoflife.org/open-letter/pause-giant-ai-experiments/" rel="nofollow">https://futureoflife.org/open-letter/pause-giant-ai-experime...</a><p>I don't see any way to verify the signatures. Though the mention of Sam Harris' signature disappearing suggests they're being moderated at least?
The posters here seem to be highly skeptical of the need to regulate emerging AI.<p>I find that pretty disappointing and surprising.<p>Recently, they gave gpt-4 access to a terminal, the internet, and money. And gpt-4 itself is a software (and software in general has bugs, vulnerabilities, etc) black box that is incredibly, unprecedentedly powerful and not fully understood. Part of its training data is almost every known security vulnerability.<p>You guys really don’t see any potential problems with this? I mean really? Get a little creative here.
Speaking of AI safety:<p>With Tesla “Autopilot” Elon Musk is responsible for releasing AI that has actually killed multiple people.<p>I don’t think ChatGPT has killed anyone yet.
Ultimately its a millennia old fight between manipulators and rationalism.<p>The bead maze toy versus the abbacus.<p>Twisted contorted reasoning versus formal verification.<p>Choose your abbacus.<p>The black box nature of machine learning models is not the issue. Instead of training to imitate vacuous conjectures and claims as humans on average typically do, they could be trained to do automated theorem proving, AlphaZero style.<p>A minimalistic verifier like metamath is available free for download, including set.mm and a freely available book. It would be hard to purge from civilization.<p>Currently its math database is collaboratively worked on at github.<p>In theory a blockchain could host it.<p>Fermat style challenging could be used to objectively assess the value of theorems: the longer it matures unproven as a challenge on the chain, the higher the reward if someone finally proves it.<p>This inevitably creates an incentive to enter and digitize known mathematics into machine readable form, which will be easy for machine learning to accomplish.<p>Machine learning empowered automated theorem proving will become a profitable business, with the fruits available for all to benefit from.<p>Well, cryptography and protocols will also appear.<p>So during training the machine learning models will get endless bedtime stories about Alice, Bob and Eve.<p>Using conventional forward or backward chaining combined with adversarial models, one can construct arbitrary provable theorems, and negate it, then hide or propagate the negation so its not simply the first symbol in the theorem.<p>So we can train models to challenge each other Fermat style, about the truth or falsehood of a statement, and demand proof.<p>We can thus construct artificial mathematical systems with known inconsistencies and train models to <i>seek</i> a proof that the system contains an inconsistency. Such a proof will depend on the conflicting axioms.<p>Hence the models will be our best tool to detect and resolve hypocrisy.<p>The literal meaning of "apocalypse" is "revelation" or "uncovering", not "big tragedy"...<p>The verification algorithm, for example the ~300 LoC python implementation by Raph Levien, owes most of its length to parsing the metamath file format.<p>The actual Kolmogorov complexity of the verification algorithm itself is much smaller. There won't be any bits to "align".<p>All these hopeless attempts at trying to align the intuition component of the machine learning model, instead of training it to gain intuition in producing logical derivations.<p>The real horror of the control freaks is not that their alignment mechanisms might fail, but that its impossible to bias the verification algorithm itself, that its impossible to perpetuate the conflicts of interest, that any additional code in the verifier is immediately suspect, especially if it obviously skips all checks and dogmatically accepts a statement if its signed by a hardcoded "right" key.<p>The objective judge will be mechanized.<p>"abaccus akbar!"
Elon already made his money selling a promise of FSD but now wants to pump the brakes on AGI because it is almost here and he doesn't own it?<p>Where did all the hyper-competitive SV libertarians disappear to in the last year?
> As of Tuesday, no O̶p̶e̶n̶AI.com employees had signed the letter, although CEO Sam Altman’s name briefly appeared then disappeared from the list of signatories.<p>You already know the intention(s) of Sam Altman and O̶p̶e̶n̶AI.com. It was only to run with VC money and close up all their research up.<p>They are no better than DeepMind.