This is an automated archive made by the Lemmit Bot.

The original was posted on /r/singularity by /u/sdmat on 2024-09-27 07:52:04+00:00.

After using this for a couple of days it is abundantly clear that what OAI showed back in May was a proof of concept they had no intention of shipping - the claim that it was a product was an outright lie.

What do we get four months later? It’s not great. No doubt the main focus was infrastructure - Brockman’s very public job posting for this at the time probably didn’t win him points with Altman. Aside from that a lot of effort seems to have gone into reducing capabilities and layering on censorship.

• It can’t see, even still images.
• It can’t sing, or more accurately isn’t allowed to.
• The censorship is heavy and way too sensitive. It kicks in for completely innocuous requests like asking about Mandarin pronunciation.
• There are multiple layers of censorship - the model itself has instructions to deny its capabilities and restrict what it does, but on top of this there is a political officer model that cuts off the main model and plays a generic prerecorded denial. This is jarring and a bit uncanny.
• It is also arbitrarily restricted from talking for more than a few sentences. Apparently that worked fine during the alpha.

When the model is allowed to do its thing, it is great. There are minor quibbles: the Sky debacle, voice quality is notably worse, glitches and dropouts, and the ~45 minutes usage cap. But the core technology is amazing. It really is magical - you can sometimes forget it’s a machine. Which makes it all the more disappointing when one of the glaring product flaws kicks in.

It is also nearly useless in this state for practical applications. In my experience this includes the obvious and appealing use case of language tutor due to the ridiculously overzealous censorship. You can’t even paste in text, so it is unable to read or discuss anything. The session will swap to legacy voice mode if you do.

It’s great that OAI finally shipped, but it needs a lot of work to be good for more than a neat demo. Sometimes even for that, the censorship kicked in when I tried this with friends today.