Top Story

• OpenAI's next generation model, GPT-5, is ahead of schedule and coming in under budget. (WSJ / MSN)
  
  Sorry, just kidding. GPT-5 is not working, may never work as planned, and each training run takes six months and costs half a billion dollars.

  OpenAI has conducted at least two large training runs, each of which entails months of crunching huge amounts of data, with the goal of making Orion smarter. Each time, new problems arose and the software fell short of the results researchers were hoping for, people close to the project say.

  Also there's the tiny problem that with GPT-4, OpenAI already looted the entire public internet. GPT-5 needs a lot more data for its training, and there isn't more data.

  OpenAI’s solution was to create data from scratch.

  It is hiring people to write fresh software code or solve math problems for Orion to learn from. The workers, some of whom are software engineers and mathematicians, also share explanations for their work with Orion.

  But, you say, the internet contains all human knowledge. Won't trying to expand that significantly take a long time? Won't it cost a huge amount of money?
  
  Yes.

  The process is painfully slow. GPT-4 was trained on an estimated 13 trillion tokens. A thousand people writing 5,000 words a day would take months to produce a billion tokens.

  What about using AI to train your new AI?

  OpenAI also started developing what is called synthetic data, or data created by AI, to help train Orion. The feedback loop of AI creating data for AI can often cause malfunctions or result in nonsensical answers, research has shown.

  
  Scientists at OpenAI think they can avoid those problems by using data generated by another of its AI models, called o1, people familiar with the matter said.

  Scientists at OpenAI are paid to think that. They are paid a lot to think that.
  
  In short, your job is safe for now.

Tech News

• If you wanted to buy an ASML EXE:5000 Lego model, you can't. (Tom's Hardware)
  
  Unless you work for ASML, so time to polish up your resume. Having 10 years experience in 2nm lithography is a good start.
  
  
  
• Asus' first all-new NUC is here after the company bought Intel's mini-PC operation. And as NUCs go, it is certainly one. (Liliputing)
  
  Actually, this was reportedly designed at Intel before the NUC division sold, which hopefully is why it's pretty meh. It's fine as a business-priced business PC, but it doesn't have a great deal to offer consumers.
  
  
  
• A new diamond battery can consistently produce energy for thousands of years, based on carbon-14 as an internal power source. (Live Science)
  
  The only problem is that it produces about 200 microwatts. So about 200 of these could run a Raspberry Pi Pico. 300 if you use WiFi.
  
  
  
• 2024 has been kind to AI startups, but less so to EV efforts. (Tech Crunch)
  
  Apple spent billions of dollars in secret on its Apple Car only to confirm the project existed the same day it was shut down. Ford and GM both radically scalef back their work on electric vehicles.
  
  And Arrival, Cake, Canoo, Cruise, Fisker, Ghost, Lilium, and Phantom are all either dead or close to it.

Posted by: Pixy Misa at 06:00 PM | Comments (3) | Add Comment | Trackbacks (Suck)
Post contains 547 words, total size 4 kb.