That's a whole lot of compute, and apparently a big chunk of that went into 'safety and alignment'
Brief · Source report
Jensen Huang says 100,000 Nvidia GPUs were used to train OpenAI's latest model, GPT-6 Astra, and there's already plans to bring quadruple that amount of hardware online
Source excerpt. Read the original for the full report.