System Error: Where Big Tech Went Wrong and How We Can Reboot

In no more than the blink of an eye, a naïve optimism about technology’s liberating potential has given way to a dystopian obsession with biased algorithms, surveillance capitalism, and job-displacing robots. Yet too few of us see any alternative to accepting the onward march of technology. We have simply accepted a technological future designed for us by technologists, the venture capitalists who fund them, and the politicians around the world who give them free rein.

It doesn’t need to be this way.

In this AGI Public Policy Seminar, Jeremy Weinstein presented his book, System Error: Where Big Tech Went Wrong and How We Can Reboot, emphasized the complex environment in which society and Big Tech now operate, arguing that while rapid innovation delivers convenience and economic benefits, it simultaneously exposes policy shortfalls, ethical blind spots, and accountability gaps. Weinstein argues that the root of the problem lies not in technology’s complexity alone, but also in how society and its leaders engage with innovation.

Three major topics were highlighted concerning the governance of technology and AI:

• Bridging the knowledge gap between policymakers and technology: policymakers often lack technical understanding, allowing private sector innovation to outpace public oversight and critical issues such as cybersecurity to be inadequately addressed
• Moving beyond narrow optimization: developers and policymakers should not focus on isolated technological “fixes” such as automating ticketing disputes without considering broader societal purposes, and should align innovation with deeper societal goals
• Establishing ethical and institutional safeguards: sustainable technological advances require privacy protection, transparency, cooperation with different stakeholders, and globally consistent frameworks to ensure that innovations contribute to, rather than undermine, shared well-being