NYC Council AI hearing: tech giants can't swear AI follows safety rules; unreviewed AI code raises alarms

The New York City Council (chaired by Speaker Menin) held an AI-focused hearing, attended by former OpenAI researcher Daniel Kokotajlo and Jacob Coxon, among others. Afterward, Gary Marcus posted a series of comments on the hearing, with a core message: the current state of unreviewed AI code is alarming, and what truly needs governance is generative AI.

Confirmed

• Kokotajlo reiterated at the hearing that most code today is already written by AI, but review and oversight are far from keeping up; combined with the industry's "move fast and break things" ethos, the whole industry is sitting on enormous risk.
• He also cited his own judgment: "Our ability to detect alignment problems was already quite poor, and it is about to get dramatically worse."
• Remarks by Jacob Coxon and others likewise stressed that "handing large amounts of work to AI without carefully checking it is irresponsible," a position Marcus explicitly endorsed.
• Marcus pushed back on the chair's opening claim that "we don't yet know how to control any AI system," calling it inaccurate—many AI systems such as AlphaGo and GPS routing pose no control problem.

Why it matters

• Marcus's correction shifts the discussion from a vague "AI is uncontrollable" to the specific risk source of generative AI; he argues that it is generative AI, not all AI, that should be paused.
• If "AI writes the code and nobody checks it closely" holds true, software quality and security risks are accumulating systemically, and practitioners self-assess their alignment-detection ability as worsening—giving regulators first-hand testimony at the hearing level for intervention.

2026-10-05 ~ 2026-10-06 · 7 related posts

• Gary Marcus: It's Generative AI We Can't Control, So Pause That Specific Technology — GaryMarcus · 2026-10-05
• Gary Marcus Agrees: AI-Written Code Goes Unchecked, and That's 'Irresponsible' — GaryMarcus · 2026-10-05
• [source] Gary Marcus flags NYC hearing claim: most code is now AI-written and under-reviewed — GaryMarcus · 2026-10-05
• Kokotajlo: our ability to notice misalignment is poor and set to get much worse — GaryMarcus · 2026-10-05
• [source] Tech firms can't testify under oath that their AI obeys safety instructions, says Gary Marcus — GaryMarcus · 2026-10-06
• Gary Marcus calls OpenAI's safety commitment 'close to perjury,' flags Astra problems — GaryMarcus · 2026-10-06

1 near-duplicate retellings: GaryMarcus