Crazy, true story: Minnesota offered an expert declaration on AI and “misinformation” to oppose our motion to enjoin their unconstitutional law. His declaration included fake citations hallucinated by AI! 🧵 1/x
Minnesota AG @keithellison offered the expert declaration in support of the state’s ban on political “deepfakes,” which we argue infringes core political speech. https://hlli.org/kohls-v-ellis... 2/
@keithellison His report claims that “One study found that even when individuals are informed about the existence of deepfakes, they may still struggle to distinguish between real and manipulated content.” I guess the struggle is real because this study does not exist! 3/
@keithellison The doi link is broken. No content with this title exists online. The journal exists, but those pages point to unrelated articles. The authors seem to be in an uncanny valley of academics who publish about deepfakes, but their names are slightly wrong. 4/
@keithellison This was really weird for me to discover; I wasn’t sure of myself. Maybe the title was slightly wrong. But I couldn’t find anything even close to the article’s description, so we filed a motion to exclude the testimony. https://hlli.org/wp-content/up... 5/
As far as we can tell, this is the first time that an expert has cited hallucinated content in court. Eugene at @VolokhC has followed AI-generated content in the courts closely. No one else seems to have called out hallucinated expert citations before. 6/
@VolokhC Volokh also discovered that there’s a second hallucinated citation in the declaration. The author & journal are real, but this article does not exist and is not at the cited location. Some puckish AI invented it! #more-8305449 class="text-blue-500 hover:underline" target="_blank" rel="noopener noreferrer">https://reason.com/volokh/2024... 7/
@VolokhC (The unplanned additional motion to exclude the expert is why I was working to the last minute on Friday night.) https://x.com/FrankBednarz/sta... 8/
Something to consider when the director of the Stanford Internet Observatory (@stanfordio) opines that fact-checks don't work & censorial laws are “critical in preserving the integrity of democratic institutions” from AI deepfakes. Did an AI write this sentence too? 9/fin
The gist of his report is that counterspeech no longer works (and therefore government censorship is necessary). I think that's incorrect, and we hopefully prove our point by calling out AI misinformation to the court. https://x.com/aeschulman/statu...
Interesting debate in @VolokhC comments about whether the fake citations might have been intentionally illustrative about the problems of deepfakes. The perjury jurat makes me think this unlikely, but we'll see how the state responds.
Reporting on the story: @_cingraham for the @MNReformer. Still no comment from the expert, who told Prof. Volokh that he would have a statement in a few days. https://minnesotareformer.com/...
Protip: LLMs are eager to please, sometimes make stuff up. https://x.com/tedfrank/status/...
Update: still no comment from the expert, but the AG will “reply in court shortly” according to reporting by @Law360 that quotes co-counsel @james_vf_dickey at @UMLawCenter and me (at @HamLincLaw). https://www.law360.com/article...
@james_vf_dickey @HamLincLaw Conservatives pounce... on fabricated citations in a declaration under oath. https://www.sfgate.com/tech/ar...
Update thread with link to new filing. cc: folks who previously wrote on this: @VolokhC, @_cingraham, @hkonnath, @ToddFeathers, @stephencouncil, @instapundit, @CSJeffrey. https://x.com/FrankBednarz/sta...
@FrankBednarz @politicalmath Seeing this more and more. Had it in a case as well.
@TXLAW24 @politicalmath With an expert, or attorney? If an expert, would you mind telling me the case?
@FrankBednarz Hey I'm a reporter with the Minnesota Reformer, I sent you an email about this earlier today. Any new information about what happened here? Wild story!
@_cingraham Oh, just replied to your email.
@FrankBednarz This is an amazing story. 😂This Minnesotan thanks you.
@AaronK_MN You're welcome. And a BYU grad! Tough loss this weekend; I saw that game.
@FrankBednarz great piece. I was just dealing with an "expert" where I found similar issues in just-off citations - I wonder if those too were AI hallucinations.
@AshmoreHardy The thing that seems to characterize these hallucinated citations is they use real journal articles, and sometimes even real relevant academics, but the studies they imagine don't exist in the journal when you look them up. https://www.amjmed.com/article...
@FrankBednarz Rule 11?
@FrankBednarz a LEADING expert
@FrankBednarz Be sure to ask him which, if any, of his studies/findings have been reproduced/replicated. https://www.nationalacademies....