San Francisco police have been quietly using artificial intelligence to write reports for weeks, and critics worry that the technology could create more opportunities for the abuse of power.
According to an internal memo obtained by The Standard, the San Francisco Police Department is participating in a pilot AI police-report program created by Axon, the company that provides body cameras to cops.
The pilot began in May and will last until August, when officers will rate the software in a survey, according to the memo. Officers are allowed to use the AI technology only for writing reports on certain infractions, such as misdemeanors without arrests and citations, as well as for informational documents.
In an internal October email obtained by the digital rights nonprofit Electronic Frontier Foundation, an SFPD representative said the department planned to pilot the program, Draft One, and had begun its “test phase.”
William Sanson-Mosier, SFPD’s chief information officer, hinted at the partnership with Axon in an interview in December, saying the department would use AI to make transcriptions from audio recorded by body-worn cameras. The AI would generate the first draft of reports.
“We’re excited to be piloting with them on that,” Sanson-Mosier said.
The SFPD said 54 officers at the Central and Southern stations are taking part in the pilot. They are required to proofread the draft reports and sign off on their accuracy. Officers must disclose when they have used the AI in a report.
Two officers told The Standard their cameras began automatically transcribing their audio recordings several months ago.
The SFPD says the district attorney is aware of the pilot, and no AI has been used to write a report involving an arrest.
“The SFPD will evaluate Draft One’s cost, efficacy, and practical implications when the pilot program ends in August to determine whether to continue or expand using the tool,” the department said in a statement.
Axon claims that Draft One will produce “high-quality draft report narratives in seconds” that will reduce paperwork and enable police to spend more time on the streets. It can also merge multiple reports. According to the company, officers are meant to review the reports before submitting them.
If expanded across the force, the AI program will be the SFPD’s latest implementation of new technology. The department has used its expanded surveillance powers to deploy drones, automatic license plate readers, and private camera networks to surveil suspects.
Civil liberty groups and defense attorneys oppose the move to incorporate AI in the writing of police reports, arguing that giving departments more high-tech tools without checks or balances will create abuses of power.
“Police reports are already full of inaccuracies, and officers often turn off the audio and video of their body-worn cameras,” said Zac Dillon, a policy analyst for the Public Defenders office. “We expect that AI-generated police reports based on body-worn camera footage will also be inaccurate and give officers another way to not properly document the arrests they make.”
EFF spokesperson Matthew Guariglia said there is no evidence that the technology has sped up police work.
Guariglia added that defense lawyers across the country have told EFF that AI raises a major problem when it comes to the veracity of police testimony. Police reports can’t be presented as evidence alone in court, so officers must testify about what they wrote. But if AI wrote a report, and a cop’s testimony is different from that report, police will be able to blame the technology.
“If a cop is caught in a lie on the stand, it’s much easier for them to say the AI made that up as opposed to them saying you caught me lying in the report,” Guariglia said.
Another problem, he said, is that there is no way to track which part of a report was written by AI versus an officer, making it difficult to parse the document if questions are raised about its veracity.
Some U.S. police departments have begun using AI but are not disclosing its use, Guariglia said.
Issues surrounding the use of AI in police work have arisen in other jurisdictions. In Seattle, prosecutors have instructed officers not to use AI for reports due to concerns about veracity. In Utah, a law that took effect this year requires officers to disclose the use of AI in their work. Police in Anchorage, Alaska, have found that the program did not speed up the writing process.
Last year, the San Francisco Board of Supervisors passed legislation requiring all city departments to report their use of AI annually. The law allows for appeals by people who believe AI created errors in their cases. It is not known if any appeals have been filed since the SFPD started its pilot.
Despite such concerns, a grand jury report issued Tuesday said the city should embrace AI with open arms.