Litigating Shotspotter Evidence

Summary

The webinar explains how ShotSpotter evidence is generated, what its reliability claims do—and do not—establish, and how defense lawyers can challenge it in criminal cases. The presenter argues that ShotSpotter’s reported accuracy figures are not supported by the kind of controlled validation testing expected for forensic methods, and that an alert alone should not justify a police stop or search.

How ShotSpotter works

According to the presentation, ShotSpotter’s process has two main stages:

1. Computer analysis

   • Microphones deployed in a neighborhood detect an impulsive sound and send data, including sound waves and recorded audio, to a central system.
   • An algorithm attempts to classify the sound as gunfire or non-gunfire.
   • The system also estimates the sound’s location using sensor data and a physics method called multilateration.

2. Human review

   • ShotSpotter employees review the event and can override the algorithm’s classification.
   • The company aims to send alerts to police quickly—typically within roughly 60 to 90 seconds.
   • The final report may state that a gunshot was detected and give a time and estimated location, but it may not reveal whether the computer and human reviewer agreed.

The presenter stresses that classification and location are distinct tasks, and that each requires its own assessment of performance.

Reliability concerns raised in the presentation

ShotSpotter has made claims including that it detects at least 80% of detectable outdoor gunfire within 25 meters, and that it is 97% accurate with a false-positive rate around 0.5%. The presenter argues that these figures are problematic because:

• A single “accuracy” figure can be misleading. A classification system has separate false-positive and false-negative rates; location accuracy is a different measure. The speaker says these cannot properly be collapsed into one percentage.
• Reliable claims require controlled validation. The presentation cites forensic-science guidance that performance should be assessed using validation data where the true source of each test sound is known—not simply information gathered from police responses to alerts.
• Testing needs to reflect real conditions. Relevant variables include:
  • Urban density and environmental noise, which can reflect, refract, or diffract sound.
  • Sensor number and placement, including whether an event occurs near the edge of a coverage area.
  • Firearm, ammunition, and the direction the firearm is pointed relative to sensors.
  • Innocent sounds such as fireworks, vehicle or construction noise, weather, helicopters, and other urban activity.
• Validation should be robust and ongoing. The speaker says it should use a sufficiently large sample, include developmental and deployment-stage testing, be peer reviewed and replicated, and be repeated when important parts of the system change.

In the Michael Williams case, the presenter says ShotSpotter reported that a gunshot occurred at a particular time and location. Williams was charged with murder despite the absence of an eyewitness identifying him, a murder weapon, or other physical evidence connecting him to the shooting. The presenter says that, after the defense demanded the algorithm’s screenshot, it showed that the algorithm had classified the sound as a firecracker with 98% confidence, but a human reviewer had overridden it. He says this information was not disclosed until the defense specifically sought it. The murder charges were later dismissed.

What the presenter says supports ShotSpotter’s accuracy claims

The presenter says ShotSpotter told the defense that it had not performed live-fire or deployment-qualification testing in Chicago. He also points to a company audit that based accuracy figures on reports from police-agency clients about possible errors.

The presentation challenges that approach:

• Police feedback is not the same as controlled validation with known ground truth.
• Officers may not know when ShotSpotter missed a gunshot, especially if no one reports the event and police never investigate it.
• Finding evidence at a scene after an alert does not, by itself, establish that the system correctly classified or located the sound.
• The presenter cites a Chicago Office of Inspector General review of about 50,000 alerts, saying officers found no evidence of a gun crime in 91% of responses. He argues that if agencies do not report such outcomes, treating silence as confirmation can make the system appear more accurate than it is.

The presenter also questions the human-review component. He says the reviewers’ job descriptions emphasized customer-service experience rather than scientific qualifications; ShotSpotter provided no formal training materials in response to a subpoena; and the defense had not been given proficiency-test results demonstrating reviewer competence. He also says the company’s protocol for when reviewers may override the algorithm did not set out a scientifically defined process, though the relevant document was under a protective order at the time.

Fourth Amendment and other litigation strategies

The presenter recommends challenging ShotSpotter evidence both in admissibility proceedings and in suppression motions.

• Argue that an alert alone does not create individualized suspicion. The presentation discusses appellate decisions treating a ShotSpotter alert as more like an anonymous tip than an eyewitness account or known informant. It also cites a D.C. court ruling that a nearby alert alone could not justify stopping and searching people in the area.
• Challenge the reliability of the alert in suppression litigation. The presenter says that when an alert is used to justify a stop, its reliability is relevant to the Fourth Amendment analysis. He cites Navarette on the reliability of anonymous tips and Florida v. Harris on a defendant’s opportunity to challenge the reliability of evidence, including through expert testimony.
• Examine what police corroborated. Relevant issues include the time between the alert and police arrival, whether officers observed specific conduct, and whether there was evidence connecting the person stopped to a crime. The presenter argues that ShotSpotter alerts generally do not identify a suspect or describe a person’s actions, so they lack the kind of person-specific information that may support an anonymous tip.
• Seek the underlying records. The presenter recommends comprehensive subpoenas to ShotSpotter, including requests for:
  • Validation and deployment testing.
  • Protocols and procedures.
  • Job descriptions, training materials, and proficiency records.
  • The algorithm’s underlying classification or screenshot for the incident.
  • Other records needed to assess how the alert was produced and reviewed.
• Bring formal admissibility challenges. The speaker urges defense lawyers to pursue applicable Frye or Daubert challenges rather than allow unvalidated forensic evidence to be presented without scrutiny.

The presenter says that in his office’s experience, pursuing discovery and litigation can lead prosecutors to exclude ShotSpotter evidence, reduce charges, or dismiss cases rather than litigate the system’s reliability.

Other points from the webinar

The introduction notes that ShotSpotter acquired predictive-policing company HunchLab, which it renamed ShotSpotter Missions, and that the technologies may be used together. In the question-and-answer session, the presenter says ShotSpotter is deployed in more than 100 U.S. cities and is also seeking international expansion. He identifies engineer John Hanson of the University of Texas at Dallas as an expert the defense has consulted on sound-wave propagation.

Speakers and sources featured

Speakers

• Jumana Musa — Director of the Fourth Amendment Center at NACDL; moderator and opening/closing speaker.
• Brendan Max — Chief of the Forensic Science Division of the Cook County Public Defender’s Office; presenter and Q&A speaker.

Sources and authorities discussed

• ShotSpotter materials, reports, protocols, and employee job descriptions.
• Ralph Clark, ShotSpotter CEO, quoted in a BBC report about the company’s reliance on agency feedback.
• BBC report on ShotSpotter’s accuracy claims.
• Edgeworth Analytics audit discussed in connection with ShotSpotter’s reported accuracy figures.
• Chicago Office of Inspector General report on ShotSpotter alerts.
• Forensic-science sources and guidance cited by the presenter: Bill Thompson, OSAC, PCAST, and Jonathan Koehler of Northwestern.
• Legal authorities discussed: a Seventh Circuit decision involving ShotSpotter and Rickman; a D.C. court decision; Navarette; and Florida v. Harris.
• John Hanson, engineer at the University of Texas at Dallas, identified during Q&A as a defense expert on sound-wave propagation.
• Michael Williams and Safarian Herring — central figures in the case example, not webinar speakers.