Featured Post

Welcome to the Forensic Multimedia Analysis blog (formerly the Forensic Photoshop blog). With the latest developments in the analysis of m...

Saturday, July 12, 2025

Coming Out of Retirement for This One: On AI, Authorship, and the Erosion of Ground Truth

It’s been a while since I posted here.

I’ve moved on from daily forensic casework and stepped back from the blog to focus on broader systems work—on justice, equity, neurodiversity, and how we reckon with broken epistemologies. But sometimes something happens that drags you out of retirement whether you like it or not. And this is one of those moments.

The Electronic Frontier Foundation (EFF) just released a thorough, scathing analysis of Axon’s “Draft One”—an AI tool currently being used in police departments to automatically generate narrative reports from body-worn camera audio. The headlines focus on missing drafts and vanishing accountability, but from where I sit, this story goes much deeper. It cuts to the core of what forensic science is supposed to be—and the mechanisms by which truth, evidence, and responsibility are disappearing in real time.

This is not just an engineering failure. It’s a perversion of process, designed to undermine transparency and suppress ground truth. And unless we start shouting about it now, loudly and persistently, we may find the damage done before it’s ever tested in court.

So here I am. Let’s talk.


What Is Forensic Science, Again?

When I was serving on the OSAC Video/Imaging Technology and Analysis Task Group, we spent a fair bit of time just trying to define our terms. “Forensic science” may sound straightforward, but once you include all disciplines—from biology to acoustics to digital traces—you need language that holds across those domains.

The final agreed definition read (it's probably changed since then):

Forensic science is the systematic and coherent study of traces to address questions of authentication, identification, classification, reconstruction, and evaluation for a legal context.

Let’s break that down.

  • Systematic and coherent: It means you use a validated methodology. You test. You document. You repeat. You don’t just guess.

  • Study of traces: Not assumptions. Not vibes. Not hallucinated context from a black box algorithm.

  • For a legal context: Which means your findings must be discoverable, explainable, and reproducible in court.

Any tool—human or machine—that doesn’t meet those standards doesn’t belong in forensic science or in the justice system.

And yet Axon’s Draft One is being used to generate police reports that:

  • Can’t be traced to an identifiable author.

  • Can’t be audited for edits or AI influence.

  • Can’t be cross-examined in court, because the drafts no longer exist.

That’s not justice. That’s marketing.


The Death of the Draft

Let’s be clear: deleting original drafts isn’t some accidental oversight—it’s a deliberate design choice. A senior Axon product manager is on record stating that storing drafts would only “create more disclosure headaches” for their customers and attorneys.

In other words, the system is intentionally structured to evade scrutiny.

In civil proceedings, experts operate under strict obligations—FRE Rule 26 (or its UK counterparts) requires comprehensive disclosure: your notes, your methodology, your tools, your working files. Everything from the History Log in Photoshop to the process report in Amped FIVE is considered part of the evidentiary record and must be turned over. This ensures transparency and allows opposing parties to challenge the foundation of any claim.

But there’s no equivalent to Rule 26 in criminal procedure.

And that absence is where much of the injustice festers. In criminal cases—especially those resolved through plea deals—discovery is often minimal, accountability mechanisms are weaker, and tools like Draft One can be quietly implemented with no real oversight. The system permits it not because it’s just, but because it’s expedient.

When a narrative is generated by AI, lightly skimmed or rubber-stamped by an officer, and submitted without retaining the drafts, who’s actually attesting to authorship?

No one.

And in a system already tilted against defendants, that’s not just a technical gap—it’s a structural abuse.


Officers of the Court, Without Responsibility

When a police officer submits a report, they do so as an officer of the court. That’s not metaphorical. They’re bound by law and professional ethics to tell the truth, to the best of their knowledge, under penalty of perjury.

The legal assumption is that the report is theirs. They saw the event. They wrote it down. They stand by it.

But if Draft One creates a first draft—based on audio the officer might not even remember clearly—and that draft is then lightly edited (or not at all), who is really the author?

If the report contains bias, omission, or invention, is it the fault of the AI? Or the officer? And how would you know?

EFF is absolutely right to call this a “smokescreen”—one that could allow officers to deny responsibility while still reaping the legal authority of a sworn report. It’s the worst of both worlds: automation without accountability.


The Market Logic Behind the Curtain

Let’s not ignore the profit motive either.

Axon is a publicly traded company. Its duty is to shareholders, not truth. Draft One is being rolled out not because it meets evidentiary standards—but because it saves time, cuts labour costs, and locks departments deeper into the Axon ecosystem.

In a context where most cases end in plea deals—where defence teams rarely get the chance to challenge reports through discovery or trial—these AI-generated narratives may never face real scrutiny. They’ll become “truth” by default, untested and untestable.

This is how you change the epistemology of the criminal legal system without ever passing a law. You let a company rewrite the workflow.

And if no one objects loudly enough, that new workflow becomes the norm.


The Real Problem: Validation and Ground Truth

When I wrote my 2020 article on Ground Truth in Digital/Multimedia Forensics, I was wrestling with many of these same questions.

Validation isn’t optional. You don’t get to skip the experiment just because the vendor says their tool is “reliable.” You have to test—under real conditions, with real data, using accepted scientific methods.

And if your tool doesn’t allow for that kind of testing—if it deletes its own outputs, obscures its authorship, or refuses to log its transformations—then you are not practising forensic science.

You’re engaging in speculative automation.

You’re telling stories with invisible authors and calling them evidence.


A Final Word (For Now)

I never thought I’d see the day when officers could submit reports they didn’t write, generated by a system that deletes its drafts, under the banner of “efficiency”—and still have those reports carry the full weight of legal authorship.

But here we are.

And so, for the moment, I’m coming out of retirement. Because the integrity of the judicial system matters. Because ground truth matters. And because the legal system cannot afford to outsource authorship and accountability to a product designed to dodge both.

This isn’t innovation. It’s abdication.

And if we don’t push back now, we may find ourselves standing in courtrooms arguing with ghosts—armed only with final reports, stripped of history, speaking with no traceable voice.

Friday, January 3, 2025

 January 2025 Update

I received a very odd notice from Google congratulating me on the traffic that this blog has received in 2024. Yes. 2024. I haven't really thought about this space in more than 4 years. But there it was. So I clicked into the stats. It turns out this blog logged over 115 thousand visits last year. WTF? Since I started this space, it's logged over 1.88 million views. Double WTF!

web site stats for this blog

I'm not going to maintain this space other than keeping it up as an archive. If you really want to know what I'm up to lately, you can find me on Substack at AutSide.Substack.com. See you there.

Jaime

Tuesday, September 22, 2020

So long, and thanks for all the fish

 This will be the final post in this space. I've retired from the practice. I'll leave this free resource up as an artifact and a reference. 

So long, and thanks for all the fish.

Friday, March 27, 2020

A cultural shift in training delivery is happening

First of all, I hope this post finds you and yours in good health. I hope that you have enough to eat and have enough resources to meet your basic needs. I know that many folks have been sent home to work, some have even lost their jobs (some temporarily, some permanently). With courts closing and pushing trial calendars out, most of the legal support world is on hold. It's rough out there. I get it. I'm living it too.

I've had ample time to get caught up on projects, write papers, and fulfill my continuing education requirements. I switched to on-line learning for my own continuing education a while back, going 100% on-line about two years ago. As a consequence, I've seen the good, the bad, and the ugly of training and education offerings.

With the current crisis, vendors in the forensic science space are stuck. The learner population has, for generations, operated under the belief that they need to be physically present in the room with the instructor and their fellow students. Providers have reinforced this by only offering in-person training. Some occasionally ventured on-line, but the learners weren't there; it wasn't profitable so it wasn't offered again.

Now, travel is restricted. People have been sent home. Without all the usual economic activities, municipalities are seeing gigantic holes in their budget projections. The state and local agencies are appealing to the federal government for help. As of today, that help is still being debated in Congress. What it will look like, what will be prioritized, is yet to be seen. But, if government agencies' reaction to this crisis mirrors that of the 2007-2010 financial crisis, personnel will face pay cuts and only essential functions will be funded. Last time, training was not deemed essential. I'm guessing that will be the case again.

The vendors understand this too. Most have announced changes in their licensing terms as well as updates to their training schedules, moving training on-line. But, it's not that easy to shift paradigms. Organizing and delivering an in-person training session is entirely different that organizing and delivering an on-line training session. I should know, I've done both.

Some vendors are offering "webinar" based training. With these, you log into a portal like Zoom, and you watch as the instructor leads you through a "broadcast" version of the usual in-person training. You might see a split screen with their talking head and their computer's screen. But, Zoom users are facing throttling issues as so many are now working from home. Zoom, and it's competitors, have offered so many "free" accounts that they're now getting swamped. A few instructors are using these "free" accounts to host these "webinars." It's not going well. Added to this is the dispersed nature of the learner population. A 9-5 class, hosted in New Jersey, means a west coast USA learner must be up and ready to learn at 6am. Not optimal. I faced this issue learning SalvationData's VIP. Given their instructor was in central China, the course ran from 6-10pm, then continued in the early morning after I got a rather brief nap. It was crazy.

The other problem is price. If a vendor is utilizing a free or low cost service to host a webinar, do you expect to pay the same price as an in-person training session where you can not only get hands-on help from the instructor, but you can interact with your fellow learners? I should think not. Yet, many vendors do not discount their on-line offerings, or offer minimal discounts. They're counting on the fact that it's often the case that the learner is not spending their own money, but drawing from training funds at their agency. They're not price conscious because they don't have to be. Now, with the current budget uncertainty, learners must be very price conscious if they're going to get their training requests approved.

I share the aspirational goals of my country's leaders in that I hope to see the country back to work by Easter. It's aspirational - a best case scenario. Given the hit that the economy of the world has taken, I don't think the old training model will ever see the light of day again. Vendors must face the reality of restricted funds and restricted travel.

Back when I was working with Amped Software, Inc., and with Axon, many of us saw this problem coming. We saw the mentality of treating training like an extended vacation as unsustainable. Training staff can not be everywhere at once. Staffing costs are quite expensive, as is travel to the training location. In an economic crisis, the first thing to get eliminated was training. It happened before. It would happen again - and it has. I spent about a year researching the best options for moving on-line, eventually arriving at LearnUpon as our LMS provider about time the deal between Amped and Axon collapsed. When Apex was born from this, we were ready to move our offerings on-line. First out of the gate was Statistics for Forensic Analysts, a fully validated course delivered on-line as micro learning. Along the way to that, I earned a Masters in Education - Instructional Design.

All of my in-person courses have been totally revamped and assembled as micro learning offerings. The design and delivery isn't negatively affected by bandwidth problems. Our more popular introductory courses are currently available and have an active learner populations. They're steeply discounted versus in-person offerings. We don't use "free" webinar services, we have invested in a state of the art LMS, which does cost us a bit of money each year. Our more advanced courses will be available soon, now that I have more time to focus on the deliverables. Deliverables? Yes, our Photogrammetry class, for example, includes a complete set of instructions for going to your local building supply store in order to get the materials you'll need to build a reference rig for creating reverse projection recordings. It also includes the printer's template for creating your own resolution / height charts. With the current travel restrictions, as well as business' focus on the current crisis, it's not responsible to require learners to venture out to do these things. Thus, we'll wait until the world recovers.

Yes, we're obviously not offering "official" "vendor approved" courses. But, in fairness, how many vendors have instructional designers on staff? How many have formally validated their on-line courses as fulfilling their stated learning goals? How many have years of experience in the on-line space and utilize the best tools to deliver their training? To that end, how many actually understand how to create learning goals and outcomes for on-line learning events, then deliver upon those goals? Given the amount of grief I'm witnessing - head over to LinkedIn to see the stories - I'm guessing that learners are experiencing pretty terrible courses. Just because one is a subject matter expert, or a good in-person instructor, does not mean that one will be successful delivering upon learning goals in the asynchronous e-learning space.

Once you get settled and are looking for something to distract you from the fact that you've run through the entirety of the Netflix catalog, we're here for you with some amazing learning opportunities. You can sign up and start learning any time, no need to wait or coordinate schedules in order to attend a "webinar" hosted in some distant location. Our learning opportunities fit your schedule, that's the beauty of micro learning.

Have hope, my friends. Make aspirational goals. We'll get through this. Let's utilize the downtime to create good habits of self-care and self-improvement. Let's not waste our most precious resource, time. Have a great day.

Wednesday, March 25, 2020

Working from home? Remember, Alexa hears everything.

As firms and agencies urge their employees to work from home during the global pandemic, their employees’ confidential phone calls run the risk of being heard by Amazon.com Inc. and Google.

Mishcon de Reya LLP, the U.K. law firm that famously advised Princess Diana on her divorce and also does corporate law, issued advice to staff to mute or shut off listening devices like Amazon’s Alexa or Google’s voice assistant when they talk about client matters at home, according to a partner at the firm. It suggested not to have any of the devices near their work space at all.

Mishcon’s warning covers any kind of visual or voice enabled device, like Amazon and Google’s speakers. But video products such as Ring, which is also owned by Amazon, and even baby monitors and closed-circuit TVs, are also a concern, said Mishcon de Reya partner Joe Hancock, who also heads the firm’s cybersecurity efforts.

“Perhaps we’re being slightly paranoid but we need to have a lot of trust in these organizations and these devices,” Hancock said. “We’d rather not take those risks.”

The firm worries about the devices being compromised, less so with name-brand products like Alexa, but more so for a cheap knock-off devices, he added.

Like Wall Street, law firms are facing challenges trying to impose secure work-from-home arrangements for certain job functions. Critical documents, including those that might be privileged, need to be secured. Meanwhile in banking, some traders are being asked to work at alternative locations that banks keep on standby for disaster recovery instead of makeshift work-from-home stations to maintain confidentiality.

Smart speakers, already notorious for activating in error, making unintended purchases or sending snippets of audio to Amazon or Google, have become a new source of risk for businesses. As of last year, the U.S. installed base of smart speaker devices was 76 million units and growing, according to a Consumer Intelligence Research Partners report.

Amazon and Google say their devices are designed to record and store audio only after they detect a word to wake them up. The companies say such instances are rare, but recent testing by Northeastern University and Imperial College London found that the devices can activate inadvertently between 1.5 and 19 times a day.

Tech companies have been under fire for compromising users privacy by having teams of human auditors listen to conversations without consent to improve their AI algorithms. Google has since said that users have to opt-in to let the tech giant keep any voice recordings made by the device. Amazon now lets its users set up automatic deletion of recordings, and opt out of manual review.

The law firm’s warning first surfaced on an Instagram account “justthequant,” where people share their intel and struggles of working from home.

This story was originally published on Bloomberg.com. Read it here.