Right to Silence and the collection of evidence

"You do not have to say anything, but it may harm your defence if you do not mention when questioned something which you later rely on in court. Anything you do say may be given in evidence." - Right to Silence (UK)

When I retired from police service, I had spent a third of my life working for the LAPD. One of the questions that someone with my resume will eventually face is the balance of case types worked, the implication being that I would have mostly worked for the prosecution of criminal defendants. But, in Los Angeles County, there's an over 90% plea rate. In reality, I worked on the major criminal cases about 50% of the time, with the remainder working in the defence of the my self-insured city. I've also worked in the defence of officers falsely accused of perjury (People v Abdullah) and have donated a lot of time for case review in support of Innocence Project type cases.

If you're an American, you will be more familiar with the Miranda warning than the warning shown above from the UK. But, I think the Miranda warning, and the 5th Amendment in general, is setting people up for failure in successfully defending themselves in court (assuming they're innocent of all charges). Here's what I mean.

Americans generally believe that it's up to the State to prove guilt. Whilst this is true in theory, there are so many stories about innocent people pleading guilty to charges based on a "risk assessment" that they make about their potential to succeed at trial. The Prosecution has their theory of a case. The State's investigative might has been focussed on the Prosecution's theory in the collection of evidence. They're not necessarily concerned with the collection of alibi evidence for you. They may stumble upon potentially exculpatory evidence, but that's not their job - it's yours.

This is where I think the UK's standard admonition is more honest. Sure, you have the Right to Silence. But, by remaining silent, you may actually harm your defence. 

If you are innocent, you know where you've been, with whom you've associated during the time in question, places you've visited, and etc. Your defence team must begin it's own evidence collection process. Your movements throughout the day all leave traces - the store, the gas station, the coffee shop, highway tolls, and etc. There's very little about one's life these days that isn't tracked or recorded. In addition to CCTV video of you going about your day, your phone has likely recorded even more details about where you've been and what you've been doing. Now, there's even the personal home assistants like Alexa and Siri that can serve as a witness to your movements.

All of this data must be preserved. However, the average person doesn't have the resources or the know-how to collect and preserve digital evidence properly. Some items, like cell tower logs, may require a warrant to acquire. You must have a capable and aggressive lawyer working on this for you. 

I'm here to help. I've seen all sides of this and have been working on standards for digital evidence collection and processing for well over a decade. I've taught classes to public and private audiences on this as well (click here for more info).

Yes, in the US we have the presumption of innocence. But, the best defense is a good offense. You must go on the offense and collect the evidence that proves your innocence. The evidence shows that it will harm your defence if you don't.

A rough guide to spotting bad science

A relevant infographic from Compound Interest.

Mind the Communication Gap

I received a panicked email and phone call last week that can easily be summarized by the graphic above. Yes, there are still vendors selling to the police services that don't understand or accommodate the agencies needs or schedules.

As most in government service know, often the ability to spend money on tools and training happens within a short window of time. Such a window had opened for this person and they reached out to a company in response to a bit of marketing on new redaction functionality in a particular piece of software.

In the email that was shared with me, the requestor made it clear that they wanted to redact footage from body worn cameras. A trial code was shared for the company's tool. The requestor eagerly downloaded and installed the software, then assembled his supervisors and stakeholders to evaluate the new software. That's when the problems began.

Redacting the visual information was relatively straight forward, but cumbersome. However, he couldn't figure out how to redact the audio portion of the file. He reached out to the company's rep. Unfortunately, it was late on Friday in California. He received no reply from the company's rep, who'd likely gone home for the weekend. Thus, having previously communicated with me on technology issues, he reached out hoping that I'd still be around.

I let him know that to the best of my knowledge, none of Amped SRL's products redact audio. Yes, they'd promised audio redaction in FIVE at the 2016 LEVA conference. But, it's never materialized. As it's not been developed, it's quite likely that there is no audio redaction functionality in Replay.

He received a note from the sales rep on Monday. It informed him that the redaction functionality was built around a CCTV use case, and as such, did not concern itself with audio. The order taker that responded to the inquiry could have saved the requestor a ton of time and stress by simply reading the request - redaction of BWC footage - adding a helpful disclaimer in the reply that the tech doesn't redact audio.

Redaction remains a huge issue in California. Agencies are looking for a quick fix, but no easy solutions have presented themselves. The marketing from Amped seemed to provide a glimmer of hope to the requestor. But those hopes were dashed when the advice from the order taker, three days late, was that there is no single redaction solution - which isn't entirely true.

As I informed the requestor on Friday evening, the Adobe Creative Suite has all the necessary tools to perform a standards-compliant redaction for California's new laws. The automatic tracking in PremierePro is the best currently available. Plus, at $52.99/month, the cost savings is substantial vs. FIVE or Replay.

Given that the requestor had until 1700 PDT to spend the allocated funds, not receiving a reply until the following Monday was not an option. Thankfully, I was able to answer his questions and get him on the right path. Next stop for him is my redaction class, featuring the Adobe tools (link).

Have a great day friends.

Generic Conditionals

In my retirement from police service, I'm busier than ever. One of the projects that I'm involved with is the creation of an instructional program in report writing for a national police service. In defining the instructional problem, I've found that the learner population has a problem with "factual conditionals." I've also noted this problem in the report writing samples of their forensic science practitioners.

Because people have problems with the relationship between the dependent and independent clauses, their reports are hard to read and interpret. What should be a clear statement - (dependent variable / action) resulted in (independent variable / result) - is often a confusion of meanings.

Often, what should be written as a conditional is written as a declarative statement. This problem hides potential meanings, and obscures avenues for inquiry.

For example, one of the sample videos that I use in my Content Analysis class (link) examines a traffic collision scene. A collision occurs as V1 attempts to turn left whilst exiting a parking garage. In the declarative statement, fault is obvious - turning left eludes to issues of right of way. What is missing is the conditional. If V1's progress is purposefully impeded, then the inquiry turns from a simple traffic collision to a "staged collision," - an entirely different line of inquiry.

When the responding officer records the statements of those involved, as well as witnesses, it becomes important to consider the statements in a "conditional sense." If Person 1's statement is true, then the scene would be arrayed thus." or "If Person 2's statement is true, then Person 1's statement is untrue." The conditional statements help frame the analysis of the statements and the evidence.

Using an example from the weekend's posts, "If headlight spread pattern analysis is a subset of digital / multimedia forensic science (comparisons), then the analysis must examine the recording of the pattern and not the pattern itself."

Just something to ponder on this beautiful Tuesday morning. Have a great day, my friends.

Fusion-based forensic science

When I first started out in multimedia forensics, there were just a few vendors offering tools to practitioners. Ten years later, there were a whole bunch. A further ten years later, and there are a relatively small group of vendors again.

When I sat down with Dr. Lenny Rudin of Cognitech a few weeks ago, to catch up and see what's new with him, we took that walk down memory lane that is customary for the "old guys." We talked all things Cognitech, and the state of the market.

I asked him about "codec / format support" in his tools and it seems that he favors the approach taken by the folks at Input-ACE - or the other way around I guess, since Cognitech has been around much, much longer than Input-Ace. The approach, seize the device and process the hard drive directly, is where Input-Ace is going in their partnership with Cellebrite and DME Forensics. Cognitech is there as well, seemingly with SalvationData.

This fusion-based approach makes sense. Do what you do, well, and let others do their thing well - then partner with them. It makes sense.

On the other end of the spectrum is Amped SRL. I remember standing in the exhibit space at LEVA Asheville when a certain developer was trying to offer his services (even his IP) to Amped. The response he received was that everything that is necessary to the task should be in FIVE. No plug-ins. No extra programs. That, and that Amped was happy with their current staff and development pace. In that conversation, Amped noted that FIVE should be the "go-to" for analysts. For a few years, this was the case. Now, increasingly, it's not.

Is everything that is necessary for an analyst's work in FIVE? Currently, there are features available in DVRConv that aren't available in FIVE. The main one that most customers wanted (when I was fielding calls in the US for Amped Software, Inc.) is the ability to "convert" everything. You see, in DVRConv, you just drag/drop a bunch of files and folders into the interface and it "converts" everything that's convertible - and maintains folder structure. FIVE, on the other hand, only "converts" one file type at a time, one folder at a time. The new "change frame rate on conversion" functionality is another example of something that is done in DVRConv that isn't done in FIVE.

Then, there's Reply. It seems that rather than continue to develop FIVE, Amped is cannibalizing it. Is Redaction moving over to Replay? It seems so. Yes, Replay now has improved tracking functions. But, what about all of those customers who bought FIVE for redaction. How are they feeling right now knowing that promises made about future functionality weren't promises kept. Promises made that weren't kept, you ask? Yes. Amped has yet to make good on their promise, made at LEVA Scottsdale, of adding audio redaction functionality to FIVE. Perhaps audio redaction will make it into Replay. Perhaps not.

Amped was once known as the provider of "specialists'" tools. Now, it's "generalizing" them as it migrates FIVE's functionality to Replay. A lot of US based agencies bought five years worth of support and upgrades when they purchased their original licenses. I'm wondering how that ROI looks now, given Amped's latest moves.

Meanwhile, Amped's competition has closed the gap and begun to overtake them in the marketplace. Dr. Rudin, sensing this, is about to make a big push with some new tools (hint: video authentication).

But, with all of this in mind, probably the oddest development is Foclar's entry into the US market. Foclar's Impress isn't a new product. It's been available in Europe for years. But, in terms of development and functionality, it's about where Amped was on FIVE's entry to the US market ten years ago. Let's see if Foclar has the wherewithal to play catch-up. As Amped raises it's prices for its products and services beyond what the market will bear, they'll leave the door open for Foclar to make some headway - should that European company choose to compete on price.

Gone - for the most part - are Signalscape, Salient Stills, and Ocean Systems. These companies still exist as fractions of their former selves. Signalscape's StarWitness is gone - replaced by interview room recorders and evidence processing machines. Salient Stills never really owned the IP in VideoFocus - which is now a part of DAC / Salient Sciences. As for Ocean Systems, it seems as though it exists at the will and pleasure of Larry Compton (who might be it's only employee). They're still offering the Omnivore, the Field Kit (Omnivore + a laptop), and Larry's training courses. Legacy products are on autopilot. Has ClearID been substantially improved / updated since Chris Russ' sacking years ago? I don't think so. I never understood the value proposition of taking 8 of the algorithms from OptiPix / FoveaPro (which were collectively priced at about $500 for about 70 algorithms) and charging much, much more. Sure, you got a new UI. But was that worth the price? I still have an old MacBook running PS CS2 just to run Chris' old stuff.

Have a great day my friends.

Review: Forensic science. The importance of identity in theory and practice

A paper was recently published over on Science Direct (link) exploring what may be a "crisis" in forensic science. Here's the abstract of the paper:

"There is growing consensus that there is a crisis in forensic science at the global scale. Whilst restricted resources are clearly part of the root causes of the crisis, a contested identity of forensic science is also a significant factor. A consensus is needed on the identity of forensic science that encompasses what forensic science ‘is’, and critically, what it is ‘for’. A consistent and cogent identity that is developed collaboratively and accepted across the entire justice system is critical for establishing the different attributes of the crisis and being able to articulate effective solutions. The degree to which forensic science is considered to be a coherent, interdisciplinary yet unified discipline will determine how forensic science develops, the challenges it is able to address, and how successful it will be in overcoming the current crisis."

The article seems an exploration of struggle for identity, as the title suggests. What is it? What are we to do with it? What happens when it's not done correctly? Who's responsible for reform? Curiously, even as the paper notes the work done on forensic science in the US, it omits reference to the 2017 A Framework for Harmonizing Forensic Science Practices and Digital/Multimedia Evidence (link). I point this out because the OSAC's document provides a solid definition of forensic science that can serve as a foundation from which to explore the paper's topic, as well as to provide a path forward for research and practice.

As a reminder, the definition of forensic science, "The systematic and coherent study of traces to address questions of authentication, identification, classification, reconstruction, and evaluation for a legal context." "A trace is any modification, subsequently observable, resulting from an event."

I think that within the OSAC's definitions, the goals outlined in the paper can be achieved.

Nevertheless, the likely root of the "crisis" as observed by the author can be seen in another area altogether - bias.

The topic of bias, and the many places it influences the justice process has been explored in depth. Particularly, these two papers (link) (link) explore the impact of bias from the standpoint of each of the stakeholders and the influence of modern media.

Another place where bias occurs is in the selection and continuation of cases by the prosecution. For example, what effect does linking a prosecutor's "win / loss" record to their promotability within their organization have on their decision making process? Once they've filed a case, is there evidence of an "escalation of commitment" when confronted with problems in proving their case? With "escalation of commitment" bias, the prosecutor may seek out "fringe techniques" in an attempt to support their theories of the case. Is it these "fringe techniques," not forensic science, that have contributed to the observed "crisis" - as was illustrated in Netflix's Exhibit A?

Still, I'm glad to see that people are starting to identify that there might be a problem in the practice of "forensic science." How that problem, or "crisis," is addressed will make all the difference in the world.

Have a good weekend my friends.

Can Amped's Authenticate assist in detecting "deep fakes?"

Short answer: no.

Long answer: everything old is indeed new again.

In 2016, I presented an information session on authentication at the LEVA Conference in Scottsdale, Az. Here's the session description:

Understanding Concepts of Image Authentication Workshop (2016)
This workshop is for those interested in the authentication of digital images. The workshop provides an overview of the techniques and skills necessary to perform basic authentication examinations using Amped Authenticate (Axon Detect) on digital images in a “forensic science” setting as well as to package, deliver, and present those findings in their local court room context.

At this year's LEVA Conference, it seems that the old topic is being dusted off, with one small addition. Can you spot the main difference between the 2016 session description (above) and this year's (below)?

Authenticate: The Beginners Guide to Image Authentication* (2019)
Image authentication techniques have multiplied over recent years. The simplicity of Image editing and the increase of bogus imagery, “Deep Fakes”, being identified in the media has, quite rightly, meant that methods to detect manipulation must be available to the legal system.

The difference: "Deep Fakes"

There's one big problem though. A "deep fake" is a video. Authenticate doesn't work on video, only images of a specific file type.

In my forensic multimedia analysis course (link), I feature a number of proposed techniques that address so-called "deep fake videos." All of the solutions work within a fusion-based methodology - requiring different tools and applications of those tools to identify the many components that make up fake videos.

But the wider question, why use the term "deep fakes" incorrectly to market an information session on an image authentication tool? Is this SEO gone wild? I don't know. At a minimum, it's confusing. At the maximum, it's deceptive. Hopefully, you weren't just going to learn about "deep fakes." If so, you're sure to be disappointed - again, Authenticate does not process or authenticate video.

Have a great day my friends.

keywords: audio forensics, video forensics, image forensics, audio analysis, video analysis, image analysis, forensic video, forensic audio, forensic image, digital forensics, forensic science, amped five, axon five, training, forensic audio analysis, forensic video analysis, forensic image analysis, amped software, amped software training, amped five training, axon five training, amped authenticate, amped authenticate training

The best artists steal

It's an old adage in the art world that good artists copy the work of their masters, but the great artists steal it (link). With that in mind, a reader of the blog pointed out the apparent similarities between the Camera Match Overlay in Input-Ace (link) and a patent on file at the USTPO (link).

If you read the marketing around the Overlay tool, you might begin to notice the similarities between it and the patent. But, don't be fooled. US Patent 9,305,401 (link) describes a system for building out a scene in 3D utilizing images from that scene, then conducting various measurements. Overlay does none of that. Overlay does exactly what the name implies - it allows you to take the user interface of Input-Ace and "overlay" on top of the user interface of the tool with which you are actually conducting your measurement exam. The idea is that you can infer the location of images present in the images loaded in ACE, and overlaid on the measurement tool, from the data in the measurement tool. Because of this, I don't think ACE is infringing on anyone's patents - in my opinion.

Yes, the patented process is operationalized into Cognitech's software, and that software is actually reconstructing the scene and performing the measurement exams. No, Input-Ace's Camera Match Overlay tool is not reconstructing a scene and is not used directly in conducting the measurement.  Yes, it does lend a hand in attempting to calculate the range of potential measurement values (and thus the error potential in the measurement), so you'll need to be extra careful in how you report and present your results using their methods. You'll also need to explain how you validated your unique results. Yes, you should validate any tool used in your work as well as the results that lead to an opinion / conclusion.

Thanks for reading. Keep the comments coming. Have a good day my friends.

Detecting 'Doctored' Images

Just a wee reminder that authenticating multimedia evidence (aka 'spotting doctored images on the internet') is a bit more complicated than finding the "doctor" in the image.

If you'd like to learn the underlying science behind authenticating this complicated evidence type, check our our course - Forensic Multimedia Authentication (link). Offered on-line as micro-learning, seats are always available and you learn at your own pace. Click on the link for more information, or to sign up today.

Everything old is new again

Last month, I took a look at the new Camera Match Overlay feature in Input-ACE. The Overlay feature can be used in conjunction with a 3D laser scanner and it's accompanying software to create demonstrative exhibits.

I'm not a big fan of the feature, preferring single image photogrammetry to conduct measurements within the evidence items without creating brand new exhibits. But, it occurred to me ... one of the nice things about getting old is you get to see history repeat itself.

When I first arrived at the LAPD in 2001, it had a video / image processing workstation from Cognitech. It was then that I first met Dr. Lenny Rudin. Last month, I was in Pasadena to present a lecture and ventured over to CalTech's amazing restaurant to have lunch with Dr. Rudin and catch up on what's new with him and Cognitech.

Our conversation bounced all over the place and eventually exceeded the allotted time that the restaurant had for lunch service. I like those conversations where time ceases to be a factor, just enjoying the topics and the company.

Of course the current state of the industry came up. Who's who and what's what. If you're reading this and you don't know the names (Cognitech and Dr. Rudin), that's a shame. Dr. Rudin is one of the founders of this thing we now call Forensic Multimedia Analysis, but has largely been written out of the history by those with a more commercial agenda. It's only the old folks who know the likes of Dr. Rudin, Dr. Russ, and the like.

Nevertheless, we discussed input-Ace's Overlay feature a bit, noting it's similarity to Cognitech's Measure package that began life in the 90's (now called AutoMeasure).

Everything old is new again. You can find the 1995 paper that describes Cognitech's Measure over at the SPIE (link). It's not an "overlay" procedure as such, but a single image photogrammetry method that builds out the 3D space within the 2D image. Unlike Overlay and its mixed-methods approach, Cognitech's Measure does have a validation history available in the literature.

Over lunch, we talked about where Cognitech's tools are now and what are the plans for the future of Cognitech's offerings. It has been a while since I've used Cognitech's tools. I'm looking forward to getting to know the new versions and the new products. Hopefully, if Dr. Rudin agrees to allow it, I'll be showcasing some of them in future blog posts.

Have a good week my friends.

Yes, you do need stats ... actually

Yesterday, I received the good news that my validation study of how a course in statistics could improve the statistical literacy of digital / multimedia forensic analysts when delivered on-line as micro-learning was published by the Chartered Society of Forensic Science in the UK. I got excited and put the good news on my LinkedIn feed.

Along with the usual emoji responses, I received the comment shown below.

Rather than simply comment there, I'd like to take the opportunity to illustrate the many ways in which it's not just me who says that the world of the digital evidence analyst can benefit from a solid foundation in statistics.

You see, the course was created because the relevant government bodies around the world have said, on a rather regular basis, that the investigative services and the forensic sciences need a solid foundation in statistics.

Starting at the US government level, there's the PCAST Report from 2016 (link): "NIST has also taken steps to address this issue by creating a new Forensic Science Center of Excellence, called the Center for Statistics and Applications in Forensic Evidence (CSAFE), that will focus its research efforts on improving the statistical foundation for latent prints, ballistics, tiremarks, handwriting, bloodstain patterns, toolmarks, pattern evidence analyses, and for computer and information systems, mobile devices, network traffic, social media, and GPS digital evidence analyses." (emphasis is mine)

CSAFE has already responded with some tools for digital forensic analysts (link).  The ASSOCR tool will help analysts "determine if two temporal event streams are from the same source by through this R package that implements a score-based likelihood ratio and coincidental match probability methods."

The HEISENBRGR toolset can be used to "match accounts on anonymous marketplaces, to figure out which of them belong to the same sellers."

What about NIST? What is the issue that NIST is taking steps to address? The PCAST report notes, "The 2009 NRC report called for studies to test whether various forensic methods are foundationally valid, including performing empirical tests of the accuracy of the results. It also called for the creation of a new, independent Federal agency to provide needed oversight of the forensic science system; standardization of terminology used in reporting and testifying about the results of forensic sciences; the removal of public forensic laboratories from the administrative control of law enforcement agencies; implementation of mandatory certification requirements for practitioners and mandatory accreditation programs for laboratories; research on human observer bias and sources of human error in forensic examinations; the development of tools for advancing measurement, validation, reliability, and proficiency testing in forensic science; and the strengthening and development of graduate and continuous education and training programs."

It's that last bit that prompted me to design and validate an instructional program in statistics for forensic analysts. But, it's the first sentence that speaks to the comment from LinkedIn. Analysts don't deal in absolutes or definite - binary. The world of the computer program may be binary, but the world certainly isn't. There is a natural variability to be found everywhere. But more to the comment's point, how does an analyst know that their "various forensic methods are foundationally valid, including performing empirical tests of the accuracy of the results."

Ahh... but, you're saying, all of your support is from the United States. It doesn't apply to the rest of the world. In that, you're wrong. Let's look at the UK.

In September 2018, Members of the Royal Statistical Society Statistics & Law section (link) submitted evidence (link) to a House of Lords Science and Technology Committee inquiry on Forensic Science. Question 2 asked, "what are the current strengths and weaknesses of forensic science in support of justice?" Here's the RSS' response. Notice the imbalance between strengths and weaknesses.

I've highlighted the relevant section as it relates to this topic. "poor quality of probabilistic reasoning and statistical evidence, for example, providing irrelevant information because the correct question is not asked. For example, an expert focused on the rarity of an event, rather than considering two competing explanations of an event."

Our course on statistics for forensic analysts seeks to teach probabilistic reasoning, exploring the differences between objective and subjective statistics, as well as the fact that most of the forensic sciences currently work in the wold of abductive reasoning (taking your best shot).

Now there's the accusation that digital analysts are often engaged in "push button forensics." We buy tools from vendors and hope that they're fit for purpose and accurate in their results. But are they? We don't know, so we validate our tools (hopefully). If you're trusting the market to deliver reliable, valid, and accurate tools, you may be disappointed. As the above referenced report notes, "What can be learned from the use of forensic science overseas? Seen from continental Europe, there has been a loss of an established institution (FSS) with a profound body of knowledge. Now research seems scattered among different actors (mainly academic), as commercial providers might have other priorities and limited resources to invest in fundamental research." (emphasis mine)

To the Royal Society's point, if you're a digital analyst and there's a challenge to your conclusions or opinions, on what do you base your response or your work? For example, you've retrieved photos from a computer or phone. Your tool automatically hashes the files. But, a cryptographic hash does not guarantee the authenticity of the file, only places a unique value into the process to handle questions of integrity. How do you conduct an authenticity examination without a knowledge of statistics? You can't. How do you validate your tools without a knowledge of statistics? You can't.

Over in Australia (link), there is agreement on the need for training and research - just what I've presented. "There is however one aspect of the report with which the Society is in complete agreement; the need for both continuous training and research in forensic science. We are also aware of the lack of funding for this research and therefore support the recommendation of PCAST that this is essential if our science is to continue to develop into the future."

To conclude, yes, you do need training / education in statistics if you're engaged in any forensic science discipline. Many practitioners arrive in their fields with advanced college degrees and thus will have had exposure to stats in college. But, on the digital / multimedia side, many arrive in their fields from the ranks of the visible policing services. They may not have a college degree. They may only have tool-specific training and may be completely unaware of the many issues surrounding their discipline. It's for this group that I've designed, created, and now validated my stats class. It's made in the US, to be sure, but it's informed by the global resources listed in this post - and many others.

I hope to see you in class.

Demonstrative exhibits and reconstruction of events

My last post generated a few responses that I want to address in a separate post, as opposed to editing the previous post. A few people got the impression that I was saying that what folks are doing with ACE's Camera Match Overlay isn't "forensic science" or "forensic video analysis." That's not at all what I was saying. Let's dive into that question to explain.

First, the definition of forensic science again: "Forensic science is the systematic and coherent study of traces to address questions of authentication, identification, classification, reconstruction, and evaluation for a legal context."

Forensic science thus includes:

• authentication
• identification
• classification
• reconstruction
• evaluation

The type of work performed in the examples on ACE's websit clearly indicate that the Camera Match Overlay is a tool for reconstruction. This is how the product is being positioned in the market. Camera Match Overlay is an addition to ACE, and not part of it's basic functionality. If you're not involved in reconstruction, you can skip the Overlay tool and save a few bucks.

What ACE's basic functionality excels at is "evaluation." What's in the container? How should it be handled? Those File Triage type questions. Once answered, it's a short trip to repackaging the data in a format that is playable for the end user. But remember, evaluation has it's own set of rules.

ACE is also really good at reconstruction - the syncing and linking separate video streams. Reconstruction has it's own rules, workflow, and toolset. Reconstruction attempts to illustrate a theory of the sequence of events in question. Reconstruction is not authentication, identification, or classification - which have their own rules, workflows, and toolsets.

With that in mind, the second set of questions deals with training and tools.

A 16 hour training session on Camera Match Overlay's operation and use is likely sufficient for a technician to be able know which buttons do what functions across a variety of use cases. What it is not is a comprehensive education on photogrammetry. Because the focus of tool-specific training is the tool, we've split off the foundational education side as separate, non-tool-specific deep dives so that you get an unbiased exploration of the discipline from a neutral third party. If you're giving technician level testimony (no opinion offered), tool-specific training is likely enough. But, if you're offering an opinion (even passively), then you need a foundational education in the discipline in which you're engaged.

The third set of questions deals with the legal aspects of evidence hearings.

Keeping in mind that I'm not an attorney, consider the evidence hearing's rules (Frye / Daubert). Both types of hearings have as a foundational element what is commonly known as the “general acceptance test.” Generally accepted scientific methods are admissible, and those that are not sufficiently established are inadmissible.

Can a tool or technique without a history of publication or validation be "sufficiently established?"

Camera Match Overlay technology is new. It's the "shiny new object" for reconstruction exercises. The resulting videos become an amazing demonstrative aid to one's testimony, using the power of stunning visuals to illustrate one's theory of a case. But, bear in mind that it's only a demonstrative illustration of a single theory. There may be other theories worthy of exploration. If you're engaged in science, Daubert requires that you explore those other theories. If you're just engaged in trial support, and thus have no opinion, then go right ahead and create those stunning visuals.

All of this requires a bit of honesty. When I've simply retrieved files, I'm engaged in technician level work. When I've clarified and enlarged a frame, I've engaged in technician level work. These activities can support an analysis, and thus help to illustrate one's opinion, but they're not "analysis" in and of themselves. From the Frye ruling, "while courts will go a long way in admitting expert testimony deduced from a well-recognized scientific principle of discovery, the thing from which the deduction is made must be sufficiently established to have gained general acceptance in the field in which it belongs." When I want to offer an opinion, I must use tools and techniques that have been sufficiently established in my field. If I want to use "reconstruction" tools to reinforce my opinion in an "identification" exam, those tools must be sufficiently established within the realm of "identification." At this time, there are no studies validating the use of the Camera Match Overlay technology and methods for "identification" or "classification."

There are no studies involving the product at all. It's brand new. I'm certainly open to participating in validation studies, if anyone want to engage in our services. But for now, Camera Match Overlay seems to belong to the world of reconstruction until validated otherwise.

Thanks for reading. Have a great day my friends.

Camera Match Overlay?

Following up on yesterday's post, we were dealing with an interesting issue that concerns the mixed methods approach of inserting a single frame taken from CCTV footage (previous event) into a laser-generated point cloud scan (current event) processed by [ fill in the blank ] software. Let's continue ...

Images from The Matrix. (c) 1999 Warner Bros.

One of my favourite scenes from the 1999 hit movie, The Matrix, gives us a clue to how this conundrum will be addressed in the near term.

Spoon boy: Do not try and bend the spoon. That's impossible. Instead... only try to realize the truth.

Neo: What truth?

Spoon boy: There is no spoon.

Neo: There is no spoon?

Spoon boy: Then you'll see, that it is not the spoon that bends, it is only yourself.

Over in southeast Washington, the folks behind iNPUT-ACE have come up with a novel approach, essentially telling us that there is no spoon to bend.

I've been curious about the Camera Match Overlay feature in ACE for a while now. I was aware of what folks were doing with point clouds and have seen some of the demonstratives in YouTube videos of Grant and Andrew's testimonial experiences. As usual, the Fredericks family has been quite open with what they're on about. It's good for business after all.

So why does the new overlay tool remind me of the spoon?

The Camera Match Overlay does not bring a point cloud into ACE. The Camera Match Overlay does not integrate with a scanner's software to insert CCTV frames or video. What the Camera Match Overlay does is quite novel, and entirely sidesteps the issue of a mixed-methods approach.

You see, when using the Camera Match Overlay tool in iNPUT-ACE, you're working in ACE to "toggle the opacity" of ACE (where the CCTV footage exists) whilst you reposition ACE's UI, and thus the footage, as an overlay to your preferred scanning tool (where the point cloud exists). This is a manual process that requires that you match the "zoom level" of the scene in your scanning tool to that of the CCTV frame(s), as well as manually positioning the ACE overlay. But, once "eyeballed," you can lock in this positioning in ACE's UI. As a technical aside, I would imagine that your GPU and monitor's performance will be critical, as well as your eyesight, in properly "eyeballing" the alignment.

Thus, there is no spoon. ACE serves up the CCTV footage (projects) with a faded opacity so that you can work in your 3D tool.  I know that the Fredericks' use the legacy tool, SceneVision-3D to work with point cloud data. But, I'm sure that the overlay will work with any vendors' tools once properly set-up.

Now, on the science side, it seems that ACE can help you make some calculations of error with the assistance of your scanning tool, producing a report to address margin of error. And ... as you should know, margin of error is directly related to your region of interest. The farther away from the camera your object of interest is, the larger the error will be ... as well as your nominal resolution. It seems that ACE reports the error in a few ways - with the final number being a function of "scanner accuracy," "calibration accuracy," and "resolution accuracy." The scanner accuracy comes from your scanner. The calibration accuracy seems to come from the overlay (not sure how it handles potential mis-alignment). The resolution accuracy is really just nominal resolution for the region of interest.

For close up work, it seems that error won't be much of a problem unless, for example, your subject is within the normal distribution for human heights. But at longer distances, the error can be in the ~2'-4' range; not much if you're trying to measure a skid mark but too big if you're trying to measure a person's height. Back to that manual process of aligning views, with a high nominal resolution, being off by just a few pixels will significantly affect your error - so again, pay close attention to that step in your process.

One problem you'll have to overcome is that most scanning software lack any sort of video output to save out your pretty demonstratives. You won't be able to export your demonstratives out of ACE either -  remember, it's just overlaying the UI. But, if you've got an Omnivore, or Camtasia, you've got all that you need. Remember, these are demonstratives. You're working in the world of the abductive - taking your best shot to demonstrate your theory of the case in which you are working.

Now for the disclaimers.

Don't try this without training and experience in every single piece of your puzzle. Given that these demonstratives will be used to illustrate your opinion, please get certified as an analyst. Make sure that you understand the foundation science and rules of this discipline. Get to ACE's training. Make sure that training includes an appropriate amount of time on the Camera Match Overlay tool. Make sure that the tool works with your chosen scanner software. Make sure you're trained and experienced on that tool as well. Make sure that you have training in Forensic Photographic Comparison as well as Forensic Photogrammetry. If you're using Camtasia, get trained and certified there as well (I did). If you're using an Omnivore, get trained. Why? Because in the world of demonstratives, you're demonstrating your theories, opinions, and conclusions. It's your opinion, thus it becomes all about you and the foundations of your work.

Have a good day, my friends.

Using FARO Scene 2019 to measure within CCTV images?

Last month, FARO updated their Scene software's user guide for 2019. I was very curious to see if the guidance had changed on a controversial issue that I addressed in a letter to the editor of the Journal of Forensic Identification (link) last year.

The heart of the issue deals with the mixed methods approach of inserting a single frame taken from CCTV footage (previous event) into a laser-generated point cloud scan (current event) processed by FARO's Scene software. The question on everyone's mind is this: can you measure items / objects / subjects that are present in the CCTV image but are not present in the point cloud scan?

Let's see what the new user guide has to say.

page 80 - 08m86e00_FARO_SCENE_User_Manual_2019.1.pdf

According to FARO (see above graphic), "There are three ways to use images in SCENE:

• Images can be added with their original resolution to the workspace and thus provide additional information about the scan environment.
• Images can be added with their original resolution to the workspace and thus provide additional information about the scan environment. These images are imported into the 3D world into virtual scans with their full resolution. Such images will be interpreted like a high resolution scan of a plane surface and can be placed on arbitrary positions in the 3D world.
• Images can be used to add color information to already existing scan points."

What is the primary reason for adding images into the scan environment? Images can be used to add "richness" and "texture" to the scan. Here's how:

page 81 of the pdf file

If you've worked with these types of scans, you know that the result is a flat grey colour. The role of images, or virtual scans, is to add information back into the scene. In the example above, the scan only registers the location of the picture frame - but not the contents of the frame. Thus, the virtual scan is utilized to add the "information" about the picture into the scan. 

But that's not what you want to do, is it. You want to measure an item that isn't present in the scan.

page 85 of the pdf file

The Place in 3D function seems like it might be it. But, it's not. The Place in 3D function allows the user to place a 2D representation of a region of interest into the scan. The user then must associate points in common between the 2D image and the 3D scan.

page 86  of the pdf file

Now that you've registered the scan and associated points in common, you're ready to measure ... or are you?

You don't want to measure items in the scan. You want to measure items not in the scan. Unfortunately, you can't with this tool. There are no corresponding points between the thing you want to measure and the point cloud. The FARO Scene manual lists the procedure for measuring points in common. It does not show users how to measure the woman in the scene above, who is not present in the point cloud scan.

Thus, the question that begs asking: if a "forensic video analyst" has a scan, some CCTV footage, and FARO Scene, where is the measurement happening? If it's happening in Scene, which doesn't support the function, how accurate are the measurements? If it's not happening in FARO Scene, where is it happening?

If the measurement is not happening in Scene, how does the "analyst" get the image out? 

Section 11 deals with exporting from Scene. "11.5.3 Exporting the images of the scans to.jpg format

• In the Structure View, right-click the scan, then select Export> PanoramicImages. Select Scan Resolution to create images that have the same color resolution as the scan, or select Full Color Resolution if you want to create panoramic images with the highest color quality possible, and which are compensated to remove the offset between the two halves of the scan, as well as any distortion cause by the scanner’s rotation. Full color resolution panoramas have a white stripe at the bottom of the picture because the proportions of the scan and the picture are different. (Scans made with FARO scanners versions M70, S70, S350 and later create 160 megapixel images. Scans from older scanners only output panoramic images with 40 megapixel images.)

Which requires another question be asked: what's the point of bringing the CCTV frame into the point cloud scan only to bring it out again to measure?

Why is this a big issue? Science. The mixed-methods approach has yet to be validated in a general sense, with that validation published in order for "the community" to investigate the methodology and results and attempt to replicate the published experiment.

Has there ever been an evidence hearing (Frye / Daubert hearing) on this? If you're aware of one, I'd sure like to know. Not that evidence hearings should replace validation studies, or that evidence hearings should come before validation studies. I'm just aware, anecdotally, that people are performing this technique and testifying as to their results / conclusions. What isn't available in the anecdotes are any sense of the science or testimony as to validity. 

Let's take a look at why this is important.

You'll remember that prior to Daubert, Frye was the law of the land. The Frye standard is commonly referred to as the “general acceptance test” under which generally accepted scientific methods are admissible, and those that are not sufficiently established are inadmissible. Can something without a history of publication or validation be "sufficiently established?"

The Frye Standard comes from the case Frye v. United States, 293 F. 1013 (D.C. Cir. 1923) in which the defendant, who had been charged with second degree murder, sought to introduce testimony from the scientist who conducted a lie detector test. The D.C. Court of Appeals weighed expert testimony regarding the reliability of lie detector test results. The court noted: Just when a scientific principle of discovery crosses the line between the experimental and demonstrable stages is difficult to define…. [W]hile courts will go a long way in admitting expert testimony deduced from a well-recognized scientific principle of discovery, the thing from which the deduction is made must be sufficiently established to have gained general acceptance in the field in which it belongs.

The last part of that sentence is where I want to go with Frye - in the field in which it belongs. 

3D laser scans of traffic collisions are wonderful. 3D laser scans of scenes where the police have used force are a good thing. These recording methods capture the smallest details of the scene - present at the time of the scan (aka "now"). They have done so well in their documentation of scenes that they've become generally accepted among "scene recinstructionists." But, "scene reconstruction" is an entirely different function than digital / multimedia forensic analysis. For the FVA community, other methods of photogrammetry are generally accepted (e.g. single image photogrammetry). If the "scene reconstruction" folks want to work in the domain of digital / multimedia forensic analysis, then they must follow it's rules ... not theirs. This is key. Remember, validity deals with the accuracy of the measure as well as the appropriateness of the process / tool.

I was initially excited to see that FARO had issued an update. But, as you can see, not much has changed from the previous version in terms of measuring objects that aren't in the scan. Perhaps next time ...

Fusion-based approach to digital and multimedia forensics

No tool is perfect. The problem for digital / multimedia forensic analysts has always been one of "does Tool X support the types of files that I see in my lab." Tool manufacturers do their best to support their customers, but there are just so darned many file types to keep track of.

Early on, we saw this in Los Angeles with mobile phones. California, being a CDMA state, wasn't well supported by the mobile forensics tools. We found a Korean company with an office in Los Angeles that made an amazing parsing tool that could find things in the physical that no one else could. I still rely upon FinalMobile and the staff at FinalData to help me with the processing and analysis of the most difficult files.

The same problem exists in the processing of evidence from DVRs. In Los Angeles, we were seeing DVRs from China that weren't being sold elsewhere in the US. Thus, for the tool makers that were building acquisition templates based upon what was around their developers, their tools never seemed to work well on the DVRs that I encountered in LA. I found a Chinese company that had SDK access to the Chinese DVR manufacturers ... and thus a more comprehensive support for the types of DVRs that I was seeing in LA. They're not one-at-a-timing DVRs. They work directly and cooperatively with the manufacturers to assure that they support the DVRs that are produced in China.

In terms of image / video processing, as innovation from Italy winds down and their prices in the US and Canada increase (in some cases quite dramatically), some really cool developments are happening in south-eastern Washington state. Do I believe that you should only have FIVE, or only have Input-Ace? Certainly not. There are things that each does really well, and things that each does poorly or not at all. If you can afford to, get both. If you can't afford both, look at the type of work that you're doing and see which is the most appropriate for you.

I've always preached a "fusion-based" approach to digital / multimedia forensic analysis - otherwise known as "buy one of each tool" if you can. In doing so, you'll have the greatest coverage possible for the evidence that arrives at your lab. With DVRs, there are things to like about SalvationData's Video Investigator Portable (that Chinese company mentioned above) and there are things to like about DVR Examiner. SalvationData's VIP can acquire NTFS discs - like those from Exacq Vision (we see those a lot in California). It can also perform the acquisition over the network (even over WiFi) for those cases where seizing the DVR isn't practical or legal (as is the case in California with the new digital privacy laws). It can find file fragments and organize them logically. On the other hand, not everyone can purchase software direct from China. DVR Examiner comes from Colorado - where there are people to pick up the phone when you're working (not in the central Asian time zone). It's convenient. It's also available as a package deal with Input-Ace and the whole universe of tools and services from Cellebrite.

Gone are the days of doing everything in Photoshop. A fusion-based approach just makes sense in today's world. With this in mind, you'll see more product reviews and deep dives on fusion-based workflows around some complex cases in future posts as well as in our on-line learning portal. Stay tuned. It's going to be fun.

Experimental Design

Before one begins any sort of research, one usually surveys the literature on the topic to see if any research has been completed and what, if anything, was concluded. Sure, the researcher has a general idea about what they want to study, but a literature review helps to inform and refine the eventual design of the study. According to Shields and Rangarajan (2013), there's a difference between the process of reviewing the literature and a finished work or product known as a literature review. The process of reviewing the literature is often ongoing and informs many aspects of the empirical research project. See what I just did, I discovered some research on literature reviews, and inserted the summary into my paragraph. Usually, there's an accompanying citation. Here it is: Shields, P., Rangarjan, N. (2013). A Playbook for Research Methods: Integrating Conceptual Frameworks and Project Management. Stillwater, Oklahoma: New Forums Press. ISBN 1-58107-247-3.

I received some feedback about the few posts I've written regarding "headlight spread pattern analysis." One was very intriguing - "... assume the premise is true, that there is uniqueness that can be discovered through experimentation. Where would you begin? What would the experimental design look like?" Hmmm....

Given the term, "headlight spread pattern analysis," there are four distinct elements - "headlamps," "the diffusion of light," "pattern matching," and a methodology for "analysis." Each of these would need to handled separately before adding the next element - the recording of this diffusion of light within a scene.

Let's just do a bit of research on the types of headlamps available to the general commercial market, leaving the other three elements for later.

Our first discovery is that we must separate the "lamp" from the "bulb." The bulb provides the "light" and the "lamp" is a system for the projection of that light.

For the lamp's housing, there are two general types: "reflector" and "projector." Of the light sources ("bulb"), there are several available types: Tungsten, Tungsten-halogen, High-intensity discharge (HID), LED, Laser. Of the "filament" type lamps, there are over 35 types available in for sale in the US, and covered by the World Forum for Harmonization of Vehicle Regulations (ECE Regulations), which develops and maintains international-consensus regulations on light sources acceptable for use in lamps on vehicles and trailers type-approved for use in countries that recognise the UN Regulations.

Given the eventual experimental design, it's important to note that the US and Canada "self-certify" compliance with the ECE Regulations. No prior verification is required by a governmental agency or authorised testing entity before the equipment can be imported, sold, installed, or used.

For a bulb's operation, there are variables to consider. There's voltage (usually 12V) and wattage (between 20w - 75w) - collectively known as "Nominal Power." Then there's "luminous flux." In photometry, luminous flux or luminous power is the measure of the perceived power of light. It differs from radiant flux, the measure of the total power of electromagnetic radiation (including infrared, ultraviolet, and visible light), in that luminous flux is adjusted to reflect the varying sensitivity of the human eye to different wavelengths of light.

Lots of big words there. But two stand out - luminous flux (the measure of the perceived power of light) and radiant flux (the measure of the total power of electromagnetic radiation). For our experiment, we'll need to differentiate between these as someone / some people are going to compare patterns (perception is subjective). We'll also need an objective measure of the total power of our samples. Luminous flux is used in the standard as the point of headlamps is to improve the drivers perception of the scene in front of them as they drive.

Luminous flux is measured in lumens. On our list of bulbs, the luminous flux values are reported as being between 800lm and 1750lm with a tolerance of between +/-10% and +/15%. This makes the range between 680lm and 2012.5lm. It's important to remember that the performance of a bulb over it's life span is not binary (e.g. 1550lm constantly until is stops working). Performance of lamps degrade over time.

Back to the lamp as a system. There are the general types of fixed lamps - they're bolted on to the front of the vehicle by at least four fasteners. These types need to be "aimed" at the time of installation, which can shift over time as the fasteners loosen. There are also "automatic" lamps, which feature some form of "beam aim control." These "beam aim control" types include, headlamp leveling systems, directional headlamps, advanced front-lighting system (AFS), automatic beam switching, Intelligent Light System, adaptive highbeam, and glare-free high beam and pixel light.

Now in the cases that I've reviewed, it seems that "headlight spread pattern analysis" was employed when a proper vehicle make / model determination failed due to a lack of available detail - usually due to a low nominal resolution.

Given what I've just shared above, about the potential variables in our study, an important revelation emerges. If there is insufficient nominal resolution to conduct a vehicle make / model determination, which considers class characteristics like presence / quantity of features like doors and windows before considering the presence / quantity / type of features within those items, then how could there be a determination as to type of lamp system and bulb that would be necessary for any comparison of headlight spread pattern? What if there's a general match of the shape of the pattern, but the quality of light is wrong? Or, what if the recorder's recording process and compression scheme corrupt the shape of the light dispersal or change the quality of the light? How then is a "scientific" comparison possible?

Short answer - it's not. This is one of the ways in which "forensics" (rhetoric) is used to mask a lack of science in "forensic science."

But, let's take a look at that question in a different way. Given all of the variables listed above, what would a normal distribution of "headlight spread patterns" "look like" (observed without recoding) for each of the possible combinations of system, bulb, and mounted position? What would they look like after being recorded on a DVR? This adds more variables to the equation.

For the recording system, there's the camera / lens combination, there's the transmission method, and there's the recorder's frame rate and compression scheme to contend with. Sure, you have the evidence item. But you don't know if the system was operating "normally" during that recording, or what "normal" even is until you produce a performance model of the system's operation in the recording of everything listed above. You'll need the "ground truth" of the recorder's capabilities in order to perform a proper experiment.

Remember, the recording may be "known" - meaning you retrieved it from the system and controlled it's custody such that the integrity of the file is not in question. But, what is unknown is the make / model of the vehicle. THIS CAN'T BE PRESUPPOSED. IT MUST BE DETERMINED.

In the cases that I reviewed, each comparison was performed against a presupposed make / model of vehicle - the "suspect's vehicle." If convenience samples were employed for a "comparison," then it was a few handy cars of the same make / model / year as the accused's vehicle. THIS IS NOT A VEHICLE DETERMINATION. This is no different than a single-person line-up, or a "show-up." This method has no relationship with science.

Back to the literature review and how it may inform a future experimental design.

What I've discovered is that the quantity of variables is quite large. Actually, the quantity of system types, then the variables within those systems, is quite large. This is before considering how these will be recorded by a given recorder. This information would be required to validate case work, aka a CASE STUDY. A case study is only applicable to that one case. If one wanted to validate the technique, then an appropriate amount of recorders would need to be included (proper samples of complete system types).

Given all of this, the cost of a single case study would be beyond the budget of most investigative agencies. It's certainly beyond my budget. The cost of testing the questions, "headlight spread pattern analysis has no validity" (H null) and "headlight spread pattern analysis has validity" (H1) would be massive.

Nevertheless, given all of the above, to conclude "match" - it is "the accused's vehicle" - one must rule out all other potential vehicles. Given that estimates put the number of cars and trucks in the United States at between 250-260 million vehicles for a country with 318 million people, then "match" says "to the exclusion of between 250-260 million vehicles" - which doesn't include the random Canadian or Mexican who drove their car / truck across the border to go shopping at Target. Because of this, "analysts" usually equivocate and use terms like "consistent with" or "can't include / exclude." Which, again, is rhetoric - not science.

Authentication education - now available on-line

I've been teaching authentication for many years. I've been all over the US and Canada presenting in classes big and small. Now, I've taken the big leap. My comprehensive educational course, Introduction to Forensic Multimedia Authentication, is available on-line as micro-learning.

This course lays the foundation for your work with your preferred tool. Current research, best practices, standards, and work flows are covered for audio, images, video, and the meta-data that are found in evidence files. If you'd like to see what is covered, here's the syllabus. This offering is the same as the in-person 40 hour course, which you can now do on-line. Because it's on-line, I can offer it to you for a much lower price than an in-person offering. Plus, you can take up to 60 days to complete it.

This course moves beyond the buttons of your preferred tool to lay the foundations for the work that you do. As such, it will assist you in explaining your work in your reports and in your testimony.

Click on the course link above. Check out the syllabus. Sign up today.

Welcome

Welcome to the Forensic Multimedia Analysis blog (formerly the Forensic Photoshop blog).

With the latest developments in the analysis of multimedia (video, audio, images, and metadata), we move the discussion beyond a single piece of software to include (in no particular order) processing and analysis fundamentals, court cases, upcoming training offerings, product reviews, current research, standards and practices, industry events and trends, and much more.

Digital / multimedia forensic analysis covers the domains of:

• Authentication
• Photogrammetry
• Photographic Comparison
• Photographic Content Analysis

Clarification, enhancement, and restoration are processes that can occur within the domains, but aren't domains in and of themselves.

We use the term digital / multimedia forensic analysis, as opposed to forensic video analysis or forensic audio analysis as we acknowledge that modern multimedia evidence potentially contains audio, images, video, as well as metadata. Thus, we need to be able to process and analyze everything that's found in the evidence files that we receive.

It's also important to define "forensic science." For this, I'll refer to "A Framework to Harmonize Forensic Science Practices and Digital/Multimedia Evidence." OSAC Task Group on Digital/Multimedia Science. 2017 (link): "Forensic science is the systematic and coherent study of traces to address questions of authentication, identification, classification, reconstruction, and evaluation for a legal context."

What is a trace? "A trace is any modification, subsequently observable, resulting from an event." You walk within the view of a CCTV system, you leave a trace of your presence within that system. You send a text, you leave a trace on your phone.

Within this framework, and wherever possible, we'll frame our discussion around standards and science. Validity, reliability, and reproducibility will be our goals ... not to present something unique and proprietary that only we can do here, but to illustrate the science behind the tools and techniques so that you can do it too.

I hope you enjoy your time here.

Jaime

First, do no harm

In an interesting article over at The Guardian, Hannah Fry, an associate professor in the mathematics of cities at University College London, noted that mathematicians, computer engineers, and scientists in related fields should take a Hippocratic oath to protect the public from powerful new technologies under development in laboratories and tech firms. She went on to say that "the ethical pledge would commit scientists to think deeply about the possible applications of their work and compel them to pursue only those that, at the least, do no harm to society."

I couldn't agree more. I would add forensic analysts to the list of people who should take that oath.

I look at the state of the digital / multimedia analysis industry and see places where this "do no harm" pledge would re-orient the relationship that practitioners have with science.

Yes, as someone who swore an oath to protect and defend the Constitution of the United States (as well as the State of California), and as someone who had Bill Bratton's "Constitutional Policing" beaten into him (not literally, people), I understand fully the relationship between the State and the Citizen. In the justice system, it is for the prosecution to offer evidence (proof) of their assertions. This simple premise - innocent until proven guilty - separates the US from many 'first world" countries.

I've been watching several trials around the country and noticed an alarming trend - junk procedures. Yes, junk procedures and not junk science as there seems to be no science to their procedures - which serve as a pretty frame for their lofty rhetoric. This trend can be beaten back, if the sides agree to stick to the rules and do no harm.

Realizing that I've spent a majority of my career as an analyst in California, and that California is a Frye state, I'll start there in explaining how we, as an industry, can avoid junk status and reform ourselves. Let's take a look.

You might remember that prior to Daubert, Frye was the law of the land. The Frye standard is commonly referred to as the “general acceptance test” under which generally accepted scientific methods are admissible, and those that are not sufficiently established are inadmissible.

The Frye Standard comes from the case Frye v. United States, 293 F. 1013 (D.C. Cir. 1923) in which the defendant, who had been charged with second degree murder, sought to introduce testimony from the scientist who conducted a lie detector test.

The D.C. Court of Appeals weighed expert testimony regarding the reliability of lie detector test results. The court noted: Just when a scientific principle of discovery crosses the line between the experimental and demonstrable stages is difficult to define…. [W]hile courts will go a long way in admitting expert testimony deduced from a well-recognized scientific principle of discovery, the thing from which the deduction is made must be sufficiently established to have gained general acceptance in the field in which it belongs."

The last part of that sentence is where I want to go with Frye - "in the field in which it belongs."

There is an emerging trend, highlighted in the Netflix series Exhibit A, where [ fill in the type of unrelated technician ] is venturing into digital / multimedia analysis and working cases. They're not using the generally accepted methods within the  digital / multimedia analysis community. They're not following ASTM standards / guidelines. They're not following SWGDE's best practices. They're doing the work from their own point of view, using the tools and techniques common to their discipline. Often times, their discipline is not scientific at all, and thus there is no research or validation history on their methods. They're doing what they do, using the tools they know, but in a field where it doesn't belong. Their tools and techniques may be fine in their discipline - but there has been no research on their use in our discipline. Thus, before they engage in our discipline, they should validate them appropriately - in order to do no harm.

Let's look at this not from the standpoint of my opinion on the matter. Let's look at this from the five-part Daubert test.

1. Whether the theory or technique in question can be and has been tested. Has the use of [ pick the method ] been tested? Remember, a case study is not sufficient testing of a methodology according to Daubert.

2. Whether it has been subjected to peer review and publication. There are so few of us publishing papers, and so few places to publish, that this is a big problem in our industry. Combine that with the fact that most publications are behind paywalls, making research on a topic very expensive.

3. It's known or potential error rate. If there is no study, there really can't be a known error rate.

4. The existence and maintenance of standards controlling its operation. If it's a brand new trend, then there really hasn't been time for the standards bodies to catch up.

5. Whether it has attracted widespread acceptance within a relevant scientific community. The key word for me is not "community" but "scientific." There are many "communities" in this industry that aren't at all "scientific." Membership organizations in our discipline focus on rapidly sharing information amongst members, not advancing the cause of science.

So pick the emerging trend. Pick "Headlight Spread Pattern." Pick "Laser Scan Enabled Reverse Projection." Jump into any research portal - EBSCO, ProQuest, or even Google Scholar. Type in the method being offered. See the results ...

The problem expands when someone finds an article, like the one I critiqued here, that seemingly supports what they want to do, whilst ignoring the article's limitations section or the other articles that may refute the assertions. This speaks to the need for a "research methods" requirement in analysts' certification programs.

If you're venturing into novel space, did you validate your tool set? Do you know how? Would you like training? We can help. But, remember that people's lives, liberty, and property are at stake (and they're innocent until proven guilty), can we at least agree to begin our inquiries from the standpoint of "first do no harm?"