Is it practical?

Continuing the discussion of certification in general, the NAS Report notes the following about certification programs, "The American Bar Association has recommended that certification standards be required of examiners, including “demanding written examinations, proficiency testing, continuing education, recertification procedures, an ethical code, and effective disciplinary procedures.” Let's break these down.

Demanding written exams. What does this mean? Are "written exams" composed of essay questions where one explains topics in depth? Are "written exams" objective tests, like multiple choice exams? What makes an exam "demanding?"

Could an examination body survey the entirety of my written work, books, papers, speeches, this blog, and so on? Could they judge my proficiency based on my written work? I'm not sure. But, pulling from my academic life, it is entirely possible for someone to be proficient in their craft, but be non-verbal. How do you conduct a "demanding written exam" on a non-verbal person? What about differences of language? Do we insist upon all tests being in US English? How do we handle the issue of testing vulnerable populations? Given my extensive work in the autistic community, both academically and professionally, these are relevant questions.

Proficiency testing. Does this belong to an independent certification program or within a vendor's training program? For example, who is a better judge of my proficiency with my chosen tools, Amped Software, Inc. (vendor) or the IAI (third party)? What does the IAI know about FIVE and Authenticate? Nothing, actually. They're still focussed on Photoshop. How do I know this? Their reading list has George Reis' Photoshop book as one of the sources for the test. Whilst still relevant, and quite good, it's a 10 year old treatise on George's favorite PS techniques in no particular order. Given that I no longer use Photoshop in my analytical workflow, how is a test on Photoshop helpful in assessing my proficiency as an examiner?

Perhaps the "requirement" could be met with a review of an analyst's case documentation. Submit a few reports, have the panel examine them in light of the ASTM and SWGDE standards / guidelines.

Continuing education. There comes a point in time, when you've been doing this for a while, that you run out of classes to take. Some, that are available, are cost prohibitive. That's a large part of why I've moved a lot of the classes that I teach to an LMS as micro-learning. As an example, if you can't get in to your local college to take a statistics course, you can take Statistics for Forensic Analysts on-line at an affordable price. Same for redaction, Redaction for Standards Compliance is offered on-line featuring a variety of software platforms.

I've been around the US and Canada quite a bit. I realize that traveling to training is not always an option. I think the internet offers some amazing options to level the playing field as regards continuing education. The question becomes, will your chosen certification program accept your completion certificate from an on-line course or provider?

Recertification procedures. I think the important part here is not only that the individual circle back and re-certify, but that the certification program continually monitors the industry and keeps the certification program up to date. Four years from now, when my AVFA certification is due for renewal, I should not be presented with the same test instrument. Laws and technology change. So too should the certification program. Again, I love George Reis to death, but is his book still relevant in a modern multimedia workflow? Perhaps. But are there other, more appropriate texts? Yes. Absolutely. What about listing Dr. John Russ' Image Processing Cookbook as well, given the preference for Photoshop? Or his Image Processing Handbook, for a more academic look at image processing? If price is an issue, they could choose Fundamentals of Digital Image Processing by Dr. Anil Jain. It's a text book that can be bought used for under $10.

Ethical code. Here's LEVA's Code of Ethics. This post is an example of article 5 in LEVA's COE, offering constructive criticism. Interestingly, article 9 is what prevented me from ever being LEVA certified. Article 9 states, "An Analyst shall maintain the confidentiality of evidence received and work undertaken in accordance with any such request by the submitting agency." It was the policy of the Los Angeles Police Dept. that case work remain confidential and no evidence leave the secure facility without permission. When I sought permission to begin the process of certification, it was flatly denied. Given the LEVA certification methodology, if you can't present one of your cases, you can't get certified. Thus, in honoring their COE, I was prevented from ever being certified whilst in police service. Now, in retirement and in private practice, LEVA discriminates against me for not being a current public service employee by charging confiscatory prices for their offerings.

For the IAI's COE, click here. It begins on pg 79 of the linked document.

Effective disciplinary procedures. One of the things that we do is review opposing counsel's analysis. The examiner on the other side of any case should expect a thorough review of not only their work but their credentials as submitted through the discovery process. I have lost track of the amount of times I have encountered embellishments and fabrication of credentials by analysts, some of which have been certified by LEVA or the IAI. Everything you write in your report should be factual. Everything you write about yourself should be verifiable. As an example of this, I note as a matter of fact that I am retired from police service, having served just shy of 15 years with the Los Angeles Police Dept. under civil service classification 3687. This is a verifiable statement. I can provide the job description for civil service classification 3687, or you can click on the link in the previous sentence. I can provide my retiree ID. I could even print out an account statement from the City's retirement plan noting how many years/months I have of retirement credits (an indication of years/months worked). But, if I simply quit my job, it is a slightly different process than retiring. Thus, I wouldn't say that I had retired, but that I had quit / separated / moved on - or some other term that would accurately describe the end of my time in police service. Same goes for my education. No embellishment, just a simple presentation of the facts.

Back to ethics, there have been a few times that I reported my findings to a certification body. To the best of my knowledge, nothing was ever done with the information that I provided. This is the problem with most professional organizations. We're all friends. No one wants to judge their friends. No waves. No trouble. Let it slide. If there are no consequences, there's no effective discipline.

As these all relate to the certification experience that I just undertook, the AVFA includes all elements except the "proficiency test." I was not given an item of evidence and asked to examine it vs. a question under enquiry. Could my body of work stand in a proficiency test's stead? How much have I written? How many courses of instruction have I designed? How many have I trained to competency? How many cases have I worked? After all, it's this body of work that prepared me to take and pass the exam.

Getting back to the NAS Report, "In essence, ‘certification’ usually means that a particular individual has completed a defined course of education, training, and experience, and has passed an examination prepared by peers which demonstrates that the individual has obtained at least the minimum level of competence required to practice the specific discipline." My education, training, and experience prepared me to take and pass the examination that was prepared by peers, thus demonstrating to the certificate program a minimum level of competence. As to the peers who prepared the exam, I may not know a single one of them, but having a list of names with contact information is a great start.

Taking the AVFA Certification Exam

I have to say that signing up and taking the AVFA certification exam was a rather pleasant experience. Yes, that seems a bit odd to say. But, it's true. It should be true that an organization treats you like an adult, respectfully walking you through the process of engaging their services.

I started on the ETA-i web site. This is where one pays the exam fees, the first stop in the process. Simply select the "Pay Exam Fees" option in their store, select the exam, and pay the fee.

You can do a bit of scouting as to a testing location ahead of your purchase, and include that information in the purchasing process. It will speed things along a bit. I selected a Prometric testing close by. After I paid the exam fees, ETA-i sent my information and the exam to the testing center. I then paid the proctoring fees to the testing center. In total, I paid $120 in fees.

I can't emphasize this enough, I paid $120 in fees to take this certification exam.

If you're sitting for the LEVA CFVA board, you will have to get there (transportation costs) and stay there at least a few days (hotel and food). Likely, you'll want to stay the whole week and, assuming you pass, get your certificate at the Friday dinner. Assuming average travel costs, you or your agency will spend about $2000 for this experience. This $2000 comes after spending thousands upon thousands in the Levels courses and the electives.

For the IAI's CFVE, the cost of testing is significantly lower than LEVA. The IAI charges $400 for the initial test. Many complain about the process of registering for the exam - the letters of recommendation, the documentation of training, and so forth. Plus, finding a proctor location can also be an issue. You can look at their list of certified examiners here. There are only 22 people on the list. Considering the previous post, and the NAS report's admonition about the need for certified analysts, 22 examiners does not even come close to addressing the need. Then, you can see that there are only 9 new examiners since the original group passed through the gates. That's not sustainable.

At $100, the ETA-i's AVFA is affordable. Many agencies do not support their employees in certification. Mine didn't. Everything I did for LEVA, NaTIA, the IRIT, FISWG, and the OSAC, I did on my own time (vacation / comp time). My decision to not get LEVA certification was as much financial as it was practical - I couldn't afford to go out of pocket on certification given the fact that my employer didn't care if I was certified or not. It wasn't a condition of employment and California does not require it for testimony.

Back to the experience.

I paid my fees to the ETA-i and to the testing center. I made an appointment to take the test and showed up a bit early to fill out the paperwork and sign all the forms. The testing center was just that, a quiet place full of cubicles and people taking a variety of tests. The proctor verified my identity. But the proctor, and the ETA-i, did not ask me about my employer or if I work primarily for "the defense." It shouldn't matter for whom I work or what types of cases I typically process. For the AVFA, the price is the price. No membership fees. No tiered pricing for government / non-government. No defense vs prosecution bias (in truth, I probably spent more time defending my self-insured public agency employer than working as a part of prosecuting criminal cases). That bias has always bugged me.

The test instrument was delivered on time and I had 2 hours to answer 100 questions. A comfortable amount of time for me, given my almost 2 decades of experience in the discipline. Upon completion, I notified the proctor who collected all of the materials and wished me well. I returned to my car and to my day. I wasn't stressed out at all. I was prepared to take the assessment and I was treated respectfully by all parties involved. In all, it was probably one of the more pleasant testing experiences I've experienced.

The testing center does not score the test. That's done back at the ETA-i offices. Thus, there's a bit of a delay in finding out if you've passed. For me, I had the results back in less than a day. I passed the test. In the next week or so, I'll receive the official letter notifying me of the results as well as the actual certificate. They also include a wallet card - perhaps necessary if confronted about one's status.

In thinking back over this whole experience, the ETA-i is separate from all of the training that I've received over the years, as is the testing center. Thus, it seems to comply with the ASTM's E2659 – 09. Standard Practice for Certificate Programs. Their accreditation provider follows ISO/IEC 17024 in providing oversight of the ETA-i's programs. This for me is a big deal. I want to know that the certificate that I just earned is legitimate. Given my experiences with NaTIA and LEVA on retirement, I also want to know that in the transition from police service to private practice I'll be treated the same. I was treated with respect throughout the entire process with the ETA-i.

Given the reach of the ETA-i, as well as the availability of proctoring sites from companies like Prometric, the AVFA seems poised to help the NAS Report's recommendations come to fruition - if there's a certification in your discipline, you should have it. The AVFA's certification is well within the reach (financially) of the average analyst / examiner. It's certainly worth a look.

The AVFA Certification

On November 22, 2005, the Science, State, Justice, Commerce, and Related Agencies Appropriations Act of 2006 became law. Under the terms of the statute, Congress authorized “the National Academy of Sciences to conduct a study on forensic science, as described in the Senate report."

In 2009, the National Academies Press published Strengthening Forensic Science in the United States: A Path Forward (aka, the NAS Report). The report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine.

On page 208 of the NAS Report, the authors address the issue of certification.

"The certification of individuals complements the accreditation of laboratories for a total quality assurance program. In other realms of science and technology, professionals, including nurses, physicians, professional engineers, and some laboratorians, typically must be certified before they can practice. The same should be true for forensic scientists who practice and testify. Although the accreditation process primarily addresses the management system, technical methods, and quality of the work of a laboratory (which includes the education and training of staff), certification is a process specifically designed to ensure the competency of the individual examiner."

The NAS Report goes on to note, "In addition to improving quality, certification programs can enhance the credibility of certificate holders."

As I noted last week, there are three certifications available in the discipline of digital / multimedia analysis. I would like to correct that here first, leaving the original post unedited.

Of the domains of what is traditionally considered Image Analysis / Video Analysis, there is no specific component within the LEVA / IAI that addresses the testing of a practitioner's competency in authentication. Photogrammetry is sometimes covered at their conventions, but there again, it's not a specified component of the certification programs.

I can say this with certainty about the LEVA certification. One may stand for certification upon successfully completing the LEVA Levels courses, the Courtroom Prep class, and some electives. For the exam, the analyst chooses a case that they've worked and presents it to the certification board. Thus, what is covered in that "exam" is unique to the analyst - varying from analyst to analyst. As a certificate program, it seems quite out of alignment with ASTM E2659 – 09, Standard Practice for Certificate Programs.

As to the domains under examination, there is no list available from either group. The IAI has a reading list that consists of a few older texts as well as some SWGDE guidelines. It's assumed that the practitioner will gain proficiency in these areas via vendor training, attending workshops at conferences, and other training opportunities.

What's missing: audio.

In Tx vs Oliver, the LEVA CFVA's testimony notes the importance of the audio track of the body-worn video recording, and how it helped in the analysts work. But, the CFVA exam does not specifically test for audio competency. I'm certainly not knocking the analyst. Just noting that the CFVA exam doesn't specifically test for competency in audio analysis and the analyst noted how important the audio track was in guiding his work and his conclusions.

Yes, forensic multimedia analysis includes everything in the multimedia container - audio, video, images, metadata, error, and junk. The CFVA - video analyst - seems to just cover the video portion of that list. Not necessarily a bad thing, but it's important to know what it does and doesn't cover.

Let's say you have an "image" from an iPhone - a Live Photo. Live Photos contain sound (link). Does your chosen software platform recognize that a Live Photo is essentially a video file? Does your chosen software platform have the ability to play the audio? What about "enhancing" the audio? Do you know how to enhance audio?

Here's where the AVFA certification has a leg up. Whilst the "competency requirements" aren't all that well written, and the format of the test (as noted on their site) does not match the language in the comps document, the analyst will definitely have an idea as to what will be on the test.

Yes, the AVFA exam includes audio. It's the first A in the acronym. The AVFA exam includes Photogrammetry and Authentication. There doesn't appear to be anything related to comparisons or content analysis, but those domains can sometimes be blended into the others, or generalized to "analysis."

Back to the NAS Report, "In essence, ‘certification’ usually means that a particular individual has completed a defined course of education, training, and experience, and has passed an examination prepared by peers which demonstrates that the individual has obtained at least the minimum level of competence required to practice the specific discipline." It seems that the AVFA meets that definition.

I'm going to take the test soon. I'll let you know how that goes as well as comment on the whole process - from sign-up to notification of results and beyond.

When the data supports no conclusion, just say so

I can't quantify the amount of requests I've received over the years where investigators have asked me to resolve a few pixels or blocks into a license plate or other identifying item. If, at the Content Triage step, nominal resolution isn't sufficient to answer the question, I say so.

When the data supports no conclusion, I try to quantify why. I usually note the nominal resolution and / or the particular defect that may be getting in the way - blur, obstruction, etc. What I don't do is equivocate. Quite the opposite, I try to be very specific as to why the question can't be answered. In this way, there is no ambiguity in my conclusion(s).

Additionally, whomever is responsible for the source of the evidence will have insight as to potential improvements to the situation. For example, if the question is "what is license plate," and the camera is positioned to monitor a parking lot, then the person / company responsible for the security infrastructure can be alerted to the potential need for additional coverage in the area of interest. This could mean additional cameras, or a change in lensing, or ...

Why this topic today?

I received a link to this article in my inbox. "Expert says Merritt truck ‘cannot be excluded’ as vehicle on McStay neighborhood video." Really? "Cannot be excluded?" What does, "cannot be excluded" mean?

At issue is a 2000 Chevy work truck, similar to the one below. Chevy makes some of the most popular trucks in the US, second only to Ford. This case takes place in Southern California, home to more than 20m people from Kern to San Diego counties.

"Cannot be excluded" is not a conclusion, it's an equivocation. Search CarFax.com for used Chevy Silverado 3500 HD trucks for sale within a 100 mile radius of Fallbrook, CA (92028). I did. I found  49 trucks for sale on that site. 49 trucks that "cannot be excluded" ... AutoTrader listed 277 trucks for sale. 277 more trucks that "cannot be excluded" ...

All of this requires us to ask a question, is the goal of a comparative analysis to "exclude" or to "include?" How would you know if you've never taken a course in comparative analysis? Perhaps we can start with the SWGDE Best Practices for Photographic Comparison for All Disciplines. Version: 1.1 (July 18, 2017) (link):

Class Characteristic – A feature of an object that is common to a group of objects.
Individualizing Characteristic – A feature of an object that contributes to differentiating that object from others of its class.

5.2 Examine the photographs to determine if they are sufficient quality to complete an examination, and if the quality will have an effect on the degree to which an examination can be completed. Specific disciplines should define quality criteria, when possible, and how a failure to meet the specified quality criterion will impact results. (This may apply to a portion of the image, or the image as a whole.)
5.2.1 If the specified quality criteria are not met, determine if it is possible to obtain additional images. If the specified quality criteria are not met, and additional images cannot be obtained, this may preclude the examiner from conducting an examination, or the results of the examination may be limited.
5.3 Enhance images as necessary. Refer to ASTM Guide E2825 for Forensic Digital Image Processing.

These steps are the essence of the Content Triage step in the workflow - do I have enough nominal resolution to continue processing and reach a conclusion?

But, there is more to this process than just a comparison of a "known" and an "unknown." How does one go from "unknown" to a "known" for a comparison? How do you "know" what is "known?" First, you must attempt a Vehicle Make / Model Determination of the vehicle in the CCTV footage.

For a Vehicle Make / Model Determination, the SWGDE Vehicle Make/Model Comparison Form. Version: 1.0 (July 11, 2018) (source) is quite helpful.

How many features are shared between model years in a specific manufacturer's product line? Class Characteristics can help get you to "truck," then to "work truck" (presence of exterior cargo containers not typically present in a basic pickup truck), then to "make" based on shapes and positions of features of the items found in the Comparison form. The form can be used to document your findings.

You may get to Make, but getting to Model in low resolution images and video can be frustrating. What's the difference between a Chevy Silverado 1500, 1500LD, 2500HD, 3500HD? There are more than 10 trim variations of the 1500 series alone. What's the difference between a 2500HD and a 3500HD?

After you've documented your process of going from "object" to "work truck" to a specific model of work truck, how do you move beyond class, to make, to model, to year, to a specific truck? Remember, an Individualizing Characteristic is a feature of an object that contributes to differentiating that object from others of its class. Before you say "headlight spread pattern," please know that there is no valid research supporting "headlight spread pattern" as an individualizing characteristic - NONE. I know that there are cases where this technique has been used, but rhetoric is not science. Many jurisdictions, such as California and Georgia, will allow just about everything in at trial, so not having one's testimony excluded at trial is not proof of anything scientific.

Taking your CCTV footage, you've made your make / model / year determination using the SWGDE's form. Now, how do you move to an individual truck?

This is where basic statistics and inferential reasoning are quite necessary. Do you have sufficient nominal resolution to pick out identifying characteristics in the footage? If not, you're done. The data supports no conclusion as to individualization.

But assuming that you do, how do you work scientifically and as bias free as possible? Unpack the biasing information that you received from your "client" and design an experiment. In the US, given our Constitutional provisions that the accused are innocent until proven guilty, it is for the prosecution to prove guilt. Thus, the staring point for your experiment is that the truck in question is not a match. With sufficient nominal resolution, you set about to prove that there is a match. If you can't, there is no match as far as you're concerned. Remember, the comparative analysis should not be influenced by any other factors or items of evidence.

In designing the experiment, you'll need a sample set of images. You see, a simple "match / no match" comparison needs an adequate sample. It perverts the course of justice to simply attempt the comparison on the accused's vehicle. We don't do witness ID line-ups with just the suspect. Neither should anyone attempt a comparison with just a single "unknown" image - the accused's. Yes, I do use this specific provision of English Common Law to explain the problem here. Perverting the Course of Justice can be any of three acts, fabricating or disposing of evidence, intimidating or threatening a witness or juror, intimidating or threatening a judge. In this case, one Perverts the Course of Justice when one fabricates a conclusion (scientific evidence) where none is possible.

Back to the experiment. How many "unknown" images would you need to approach 99% confidence in your results, thus assisting the course of justice? Answer = 52. How did I come up with 52?

Exact - • Generic binomial test
Analysis: A priori: Compute required sample size
Input: Tail(s)                   = One
Proportion p2 = 0.8
α err prob = 0.01
Power (1-β err prob)     = .99
Proportion p1             = 0.5
Output: Lower critical N = 35.0000000
Upper critical N         = 35.0000000
Total sample size         = 52
Actual power             = 0.9901396
Actual α = 0.008766618

A generic binomial test is similar to the flip of a coin - only two possible outcomes, heads / tails or match / no match. It's the simplest test to perform.

The error probability is your chance of being wrong. At 52 test images, you've got a 1 in 100 chance of being wrong (.99). As you move below 15 test images, you have a greater chance of being wrong than being right. With a sample size of 1, you're likely more accurate tossing a coin.

The 52 samples help us to get to make / model / year. You may chose to refresh those samples with new ones to perform a "blind comparison," and attempt to "include" the suspect's vehicle in your findings. To do this, you'd need the specific description of the "known" vehicle that makes it unique vs the others in the sample.

If I were performing a make / model / year determination, and then a comparison, I would note any errors or limitations in my report. If the data supported no conclusion, or if the limitations in the data prevented me from arriving at a determination, I would note that the data supported no conclusion. If I was able to make a determination, I would have noted my process and how I arrived at the conclusion (in a reliable, valid, and reproducible fashion).

The problem with the reporting of the case is the "cannot be excluded" portion is in the headline. One has to read deeper into the article to find, "... Liscio denied  (that his conclusions may have been formed to fit the bias of the prosecution, who was paying him...), and reminded McGee more than once that he had not identified the truck specifically as Merritt’s..."

Which requires another question be asked, if the analyst had not identified the vehicle, what was he doing there in testimony?

"Among the items that helped to reach the conclusion that the vehicle was “consistent” with Merritt’s truck was a glint caught by the video that matched the position of a latch on a passenger-side storage box toward the rear of the truck, said Liscio,who uses 3D imagery."

Here we move from "cannot be excluded" to "consistent with," another equivocation. How does one not identify a vehicle, but find that said unknown vehicle is "consistent with" the "known" vehicle? This is the problem with Demonstrative Comparisons. When you place a single "known" against a single "unknown" in a demonstrative exhibit, you are making a choice as to what to include in your exhibit - thus you have concluded.

Back to the demonstrative. What is it about the latch on the side of the truck that is unique? Won't all work trucks of this type have latches on their cargo containers? Why is this one so special that it can only be found on the accused's truck? Of these questions, the article does not give an answer.

"I’m not saying that this your client’s vehicle,” Liscio repeated. “All I am saying is that the vehicle in question is consistent with my report, and if there is another vehicle that looks similar, that is possible.” How about at least 326 vehicles found on just two used car web sites?

If you'd like to explore these topics in depth, I'd invite you to sign up for any one (or all) of our upcoming training sessions. Our Statistics for Forensic Analysts course is offered on-line as micro learning and thus enrollment can happen at your convenience. Our other courses can be facilitated at your location or at ours, in Henderson, NV.

What new certification program?

We had a lively discussion in last week's Advanced Processing Techniques class. This class seems to bring out these types of conversations.

We were discussing certifications in our discipline: digital / multimedia analysis. When I said that I know of three, the students seemed puzzled. Three? They were aware of LEVA's CFVT and CFVA programs and the IAI's CFVE program. LEVA counts as one, the IAI, two. Where's three?

Whilst you were not paying attention, a group of analysts got together with the Electronics Technicians Association, International, to create the Audio Video Forensic Analyst (AVFA) certification. Hat's off to them for pulling that one off.

Who is the ETA-i? What is the AVFA? Let's take a look.

The ETA-i is a legit organization that certifies electricians, cable splicers, and other comms techs. It's been around since 1978 and has over 90 certification programs. I spoke with the office manager there about the AVFA certification and received an interesting answer about it's genesis. I was told that a few years ago, they began an expansion into other related industries. They added a Digital Video (DVE) Editor certification. From there, they added the AVFA certification. Some of those involved in the DVE certification were involved in the creation of the AVFA program.

Before you grab your pitch fork and start marching towards the town square, let's take a quick look at the program and compare it to what we know about the existing market for certifications.

Validity is defined as the extent to which an assessment accurately measures what it is intended to measure. Validity refers to the accuracy of measurement. In this case, the assessment is the certification exam. Determining validity can be viewed as constructing an evidence-based argument regarding how well a tool measures what it is supposed to do. Evidence can be found in content, response process, relationships to other variables, and consequences (source).

Reliability refers to whether an assessment instrument gives the same results each time it is used in the same setting with the same type of subjects.  For test/retest reliability, the test should give the same results each time, assuming there are no interval changes in what you are measuring, and they are often measured as correlation.

Face Validity - the extent to which a test is subjectively viewed as covering the concept it purports to measure.

Is the AVFA reliable? As a multiple choice exam, it is certainly possible. One would hope that it's written in such a way that you'd answer the same questions in the same way on successive administrations of the test.

Does the AVFA certification test have face validity? You can find out what it's supposed to measure on it's web site (link). You might look at the competencies sheet and note that there are programs listed that you're likely not familiar with. But, every analysis requires a platform. The creators of the AVFA had to choose tools and terminology. They seem to have chosen freeware and sources of information not hidden behind paywalls. Good for them. The ASTM's glossary is great, but how much do you have to pay to use it?

You can sign up and take the test yourself. No membership required. The test is affordable ($100 + proctor fees). The test is entirely multiple choice. Pass it, you're certified. Fail it, re-take it for free. But, on the face of it (pun intended), it seems to have face validity.

Accredited? Yes. The program at ETA-i is accredited by International Certification Accreditation Council (ICAC). Who are they? I didn't know either, so I called them as well. I had a nice and long conversation with a representative who explained their program and their long relationship with the ETA-i. According to them, they follow ISO 17011 / 17024 in the administration of their accreditation program. So far so good.

What's the deal with accreditation? One of the things that ASTM E2659-09 as well as ISO 17011 / 17024 emphasize is the separation of the training event / training provider and the certifying body. There is a risk of bias if the training provider also runs the certification process - as in teaching the test. This is a large part of why you take Photoshop classes from me, George, Dorothy, Scott Kelby, and etc. ... then trek on over to ProMetric to take the Adobe Certified Expert exam.

This contrasts with LEVA, as an example. LEVA is both training provider and certifying body. As such, it does not follow or comply with ASTM E2659-09 as well as ISO 17011 / 17024 in the design of its program. Thus, it's program is not accredited. There's no way to simply sit for an objective certification exam with LEVA. You must take their classes, all of them, in order to sit for the test. Additionally, it's impossible to judge the reliability or validity of the certification exam as all LEVA certification exams are unique experiences - and are closed to the public. I'm not saying that there's no value to the experience, just that it's not valid from a design / standards standpoint.

The IAI is in the middle. I'm aware that they are seeking accreditation for their programs. They are not a training provider, per se. They do offer training at their conferences, but you can't call that "training to competency." It's more like information sessions, as the ASTM defines training events. Thus, it is possible for the IAI to get their programs accredited. They should, however, remove the discriminatory membership requirement in order to sit for their test.

Thus, as someone who is trained / educated in curriculum and instructional design, who has created many assessments, and has reviewed countless others, I'm excited to see how this new program shakes out. I'll likely sit for the exam soon, just to see what it's like.

If you decide to take the test, let me know what you think of it. Try to go into the process with an open mind. I'm a big fan of democratizing science. If this is a legit certification, I'm all for it.

The need for objective measures in forensic evidence

The latest edition of Significance, the journal of the Royal Statistical Society, features an article on the importance of objective measures and standards in evaluating evidence. Whilst many of the points made by the author are valid and applicable, practitioners within the digital / multimedia forensic sciences already have standards and guidelines for basing their work on an objective footing.

Investigators want to use video evidence to "identify" an object in a video - gun, knife, etc. But this "identification" is often complicated by a lack of nominal resolution in the region of interest. Many will simply state that the object has been identified, without actually taking the steps to formally conclude (prove) their assertions. They go with what they "know" (abductive reasoning), as opposed to what they can prove. There's a better way.

First off, here are some relevant definitions from the SWGDE Digital & Multimedia Evidence Glossary. Version: 3.0 (June 23, 2016)

Artifact: A visual/aural aberration in an image, video, or audio recording resulting from a technical or operational limitation. Examples include speckles in a scanned picture or “blocking” in images compressed using the JPEG standard.

Image Analysis: The application of image science and domain expertise to examine and interpret the content of an image, the image itself, or both in legal matters.

Image Comparison (Photographic Comparison): The process of comparing images of questioned objects or persons to known objects or persons or images thereof, and making an assessment
of the correspondence between features in these images for rendering an opinion regarding identification or elimination.

Image Content Analysis: The drawing of conclusions about an image. Targets for content analysis include, but are not limited to: the subjects/objects within an image; the conditions under which, or the process by which, the image was captured or created; the physical aspects of the scene (e.g., lighting or composition); and/or the provenance of the image.

Nominal resolution: The numerical value of pixels per inch as opposed to the achievable resolution of the imaging device. In the case of flatbed scanners, it is based on the resolution setting in the software controlling the scanner. In the case of digital cameras, this refers to the number of pixels of the camera sensor divided by the corresponding vertical and horizontal dimension of the area photographed.

Video Analysis: The scientific examination, comparison, and/or evaluation of video in legal matters.

It's also important to define "forensic science." For this, I'll refer to "A Framework to Harmonize Forensic Science Practices and Digital/Multimedia Evidence." OSAC Task Group on Digital/Multimedia Science. 2017: "Forensic science is the systematic and coherent study of traces to address questions of authentication, identification, classification, reconstruction, and evaluation for a legal context."

What is a trace? "A trace is any modification, subsequently observable, resulting from an event. You walk within the view of a CCTV system, you leave a trace of your presence within that system."

SWGDE Best Practices for Photographic Comparison for All Disciplines. Version: 1.1 (July 18, 2017) provides a few more definitions.

Class Characteristic: A feature of an object that is common to a group of objects.

Individualizing Characteristic: A feature of an object that contributes to differentiating that object from others of its class.

With the definitions in mind, SWGDE Best Practices for Image Content Analysis. Version: 1.0 (February 21, 2017) provides the framework to begin the Content Triage step in the workflow - can I answer the question with the evidence file?

5. Evidence Preparation
5.2 Based on the request for examination, determine if submitted imagery is available to complete requested analysis. Determine whether submitted imagery is of sufficient quality to complete the requested examination, or if the image quality will have an effect on the degree to which an examination can be completed.
5.2.1 If the specified quality criteria are not met, determine if it is possible to obtain additional images. If the specified quality criteria are not met, and additional images cannot be obtained, this may preclude the examiner from conducting an examination, or the results of the examination may be limited.

(do I have sufficient nominal resolution within the target area to fulfill the request - answer the question? The nominal resolution, measured in pixels per inch or inches per pixel - or it's metric equivalent, is the objective measure.)

6. Examinations Method
There is no one specific methodology for content analysis. The methodology for analysis will primarily be derived to answer the requested examination. However, any methodology applied to content analysis should incorporate an analysis of the imagery, the cataloguing of relevant features, an evaluation of the significance of the detected features, an evaluation of the limiting factors of the imagery, the formation of a conclusion, and a verification of the analysis. The repeatability (and / or reproducibility) of the procedure and documentation of the workflow is of paramount importance. Documentation should be performed contemporaneously.
6.1 Assess the contents of the image, to determine whether factors are present that can answer the examination request. The examination request generally will fall into one of the following categories:
6.1.1 Analysis to determine the conditions under which, or the process by which, the image was captured or created. Examples include, but are not limited to, the limitations of the recording device, and the inclusion of artifacts based on the file format or compression. This can help to answer the question “How does the recording system affect what is visible in the scene?”
6.1.2 Analysis to determine the physical aspects of the scene, including events captured. Examples include, but are not limited to, the lighting and composition of the scene, the presence of specific objects within the scene, a determination of the interaction between objects in the scene, and a description of events within a scene. This can help to answer the questions “Is a specific object visible in the scene?” or “What happened in the scene?”
6.1.3 Analysis to determine the classification of an object within an image. Examples include, but are not limited to, the make, model, and year of a vehicle, the determination of a manufacturing logo, and the determination of the brand and model of a weapon. This can help to answer the question “What is the object visible in the scene?”
6.1.4 Analysis to determine the location or setting of the image content. Examples may include either a general setting (e.g. Portland, Oregon) or a specific setting (e.g. Conference Room 23, the Northwest Corner). This can help answer the question “Where is the scene?”
6.3 Assess the image for features that contribute to the ability to form a conclusion, and record observed features. Consider the weight or importance of identified features, in order to determine the focus of the examination. Examples of features may include logos, shapes, reflections, or specific items.

How does one move from "artifact" (6.1.1) to "object" (6.1.3)?

As regards “artifact:"

When nominal resolution is low, the analyst must be certain of the provenance of the features within the region of interest in the evidence item. Are the “items of interest” actual representations of things, or are they an “aberration” in the image, resulting from problems in compression and etc. In order to move from “a few dark blocks of pixels” is not an “artifact” to the "few dark blocks of pixels” is an “object” - the rules for classifying an object apply. Additionally, one must rule out the possibility that what one sees is not an artifact. This must be done in a systematic / coherent fashion in order to be a forensic science exercise.

As regards “object:"

In order to move from “a few dark blocks of pixels” that are not a result of an “aberration" to firearm, there needs to be Class Characteristics present in the view that would lead to the conclusion of "object." In order to form a conclusion, those Class Characteristics should be identified by the analyst - overall shape: the presence of features of an object that contributes to differentiating that object from others of its class  Is the nominal resolution in the region of interest sufficient to conclude as to class and individualizing characteristics? This is the objective measure.

The standards and guidelines are there. It is left to the investigator, the analyst, the attorney, and the Trier of Fact to follow the rules and accept the facts as they present themselves. What does the investigator do when told by the analyst that the nominal resolution does not support a conclusion? Does he / she shop around for another analyst? The investigator must not fall victim to cognitive perseverance (bias) when faced with this new information.

Additionally, these stakeholders must understand when they're on sound scientific footing or when they're just taking their best shot.

Types of Resoning

Abductive reasoning: taking your best shot
Deductive reasoning: conclusion guaranteed
Inductive reasoning: conclusion merely likely

What most in the forensic sciences don't quite acknowledge is that they operate in the world of abductive reasoning most of the time.

From a US Constitution standpoint, from the standpoint of Constitutional Policing, it is for the side offering an item as “proof” of some condition (e.g., object = firearm) to actually prove class = firearm. It perverts the cause of justice to require the defense to prove that it isn’t.

This is the level of detail that we get into when I present training in the digital / multimedia forensic sciences, as well as my other educational offerings. If you need help on a case, or want a second opinion, feel free to contact me today.

Adobe Tells Users They Can Get Sued for Using Old Versions of Photoshop

It was just a month ago that I warned you to keep your installer files. Now this from Adobe:

"Users of older versions of Adobe Creative Cloud apps including Photoshop have been told to stop using them or face potential "infringement claims" from third-party companies who are unnamed but suspected to be Dolby. Adobe cites only "ongoing litigation" as the reason for the abrupt announcement."

"Adobe recently discontinued certain older versions of Creative Cloud applications. Customers using those versions have been notified that they are no longer licensed to use them and were provided guidance on how to upgrade to the latest authorized versions," said Adobe in a statement to AppleInsider."

"Unfortunately, customers who continue to use or deploy older, unauthorized versions of Creative Cloud may face potential claims of infringement by third parties. We cannot comment on claims of third-party infringement, as it concerns ongoing litigation."

"Instead, users are receiving the equivalent of a cease and desist email, informing them that the apps that they are using are discontinued."

If you're an individual user in a private lab, you may not become the focus of Adobe's attention. But, if you're a part of a large organization or government lab, I'll bet that Adobe will see the potential profit in coming after your agency.

Stay tuned ...

Res ipsa loquitor

In the common law of torts, res ipsa loquitur (Latin for "the thing speaks for itself") is a doctrine that infers negligence from the very nature of an accident or injury in the absence of direct evidence on how any defendant behaved. Although modern formulations differ by jurisdiction, common law originally stated that the accident must satisfy the necessary elements of negligence: duty, breach of duty, causation, and injury. In res ipsa loquitur, the elements of duty of care, breach, and causation are inferred from an injury that does not ordinarily occur without negligence.
Res ipsa loquitur is often confused with prima facie ("at first sight"), the common law doctrine that a party must show some minimum amount of evidence before a trial is worthwhile.

(Source)

What on earth does this have to do with multimedia analysis? In so many cases, attorneys argue and judges agree that a video / image is what it is - that it speaks for itself - that an analyst is not needed to explain crucial elements of the evidence item.

Take this image:

The upper section is a depiction of the Chuvash State Opera and Ballet Theater in Cheboksary, Russia. This icon of Brutalist architecture is one of the great examples of Brutalism in Russia, and a must see for tourists.

The lower section layers in (Photoshops) elements from Star Wars.

Under res ipsa loquitur - the lower section speaking for itself - it could be argued that Imperial troops have occupied central Russia. This is, of course a ridiculous idea. Here, we're using absurdity to illustrate the absurd.

In terms of modern video evidence - can the video indeed speak for itself? Is the object of interest an artifact of compression, a result of noise, or an element of the scene? How would you know? You would engage an analyst - like me.

Any case involving Forensic Photographic Comparison or Forensic Content Analysis must necessarily start with a "ground truth assessment;" what are the elements of the area of interest. A macro block analysis is often performed. Is the area of interest populated by lossless encoded data, copied data, predicted data, or error? How would you know? You would know by using validated tools and your trained mind / eyes.

In Authentication cases, this is no different. Fakes are getting sophisticated and easier to perform. Analysts need valid tools, training, and education. This is why the training / educational sessions that I offer move beyond simple button pushing into a deeper understanding of the underlying technology as well as how forgeries are created in the first place. Because "false positives" and "false negatives" can be frequent in authentication tools that are built around data sets, one must know how to interpret results as well as how to successfully validate them across a wider variety of methods that do not only involve the software being used. This is why statistics goes hand in hand with the domains of analysis.

If you're interested in training / education, you can sign up for our on-going / upcoming offerings on the web site. Seats are available. Also, with our on-line micro learning offerings, seats are always available as you learn at your own location.

UX Talk: Amped Authenticate - Batch File Format Comparison

One of the problems with products that generate results based on a comparison to an internal database  is errors - false positives / false negatives. As much as I like Amped's Authenticate, it does suffer from this problem. The new reporting feature makes this worse, not better.

Here's the scenario.

You load an image and check the File Information, or you run the new "Smart Report." One of the results in this panel indicates a mismatch of the JPEG's Quantization Table - "Compression signature is incompatible with the actual camera make-model."

How do you know if this result comes from an actual mismatch? How do you know if this result is in fact a "non match," meaning the evidence file comes from a camera who's JPEG QT information isn't actually in the product's database - in this case Amped Authenticate?

Answer: run a batch File Format Comparison against a valid sample set of random images taken by the same model of camera.

I've written previously about the utility of the Camera Forensics service. This is one of those instances where the service comes in handy. Assemble your sample and drop a copy of your evidence image into the folder of samples. Then run the Comparison.

When the report is generated, scroll across to find the column for JPEG QT hash. If, in fact, your evidence image is a mismatch - "Compression signature is incompatible with the actual camera make-model" - then the evidence image's JPEG QT hash should not match the JPEG QT of any of the sample file. In my case, all of the samples matched each other, and matched the evidence file. The result from the software was false.

Were this your case, an actual case, you would want to thoroughly document this process of validating the results. You wouldn't want to leave the statement of "mismatch" untested. Yes, the software returned the result. However, we assembled a valid sample set to test the results. Here, we found the software in error (likely the software's internal database was incomplete). We found that the evidence file's JPEG QT matched the JPEG QT for all of the sample files.

The pace of change in camera technology is quite brisk. The issue is further complicated by the fact that camera technology is a bit regional - meaning there are cameras sold in some areas but not others. Additionally, not all manufacturers make their QTs available to developers. Very few do, actually. Thus, is it realistic to find all QTs present in a particular software's database? Of course not. Therefor, you'll need to remember to validate your results - especially if you get a "false" result.

UX Talk: Amped FIVE's Change Frame Rate filter

The Change Frame Rate filter was introduced into FIVE in 2014. It's functionality is quite simple. You can adjust a file's frame rate, targeting a specific frame rate or a specific time.

For today's post, I have some old files that I use in my Advanced Processing Techniques class. The old DVR that is the source of the class' files only had enough processing power to generate a file that covered 1 minute per camera. Each camera view's recordings were stored in separate 1-minute chunks of data, in separate folders, in a crazy folder scheme. Using Concatenate would be a nightmare as the video files are all named the same.

When creating a proxy file for these files, FFMPEG does not recognize the fps tag in the container. Thus, it defaults to 25fps - which is not correct. Notice the screen shot above. At 25fps, the total time is about 10 seconds. Yet, the file captured 1 minute of time. Obviously, the frame rate is wrong. This type of use case is exactly why I like having this functionality in my toolset. I can manually enter the Total Time value and FIVE will calculate the resulting Frame Rate (ish).

The (ish) - the caveat - using this filter is not without it's problems and pains. Being divorced from the Amped team means I can show you the problems and work-arounds without getting in trouble from the boss.

For this project, you might be tempted to load each 1-minute file, use the new Timeline filter, then change the frame rate as the final step (after using Timeline). The problem here, and the reason I feature this in training, is the high variability in the number of frames within each of the one minute long data files. Thus, I'd want to utilize the Change Frame Rate filter in each minute's processing chain in order to accurately represent time. It's a slog, but it's doable.

If you're a savvy user, you might be thinking, "Hey Jimmy. Just copy the filter instance from one chain and paste it into the other chains." You might think that. You'd be wrong. It doesn't work that way. If you copy / paste the filter, it retains the settings from the source and doesn't update itself with the destination chain's values. You'll want to change the values, but it still won't update. So, it's best to just use a new instance of the filter for each chain.

Another issue with the filter (I'm hoping that all of this gets fixed, in which case I'll post an update to this post) deals with the way in which users enter values in the Total Time field. In the screen shot below, you'll notice that the Frame Rate is 3.7415. I got this incorrect value because of the way in which I edited Total Time. I selected only the "tens" and changed it to zero, the "ones" value was already zero so I left it alone. Doing so caused an incorrect result. To get a correct result, you'll need to select the entirety of "seconds" (or minutes / hours if applicable) and make the change. Yes, I know, this is a lot to keep track of.

Incorrect

Correct

If you've restored the frame rate in each processing chain separately, combining the processing chains with the Timeline filter will be necessary. You might think that doing so will result in a file where the Total Time for each chain will add up to a nice round number in the Timeline filter's results. Again, you would be wrong.

For my project, I have 15 files in 15 folders. Each file represents 1 minute. Each file gets it's own processing chain. Each chain is adjusted to 1 minute of total time. Each chain's frame rate is somewhere around 4fps (see above).  But, in combining the chains with the Timeline filter, the program seems to favor the frame rates. The resulting duration, after using the Timeline filter, is not 15 minutes. It's closer to 14 minutes. Timeline seems to do some simple math in spite of all the work that I did in each processing chain - accounting for total frames and frame rate - ignoring the declared Total Time per chain.

These are the types of questions I ask when validating a tool. They're also the types of things that I check when validating a new build / release of a known tool. I don't assume anything. I test everything. In testing - I found this issue.

Does one frame per second make a difference? Maybe not. But, in my project, the uncorrected difference is 1 minute in 15 - or 4 minutes per hour. That's significant. Given that other filters, like Add Timestamp, rely upon correct timing, yes - it will make a difference.

So, do you process everything and change the frame rate on the combined results?

Or, do you change the frame rate in each processing chain, then correct the results after combining the chains utilizing the Timeline filter? You'll get a different frame rate for each - thus the "ish." Which is correct? Your validation of your individual case will tell you. Because of the variability, my results can only advise you of the affect, they can't inform your results or your experiments other than that.

UX Talk: Amped FIVE's Timeline Filter

A few months ago, Amped released Build 12727. That build featured some new functionality - the Timeline Filter.

What the Timeline Filter tries to do is to turn FIVE into an NLE by allowing you to combine processing chains. You can process segments individually, then combine them at the end of your work before writing out the finished results.

Timeline works a bit like Concatenate. The difference being that Concatenate works as a part of the "conversion" process and Timeline works as part of the processing tasks. With Concatenate, you get what you get. With Timeline, you can fine tune the results by selecting specific processing chains as well as the processing steps within those chains.

Timeline is part of the Link filter group. Like the other filters in that group, it doesn't support audio. If you're wondering where your audio track is after using the Timeline, Multiview, or Video Mixer filters, you're likely in North America. It seems, from observing the development of FIVE from here (the western US) that no one else in the world has to deal with audio in multimedia files.

The Filter Settings for the Timeline Filter are pretty simple. One thing to note, when you choose it, the settings will include all your project's processing chains. You'll want to sort through the Inputs and Remove any chains that don't belong (click on the Input to remove and click on the Remove button).

Once combined, write the resulting file out to your desired codec / container.

If your files do have an audio track, and you want the results to also have the audio track, you'll need to strip out the audio first (part of the Convert DVR functionality). Once you've performed your processing / analysis tasks, you'll want to re-combine the audio and video segments in an NLE like Vegas or PremierePro.

Hang on to those installers

Early on in my career, I learned to hang on to the installer files for the programs that I encountered. At the time of the retrieval, I thought (wrongly) that I had the correct installer file already and didn't need to hang on to the file that came out of the evidence DVR. That decision came back to haunt me later when doing the analysis as the DVR's owner replaced the DVR before we could go back to retrieve the program's installer from the machine. This was a long time ago, but the lesson stuck. Assume nothing. Keep everything together.

Fast forward a decade and a half.

Amped SRL made a big deal about their incorporation of the Genetec SDK in FIVE, and FIVE's support of the .g64 and .g64x file types in 2017 (Build 10039). Indeed it was a big deal. Genetec is a popular company and agencies around the western world face these file types quite often. Genetec makes a player for their files, but to export individual camera views from their player takes a lot of time to accomplish - especially in major cases with many relevant camera views.

Fast forward to yesterday.

As you may have heard, I'm no longer part of the Amped team. Amped support is now handled through their support web site. There's no more phone support for North American customers (announced via a change to their EULA that customers likely accepted without reading). But, with an unpronounceable Scandinavian last name, I'm easy to find. I've been getting many calls per day about various analysis-related / software things. Although I can't support software directly, I can still answer UX (how-to) type questions. I also still conduct training sessions around Amped's software (click here for the calendar of offerings).

Yesterday, an old friend called me in a panic. He had a major case, the files were .g64x, and FIVE wasn't working with them anymore. Yes, he knew that I'm no longer with Amped Software. But, he was running out of options and time.

What he didn't know, what most of Amped's customers don't know, is that in March of 2018, Amped SRL quietly removed the Genetec functionality from it's product line (Build 10717). It's normally verbose change log notes only that this build was an interim update - nothing about what was changed. The functionality was not only removed from 10717, but also 10039. Any mention of the Genetec functionality was stricken from the logs. If you search for Genetec on the Amped blog, you will not find anything. It's as if the Genetec integration never happened.

A few weeks ago, another old customer called with a similar story. In his case, he had performed the original work with the .g64 files in Build 10039. The case is moving to trial and he's updated FIVE faithfully. He's been asked to do some additional work on the case. But, the project file doesn't work. Usually, it's quick to duplicate your work and recreate a corrupted project file. But in this case, he can't. He no longer has the ability to work with these files in the same way. It's caused a huge issue with his case - a huge issue he didn't know was coming.

That brings me back to that initial lesson in dealing with the first generation of DVRs - save everything together. Yes, Amped's support portal gives users under support contracts the ability to download previous versions. But, what several have discovered, Amped has changed functionality to at least one of those installer builds (preserving the old build number in rolling out the updated version doesn't help). What else has changed in what's hosted there? How would you know?

All hope was not lost. An older computer of mine was still configured for support of Axon's sales team. I still had the original Axon FIVE Build 10039 on that laptop for a demo done quite some time ago in support of Axon's sales team.  I was able to help the users solve their problems with the Genetec files. I was happy to do so. If you need some help as well, let me know.

The "NAS Report," 10 years later

It's been a bit over 10 years since Strengthening Forensic Science in the United States, a Path Forward was released (link). What's changed since then?

As a practitioner and an educator, Chapter 8 was particularly significant. Here's what we should all know - here's what we're all responsible for knowing.

"Forensic Examiners must understand the principles, practices, and contexts of science, including the scientific method. Training should move away from the reliance on the apprentice-like transmittal of practices to education ..."

10 years later, has the situation changed? 10 years later, there are a ton of "apprentice-like" certification programs and just a handful of college programs. College programs are expensive and time consuming. Mid-career professionals don't have the time to sit in college classes for years. Mid-career professionals don't have the money to pay for college. Those that have retired from public service and "re-entered" their profession on the private side face the same challenges.

Years ago, I designed a curriculum set for an undergraduate education in the forensic sciences. It was designed such that it could form the foundation for graduate work in any of the forensic sciences - with electives being being discipline specific. I made the rounds of schools. I made the rounds of organizations like LEVA. I made my pitch. I got nowhere with it. Colleges are non-profit in name only, it seems.

To address the problems of time and money, as well as proximity, I've moved the box of classes on-line. The first offering is out now - Statistics for Forensic Analysts. It's micro learning. It's "consumer level," not "egg head" level. It's paced comfortably. It's priced reasonably. It's entirely relevant to the issues raised in the NAS Report, as well as the issues raised in this month's edition of Significance Magazine.

I encourage you to take a look at Statistics For Forensic Analysts. If you've read the latest issue of Significance, and you have no idea what they're talking about or why what they're talking about is vitally important, you need to take our stats class. Check out the syllabus and sign-up today.

Why do you need science?

An interesting morning's mail. Two articles released overnight deal with forensic video analysis. Two different angles on the subject.

First off, there's the "advertorial" for the LEVA / IAI certification programs in the Police Chief Magazine.

The pitch for certification was complicated by this image:

The caption for the image further complicated the message for me: "Proper training is required to accurately recover or enhance low-resolution video and images, as well as other visual complexities."

Whilst the statement is true, do you really believe that the improvements to the image happened from the left to the right? Perhaps, for editorial purposes, the image was degraded, from the original (R) to the result (L). If I'm wrong about this, I'd love to see the case notes and the specifics as to the original file. Can you imagine such a result coming from the average CCTV file? Hardly.

Next in the bin was an opinion piece in the Washington Post's Radley Balko - "Journalists need to stop enabling junk forensics." It's seemingly the rebuttal to the LEVA / IAI piece.

Balko picks up where the ProPublica series left off - an examination of the discipline in general, and Dr. Vorder Brugge of the FBI in particular. It's an opinion piece, and it's rather pointed in it's opinion of the state of the discipline. Balko, like ProPublica, has been on this for a while now (here's another Balko piece on the state of forensic science in the US).

I don't disagree with any of the referenced authors here. Not one bit. Jan and Kim are correct in that the Trier of Fact needs competent analysts working cases. Balko is correct in that the US still rather sucks at science. That we suck as science was the main reason the Obama administration created the OSAC and the reason Texas created it's licensing scheme for analysts.

Where I think I disagree with Jan and Kim is essentially a legacy of the Daubert decision. Daubert seemingly outsourced the qualification process to third parties. It gave rise to the certification mills and to industry training programs. Training to competency means different things to different organizations. For example, I've been trained to competency on the use of Amped's FIVE and Authenticate. But, none of that training included the underlying science behind how the tool is used in case work. For that, I had to go elsewhere. But, Amped Software, Inc, certified me as a user and a trainer of the tools. That (certification) was just a step in the journey to competency, not the destination.

Balko, like ProPublica, notes the problems with pattern evidence exams. Their points are valid. But, it doesn't mean that image comparison can't be accomplished. It does mean that image comparisons should be founded in science. One of those sciences is certainly image science (knowing the constituent parts of the image / video and how the evidence item was created, transmitted, stored, retrieved, etc. But another one of the sciences necessary is statistics (and experimental design).

As I noted in my letter to the editor of the Journal of Forensic Identification, experimental design and statistics form a vital part of any analysis. For pattern matching, the evidence item may match the CCTV footage. But, would a representative sample of similar items (shirts, jeans, etc) also match? Can you calculate probabilities if you're unaware of the denominator in the function (what's the population of items)? Did you calculate the sample size properly for the given test? Do you have access to a sample set? If not, did you note these limitations in your report? Did these limitations inform your conclusions?

Both LEVA and the IAI have a requirement for their certified analysts to seek and complete additional training / education towards eventual recertification. This is a good thing. But, as many of us know, there are only so many training opportunities. At some point, you kind of run out of classes to take is a common refrain. Whilst this may be true for "training" (tool / discipline specific), this is so not true for education. There are a ton of classes out there to inform one's work. The problem there becomes price / availability. This price / availability problem was the primary driver behind my taking my Statistics class out of the college context and putting it on-line as micro learning. My other classes from my "curriculum in a box" concept will roll out later this year and into the next year.

So to the point of the morning's articles - yes, you do need a trained / educated analyst. Yes, that analyst needs to engage in a scientific experiment - governed both by image science as well as experimental science. Forensic science can be science, if it's conducted scientifically. Otherwise, it becomes a rhetorical exercise utilizing demonstratives to support it's unreproducible claims.

Luminar 3 - First Look

The market for image editors is changing again. Your favorites are now available by subscription only. Some tools haven't been updated in quite some time. Others, are virtually unobtainable for a variety of reasons.

The market is also growing. Folks dumping cell phones and computers just need a simple photo viewer and editor - one that's easy to use and doesn't cost a fortune. You might have heard about the alliance between Cellebrite and Input-Ace. That's cool, but the licenses are a bit pricey and not everyone will have access to those tools all of the time.

Years ago, I ran a very long beta test and proof of concept at the LAPD of what was first called Amped First Responder. Then, with the partnership between Axon and Amped, that product was shifted into the Axon ecosphere and renamed - Axon Investigator. Another round of beta tests. The partnership dissolved. The project went into dormancy. Amped later released Replay. But, the concept is different. Replay isn't an affordable bulk purchase. I wanted a simple editor that was affordable for the average investigator - so everyone could have a copy at their desktop.

My old colleagues need something simple and effective. Something affordable. The latest update of Luminar 3 from Skylum might just be it.

Luminar 3 is a photo editor. It's not for crazy CCTV codecs, or video for that matter. It's a competitor to Photoshop or Lightroom, not PremierePro.

Simple things like focus, colour, sharpening, are all built in and easy to use. To reference the Adobe UI, the tools in Luminar 3 seem like a blend of ACR and Lightroom.

I loaded one of my test files into the program. There's glare from the window. There's a colour cast. The license plate is far away, but there's enough nominal resolution to be able to resolve it. It's a typical task - here's a photo from my mobile, fix it and find the suspects.

30 seconds later, I had clarified the image and saved out a copy. I even took the time to save the settings (known as Looks in Luminar 3).

Once saved, the Look shows up in the UI under Luminar Looks: User Luminar Looks. You can have Looks for all of your typical tasks - which really speeds things up. Plus, if your saved Look is too strong, you can vary the amount and dial back the effect.

Luminar 3 is fast, it's user friendly, it's easy to learn, and it's incredibly affordable. Luminar 3 is just $69. Not per month - total. You own it. Plus, your purchase allows up to 5 installs (either Mac or PC). That's a bargain.

I'm going to be exploring some of the features of this handy tool that I think will work for various tasks in the justice system - from the crime scene to the court room - in future posts. In the meantime, it's worth a look. They even offer a free trial.

My Training Calendar

I've received a few emails about my training schedule. It's over in the Apex Partners web site's  calendar (click here).

As of today, here's what's been booked:

April 29-May 1 - Introduction to Forensic Multimedia Analysis (with Amped FIVE).
April 2-3 - Advanced Processing Techniques (with Amped FIVE).
April 29 - May 3 - Book the whole week and save.

May 16-17 Advanced Processing Techniques (emphasis on system engineering / optimization).

July 15-17 - Introduction to Forensic Multimedia Analysis (with Amped FIVE).
July 18-19 - Introduction to Image Authentication (with Amped Authenticate).
July 15-19 - Book the whole week and save.

Dec 9-11 - Introduction to Forensic Multimedia Analysis (with Amped FIVE).
Dec 12-13 - Advanced Processing Techniques (with Amped FIVE).
Dec 9-13 - Book the whole week and save.

But wait, there's more...

Statistics for Forensic Analysts - on-line micro learning. Click here for information or to sign up.

Redaction of Multimedia Evidence - on-line micro learning. If you're an Adobe, Magix, or Amped customer, there's a course dedicated to those tools. If you haven't chosen a toolset yet, there's a course that incorporates all three (plus a bonus module). Click here for more information or to sign up.

If you're interested in booking a training at your location, just let me know.