Featured Post

Welcome to the Forensic Multimedia Analysis blog (formerly the Forensic Photoshop blog). With the latest developments in the analysis of m...

Thursday, January 23, 2020

What is Super Resolution?

Back in early 2017, I wrote an article for Axon in support of their now-dissolved partnership with my former employer about Super Resolution. It seems that Super Resolution is back in the news. By way of updating that post, let's revisit just what's going on with the technology and the few problems it may cause if you don't understand what's happening.

Vendor reps note that Super Resolution works at the "sub-pixel" level, and people's eyes roll. If the pixel is the smallest unit of measure, a single picture element, how can there be a "sub-pixel?" That's a very good question. Let's take a look at the answer.


From the report in Amped SRL's Five: The Super Resolution filter applies a sub-pixel registration to all the frames of a video, then merges the motion corrected frames together, along with a deblurring filtering. If a Selection is set, then the selected area will be optimized.

Ok. What is sub-pixel registration?

First, let's look at how the authors of Super-Resolution Without Explicit Subpixel Motion Estimation set up the premise: "The coefficients of this series are estimated by solving a local weighted least-squares problem, where the weights are a function of the 3-D space-time orientation in the neighborhood. As this framework is fundamentally based upon the comparison of neighboring pixels in both space and time, it implicitly contains information about the local motion of the pixels across time, therefore rendering unnecessary an explicit computation of motions of modest size. The proposed approach not only significantly widens the applicability of super-resolution methods to a broad variety of video sequences containing complex motions, but also yields improved overall performance." That's quite a mouthful.

Here's the breakdown.

The first thing we must understand is the pixel neighborhood. The neighbourhood of a pixel is the collection of pixels which surround it. The connected pixels are neighbors to every pixel that touches one of their corners.


Next, we must understand what registration means. Image registration is the process of aligning two or more images of the same scene. This process involves designating one image as the reference (also called the reference image or the fixed image), and applying geometric transformations to the other images so that they align with the reference.

Let's put it together. A static pixel (P) in a single image is easy to understand. But, what about video? That pixel represents some place in 4D space-time. That 4D space-time orientation will change as time progresses. We want to line-up (register) that pixel across the multiple frames. Super Resolution thus tracks implicit information about the motion of the pixel across 4D space-time, and corrects for that motion. The result of the process is a single higher-resolution image.

The practical implications are this:


  • Frame Averaging works well when the object of interest doesn't move. The frames are averaged and the things that are different across frames are removed and the things that are the same remain.
  • To help with a Frame Averaging exercise, we can use a perspective registration process to align the item of interest - a license plate for example - across frames. This works well when the item has moved to an entirely new location, like in low frame rate video.

But, when the motion is subtle, super resolution is a better choice.

Here's an example. The park service was investigating a vandalism and poaching incident. There's a video that they believe was taken in the area of the incident. Within the video, there's a sign in the background that contains location information (text) that's blurred by the motion of the shaking, hand-held camera. There's enough motion to eliminate Frame Average as a processing choice. There's not enough motion to use a perspective registration function to align the sign correctly. Super resolution is the best choice to correct for the motion blur and register the pixels that make the text of the sign.

In this case, super resolution was indeed the best choice. The sign's information was revealed and the location was determined.

And now the potential pitfalls ...

  • Brand new pixels and pixel neighborhoods are created in this process.
  • A brand new piece of evidence (demonstrative) is created in this process.
Whenever you perform a perspective registration, your geometric transform necessarily creates new pixels and neighborhoods. In FIVE, during the process of using the filters, the creation is "virtual" in that it all happens in CPU and RAM. These new pixels and neighborhoods are really only created when you write the results of your processing out as a new file.

That brand new piece of evidence that you've created - the results written out - is a demonstrative that you've just created. You must explain it's relationship to the actual evidence files and how it came to be. Indeed, you've just added a new file to the case. This fact should be disclosed in your report.

With the reports in FIVE, there is the plain English statement about the process that is lifted from the many academic papers from which Amped SRL gets their filters. Sure, when you're asked about the process  performed, you can likely just read the report's description. But, what if the Trier of Fact wants to know more? How confident are you that you can explain super resolution?

Consider super resolution's main use - license plate enhancement. Your derivative file is a demonstrative in support on one side's theory of the case. Your derivative is illustrative of your opinion. Did you use the tool correctly? Are the results accurate? Is seeing believing? Given the ultra low resolutions we're usually dealing with, a slight shift in pixels can make a big difference in rendering alpha-numeric characters. This is part of the reason Amped SRL likes to use European license plates in their classes and PR - they're easy to fix. Not so in the US.

Advice like that shown above is the value of independence. A manufacturer's rep can really only show you the features. I'll show you not only how a tool works, but how to use it in different contexts, why it's sometimes inappropriate to use, and how to frame it's use during testimony. If you're interested in diving deep into the discipline of video forensics, I invite you to an upcoming course. See our offerings on our web site.

Have a great day, my friends.

Tuesday, January 7, 2020

The FTC vs Axon? Axon vs the FCT? Wow!

Whilst we were all minding our own business, it seems that the US Federal Trade Commission was busy investigating Axon for anti-competitive behavior. Last Friday, Axon CEO, Rick Smith, penned a piece on LinkedIn to make his case to the public. According to Smith, the FTC believes that Axon's acquisition of VieVu in 2018 was anti-competitive.

I'm not a fly on the wall. I only know what I've read in Smith's post and the subsequent reporting and interviews. To be fair, the press has let Smith tell Axon's side of the story. For the government's side, we have only a press release on the FTC's web site.

In terms of disclosure, it should be noted that within the scope of my prior employment with Amped Software, Inc. (an Axon partner), I worked closely with several internal business units within Axon.

But, I want to break down the FTC's press release to attempt to determine what's really the problem here.

Paragraph 1: "The Federal Trade Commission has issued an administrative complaint (a public version of which will be available and linked to this news release as soon as possible) challenging Axon Enterprise, Inc.’s consummated acquisition of its body-worn camera systems competitor VieVu, LLC. Before the acquisition, the two companies competed to provide body-worn camera systems to large, metropolitan police departments across the United States."

Analysis: Yes, they were in the same market. But, given the many quality issues with VieVu's product line, they weren't really competing - in the same way that the best Texas high school football team is really no competition for the worst of the NFL in any given year. My opinion on how VieVu got "competitive" on deals was to compete on price, not on quality. It was their low price that got them in the door at police departments, but it was their lack of quality that ruined the company.

Paragraph 2: "According to the complaint, Axon’s May 2018 acquisition reduced competition in an already concentrated market. Before their merger, Axon and VieVu competed to sell body-worn camera systems that were particularly well suited for large metropolitan police departments. Competition between Axon and VieVu resulted in substantially lower prices for large metropolitan police departments, the complaint states. Axon and VieVu also competed vigorously on non-price aspects of body-worn camera systems. By eliminating direct and substantial competition in price and innovation between dominant supplier Axon and its closest competitor, VieVu, to serve large metropolitan police departments, the merger removed VieVu as a bidder for new contracts and allowed Axon to impose substantial price increases, according to the complaint."

Analysis: Given the analysis of the first paragraph, VieVu was certainly not "particularly well suited" to deliver on any department's needs. Additionally, it wasn't the "competition" that drove prices down, it was VieVu's essentially offering their goods below cost to get in the door that drove prices down. Selling below cost isn't sustainable, and police agencies must look at all factors of a vendor - like the fact that unsustainable business practices will likely mean that the company won't be around throughout the lifecycle of the product.

Paragraph 3: “Competition not only keeps prices down, but it drives innovation that makes products better,” said Ian Conner, Director of the FTC’s Bureau of Competition. “Here, the stakes could not be higher. The Commission is taking action to ensure that police officers have access to the cutting-edge products they need to do their job, and police departments benefit from the lower prices and innovative products that competition had provided before the acquisition.”

Analysis: the market is still chocked full of offerings. There's Motorola/Watchguard, Panasonic, Getac, Utility, Coban, Visual Labs/Samsung, L3/Mobile Vision, and Digital Ally, plus over 10k results from China on alibaba.com. You can get a body camera from China's LS Vision for under $100/unit. That's a lot of competition.

Paragraph 4: "The complaint also states that as part of the merger agreement, Axon entered into several long-term ancillary agreements with VieVu’s former parent company, Safariland, that also substantially lessened actual and potential competition. These agreements barred Safariland from competing with Axon now and in the future on all of Axon’s products, limited solicitation of customers and employees by either company, and stifled potential innovation or expansion by Safariland. These restraints, some of which were intended to last more than a decade, are not reasonably limited to protect a legitimate business interest, according to the complaint."

Analysis: This part is just silly. Axon says to Safariland, known for their holsters and gear, stay with what you're good at (holsters and gear) and we'll stay with what we're good at. Stay out of our lane, and we'll stay out of yours. This is a good business decision, not anti-competitive behavior. You also have to be myopic to not consider that Safariland only bought VieVu in 2015. According to the WSJ, "Vievu LLC, a maker of police body cameras, has been acquired by Safariland LLC, which is bulking up its portfolio of security products ahead of a planned initial public offering next year." Safariland's entry into other vertical markets followed a similar pattern. But, at their heart, they're a holster and gear company, so their exit from the technology sector is no great loss.

Paragraph 5: "The Commission vote to issue the administrative complaint was 5-0. The administrative trial is scheduled to begin on May 19, 2020."

Analysis: What is missing is a specific citation as to which federal laws were violated. Likely, there was no specific violation of US law, but rather a violation of an FTC Rule. The FTC has the authority to pass and enforce it's own rules outside of the normal US law making process. Smith outlines the administrative hearing process in his op-ed. Smith is correct, this won't see a "court room," as the vast majority of FTC processes are kept in-house.

An examination of the FTC's "Competition Enforcement Database" found only 25 competition enforcement actions for 2018, which was down from 2017's 32 actions. Given the totality of commercial activity in the US, this is an incredibly small number. The assumption is that they only go after the most egregious of behaviors. If that's the case, what's really behind this action against Axon. VieVue was delivery faulty products. It was losing deals on it's own. Axon did a mutually beneficial deal with Safariland to take VieVu off their hands. What's actually wrong with this? Does this rise to a Standard Oil or AT&T level? Hardly. So why this case? That's the problem with administrative processes, we'll never know. There's a complete lack of transparency into their decision making or procedures.

I do tend to agree with Smith that this issue rises above brands and technology. It's a peek into the workings of the Administrative State in the US. What remains to be seen is if the US government grants Axon permission to sue the FTC. Stay tuned.

Monday, December 30, 2019

An Amped FIVE UX tip to wrap up the year

The recent update to Amped SRL's flagship tool, FIVE, brought some UX headaches for many US-based users. You see, the redesigned reporting function does a something new and unexpected. Let's take a look, and offer a couple of work-arounds.

Let's say you're used to your Evidence.com work flow, the one where all your evidence goes into a single folder for upload, and you're processing files for a multi-defendant case. If there are files featuring only one of the defendants, which happens often, you'll want to have separate project (.afp) files for each evidence item. This will make tagging easier. This will make discovery easier. This will make the new reporting functionality become non-functional.

You see, the new reporting feature doesn't just create a report, it creates a folder to hold the report and the Amped SRL logos - and calls the folder "report." That's fine for the first file that you process. But the next project? Well, when you go to generate the report, FIVE will see that there's a "report" folder there already. What does it do? Does it prompt you to say, "hey! there's already a folder with that name. What do you want to do?" Of course not. Not expecting a new reporting behavior, and a complete lack of documentation of this new reporting format, you'll just keep processing away. At the end of your work, there's only one folder and only one report file.

The work-around on your desktop is to put each evidence item into it's own folder, within the case folder. It's an extra step, I know. You'll also have to modify the "inbox" that E.com is looking for.

The other weird issue is that FIVE now drops some logos and a banner as loose files in the report folder. I'm sure that this is due to FIVE's processing of the report - first to HTML - then to PDF via some freeware. One would think that in choosing PDF you wouldn't receive the side-car files, but you would be thinking wrong. Again, this has to do with the way the bit of freeware processes the report.

As an interesting aside, in Daubert Hearing, I actually got a cross examination question that hinted at Amped SRL being pirates of software. Is anything in the product an original creation or is it just pieced together bits and such? But, I digress.

Remember, in the US, anything created in the process of analysis should be preserved and disclosed. One customer complained that it seems as though Amped SRL is throwing an extra business card in on the case file. I don't know about that. But, it does seem a bit odd for a forensic science tool to behave in such a way.

You can always revert to the previous version if you want to save time and preserve your sanity. This new version doesn't add much for the analyst anyway. You can easily install previous versions. It takes only a few minutes each time.

As with anything forensic science, always validate new releases of your favorite tools prior to deploying them on casework. If you're looking for an independent set of eyes, and would like to outsource your tool validation, we're here to help.

Have a great new year my friends.

Sunday, December 22, 2019

Human Anatomy & Physiology

In my private practice, I often review the case work of others. This "technical review" or validation is provided for those labs that either require an outside set of eyes by policy / statute, or for those small labs and sole-proprietors who will occasionally require these services.

Recently, I was reviewing a case package related to a photographic comparison of "known" and "unknown" facial images. In reviewing the report, it was clear that the examiner had no training or education in basic human anatomy & physiology. Their terminology was generic and confusing, avoiding specific names for specific features. It made it really hard to follow their train of though, which made it hard to understand the basis for their conclusion.

Many in police service either lack a college degree, or have a degree in criminal justice / administration of justice. These situations mean that they won't have had a specific, college level anatomy / physiology course. Or, it may be that, like me, your anatomy course was so long ago that you find yourself needing a refresher.

I got my refresher via a certificate program from the University of Michigan, on-line. But, if you don't have the time or patience to work through the weekly lessons, or if you're looking for a lower cost option, I've found an interesting alternative.

In essence, I've found a test-prep course that's designed to help doctors and medical professionals learn what they're supposed to know, and help them pass their tests. It's reasonably priced and gives you the whole program on your chosen platform.

The Human Anatomy & Physiology Course by Dr. James Ross (click here) even comes with illustrations that you could incorporate in your reports. Check it out for yourself.

You don't necessarily have to have training / education in anatomy to opine on a photographic comparison. Not all jurisdictions will care. But, it really ups your reporting game if you know what the different pieces and parts are called, and if you use professional looking graphics in your report.

If you need help or advice, please feel free to ask. I'm here to help.

Have a great day my friends.

Monday, December 16, 2019

Independence Matters

Independence matters. When learning new materials, it's important to get an unvarnished view of the information. Too many "official" vendor-sponsored training sessions aren't at all training to competency - they're information sessions, at best, or marketing sessions, at worse.

Vendors generally don't want their employees and contractors to point out the problems with their tools. But as a practitioner, you'll want to know the limitations of that thing you're learning. You'll want to know what it can do, what it does well, and what it doesn't do - or does wrong.

Now, I understand that when people start showing you their resume, they've generally lost the argument. But, in this case, it's an important point of clarification. I do so to show you the value proposition of independent training providers like me. Having been trained and certified by California's POST for curriculum development and instruction means that I understand the law enforcement context. Yes, I was a practitioner who happened to be in police service. The training by POST takes that practitioner information and focuses it, refines it, towards creating and delivering valid learning events. Having an MEdID - a Masters of Education in Instructional Design, means that I've proven that I know how to design instructional programs that will achieve their instructional goals. The PhD in Education is just the icing on the cake, but proves that I know how to conduct research and report my results - an important skillset for evaluating the work of others.

Unlike many of the courses on offer in this space, the courses that I've designed and deliver for the forensic sciences are "competency based." I'm a member of the International Association for Continuing Education and Training (IACET) and follow their internationally recognized Standard for Competency Based Learning.

According to the IACET's research, the Competency-Based Learning (CBL) standard helps organizations to:
  • Align learning with critical organizational imperatives.
  • Allocate limited training dollars judiciously.
  • Ensure learning sticks on the job.
  • Provide learners with the tools needed to be agile and grow.
  • Improve organizational outcomes, return on mission, and the bottom line.
The Trier of Fact, in judging your qualifications, will want to know if you are competent in your work as an analyst. Answering that your behind was in a seat for X amount of hours is not an appropriate response. Yes, you were present at a specific event. But was that event actually a training to competency event, with a properly designed curriculum, delivered towards specific learning goals?  Or, did you report an information session at a conference as training to competency? Yes, you were there. But, did you learn what you were supposed to learn? Did you apply that learning in your work? Are you, in fact, competent?

This is why, when delivering "product training," I've always wrapped a complete curriculum around the product. I've never just taught FIVE, for example. I've introduced the discipline of forensic digital / multimedia analysis, using the platform (FIVE) to facilitate the work.  This is what competency based learning looks like.
Yes, you need to know what buttons to push, and in what order, but you also need to know when it's not appropriate to use the tool and when, how, and why to report inclusive results. You'll need to know how to facilitate a "technical review" and package your results for an independent "peer review." You'll also need to know that there may be a better tool for a specific job - something a manufacturer's representative won't often share.

Align learning with critical organizational imperatives - I've studies the 2009 NAS report. I understand it's implications. The organization that the analyst serves is the Trier of Fact, not the organization that issues their pay. The courses that I've designed and deliver match not only with the US context, but also align with the standards produced in the UK, EU, India, Australia, and New Zealand (many other countries choose to align their national standards with one of these contexts).

Allocate limited training dollars judiciously - Whilst most training vendors continue to raise prices, I've dropped mine. I've even found ways to deliver the content on-line as micro learning, to get prices as low as possible. Plus, it helps your agency meet it's sustainability goals.

Ensure learning sticks on the job - Building courses for all learning styles helps to ensure that you'll remember, and apply, your new knowledge and skills.

Provide learners with the tools needed to be agile and grow - this is another reason for building on-line micro learning offerings. Learning can happen when and where you're available, and where and when you need it.

Improve organizational outcomes, return on mission, and the bottom line - competency based training events help you improve your processing speed, which translates to more cases worked per day / week / month. This is your agency's ROM and ROI. Getting the work done right the first time is your ROM and ROI.

I'd love to see you in a class soon. See for yourself what independence means for learning events. You can come to us in Henderson, NV. We've got seats available for our upcoming Introduction to forensic multimedia analysis with Amped FIVE sessions (link). You can join our on-line offering of that course (link), saving a ton of money doing so. Or, we can come to you

Have a great day, my friends.

keywords: audio forensics, video forensics, image forensics, audio analysis, video analysis, image analysis, forensic video, forensic audio, forensic image, digital forensics, forensic science, amped five, axon five, training, forensic audio analysis, forensic video analysis, forensic image analysis, amped software, amped software training, amped five training, axon five training, amped authenticate, amped authenticate training