UX Talk: Amped Authenticate - Batch File Format Comparison

One of the problems with products that generate results based on a comparison to an internal database  is errors - false positives / false negatives. As much as I like Amped's Authenticate, it does suffer from this problem. The new reporting feature makes this worse, not better.

Here's the scenario.

You load an image and check the File Information, or you run the new "Smart Report." One of the results in this panel indicates a mismatch of the JPEG's Quantization Table - "Compression signature is incompatible with the actual camera make-model."

How do you know if this result comes from an actual mismatch? How do you know if this result is in fact a "non match," meaning the evidence file comes from a camera who's JPEG QT information isn't actually in the product's database - in this case Amped Authenticate?

Answer: run a batch File Format Comparison against a valid sample set of random images taken by the same model of camera.

I've written previously about the utility of the Camera Forensics service. This is one of those instances where the service comes in handy. Assemble your sample and drop a copy of your evidence image into the folder of samples. Then run the Comparison.

When the report is generated, scroll across to find the column for JPEG QT hash. If, in fact, your evidence image is a mismatch - "Compression signature is incompatible with the actual camera make-model" - then the evidence image's JPEG QT hash should not match the JPEG QT of any of the sample file. In my case, all of the samples matched each other, and matched the evidence file. The result from the software was false.

Were this your case, an actual case, you would want to thoroughly document this process of validating the results. You wouldn't want to leave the statement of "mismatch" untested. Yes, the software returned the result. However, we assembled a valid sample set to test the results. Here, we found the software in error (likely the software's internal database was incomplete). We found that the evidence file's JPEG QT matched the JPEG QT for all of the sample files.

The pace of change in camera technology is quite brisk. The issue is further complicated by the fact that camera technology is a bit regional - meaning there are cameras sold in some areas but not others. Additionally, not all manufacturers make their QTs available to developers. Very few do, actually. Thus, is it realistic to find all QTs present in a particular software's database? Of course not. Therefor, you'll need to remember to validate your results - especially if you get a "false" result.