Using FARO Scene 2019 to measure within CCTV images?

Last month, FARO updated their Scene software's user guide for 2019. I was very curious to see if the guidance had changed on a controversial issue that I addressed in a letter to the editor of the Journal of Forensic Identification (link) last year.

The heart of the issue deals with the mixed methods approach of inserting a single frame taken from CCTV footage (previous event) into a laser-generated point cloud scan (current event) processed by FARO's Scene software. The question on everyone's mind is this: can you measure items / objects / subjects that are present in the CCTV image but are not present in the point cloud scan?

Let's see what the new user guide has to say.

page 80 - 08m86e00_FARO_SCENE_User_Manual_2019.1.pdf

According to FARO (see above graphic), "There are three ways to use images in SCENE:

• Images can be added with their original resolution to the workspace and thus provide additional information about the scan environment.
• Images can be added with their original resolution to the workspace and thus provide additional information about the scan environment. These images are imported into the 3D world into virtual scans with their full resolution. Such images will be interpreted like a high resolution scan of a plane surface and can be placed on arbitrary positions in the 3D world.
• Images can be used to add color information to already existing scan points."

What is the primary reason for adding images into the scan environment? Images can be used to add "richness" and "texture" to the scan. Here's how:

page 81 of the pdf file

If you've worked with these types of scans, you know that the result is a flat grey colour. The role of images, or virtual scans, is to add information back into the scene. In the example above, the scan only registers the location of the picture frame - but not the contents of the frame. Thus, the virtual scan is utilized to add the "information" about the picture into the scan. 

But that's not what you want to do, is it. You want to measure an item that isn't present in the scan.

page 85 of the pdf file

The Place in 3D function seems like it might be it. But, it's not. The Place in 3D function allows the user to place a 2D representation of a region of interest into the scan. The user then must associate points in common between the 2D image and the 3D scan.

page 86  of the pdf file

Now that you've registered the scan and associated points in common, you're ready to measure ... or are you?

You don't want to measure items in the scan. You want to measure items not in the scan. Unfortunately, you can't with this tool. There are no corresponding points between the thing you want to measure and the point cloud. The FARO Scene manual lists the procedure for measuring points in common. It does not show users how to measure the woman in the scene above, who is not present in the point cloud scan.

Thus, the question that begs asking: if a "forensic video analyst" has a scan, some CCTV footage, and FARO Scene, where is the measurement happening? If it's happening in Scene, which doesn't support the function, how accurate are the measurements? If it's not happening in FARO Scene, where is it happening?

If the measurement is not happening in Scene, how does the "analyst" get the image out? 

Section 11 deals with exporting from Scene. "11.5.3 Exporting the images of the scans to.jpg format

• In the Structure View, right-click the scan, then select Export> PanoramicImages. Select Scan Resolution to create images that have the same color resolution as the scan, or select Full Color Resolution if you want to create panoramic images with the highest color quality possible, and which are compensated to remove the offset between the two halves of the scan, as well as any distortion cause by the scanner’s rotation. Full color resolution panoramas have a white stripe at the bottom of the picture because the proportions of the scan and the picture are different. (Scans made with FARO scanners versions M70, S70, S350 and later create 160 megapixel images. Scans from older scanners only output panoramic images with 40 megapixel images.)

Which requires another question be asked: what's the point of bringing the CCTV frame into the point cloud scan only to bring it out again to measure?

Why is this a big issue? Science. The mixed-methods approach has yet to be validated in a general sense, with that validation published in order for "the community" to investigate the methodology and results and attempt to replicate the published experiment.

Has there ever been an evidence hearing (Frye / Daubert hearing) on this? If you're aware of one, I'd sure like to know. Not that evidence hearings should replace validation studies, or that evidence hearings should come before validation studies. I'm just aware, anecdotally, that people are performing this technique and testifying as to their results / conclusions. What isn't available in the anecdotes are any sense of the science or testimony as to validity. 

Let's take a look at why this is important.

You'll remember that prior to Daubert, Frye was the law of the land. The Frye standard is commonly referred to as the “general acceptance test” under which generally accepted scientific methods are admissible, and those that are not sufficiently established are inadmissible. Can something without a history of publication or validation be "sufficiently established?"

The Frye Standard comes from the case Frye v. United States, 293 F. 1013 (D.C. Cir. 1923) in which the defendant, who had been charged with second degree murder, sought to introduce testimony from the scientist who conducted a lie detector test. The D.C. Court of Appeals weighed expert testimony regarding the reliability of lie detector test results. The court noted: Just when a scientific principle of discovery crosses the line between the experimental and demonstrable stages is difficult to define…. [W]hile courts will go a long way in admitting expert testimony deduced from a well-recognized scientific principle of discovery, the thing from which the deduction is made must be sufficiently established to have gained general acceptance in the field in which it belongs.

The last part of that sentence is where I want to go with Frye - in the field in which it belongs. 

3D laser scans of traffic collisions are wonderful. 3D laser scans of scenes where the police have used force are a good thing. These recording methods capture the smallest details of the scene - present at the time of the scan (aka "now"). They have done so well in their documentation of scenes that they've become generally accepted among "scene recinstructionists." But, "scene reconstruction" is an entirely different function than digital / multimedia forensic analysis. For the FVA community, other methods of photogrammetry are generally accepted (e.g. single image photogrammetry). If the "scene reconstruction" folks want to work in the domain of digital / multimedia forensic analysis, then they must follow it's rules ... not theirs. This is key. Remember, validity deals with the accuracy of the measure as well as the appropriateness of the process / tool.

I was initially excited to see that FARO had issued an update. But, as you can see, not much has changed from the previous version in terms of measuring objects that aren't in the scan. Perhaps next time ...