What new certification program?

We had a lively discussion in last week's Advanced Processing Techniques class. This class seems to bring out these types of conversations.

We were discussing certifications in our discipline: digital / multimedia analysis. When I said that I know of three, the students seemed puzzled. Three? They were aware of LEVA's CFVT and CFVA programs and the IAI's CFVE program. LEVA counts as one, the IAI, two. Where's three?

Whilst you were not paying attention, a group of analysts got together with the Electronics Technicians Association, International, to create the Audio Video Forensic Analyst (AVFA) certification. Hat's off to them for pulling that one off.

Who is the ETA-i? What is the AVFA? Let's take a look.

The ETA-i is a legit organization that certifies electricians, cable splicers, and other comms techs. It's been around since 1978 and has over 90 certification programs. I spoke with the office manager there about the AVFA certification and received an interesting answer about it's genesis. I was told that a few years ago, they began an expansion into other related industries. They added a Digital Video (DVE) Editor certification. From there, they added the AVFA certification. Some of those involved in the DVE certification were involved in the creation of the AVFA program.

Before you grab your pitch fork and start marching towards the town square, let's take a quick look at the program and compare it to what we know about the existing market for certifications.

Validity is defined as the extent to which an assessment accurately measures what it is intended to measure. Validity refers to the accuracy of measurement. In this case, the assessment is the certification exam. Determining validity can be viewed as constructing an evidence-based argument regarding how well a tool measures what it is supposed to do. Evidence can be found in content, response process, relationships to other variables, and consequences (source).

Reliability refers to whether an assessment instrument gives the same results each time it is used in the same setting with the same type of subjects.  For test/retest reliability, the test should give the same results each time, assuming there are no interval changes in what you are measuring, and they are often measured as correlation.

Face Validity - the extent to which a test is subjectively viewed as covering the concept it purports to measure.

Is the AVFA reliable? As a multiple choice exam, it is certainly possible. One would hope that it's written in such a way that you'd answer the same questions in the same way on successive administrations of the test.

Does the AVFA certification test have face validity? You can find out what it's supposed to measure on it's web site (link). You might look at the competencies sheet and note that there are programs listed that you're likely not familiar with. But, every analysis requires a platform. The creators of the AVFA had to choose tools and terminology. They seem to have chosen freeware and sources of information not hidden behind paywalls. Good for them. The ASTM's glossary is great, but how much do you have to pay to use it?

You can sign up and take the test yourself. No membership required. The test is affordable ($100 + proctor fees). The test is entirely multiple choice. Pass it, you're certified. Fail it, re-take it for free. But, on the face of it (pun intended), it seems to have face validity.

Accredited? Yes. The program at ETA-i is accredited by International Certification Accreditation Council (ICAC). Who are they? I didn't know either, so I called them as well. I had a nice and long conversation with a representative who explained their program and their long relationship with the ETA-i. According to them, they follow ISO 17011 / 17024 in the administration of their accreditation program. So far so good.

What's the deal with accreditation? One of the things that ASTM E2659-09 as well as ISO 17011 / 17024 emphasize is the separation of the training event / training provider and the certifying body. There is a risk of bias if the training provider also runs the certification process - as in teaching the test. This is a large part of why you take Photoshop classes from me, George, Dorothy, Scott Kelby, and etc. ... then trek on over to ProMetric to take the Adobe Certified Expert exam.

This contrasts with LEVA, as an example. LEVA is both training provider and certifying body. As such, it does not follow or comply with ASTM E2659-09 as well as ISO 17011 / 17024 in the design of its program. Thus, it's program is not accredited. There's no way to simply sit for an objective certification exam with LEVA. You must take their classes, all of them, in order to sit for the test. Additionally, it's impossible to judge the reliability or validity of the certification exam as all LEVA certification exams are unique experiences - and are closed to the public. I'm not saying that there's no value to the experience, just that it's not valid from a design / standards standpoint.

The IAI is in the middle. I'm aware that they are seeking accreditation for their programs. They are not a training provider, per se. They do offer training at their conferences, but you can't call that "training to competency." It's more like information sessions, as the ASTM defines training events. Thus, it is possible for the IAI to get their programs accredited. They should, however, remove the discriminatory membership requirement in order to sit for their test.

Thus, as someone who is trained / educated in curriculum and instructional design, who has created many assessments, and has reviewed countless others, I'm excited to see how this new program shakes out. I'll likely sit for the exam soon, just to see what it's like.

If you decide to take the test, let me know what you think of it. Try to go into the process with an open mind. I'm a big fan of democratizing science. If this is a legit certification, I'm all for it.