• Skip to main content
  • Skip to secondary menu
  • Skip to primary sidebar
  • Skip to footer
  • Our Services
  • Contact Us
  • Newsletter
  • Top Nav Social Icons

FindBiometrics

FindBiometrics

Global Identity Management

  • Biometrics
    • What are Biometrics?
    • FAQ
    • Biometric Associations
    • Companies
    • Premier Partners
  • News
    • Featured Articles
    • Interviews
    • Thought Leadership
    • Podcasts
    • Webinars
    • Year in Review
  • Applications
    • Biometric Security
    • Border Control and Airport Biometrics
    • Consumer and Residential Biometrics
    • Financial Biometrics
    • Fingerprint & Biometric Locks
    • Healthcare Biometrics
    • Justice and Law Enforcement Biometrics
    • Logical Access Control Biometrics
    • Mobile Biometrics
    • Other Biometric Applications
    • Physical Access Control Biometrics
    • Biometric Time and Attendance
  • Solutions
    • Behavioral Biometrics
    • Biometric Sensors and Detectors
    • Facial Recognition
    • Biometric Fingerprint Readers
    • Hand Readers & Finger Scanners
    • Iris Recognition
    • Biometric Middleware and Software
    • Multimodal Biometrics
    • Physiological Biometrics
    • Smart Cards
    • Vein Recognition
    • Voice and Speech Recognition
  • Stocks
  • Events
  • Companies
  • Podcasts

NIST’s FRVT Isn’t Up to Task of Evaluating Video Surveillance Algos, says Oosto

July 11, 2022

NIST's FRVT Isn't Up to Task of Evaluating Video Surveillance Algos, says Oosto

While virtually every biometrics vendor now agrees on the importance of standards-based, third party testing, Oosto is seeking to clarify some of the shortcomings of one of the most respected testing programs in the business.

In a new blog post, Oosto’s Chief Marketing Officer, Dean Nicolls, respectfully lays out the five major shortcomings of the National Institute of Standards and Technology’s Face Recognition Vendor Test (NIST FRVT).

As Nicolls explains, the testing program is designed to measure the effectiveness of facial recognition algorithms with an eye toward government-focused application areas such as border control and law enforcement. And Nicolls readily acknowledges that “FRVT serves its purpose,” providing assessments relevant for the use cases concerned. The problem, from Oosto’s perspective, is that the NIST’s FRVT testing isn’t adequate for assessing and comparing facial recognition solutions based on video surveillance, for five key reasons:

  1. FRVT doesn’t evaluate solutions using real-time videos. That means that it can’t adequately assess a solution like Oosto’s OnWatch, which uses AI and neural network technology that is “fundamentally different” from the kind of tech used in the basic 1:1 and 1:n matching algorithms that FRVT is meant to evaluate.
  2. FRVT’s “in the wild” category for evaluating partially obscured faces doesn’t come close to the difficulty of conditions at which Oosto’s technology is adept at navigating. “NIST’s wild images may be blurry, or of low quality, but the images are still relatively clear, the camera is generally at face level, and the lighting is good,” explains Nicolls. “In the real world, perhaps within a casino or stadium, subjects are far less cooperative and do not look directly at the camera.”
  3. FRVT was never designed to evaluate the performance of a facial recognition system operating against a crowd. Systems that perform well in the NIST’s testing program may end up with a number of false positives and false negatives when applied to surveillance feeds scanning large numbers of subjects at a time.
  4. FRVT makes assumptions about the quality of cameras being used for face capture that may not be relevant for organizations deploying “smart facial recognition cameras” that can deliver dramatically improved performances even in challenging circumstances.
  5. FRVT mounts cameras at face-level, artificially boosting the performance of algorithms that would otherwise struggle when processing images from an overhead surveillance camera. This is fine for highly controlled scenarios such as border checkpoints, but not for general video surveillance across a range of contexts.

To be clear, none of this is meant as a criticism of the NIST or even of its FRVT program in and of itself. Nicolls’ aim is rather to help to better inform organizations that are considering the use of face-based surveillance about what FRVT performance really means, and what it doesn’t mean.

That having been said, Oosto’s post concludes by calling on the NIST to create a new testing methodology that can compare video surveillance-based facial recognition solutions. It would be a welcome complement to the renowned testing programs that the NIST already has in place.

With its constructive criticism of the biometrics industry’s testing landscape, Oosto joins other face biometrics specialists in a wider call for evaluation programs that take into account new and recently evolved technologies. Earlier this year, FaceTec – a 3D face biometrics company that specializes in ID verification and user authentication – announced a contractual relationship with A.C.C.S. Ltd. intended to bring forward the world’s first 3D face matching assessment.

–

July 11, 2022 – by The FindBiometrics Editorial Team

Related News

  • Rank One Continues to Improve in NIST EvaluationsRank One Continues to Improve in NIST Evaluations
  • Paravision Takes Top Spot in Latest NIST EvaluationParavision Takes Top Spot in Latest NIST Evaluation
  • Corsight Shows Significant Bias Reduction in Latest NIST TestCorsight Shows Significant Bias Reduction in Latest NIST Test
  • Remark Posts Top Five Score in Latest NIST TestRemark Posts Top Five Score in Latest NIST Test
  • Paravision Posts Another Strong Score in Latest Round of NIST Biometric TestingParavision Posts Another Strong Score in Latest Round of NIST Biometric Testing
  • Innovatrics Celebrates NIST FRVT Leaderboard PositionInnovatrics Celebrates NIST FRVT Leaderboard Position

Filed Under: News Tagged With: Biometric, biometric surveillance, biometrics, corporate blog posts, corporate blogs, face biometrics, Face Recognition Vendor Test, facial recognition, FRVT, National Institute of Standards and Technology, NIST, Oosto

Primary Sidebar

Identity is Shaping Air Travel – Time to Invest

Sponsored Links

facetec logo

FaceTec’s patented, industry-leading 3D Face Authentication software anchors digital identity, creating a chain of trust from user onboarding to ongoing authentication on all modern smart devices and webcams. FaceTec’s 3D FaceMaps™ make trusted, remote identity verification finally possible. As the only technology backed by a persistent spoof bounty program and NIST/iBeta Certified Liveness Detection, FaceTec is the global standard for Liveness and 3D Face Matching with millions of users on six continents in financial services, border security, transportation, blockchain, e-voting, social networks, online dating and more. www.facetec.com

TECH5 logo

TECH5 is an international technology company founded by experts from the biometrics industry, which focuses on developing disruptive biometric and digital ID solutions through the application of AI and Machine Learning technologies.

TECH5 target markets include both Government and Private sectors with products powering Civil ID, Digital ID, as well as authentication solutions that deliver identity assurance for various use cases. 

Learn more: www.tech5.ai

Mobile ID World Logo

Mobile ID World is here to bring you the latest in mobile authentication solutions and application providers. Our company is dedicated to providing users with the best content and cutting edge information on technology, news, and mobile solutions for your mobile identity management needs.

Recent Posts

  • Lawsuits, Passkeys, and a New VP: Identity News Digest
  • Digital Identity Tech Demo Online Event
  • Biometric Ticketing Comes to Osaka Station: Identity News Digest
  • TikTok, Robots, and a $10 Facial Recognition Device: Identity News Digest
  • Register for the Digital Identity Tech Demo Online Event

Biometric Associations

IBIA and fido

Tweets

Footer

  • About Us
  • Company Directory
  • Advertise With Us
  • Contact Us
  • Privacy Policy
  • Terms of Use
  • Archives
  • CCPA: Do not sell my personal info.

Follow Us

Copyright © 2023 FindBiometrics