For seven years, according to court records, Anthony Sena had been stealing from retail stores in Albuquerque, New Mexico. He’d hit Walmart, Sportsman’s Warehouse, several grocery stores, and a bunch of Targets. But when he walked into an Albertsons in February, the store was ready. Its security surveillance system produced an alert, autonomously and instantaneously recognizing him as someone responsible for multiple past shoplifting incidents. Armed with the warning, loss prevention agents asked a police officer working on site to intercept him. When the officer approached Sena, he ran off. Police chased down the serial shoplifter, who had two misdemeanor warrants tied to other criminal cases, and charged him with resisting, evading, or obstructing an officer, according to the criminal complaint and reports from local station KRQE.
Face recognition technology can do that. And despite hesitancy among retailers to admit it, it is doing that. Dara Riordan, chief revenue officer at FaceFirst, said the company boasts one US deployment with hundreds of locations. Its largest customer database is in Latin America with nearly 30,000 enrolled templates.
Uses Cases
Lian Jye Su, principal analyst for artificial intelligence at ABI Research, a global technology market advisory, described some real-world use cases for face recognition in retail.
Theft Monitoring. “Many existing supermarkets and retail stores already have surveillance cameras for theft monitoring, and facial recognition is installed as an upgrade to the existing system.” In addition to real-time alerts of past bad actors—the frequent fraudulent returner, serial shoplifter, or potentially violent individual—he said tech firms are developing context-aware alert systems to detect relevant events with retailers’ CCTV streams.
Point of Sale (POS) and Self-Service Checkout. “This is very common in China, where Chinese are checking out using their digital wallet, such as AliPay and WeChat Pay, which has built-in biometric identification information,” he explained. “Instead of paying with credit cards, Chinese shoppers will scan their faces at the POS terminals during checkout at various unmanned stores, such as Hema Xiansheng (by Alibaba) and Suning Biu.”
Enhanced Customer Service. “When integrated with other systems, facial recognition can provide access to a wide range of data on individual customers,” he said. Facial recognition allows retailers to identify individual customers and link them to existing loyalty programs to generate personalized marketing for individual customers or provide discounts based on their loyalty.
According to Hedgie Bartol, then-retail segment development manager at Axis Communications, “For brick-and-mortar retailers, there are many use cases in which facial recognition can help improve safety and security. Primary use cases include identification of VIP and valued customers, access control for employees and associates, identification of known offenders to reduce crime and retail shrink, and the prevention of armed robbery.”
Deterrence has been the primary benefit from face recognition deployed at five high-risk Jacksons convenience stores in Portland and Tacoma, Washington, according to a store spokesperson. During nighttime hours, store doors remain locked until an approaching customer looks squarely into an external camera, so their image can be compared to a database of flagged individuals in real time. If no match is made, the door immediately unlocks. Images are maintained for forty-eight hours by the security provider and then deleted unless the store notifies them that a specific person’s image should be kept and flagged. To date, it has only been necessary to keep a few images on file, but Jacksons spokesperson Russ Stoddard told the Oregonian he thinks the store-entry process, combined with a sign notifying individuals that facial recognition is in use, reduces risk and limits the number of security issues that equipped stores face. “We’ve found that it precludes certain types of behavior,” he said, “because they see the camera and sign on the front door and know this is not a place to cause a ruckus.”
Facial recognition technology (FRT) also has value after a crime. “A retailer would be able to easily identify an individual that has stolen goods or otherwise caused damage to an establishment, which would allow the retailer to pursue further action against the individual,” explained attorney Adil D. Kolovic, an associate at Marshall Gerstein in Chicago.
Riordan said she sees asset protection teams using the technology successfully in just this way, such as to surface an individual running a credit card scheme wherever and whenever he’s been to a store, to alert those stores that he hasn’t yet hit, and to create a compelling case. “It allows you to send to police a really strong storyboard,” she said in her presentation at the Global Retail Crime Summit this summer.
Adoption
Adoption of FRT is happening all over. Grocers in the UK are working with tech companies to integrate facial recognition at self-checkout to verify ages for alcohol purchases. London’s Metropolitan Police rolled out “live” facial recognition across London earlier this year to reduce serious crime. The city of Ontario, Oregon, has done something similar during its recent surveillance camera upgrade. Catch&Go, the registerless digital store concept, added a facial recognition feature to “deliver a frictionless shopping experience” using facial recognition without other engagement, said NTT Data Corporation in an announcement earlier this year. Rather than keeping track of a physical ticket, guests at Universal Studios Singapore will be registered into its facial recognition systems for identification.
The technology is full of potential, said Lian Jye Su. He can see stores using AI and facial recognition to identify shoppers within the context of their buying history to make personalized recommendations, to free sales staff from store monitoring and POS functions and provide enhanced customer service, and as the cornerstone of the future’s unmanned retail store. “This is highly attractive for large e-commerce retailers that are looking to enhance their footprint in brick-and-mortar retail but are reluctant to emulate the existing brick-and-mortar retailers,” he told LP Magazine.
“When it comes to direct store deliveries, facial recognition could be utilized to allow secured access by the delivery drivers,” said Bartol. “Retailers will also look to the technology to help reduce the impact of organized retail crime (ORC) by identifying and deterring known offenders.”
Although already mature, the technology still has room to grow, especially as a multichannel solution, according to Paul Wilson, product manager for risk-based authentication at AppGate. “For example, continuous face authentication may be feasible, meaning that the user isn’t just verified when logging into an app but also as the customer uses the application or progresses through the user journey, allowing the customer to perform more sensitive tasks without interruption,” he explained. “Whilst there are obvious privacy concerns, this does suggest a future where it would be technically feasible to verify shoppers transparently as they pick up their groceries and automatically bill them as they leave.”
Maurizio Pejoves, director of P&O Global Technologies, a maker of surveillance camera systems, is a proponent of FRT in retail. “I would recommend for retailers to use facial recognition because it detects biometrics and identifies persons of interest in real time—supporting security staff, marketing departments, and operations management in one place.” He also sees value in using the technology to control access in restricted spaces, such as company buildings and warehouses.
Kolovic acknowledges the risks associated with implementing FRT but thinks they will, in time, be overcome. “It seems that it is only a matter of time before the technology is more widespread,” he said. From unlocking a phone to event security, FRT applications are already multiplying, and the prospects of a future payday “is why some of the biggest names in tech are building patent portfolios in FRT.”
It may take some time to gain a foothold in retail, however. Despite advocates and advantages, the technology still seems to be climbing uphill. The tone of news reports suggests one reason why.
In covering the apprehension of that shoplifter in Albuquerque, KQRE’s reporting stressed the surprise of shoppers upon learning the technology was in use at the store. In late July, Reuters reported results of its “investigation” into Rite Aid “quietly” adding facial recognition technology at 200-plus locations. The story hit on multiple flashpoints: it was mainly placed in stores in nonwhite neighborhoods; its technology provider had links to China; and early store deployments misidentified people of color.
Emeritus Professor Adrian Beck has been researching facial recognition in retail, including extensive interviews with LP executives who have tested the technology, and he’s found both hope and hesitancy. “Certainly, when I spoke to a number of retailers who’d looked at this, they were very excited at the potential for this technology to help them begin to try and identify persistent thieves entering into their stores,” he said in a recent discussion of his research at an industry webinar. “It’s often seen by many as a great opportunity to try and reduce crime,” but it remains a “very controversial area,” he said. Excitement is tempered by “an incredible amount of concern about reputational issues.”
It has proved too hot to touch for many. UCLA dropped its plan to use facial recognition technology on its campus. Home Depot told Reuters it tested facial recognition to reduce shoplifting but stopped its trial this year. And Reuters said Rite Aid defended its program but then told the news agency it had shut off the technology. “This decision was in part based on a larger industry conversation,” the company told Reuters in a statement, adding that “other large technology companies seem to be scaling back or rethinking their efforts around facial recognition given increasing uncertainty around the technology’s utility.”
Lowe’s never deployed facial recognition in any meaningful way, but it did test the technology several years ago. “I’ll tell you, in combatting ORC, it proved to be effective,” said Scott Draher, vice president of asset protection of safety, in a June webinar on video analytics by the ECR Retail Shrink Group. Regarding apprehending boosters, enrolling them in the system, and identifying their presence in a store, he said, “That technology does work with organized retail crime.” As is often the case, however, there was a “but.” In Lowe’s case, negative consumer sentiment was a significant reason why, despite the positive technology tests, they didn’t do anything further with it, said Draher.
Clearly, there are obstacles, but the powerful technology has grown and still beckons.
Technology
Face recognition has followed a typical technology trajectory—from hype and inflated promises to operational viability. The National Institute of Standards and Technology recently released a study concluding that algorithms today are twenty times better at searching databases and detecting matches than in 2014, with the most accurate algorithms spotting matches with a 0.2 percent error rate thanks to deep learning neural networks. “As such, face recognition has undergone an industrial revolution, with algorithms increasingly tolerant of poor-quality images,” the study concluded.
Efficiency gains have kept pace with performance improvements. The latest, say experts, is the ability to process data at the “edge,” or on board a network camera, which allows for the camera to process and provide metadata to a system’s server or cloud and reduces the processing power required on the server side and lowers bandwidth requirements.
The underlying suggestion today from solutions providers is that the technology has reached maturity, and the questions that remain are on how best to use it and extract a return on investment. More impartial judges of the technology don’t specifically refute that positioning of the technology, but they warn that performance questions aren’t exactly moot.
Face matching software can typically work with images from any camera, but the quality counts. “As with anything related to surveillance or any analytic task, you can’t obtain a low resolution from 100 yards away and expect to accomplish anything,” said Bartol. “It’s all about the pixels, camera placement and angle, and slope—and a lot is dependent on what the FRT software is requiring from its standpoint.”
For its part, recognition software is impressive but not perfect. “I would say the technology is ready in general,” said Lian Jye Su. “However, retailers need to bear in mind that it is within a certain level of accuracy. It’s identical to the facial recognition software deployed in public safety, which is suffering a lot of criticism at the moment.”
Attorney Adil Kolovic agreed: “If a retailer is considering using FRT in a security system, then the retailer should consider the possibility that flaws in the technology can lead to misidentification or even biases against certain groups.” One particular system infamously had a 31 percent error rate with dark-skinned women, he noted.
Complaints have always lurked that the technology performs less well with darker complexions, but they’ve been amplified as the nation undergoes an internal audit of its racial divisions. “Last month, major tech companies instituted moratoriums on the development of facial recognition technology in response to those accuracy and bias concerns, as well as its surveillance capabilities, against the backdrop of both public sentiment and proposed legislation,” said Kari Prochaska, a global privacy and cyber-security lawyer at McDermott Will & Emery.
Problematically, bias is often built into a system. “Most facial recognition technologies nowadays are based on deep learning,” explained Lian Jye Su. “Depending on how the technology suppliers train their AI models, the facial recognition technology may have inherent flaws or biases. Retailers must test their solutions in a rigorous manner before deploying them in the field to avoid any potential hiccups,” he warned.
But fixing those flaws is a possibility, according to Kolovic, as are other technology enhancements. “Despite the age and exploding prevalence of the technology, there is still opportunity to innovate in the space,” he said. “There is still room for advances in both the hardware and software being used to operate facial recognition technology,” and software and/or algorithms that remove racial bias from previous systems is a possibility, he said.
Some technology hurdles, which can help drive ROI, are still a work in progress. For example, face recognition systems can also be used to measure emotion through facial expressions, providing potentially valuable marketing data. But such uses may be problematic, “as a facial gesture often does not correlate with a specific feeling,” noted scientists from the Haaga-Helia University of Applied Sciences in Helsinki in their 2019 study, AI-based Facial Recognition in Emotional Detection. Researchers have also shown that it is possible to fool some mobile-payment kiosks’ facial recognition systems using realistic paper masks.
COVID-19 has thoroughly impacted retail, and it has had implications for facial recognition technology as well. Most obviously, the pandemic has revealed a core system vulnerability: when customers wear masks, facial matching systems are significantly less accurate. If mask wearing becomes normalized, that could present a performance hurdle every flu season. But the pandemic has also unearthed some of the technology’s benefits, including being a touchless biometric. FRT could get a boost as platform for operating kiosks or for access control and opening doors. “Retailers may look to incorporate the ability to provide touch-free access control by identifying employees or known associates from a digital image or video,” said Bartol.
“Now with facial recognition technology and the COVID-19 pandemic, facial recognition technology has new integration [possibilities],” explained Maurizio Pejoves at P&O Global Technologies. Not surprisingly, since thermal face recognition has been around for more than a decade, the current pandemic has propelled some businesses to combine temperature reading and facial recognition (FR). By June, facial recognition system provider PopID said it had sold 600 FR-equipped thermal cameras, including to some Subway and Taco Bell locations, so stores can simultaneously clock-in and scan workers for a fever. Dara Riordan at Face First said a future feature of its integrated systems will provide the ability for systems to alert if a person with a high temperature were to return to a store before a two-week quarantine period.
And mask-wearing is a two-way street. Making an identification is harder, certainly, but software can also “identify if a person is wearing a mask or not,” said Pejoves, who also noted that anonymous analysis of all face streams facilitates people counting, which can alert if the number exceeds what is necessary to maintain in-store social distancing.
Finally, the pandemic has accelerated desire for touchless technologies and highlighted the need for remote authentication to prevent fraud—and facial recognition fits into both. In a fintech world, where alternative payment methods are gaining speed, FRT authentication could provide both important support and ease the path to consumer approval of the technology.
Acceptance, though, is still a work in progress.
Sentiment
The ick factor that some people associate with recognition technologies has clearly waned. People routinely use their faces to unlock and securely sign-in to their phones or computers or use face authentication to access mobile applications. Increasingly, customers are getting used to receiving a face authentication request on their device, “to remotely authenticate an action, for example, or to authenticate a card payment performed in a store,” explained AppGate’s Paul Wilson.
Generally, customers appreciate the security and convenience when they use FR—and that seems to be spilling over into being more accepting of it for other interactions. Not long ago, FRT was clunky, inaccurate, and an even greater privacy concern over image capture, according to Wilson. But things have changed dramatically, he believes, both technically and with customers becoming more accepting of the technology. “The technology is now much easier to use, able to cover more use cases, and implemented to prevent privacy issues,” he said, noting that newer technologies can employ a number of techniques in certain applications to help reduce privacy concerns, such as encryption techniques or working from data that cannot be reconstructed into images of users.
A 2019 survey by the Center for Data Innovation found little support for strict limits on facial recognition technology, especially if it meant airports would be unable to use it to screen passengers. And there was also general support for its use in a retail context. According to those surveyed, only 24 percent of Americans support strict limits on facial recognition if it were to prevent stores from using the technology to stop shoplifting, while 49 percent said they would oppose limitations if that was the result. Another recent study, by Wirecard, also found general support, with 68 percent of respondents willing to use biometrics-like face recognition to make in-store purchases. Globally, there is similar indications of support for facial recognition. Concerned about fraud in online transactions, for example, 93 percent of consumers in Brazil say they’d like more sophisticated security in the form of facial recognition, according to Experian’s February 2020 Global Identity and Fraud Report.
Of course, sponsored technology surveys can be worded to shape favorable responses, and examples of blowback persists. When residents of Atlantic Plaza Towers in Brooklyn learned their landlord planned to modernize the building’s security system and replace key fob entry with face recognition, they filed an objection with the state’s housing regulator. Once a month in London, a group dons camouflage makeup designed to confound camera analytics and conduct a silent public protest of the use of live facial recognition police cameras. At a grocer in Oregon, where facial recognition image captures are required for patrons to enter, a local reporter asked customers about their attitudes toward it. “I don’t know where the pictures of my face go,” said one, but he entered anyway.
Solution providers try various methods to help retailers navigate the privacy debate. For example, by using cameras and software to assign a unique tracking ID to each person that enters a store, a system can collect analytics every time someone visits—how long they shop, suspicious activity, if they visit other of their locations—but never associate the collected data with personally identifiable information. “Faceless recognition” reduces a person’s face to its mathematical components, so no actual images are stored in the system. And it’s possible to take faces out of the recognition equation altogether, for example, by tracking a specific shopper’s journey through Bluetooth beacons that communicate with the shopper’s smartphone via a store app or a third-party app enabled to provide location data. Such systems would recognize the shopper in a return visit but just track their interactions via an assigned number.
Interviews with some shoppers indicate a level of sympathy with retailers who are trying to fight back against persistent thieves, but other interviews have suggested skepticism, distrust, and a mistaken belief that local police were populating a store’s database of security images.
Concern over public attitudes is keeping many retailers away from the technology, according to new research from the ERC Retail Loss Group conducted by Emeritus Professor Adrian Beck. In interviews with LP and retail executives, Beck said he found significant enthusiasm for the principle of facial recognition and for using it in retail environments but that the potential for a negative hit to the business brand is holding them back. “The reality is that many were very concerned about the negative impact of them being known to be using facial recognition in their organizations,” Beck said in a presentation at the Global Retail Crime Summit in June. One LP leader said his company is under a mandate to “not go anywhere near facial recognition because of reputational issues.” Colin Peacock, group strategy coordinator for the ECR Retail Loss Group, thinks more barriers need to come down before adoption becomes widespread. “I think we are going to need to look at the public sector to break down some of these barriers before retailers put their reputations on the line,” he said in the conference presentation, “Exploring the Use of Video Analytics in Retailing: Prospects, Problems, and Practicalities.”
Axis Communication’s Hedgie Bartol sees privacy concerns as the technology’s chief obstacle. “There are a lot of potential applications and interest. The hurdles, primarily, are from a privacy standpoint. The general public wants to know what you are doing with facial recognition,” he said. “More than technology, the political and sociological aspects behind is likely to be the bigger drag. ‘Is it going to be an intrusion?’”
Some good press would help, added longtime industry expert Walter Palmer. Actions by some tech firms, which have included scraping of millions of images from social media for use in FR, has contributed to confusion and distrust over the technology generally, making it harder for retailers to make the case for using it for their limited purposes. There are lingering concerns that a retailer using biometrics might later sell that information to a broker, where it could end up being used by any number of organizations.
“But public opinion can be fickle,” observed Palmer, suggesting a viral story of face recognition foiling a child abduction or neutralizing a terrorist threat could help swing the debate. The Security Industry Association (SIA) is trying to do just that, promoting examples of how FRT is enhancing safety and security in society. In opposition to legislation to ban federal agency use of facial recognition, SIA described how the nonprofit group Thorn has used facial recognition as part of a tool to help rescue children and identify human traffickers. “For example, after seeing an online post about a missing child from the National Center for Missing and Exploited Children, a law enforcement officer used Spotlight to return a list of sex ads featuring the girl,” which resulted in her being rescued. SIA cited numerous other examples of how the technology is benefitting society.
Normalization, through regular, positive interaction with facial recognition also would likely help to erase negative public sentiment as consumers have repeatedly shown themselves willing to exchange personal information for a better customer experience. “We will download a Waze app that says where you’re going, how fast you’re driving, and where you live and work, and we opt into it because of the benefits,” explained Peter Trepp, CEO of FaceFirst and author of Breaking Through the Privacy Barrier.
“At the moment, the concern of retailers over how it will be perceived by the public is certainly dogging the technology at the moment,” according to Beck. But he and other industry leaders we heard from believe it is inching closer to becoming palatable. “I often talk about the frontier of acceptability with technology,” Beck said. “We’ve seen it in the past with CCTV in public places and with RFID in the 2000s.” Each time, he said, the public eventually came around, and that’s where we are with facial recognition. “We’re on the boundary of acceptability at the moment.”
Regulation
Though complicated, the progression of the technology and public attitudes have been straightforward compared to the legal environment. Technology typically outpaces regulation, and that is certainly the case with facial recognition. Only now are lawmakers and courts coming to terms with what the technology can do. “Technology tends to find its own path, and in almost all cases it is ahead of legislation, and so it becomes important to educate on the potential uses of the technology,” Trepp told attendees at the Global Retail Crime Summit.
Behind in the race, legislators now seem to be scrambling.
“The current protest movement and climate over the past few months has raised awareness of facial recognition technology and, in particular, the power of its identification capabilities,” said Attorney Kari Prochaska. Fueled in part over the protests and rising privacy concerns, legislators at every level are examining the technology and, often, crafting and passing legislation to restrict or regulate it, she told LP Magazine.
It’s not likely to be settled anytime soon. Courtroom battles are likely to be commonplace for some time to come. “There are certainly legal headwinds to adoption of facial recognition, and we’re really not going to know for a couple of years how it all turns out as things wind through the courts,” said Palmer.
Nowhere is the legal battle more fully engaged than in Illinois. The state’s strict biometric privacy law has sparked fierce debate and multiple legal challenges. To date, most Illinois cases involving retailers have centered around employees, who have bristled at the collection of biometric data and argue that potential data breaches of unique personal identifiers expose them to irreversible privacy risks, unlike a lost or stolen key fob or ID badge.
Plaintiffs filed suit against the Wendy’s restaurant chain, for example, alleging the company violated state law for not informing employees in writing of the specific purpose and length of time that fingerprint data—collected as part of clocking in and out—would be stored and used. Walmart, too, has been dragged into the fray, over employee handprint scans for taking out and returning cash register drawers, which it argued wasn’t a violation because they weren’t reduced to images that were stored or retained. An Albertson’s pharmacy employee sued over that company’s collection of fingerprints, while the company argued that the case should be tossed because there is no difference between it and the types of companies excluded from liability under the law. After failing to convince a lower court, Albertson’s has been trying to get the Illinois Supreme Court to rule on the constitutionality of the state’s Biometric Information Privacy Act (BIPA).
BIPA initially passed in 2008 but remained relatively benign for more than a decade until a state Supreme Court ruling shifted the notion of harm and loss under the act, according to Brian Weinthal, an attorney with Burke, Warren, MacKay & Serritella. Previously, a plaintiff needed to demonstrate some sort of loss associated with the taking of biometric data, like an identity theft, but that changed in 2019. “The Illinois State Supreme Court said the taking of the information without complying is the damage, and immediately there were dozens of new class action claiming injury by people because their biometric information was collected without the necessary approvals,” explained Weinthal. “Under the statute, you can allege a violation when a biometric has been taken.”
Uniquely, BIPA allows private individuals to bring suit and recover damages of $1,000 or the amount of actual losses. While seemingly modest, the penalty could potentially be applied to each time biometric data is collected in violation of the law. Applied to employees clocking in and out every day or store patrons coming in and out, the accumulated amount could be eye-popping. “Once you have my face in a database in violation, then every time my face is scanned is a separate violation. Then multiply that by every time I come in and out of a branch for years,” said Weinthal. “You can only imagine the number of people who walk into a retail store and the potential exposure.”
The prospects of a big payday have been the driving force in Illinois, said Weinthal. “It seems there are new lawsuits daily, and aggressive plaintiff’s funds are set up to look for cases. Retailers have been sued, security companies have been sued, and anyone else you can think of is being sued.”
In the Wendy’s BIPA lawsuit, for example, the restaurant chain was named, as well as Discovery NCR Corporation, which supplies Wendy’s with its biometric clocks and cash register access systems and plaintiffs allege may hold fingerprint information on employees.
“Illinois is a dangerous pothole,” said Weinthal. “What I see most often is companies buying an expensive security system with facial recognition at the corporate level and then pushing it out and then running across the Illinois law. I get the sense many businesses aren’t aware of it.”
The fine points of “violation” remain squishy because class action suits seeking millions in damages are being settled out of court, and no case has gone the distance to provide greater clarity. However, a retailer that without notification conducts face scans of individuals as they come into a store or make a return is absolutely in violation of Illinois law, said Weinthal. “In terms of who is winning the fight, it has been plaintiff attorneys,” he said. “Defendants have been settling, and the cost of litigating is enormous,” adding that he sees no end in sight for class action filings.
In terms of where the law is going, there isn’t much momentum behind legislative initiatives to reign in Illinois law. There is certainly no rush to do so, said Weinthal. He thinks the key will be whether businesses can be successful in creatively securing the necessary consent required under the law, which includes getting written permission for face scans, notice regarding use and retention of images, and strict restriction on transferring data. Defendants have argued that the law’s various consent requirements should be met by other means, scrolling through electronic permissions on a store app, website notification, or perhaps signage in stores. “So far, though, no court has found that,” said Weinthal.
With no immediate changes on the horizon, Weinthal doesn’t see great utility in “waiting out” the controversy over Illinois’ strict biometric law, suggesting instead that operational executives working in concert with general counsels to ensure they stay within the confines of the law may be the best current option.
For the moment, Illinois is an outlier, especially in its classification of what constitutes harm under the law. The legal environment is subject to change, however, and other states, including Texas, are picking up the issue, with several currently considering laws specifically regulating biometric data usage. Washington state has passed a law to regulate the use of facial recognition software, and Europe presents an altogether different challenge. It all adds up to a buyer-beware legal landscape, experts suggested. “My main concern for retailers looking at adoption [of face recognition technology] is that they go into it with their eyes wide open,” advised Walter Palmer.
Attorney Brian Weinthal has similar advice for retailers. “If you’re looking to get into it, you need to be aware of the law and what’s happening,” he said. “Some companies have been settling for big amounts of money.”
The Illinois Supreme Court ruled recently that Six Flags must pay money damages to a boy for collecting his thumbprint without proper consent, according to Ryan Phelan, an attorney with IP law firm Marshall Gerstein.
Additionally, a state need not have restrictive biometric and privacy laws for facial recognition to have its day in court. Harm resulting from misidentification can land the technology and its owner it in front of a judge or jury.
In April, an eighteen-year-old New York student filed a $1 billion suit against Apple, alleging that its use of facial recognition technology falsely linked him to several theft incidents at the tech company’s retail stores. Ousmane Bah’s suit claims he received a summons from a court in Boston for stealing $1,200 worth of Apple products in 2018 at a time when he was at his senior prom in New York. He was also traumatized by the need to respond to false allegations of store robberies in three states, the suit claims. What happened, the student surmised, is that he lost his identification and that whoever found it must have presented it upon being apprehending for stealing at an Apple retail store. The thief’s face was then enrolled in Apple’s facial recognition security system under Bah’s name, resulting in a series of false criminal allegations against him. Seeking to get the lawsuit tossed, Apple said the suit fails to show any misidentification on its part was intentional, according to reports.
The actions of police—and the tools and money at its disposal—have been under intense scrutiny since the killing of George Floyd, and facial recognition was already being targeted by critics for supposed inherent bias in the technology. So as calls ramp up for reining in police departments, facial recognition technology is being scrutinized by municipalities.
The SIA makes the case that responsible law enforcement use of facial recognition “can actually limit the effects of inherent human bias” when used as a secondary tool in investigations. But it seems some political leaders are tilting toward a perspective that police can’t be trusted with it. In May, the San Francisco Board of Supervisors banned the use of facial recognition software by police and other local government agencies. Boston followed suit. Oakland and Berkeley, California, are on board. Oregon has banned police from using body cameras with facial recognition. Responding to public pressure, Amazon announced this summer that it would temporarily restrict police departments from using its FRT, Rekognition, for one year.
National lawmakers are also taking up the issue. Proposed legislation, the Facial Recognition and Biometric Technology Moratorium Act of 2020, would place a prohibition on the use of facial recognition technology by federal entities. More importantly, it tries to compel others to follow suit by conditioning federal grant funding to state and local entities, including law enforcement, on those entities enacting their own moratoria on the use of facial recognition and biometric technology. In late June, SIA announced its strong opposition to the recently introduced legislation, which is “not a workable solution to address reasonable concerns about the use of facial recognition,” according to SIA CEO Don Erickson. While the legislation targets governmental use of image analytics technology, passage of the legislation would be an important marker of how the privacy versus security battle is tilting.
In one city, at least, opponents of facial recognition are making headway in widening the battleground. City officials in Portland are considering a strict ban on the technology for private businesses, not just government agencies, and would prevent them from collecting, using, or storing people’s facial or biometric information. “My problem is that you should not have your image stored if you are not involved in criminal behavior,” Commissioner Jo Ann Hardesty told the Oregonian. The push is finding an audience because of alleged bias in the technology, as cited by Hector Dominguez, in Portland’s Bureau of Planning and Sustainability. “We felt a moral obligation to develop a broader approach, recognizing that any use of a surveillance technology that is biased against people of color, lacks consent, lacks due process, and can be used on minors is unacceptable.”
Accuracy issues sparked regulation in Washington state, according to Attorney Kari Prochaska. “To address the argument of the risk of bias in the technology, earlier this year Washington state passed significant legislation mandating human review of the technology, accountability reports, and training requirements,” she explained. “The primary concern and argument against using the technology are accuracy issues in identifying people of color.”
Some embroiled in the Portland debate—the city has held several work sessions on the matter—have complained that the speed of the technology’s advancement suggests that any discrepancy in technology performance is likely to be long resolved by the time city officials take action. Others argued that banning technology sends the wrong message for a city trying to attract tech businesses and that its focus on facial recognition ignores the potential for bias in the multitudes of personally identifiable information that can be collected. To supporters of FRT, a ban on private business use would certainly strike a significant blow and set a dangerous precedent, but it is in a holding pattern for now. In June, the Portland City Council punted a vote on a more limited ban specific to city agency use of facial recognition.
Amidst the regulatory churn, there is a burgeoning industry effort to self-regulate. Some question whether it will be broad enough to make much of a difference, but it could help, according to Lian Jye Su at ABI Research. “If the industry can come together and develop a framework—to address the issues of privacy, cyber security, and data storage that can be adopted across the industry—it will at least reduce some of the anxiety.” The broader this effort becomes, and the more stakeholders that get involved, the more likely it will be to help carve a path to using the technology in stores, suggested several LP executives we interviewed.
Deployment
“Retailers need to think about how they will use actually it,” advised Emeritus Professor Adrian Beck. He said several LP executives he’s spoken with have raised questions about how, exactly, you can effectively operationalize facial recognition “to actually utilize it within a store and what you want people to do with that technology.” Especially as a tool to identify potentially troublesome individuals, identifying who they are and that they are in the store isn’t the whole of it.
“I think there are real concerns here about how you are trying to protect your staff and at the same time beginning to try to ID people who might cause issues in your store,” Beck said. “You’re suddenly saying to staff, ‘We have now found someone we know could be a potentially dangerous person, here is the information we know, and now I want you to do something with that.’ So there is this potential conflict between, on the one hand, retailers wanting to protect their staff and make sure they’re not putting them in dangerous situations and a technology that potentially puts them into conflict. So that is a really interesting conundrum, one that I think retailers are going to have to try and face if they are going to use this technology.”
LP Magazine interviews elicited other advice for LP teams looking to capitalize on the substantial promise of face recognition while avoiding pitfalls.
Evaluate your use cases. What are the purposes for the technology? What kinds of events require you to authenticate a user? If you’re just trying to prevent a crime, do you need to know who it is? Do you need face recognition to identify people and requiring data storage, or is face detection, which does not capture images or store data, enough for your purposes? Are you looking for a multichannel solution that needs to work for a customer both in store and online? With clarity on these and other questions, you can identify what solution might help meet your needs or if alternative solutions that don’t involve face capture might meet them.
Make it part of the picture. Included as an element within enhancing video analytics capabilities more broadly, the business case improves for face recognition. Maurizio Pejoves said FR is just a small part of a larger need for LP teams to gain location-based insights and to detect anomalies: loitering or excessive dwell times; objects left behind; a gathering of large groups for an early warning of public unrest and disorder; people or vehicles entering a restricted area; individuals running; and objects moving in the wrong direction. “Also, owners can use the data collected from facial recognition, when enhanced with artificial intelligence, to detect which aisles and discounts are receiving the most foot traffic,” he said, as well as other valuable operational and sales data, including demographic information and tracking how people move in time and space.
Assess your infrastructure. “It is a rarity that you’ll be able to just load on the software and leverage system infrastructure as is,” said Bartol. Retailers need to consider whether to deploy the entire system on premise or leverage cloud computing, added Lian Jye Su. “Leveraging cloud computing will result in better flexibility and scalability in terms of cost,” he said.
Cost it out. “Some love it, and others have looked at it and said, ‘No way, I can’t afford it,’” said one longtime LP executive. Lian Jye Su noted that deploying AI technology requires a lot of spending on AI hardware, which may not be ideal for retailers that are on tight budget. A potential option is an “as-a-service” model, where a retailer is charged a subscription fee associated with continuing usage of the system, he said.
Make an honest business case. “If you’re considering FRT make sure to document the benefits for the company but also identify how it might go wrong. Worst-case scenarios are worth thinking about, so you’re prepared,” said one industry executive. Another added, “You can have clarity on how much it’s going to cost you, but you have to have just as detailed an answer for, ‘What does it get me?’” As with any analytics, Hedgie Bartol advised LP executives to be sure to clearly identify the intended use cases and engage with the appropriate internal stakeholders. “Set the metrics that will define success and determine how the technology can be used beyond security for enhancing operations and sales,” he said.
Know the law. So that a retailer doesn’t run afoul in obtaining or using sensitive data, “It is absolutely critical to understand data privacy laws and regulations of the jurisdiction that the retailer operates in or plans to operate in,” advised Attorney Ryan Phelan. California, Washington, New York, Arkansas, and Texas have also enacted laws, or expanded existing laws, to regulate collection of biometric data, he noted.
Minimize risk. “Retailers need to ensure that data capturing and storage are done locally, with very minimal information shared between different stores and branches. This is to prevent data being stolen during transfer and potential data abuse by malicious actors,” according to Lian Jye Su. He said retailers must minimize the storage of a customer’s original face data and delete customer face data and redundant and residual information in a timely manner after data analysis and processing, ideally not more than seventy-two hours. “Retailers must also set up appropriate standard operating procedures for data security and data management,” he advised. “A third-party privacy and cyber-security audit is a must to build public trust and integrity and prevent any lapse.”
Be sensitive to privacy concerns. “Be abundantly clear about the purpose of facial recognition technology, whether it is for theft monitoring, payment, service personalization, operation optimization, or [other reasons],” said Lian Jye Su, adding that best practice is to set up appropriate customer opt-in/opt-out procedures and a feedback mechanism to allow customers to enquire, change, and delete any personal information, and to be transparent about procedures to the public. Transparency requirements should then be assessed against a loss prevention use case as certain permission requirements “could restrict facial recognition’s effectiveness as an investigation tool,” said Attorney Brian Weinthal. Face First’s Peter Trepp acknowledged the tough spot LP is in: “You don’t want to give away trade secrets, but first and foremost you want to be compliant with the law.”
Establish a robust policy framework. To ensure universal alignment around FRT, it’s critical to clearly define how staff will engage with consumers and to provide in-depth training on the software and company protocols before implementing facial recognition technology, said several industry leaders. Choosing a vendor partner that is committed to training system integrators on how to use facial recognition technology in the proper way it also important, one leader added.
Eyes on the Future
As a controversial tool in a particularly controversial time in society, facial recognition seems to have hit a pause in retail loss prevention. But even for asset protection professionals who don’t think now is the right time, there is ample reason to stay engaged.
Stores must continue to search for the best ways to prevent shrink and violent crime in stores and to balance that against the rights of individuals walking into the stores and in preservation of relationships stores have with their customers. And those are all points that are subject to shift as new crimes emerge, technology changes, and attitudes and laws evolve.
“It’s an ongoing conversation that we will continue to have for years,” said Trepp, “as we ensure we are putting in the right technology to attack the right problems.” He advises retailers to lean into the discussion and to participate in strategic conversations about striking a palatable balance between competing forces.
Overall, the most common advice provided during interviews with LP executives and industry experts is for retailers to continue to examine facial recognition—but to do so with eyes wide open. A business policy and clarity of purpose must accompany any plan to implement this technology. Intentionality is the watchword.
Face recognition may not be the right technology right now for many retailers, but there is no retreating from it as the force behind it is stronger than ever. Recently, Interface Security Systems hosted a webinar on how to maximize ROI on security. In it, Lyle Forcum, executive director of asset protection at Panda Restaurant Group, indicated that his company is not in the market for FRT currently, but he put it this way in recommending an approach to the technology it represents: “If you’re not comfortable with change in the automated environment, you really shouldn’t be in this business. And the future of this business is video analytics and artificial intelligence.”
This story was first published in November 2020.