14182 - Facial Recognition: The Reality -

Facial Recognition: The Reality

• 27 June 2019
• Sally Ward-Foxton

AI is making automated facial recognition for mass surveillance a reality - but at what cost?

The UK is one of the most surveilled countries in the world, with closed-circuit televisions (CCTV) cameras everywhere, from shops to buses to private homes. In the last couple of years, AI techniques, such as neural networks, have propelled large-scale automated facial recognition. Combined with the UK’s huge network of existing CCTV cameras, this could hold vast potential for security applications.



A promotional picture for NEC’s NeoFace facial recognition technology. (Source: NEC)



However, there are worrying implications for privacy. Biometric facial data taken from photos or CCTV footage is particularly troubling because it can be taken without the person’s consent and without them knowing about it. This data may be collected indiscriminately — whether the subject is matched with a watch list or not — and it allows people to be located and for their movements to be tracked. It’s easy to imagine photos of crowds with every face linked to the person’s identity, then potentially connected to all kinds of other data about them.

Law Enforcement

If we are going to trade our privacy for increased security, the benefits must outweigh the costs. Is facial recognition technology actually effective in catching criminals?

Police forces around the UK have performed trials of live facial recognition technology in public places such as shopping malls, sports stadiums, and busy streets. The Metropolitan Police have undertaken 10 trials of the technology so far, using dedicated camera equipment along with NEC’s NeoFace algorithm, which measures the structure of each face, including distance between the eyes, nose, mouth, and jaw. According to NEC, this algorithm can tolerate poor quality image such as compressed surveillance video, with the ability to match images with resolutions as low as 24 pixels between the eyes.

Despite winning four consecutive NIST FIVE (U.S. National Institute of Standards and Technology, Face in Video Evaluation) awards for performance, NeoFace didn’t do terribly well in the field. In the eight deployments between 2016 and 2018, 96% of the Met’s facial recognition matches misidentified innocent members of the public as being on the watch list.

A major criticism of the trials has been that the public were not sufficiently alerted to them and were not given the opportunity to opt out. The BBC filmed one passerby in North London covering his face — the police photographed him anyway and gave him a £90 fine for disorderly conduct.



A South Wales Police van with facial recognition cameras. Text on the side of the van reads “Facial recognition fitted” in English and Welsh, but vans used by the Metropolitan Police were less clearly marked. (Source: Liberty)



South Wales Police’s more extensive trials of the technology were marginally more successful: only 91% of matches were misidentifications of innocent members of the public. Images of all passersby, matched or not, were stored by the police for 31 days.

This trial prompted one member of the public to take legal action against South Wales Police, having been photographed while out shopping and while at a peaceful anti-arms protest. He claims that automated facial recognition technology violated his right to privacy, interfered with his right to protest, and breached data protection laws (judgment is due this fall).

Ethical Concerns

A recent panel discussion at the AI conference CogX in London tackled the ethics of automated facial recognition technology. Martha Spurrier, director of human rights organization Liberty, highlighted research findings that surveillance distorts even non-criminal behavior.

“This technology affects all of us. It affects how we all interact with public spaces. It affects whether we decide to go to a protest, whether we decide to go and pray somewhere, or whether we hang out with people who might have been in trouble with the police,” she said.

Well-known issues with bias in facial recognition whereby women and people of color are more likely to be misidentified by algorithms are also a cause for concern, Spurrier said.

“[We know that] law enforcement marginalizes and demonizes particular communities. And this technology, whether it’s biased in and of itself, it will be used by people where bias is already entrenched. We have unfortunately no answer to dealing with police racism. So, we shouldn't equip [the police] with even more invasive tools to be able to enact that racism on an even greater scale,” she said.

Griff Ferris, legal and policy officer for civil liberties and privacy organization Big Brother Watch, said the public should be concerned about trials of live facial recognition like the ones carried out by UK police.

“It is mass surveillance. It biometrically scans every single face within a public space, checking against the watch list… It treats your face like a fingerprint, or DNA swab,” Ferris said, asking the audience how many would be willing to give fingerprints or have DNA swabbed when entering the auditorium (no hands went up).

“That's how it treats a public space, or a concert, or a protest. That's what it's doing and it's doing it without your knowledge and without your consent,” he added.

Proprietary Tech

Shaun Moore, CEO of facial recognition company Trueface, was the sole representative of a facial recognition company on the CogX panel. Trueface started with a smart doorbell in 2013, but now provides facial recognition to banks for secure ATM transactions, to the hospitality industry for VIP recognition, and to law enforcement.

“The [U.S.] government does not need facial recognition to surveil you. They have social media. They have your phone records. They have your credit card statements. So, you really need to think about this more holistically than just surveillance,” said Moore.

While he conceded that testing the technology on people who didn’t consent was “inappropriate,” he said there was no way around it if we want the technology to work.

“That is the only way to collect real information, and without collecting real information you will not know where the issues lie, so it's a bit of a [quandary],” he said. “But without real deployment of this technology with people who don't know that it's watching them, you will not get the information you need to see if it's accurate enough to identify people that you are looking for.”



Trueface’s facial recognition and spoof detection technology was showcased at ISC West in Las Vegas earlier this year. (Source: Trueface)



Moore said Trueface prides itself on enforcing responsible use of its technology, primarily by screening customers carefully and supplying to opt-in applications.

“We sell directly to our customers into their local infrastructure, so it's not a cloud API, you can't just go onto our site and buy our technology,” he said. “We spend at least six months with our customers to know exactly how they're using it, and we work in… primarily opt-in applications.”

Trueface’s technology protects those who haven’t opted in by blurring all faces in real time, Moore said. Imagery is not collected from faces other than for the moment that the system performs its matching, with non-matching faces completely erased from the system. He also pointed out that facial recognition does not mean storing images of faces — the Trueface system makes a mathematical representation of that face in a proprietary way, so if someone were to hack in to the database, it wouldn’t be possible to reverse engineer those numbers to plot a particular face.

Meanwhile, other private sector companies are divided on whether the technology in its current form should be used for surveillance applications. Amazon has provided its Rekognition facial recognition tool to law enforcement across the United States, despite an open letter from leading AI researchers calling for them to stop on the grounds that it is discriminatory. Microsoft has said that it doesn’t want its facial recognition technology to be used by law enforcement, with Microsoft President Brad Smith calling for regulation of the technology.

What Does the Law Say?

“What’s really clear is that [UK] policy in this area is a hot mess,” said Hetan Shah, executive director of the Royal Statistical Society. “It is really complicated, and that’s why we don’t have easy answers in this space.”

UK law stipulates the conditions for police collection of fingerprints and DNA, and what happens to that data after it is taken. It does not cover any other form of biometrics such as facial recognition images and data. While this type of data may be loosely covered under the Data Protection Act or the European GDPR regulation, biometric data other than DNA and fingerprints is not named specifically in either law. A High Court ruling in 2012 found that it was unlawful for the UK’s Home Office to store facial images of innocent people, but the Home Office only deployed a workaround, offering to delete the images of “unconvicted persons” who applied to have them removed.



Hetan Shah addresses a panel on the ethics of facial recognition technology at CogX 2019. (Source: EE Times)



The Ada Lovelace Institute, of which Shah is deputy chair, was created to investigate ethical issues related to AI technology. The Institute has commissioned a study of public perception of facial recognition technology used in public places.

“One thing we’re talking to stakeholders about is calling for a moratorium on this technology, which we think is in the private sector’s interest as much as citizens’ interest, because if we get the regulation wrong we’ll quickly be in the wrong place,” Shah said, citing a precedent set by the moratorium on genetic profiling in the insurance industry.

“In the same way that we've seen the science of genetically modified foods get set back 10 years because the public in Europe wouldn't support it, we need to find a path which gets social license to operate,” he continued. “The best way to do that is to pause on technology, and to have this discussion in many forums around the country and to think through them overall.”

Worst-Case Scenario

The situation overseas varies widely. San Francisco is one of the first cities to explicitly restrict use of the technology, following a grassroots campaign supported by the American Civil Liberties Union of Northern California. On the other extreme, frequently cited as a worst-case scenario is China. While China is keen to show off sunglasses equipped with facial recognition cameras on its police officers, there has been an international outcry about the use of facial recognition to track its Uighur Muslim community. China reportedly uses facial recognition technology on an enormous country-wide network of CCTV cameras to classify the ethnicity of its citizens, with alarms sounding if too many Uighurs appear in one area, for example. This level of social control, while alarming to Western ideals, is perfectly possible with today’s technologies.

While this is far from the situation on British streets, without any form of regulation, it’s easy to imagine the technology invading people’s right to privacy, or at worst, being deliberately misused.

The low accuracy and high rates of misidentification demonstrated in UK trials are particularly concerning, especially for women and people of color, who are disproportionately misidentified by today’s facial recognition algorithms. The argument “if you have nothing to hide, you have nothing to fear,” may no longer hold true.

13750 - UIDAI says Aadhaar Act prohibits police use of its biometric database - Biometric Update

UIDAI says Aadhaar Act prohibits police use of its biometric database

Jun 26, 2018 | Stephen Mayhew

CATEGORIES Biometrics News  |  Civil / National ID  |  Law Enforcement

The Unique Identity Authority of India (UIDAI) has responded to a pitch from the National Crime Records Bureau (NCRB) for limited Aadhaar access as a potential tool to aid police investigations, saying that the Aadhaar Act does not allow law enforcement use of the biometric database, and that it has never shared data with any law enforcement agency, The Times of India reports.

NCRB Director Ish Kumar said that gaining limited access to the Aadhaar database could help police catch first-time offenders and identify bodies.

Smarter  Authentication Password-Free, High Assurance Authentication for Active Directory domains. Get Your Demo. doubleoctopus.com

“There is need for access to Aadhaar data to police for the purpose of investigation,” Kumar said. “This is essential because 80 to 85 per cent of the criminals every year are first time offenders with no records (of them available) with the police. But, they also leave their fingerprints while committing crime, there is need for limited access to Aadhaar, so that we can catch them.”

The UIDAI answered that Section 29 of the Aadhaar Act of 2016 prohibits access to Aadhaar for criminal investigation purposes. Section 33 of the Act allows very limited exceptions in cases of national security, but only after authorization by an oversight committee.

“It may be underlined here that when Mumbai high court gave orders to share biometric data with an investigating agency in a particular case, the matter was taken up to the Supreme Court which stayed that order,” the UIDAI said in a statement.

The UIDAI also pointed out that this position is part of the defence of Aadhaar before India’s Supreme Court, on which a ruling is expected soon, and that the Act only authorizes biometric data to be used to generate Aadhaar numbers and authenticate people 1-to-1.

Facial recognition is expected to be added to the Aadhaar program on August 1, which could also increase resistance to providing police access to the database, based on reactions to similar uses of biometrics in other countries.

13463 - Welsh police wrongly identify thousands as potential criminals - The Guardian

Welsh police wrongly identify thousands as potential criminals

South Wales force defends use of facial recognition technology at 2017 Champions League final

Press Association

Sat 5 May 2018 19.01 AESTLast modified on Sat 5 May 201819.35 AEST

Shares

602

 Football fans pass through security gates before the match at the National Stadium in Cardiff. Photograph: Martin Rickett/PA

A police force has defended its use of facial recognition technology after it was revealed that more than 2,000 people in Cardiff during the 2017 Champions League final were wrongly identified as potential criminals.

Revealed: how facial recognition has invaded shops – and your privacy

 

Read more

South Wales police began trialling the technology in June last year in an attempt to catch more criminals. The cameras scan faces in a crowd and compare them against a database of custody images.

As 170,000 people arrived in the Welsh capital for the football match between Real Madrid and Juventus, 2,470 potential matches were identified.

However, according to data on the force’s website, 92% (2,297) of those were found to be “false positives”.

South Wales police admitted that “no facial recognition system is 100% accurate”, but said the technology had led to more than 450 arrests since its introduction. It also said no one had been arrested after an incorrect match.

“Successful convictions so far include six years in prison for robbery and four-and-a-half years imprisonment for burglary. The technology has also helped identify vulnerable people in times of crisis.A spokesman for the force said: “Over 2,000 positive matches have been made using our ‘identify’ facial recognition technology, with over 450 arrests.

“Technical issues are common to all face recognition systems, which means false positives will be an issue as the technology develops. Since initial deployments during the European Champions League final in June 2017, the accuracy of the system used by South Wales police has continued to improve.”

Guardian Today: the headlines, the analysis, the debate - sent direct to you

 

Read more

The force blamed the high number of false positives at the football final on “poor quality images” supplied by agencies, including Uefa and Interpol, as well as the fact it was its first major deployment of the technology.

Figures also revealed that 46 people were wrongly identified at an Anthony Joshua fight, while there were 42 false positives from a rugby match between Wales and Australia in November.

All six matches at a Liam Gallagher concert in Cardiff in December were valid.

The chief constable, Matt Jukes, said the technology was used where there were likely to be large gatherings, because they were “potential terrorist targets”.

“We need to use technology when we’ve got tens of thousands of people in those crowds to protect everybody, and we are getting some great results from that,” he told the BBC. “But we don’t take the use of it lightly and we are being really serious about making sure it is accurate.”

The force said it had considered privacy issues “from the outset”, and had built in checks to ensure its approach was justified and proportionate.

However, the civil liberties campaign group Big Brother Watch criticised the technology.

In a post on Twitter, the group said: “Not only is real-time facial recognition a threat to civil liberties, it is a dangerously inaccurate policing tool.”

13293 - Facial identification makes Aadhaar biometrics for elderly nearly foolproof - TNN

Facial identification makes Aadhaar biometrics for elderly nearly foolproof

Mahendra Kumar Singh and Rajeev Deshpande | TNN | Apr 15, 2018, 04:49 IST

HIGHLIGHTS

• Acceptance of Aadhaar identification rose to 99% for senior citizens once facial recognition was mated with finger-prints
• Finger-prints of senior citizens are some times worn out due to age and can't be captured by machines
• The findings are encouraging for UID as it will get rid of hurdles in access to pensions and welfare benefits

NEW DELHI: The introduction of facial identification along with use of finger-prints promises to substantially address complaints of failed bio-metrics in Aadhaar authentication with a study of elder persons showing that when used in combination, the processes deliver a success rate of 99%. 

A proof of conduct study carried out for Unique Identification Authority of India (UIDAI), the results of which were presented to the Supreme Court, showed that acceptance of Aadhaar identification rose from 83% for senior citizens to close to 100% once facial recognition was mated with finger-prints for authentication. 

The findings are encouraging for UID as the problems faced by elder citizens, whose finger-prints are some times worn out due to age and cannot be captured by authentication machines, are hurdles in access to pensions and welfare benefits. The problems of smudged bio-metrics also crop up in the case of persons involved in hard manual labour with fingers getting cracked and calloused.

The study, conducted in old-age homes in nine states, and covering about 4,500 respondents, tested authentication processes with the use of fingerprints, iris, finger-print with iris and facial recognition with finger-print. The finger print with iris method also scored a success rate of 95% acceptance.

The iris method of authentication, in the case of senior citizens, suffers from drawbacks similar to the use of finger prints. The capture of bio-metric confirmation is problematic with changes in the iris due to age and insertion of medical devices like lenses or conditions like cataract or glaucoma. Yet, the iris method scored higher than the finger-print identification alone.

TOP COMMENT

When will UIDAI will stop bandaiding this dubious technology. We dont know whether fingerprints are unique but the way they are stored and matched in biometrics, Apple found there is one in 50000 chance of somebody elses biometrics matching yours so they moved to facial recognition where chance is 1 in million as per their white paper. But there is big catch. But as per University of Michigan study unlike fingerprint which does not change over life (excepting damage to ridges over decades in old age), failure rate of facial recognition increases dramatically after 6 years based on aging. But that was not problem for Apple since average period user holds IPhone is 3-4 years. But Aaddhar is touted from birth to cremation. So how are they going to change biometrics every five years and malpractices that can happen. More importantly capturing facial recognition biometrics requires higher skills where angle of incidence and ambient temperatures plays greater role in capturing as well as matching. Not only UK but California and New York States were not technically less competent than India in abandoning this unreliable technology which makes sense of limited use like mobile phones, passports, smart cards but never for national identity storing data of billion from birth to cremation to be collected by large number of ill-prepared agencies.

Girish Kamat

UID has worked to increase options and sophistication of technology to reduce authentication failures among groups like seniors and labourers as this takes away from important objectives associated with Aadhaar such as facilitating bank access and direct transfers to vulnerable groups. The use of Aadhaar is intended to make transfer and withdrawal of pensions and life certificate authentications easier.

The authentication process remains vulnerable to lack of internet connectivity in rural India and UID has issued advisories that lack of Aadhaar cannot result in denial of services. It hopes that with the increasing use of Aadhaar the system will improve.

12706 - Virtual Id, Limited KYC, face recognition: New security features to help strengthen Aadhaar - Business Today

Virtual Id, Limited KYC, face recognition: New security features to help strengthen Aadhaar

 BusinessToday.in   New Delhi     Last Updated: January 16, 2018  | 18:02 IST

In the past week UIDAI has introduced three new features to improve the  security of Aadhaar database amid concerns over the safety of the world's biggest online database. Both the government and the UIDAI, which administers Aadhaar database, are confident about the safety of data. Some government officials have dismissed the criticisms over Aadhaar saying it's a foolproof system.

Trai (Telecom Regulatory Authority of India) chairman RS Sharma, in a recent interview with ET, said there's never been a breach of data, saying it's just "breach of trust". He went to justify his claim giving an example about how out of total 15 billion biometrics, not even a single biometric information had been stolen since the Aadhaar service was launched seven years back.     

UIDAI architect Nandan Nilekani also hailed the Aadhaar security structure recently saying the newly introduced security features like Virtual Ids and Limited KYC were significant to protect the Aadhaar system. About the recent incident of data breach, Nilekani said the incident was not a data leak but a "privileged leak", which the UIDAI was able to stop.

After a media report last month alleged a major data leak of Aadhaar Card holders' information on paying just Rs 500, UIDAI Chief Executive Ajay Bhushan Pandey strongly defended the system, rubbishing the claim before launching an inquiry into the incident.

A few days later, Virtual Id (VID) system and Limited KYC services were introduced as part of the added security measures. The VID is a 16-digit temporary random number that an Aadhaar holder can use for authentication or KYC services with fingerprint instead of the original Aadhaar number. Under limited KYC, authentication user agencies will neither get access to full KYC nor will they be able to store the Aadhaar number on their systems. Instead, they will get a tokenised number issued by UIDAI to identify their customers.

The UIDAI also came up with face recognition feature for Aadhaar authentication on Monday. This is the second layer of verification to ensure that those having difficulty with their fingerprints/iris authentication are no longer excluded. Pandey tweeted Tuesday said, "UIDAI introduces yet another landmark technology for authentication - Face Authentication. #AadhaarFaceAuth will help all elderly or others facing issues with fingerprint authentication. Service to be launched by 1 July 2018."  

The move will improve inclusion and benefit the people whose fingerprints no longer match to their original biometrics due to age. Since residents are already photographed at the time of Aadhaar enrolment, this new measure does not require any additional effort from the existing Aadhaar holders. However, the circular specifies that this new measure will only be allowed in "fusion mode along with one more authentication factor", be it fingerprints, iris scan or OTP (one-time-password).

12015 - Firms involved in biometric database in India contracted by Irish government - Irish Times

Firms involved in biometric database in India contracted by Irish government

Morpho and Daon provide services for Irish public services card and passport systems

Sat, Sep 9, 2017, 06:33

Elaine Edwards

Dermot Desmond’s firm Daon has provided services to the Irish Government for the public services card and passports.

Two tech firms – one owned by businessman Dermot Desmond – involved in the creation of a controversial biometric database in India, are providing services for the Government’s public services card and passports.

Known as the Aadhaar project, the Indian scheme is the world’s largest ever biometric database involving 1.2 billion citizens. Initially voluntary, it became mandatory for obtaining state services, for paying taxes and for opening a bank account.

However, India’s supreme court ruled last month that privacy was a fundamental right, following a challenge to the mandatory nature of the huge biometric identification system. The ruling is likely to test the validity of the Aadhaar project.

Desmond’s firm Daon produced software selected in 2010 for use in the Indian scheme. It has since provided services to the Irish Government for the public services card and passports.

Another firm Morpho described as a major contributor to the Aadhar project, formed Biometric Card Services in 2009 along with DLRS Group and Conduit to produce some three million Irish public services cards.

Daon, which describes itself as a “biometric enabling technology company” was also awarded a €1.9 million contract by the Department of Foreign Affairs last year to provide a “facial recognition solution” for the passport service.

Watchlists

Tender documents show the Department of Foreign Affairs required software that would, among other things, match photographs with other photos in its database, allowing it to create “one or more watchlists”.

The Department of Social Protection also uses facial identification management software provided by 3M Ireland.

Privacy and technical experts have called for clarity about the biometric software and hardware being used by Irish officials after Minister for Social Protection Regina Doherty said last weekend the Government was “not holding biometric data”.

Dermot Casey, a former chief technology officer of Storyful, said that if the Daon system was used to store the data and carry out the facial matching then the Government “appears to have purchased a biometric database system which can be extended to include voice, fingerprint and iris identification at a moment’s notice”.

Katherine O’Keefe, a data protection consultant with Castlebridge, said if the departments were using images of people’s faces to single out or identify an individual, they were “by legal definition processing biometric data”.

There was “a much larger question of what other data is being shared”, she said.

“There may be many potential benefits to having a “single view” or “universal identifier” for interacting with the State, but it would have to be extremely carefully constructed with risks assessed and mitigated to ensure it does not violate fundamental rights.”

5528 - Facial recognition: is the technology taking away your identity? - The Guardian

Facial recognition: is the technology taking away your identity?

Facial recognition technology is being used by companies such as Tesco, Google and Facebook, and it has huge potential for security. Concerned? It may be too late to opt out…

Luke Dormehl 

The Observer, Sunday 4 May 2014

• 
  

Facial recognition data points: 'While facial recognition algorithms may be neutral themselves, the databases they are tied to are anything but.'

This summer, Facebook will present a paper at a computer vision conference revealing how it has created a tool almost as accurate as the human brain when it comes to saying whether two photographs show the same person – regardless of changes in lighting and camera angles. A human being will get the answer correct 97.53% of the time; Facebook's new technology scores an impressive 97.25%. "We closely approach human performance," says Yaniv Taigman, a member of its AI team.

Since the ability to recognise faces has long been a benchmark for artificial intelligence, developments such as Facebook's "DeepFace" technology (yes, that's what it called it) raise big questions about the power of today's facial recognition tools and what these mean for the future.

Facebook is not the only tech company interested in facial recognition. A patent published by Apple in March shows how the Cupertino company has investigated the possibility of using facial recognition as a security measure for unlocking its devices – identifying yourself to your iPhone could one day be as easy as snapping a quick selfie.

Google has also invested heavily in the field. Much of Google's interest in facial recognition revolves around the possibilities offered by image search, with the search leviathan hoping to find more intelligent ways to sort through the billions of photos that exist online. Since Google, like Facebook wants to understand its users, it makes perfect sense that the idea of piecing together your life history through public images would be of interest, although users who uploaded images without realising they could be mined in this manner might be less impressed when they end up with social media profiles they never asked for.

Google's deepest dive into facial recognition is its Google Glass headsets. Thanks to the camera built into each device, the headsets would seem to be tailormade for recognising the people around you. That's exactly what third-party developers thought as well, since almost as soon as the technology was announced, apps such as NameTag began springing up. NameTag's idea was simple: that whenever you start a new conversation with a stranger, your Google Glass headset takes a photo of them and then uses this to check the person's online profile. Whether they share your interest in Werner Herzog films, or happen to be a convicted sex offender, nothing will escape your gaze. "With NameTag, your photo shares you," the app's site reads. "Don't be a stranger."

While tools such as NameTag appeared to be the kind of "killer app" that might make Google Glass, in the end Google agreed not to distribute facial recognition apps on the platform, although some have suggested that is no more than a "symbolic" ban that will erode over time. That is to say, Google may prevent users from installing facial recognition apps per se on Glass but it could well be possible to upload images to sites, such as Facebook, that feature facial recognition. Moreover, there is nothing to prevent a rival headset allowing facial recognition apps – and would Google be able to stop itself from following suit?

Not everyone is happy about this. US senator Al Franken has spoken out against apps that use facial recognition to identify strangers, going so far as to publish an open letter to NameTag's creators. "Unlike other biometric identifiers such as iris scans and fingerprints, facial recognition is designed to operate at a distance, without the knowledge or consent of the person being identified," he wrote. "Individuals cannot reasonably prevent themselves from being identified by cameras that could be anywhere – on a lamp post, attached to an unmanned aerial vehicle or, now, integrated into the eyewear of a stranger."

To proponents of facial recognition, of course, this is precisely the point. Like the club doorman who knows you by name and can spot you in a busy crowd, facial recognition can make everything that bit more personal. In Steven Spielberg's 2002 sci-fi film Minority Report, ads are made more personal by using facial recognition technology. As Tom Cruise's character walks down the street, he is bombarded with customised adverts for everything from new cars to alcoholic drinks. In 2014, a number of companies are already bringing these ideas to (digital) life. Late last year, Tesco announced plans to instal video screens at its checkouts around the country. These screens will use inbuilt cameras equipped with facial recognition algorithms to ascertain the age and gender of individual shoppers.

Personal targeted advertsing in Spielberg's Minority Report starring Tom Cruise.

A Californian startup called Emotient meanwhile focuses on the area of facial expression analysis. Incorporated into next-generation TVs by way of a webcam, this technology could potentially be used to monitor viewer engagement levels with whatever entertainment is placed in front of them. The answer to questions such as "how many times did your face register interest during a programme?" can then be fed back to television companies to help them make creative decisions concerning programming.

"It is time for a step-change in advertising," says Lord Sugar's son, Simon, chief executive of Amscreen, which developed the OptimEyes technology behind Tesco's facial recognition screens. "Brands deserve to know not just an estimation of how many eyeballs are viewing their adverts, but who they are, too. Through our Face Detection technology, we want to optimise our advertisers' campaigns, reduce wastage and in turn deliver the type of insight that only online has previously been able to achieve."

Putting aside the question of whether or not brands do "deserve" to know anything and everything about their customers, companies such as OptimEyes and Emotient are far from the creepiest application of facial recognition. In the US, the startup SceneTap (previously known as BarTabbers) has installed cameras in more than 400 bars; they use facial recognition to help bar-hoppers decide which locations to visit on a night out. SceneTap offers real-time information on everything from gender ratios to the average age of patrons. A patent filed by the company even suggests plans to link identified people with their social networking profiles to determine "relationship status, intelligence, education and income".

Although the use of facial recognition tools is still relatively new in the consumer sector, that is where much of the visible innovation will take place over the coming years. "The stakes are lower, so companies are free to take more risks," says Kelly Gates, professor in communication and science studies at UC San Diego and author of Our Biometric Future: Facial Recognition Technology and the Culture of Surveillance. "As a result, there are a lot of experiments in the commercial domain. So what if you identify the wrong person by accident when you're targeting an ad? It's not that big a deal. It happens all the time in other forms of advertising."

Mohammed Atta (right) in the airport surveillance tape from Portland, Maine, 11 September 2001. Photograph: Reuters

There are, naturally, problems, and most relate to privacy concerns. Although privacy is an issue with every form of data mining, at least online the majority of information absorbed by companies is anonymised. Facial recognition, of course, is precisely the opposite. And since facial recognition takes place in public spaces, it is not even necessary for the person surveilled actively to "opt in" to the service.

This, in turns, links to the subject of security, which for many companies and organisations is the ultimate application for facial recognition. Hitherto, most facial recognition research has been funded by governments interested in its potential for streamlining surveillance. That emphasis has only increased over the past decade, provoked by events such as the 9/11 attacks and the 7/7 London bombing in 2005.

One of the most poignant images that came out of 11 September was a grainy frame of surveillance tape footage showing hijacker Mohamed Atta as he passed through an airport metal detector in Portland, Maine. Unlike the horrifying images of the collapse of the Twin Towers, this quieter picture was dramatic because of what it implied: that if only the right technology had been available, that day's tragic could have been averted.

The idea that data mining algorithms have any place in helping us stop the next 9/11 or 7/7 has been criticised in some quarters. But there is no doubt that facial recognition plays an ever more important part in control and surveillance – both in England and overseas. On 5 April 2011, 41-year-old John Gass received a letter from the Massachusetts Registry of Motor Vehicles informing him he should stop driving, effective immediately. A conscientious driver who had not received so much as a traffic violation in years, Gass was baffled. After several frantic phone calls, followed up by a hearing with registry officials, Gass learned his image had been flagged by a facial recognition algorithm, designed to scan through a database of millions of drivers' licences looking for potential criminal false identities. The algorithm had determined that he looked sufficiently like another Massachusetts driver that foul play was likely involved, so he received the automated letter. The RMV was unsympathetic, claiming it was the accused individual's "burden" to clear their name in the event of any mistakes, arguing that the pros of protecting the public outweighed the inconvenience to the wrongly targeted few.

"The dream is for governments to be able to set up networked cameras in public locations, capable of constantly searching through the faces of people who are photographed," says Xiaoou Tang, professor in the department of information engineering at the Chinese University of Hong Kong and one of the world's leading experts in facial recognition. "Once this is done, the images can then be matched to a database looking for suspects or potential terrorists, so that [pre-emptive] arrests can be made."

Perhaps the most notable thing about our faith in facial recognition is what it says regarding belief in the inherent neutrality (or even objectivity) of such systems. "One of the things that troubles me is the idea that machines don't have bias," says Gates. Of course, in a real sense, they might not. Unless a programmer is personally prejudiced and decides deliberately to code that bias into whatever system he or she is working on, it is unlikely that a facial recognition algorithm will exhibit prejudice against certain groups for the reasons that a human might.

But that doesn't mean that prejudice can't occur. It could be, for example, that facial recognition tools show a higher rate of recognition for men than for women and for individuals of non-white origin than for whites. (This has been shown to be true in the past.) A facial recognition system might not target a black male for reasons of overt prejudice in the way that a racist person might, but the fact that it could be more likely to do this than it is to target a white female means that the biased end result is no different.

And while facial recognition algorithms may be neutral themselves, the databases they are tied to are anything but. Whether a database concerns criminal suspects or first-class travellers, they are still designed to sort us into categorisable groups.

"These databases are what define our social mobility and our ability to move through the world," says Gates. "Individual identification is always tied to social classification. It's always there for some specific purpose, and that's usually to determine someone's level of access or privilege. The ethical questions in facial recognition relate to those social hierarchies and how they're established."

"I think it worries people because there's something very permanent about it," says Xiaoou Tang. "Even when you're talking about using your face or your fingerprints to unlock a phone, this is a password we can never change. We only have one, and once it's set up it's going to be your password for life."

This isn't to suggest that facial recognition doesn't have its positives. As computer vision continues to get better over the coming months and years, we'll reap the benefits as computer users. The idea that we can take the giant, anonymous world we live in and transform it into a place as knowable as a small town is, at root, a utopian/naive one. "Ultimately we need to ask ourselves whether a world of ubiquitous automated identification is really one that we want to build," says Gates.

It's important to understand the scale of change that is under way, because it is going to dictate what happens. Knowing about facial recognition, and how it is used by both governments and companies, is key to helping us face the future. No pun intended.

Luke Dormehl is the author of The Formula: How Algorithms Solve All Our Problems (And Create More), published by WH Allen £20