Big Brother is Watching You.png


China is making advances in biometrics research applications such as intelligent surveillance and infrared facial recognition—and raising concerns among privacy advocates. Originally published in Science in 2012.

Security cameras are ubiquitous in Chinese cities: peering along corridors, surveying busy streets, scanning public spaces. Over the past decade, the country has steadily ramped up surveillance efforts, setting a target of installing 30 million security cameras by 2015, or roughly one for every 45 people.

The sheer number of cameras dotting urban China, coupled with the limited power of civic groups to safeguard the rights of those being watched, gives privacy advocates pause. But the proliferation also poses problems for the watchers: Footage is typically reviewed manually, by police officers sitting in front of a screen searching for a specific crime, says Stan Z. Li, director of the Center for Biometrics and Security Research at the Chinese Academy of Sciences' Institute of Automation (CASIA) in Beijing. Hunting down a suspect's mug can take weeks or even months, he says: “It's very tedious. There is a huge, urgent demand for intelligent video analysis.”

That's where Li and his colleagues come in. Li, who advises the central government on building an intelligent video surveillance infrastructure, has developed an application called VSearch that “makes the impossible possible,” he says. Through pattern recognition and data mining, it allows users to scan footage for people that fit certain descriptions. With closed-circuit television (CCTV) footage of a suspect wearing a red T-shirt and jeans, for example, officers can search other footage for individuals who fit the bill. VSearch works similarly for cars, searching by color and shape. “We can help the user find what he needs from a huge amount of data,” Li says. “We've reduced the workload significantly.”

Biometrics—a field that grew out of fingerprinting to encompass emerging areas like gait and heartbeat recognition—is in its heyday in China. A large research force, generous government support, and advances in the algorithm speeds that enable biometric applications have pushed innovation in areas like intelligent surveillance and infrared face recognition. The leading Chinese image-processing and machine-learning labs now “compete with the top institutions in the U.S.,” says Anil K. Jain, a computer scientist at Michigan State University in East Lansing who sits on the advisory board of CASIA's Center for Biometrics and Security Research and makes yearly visits to Beijing to meet with Chinese counterparts. The number of publications from China is “quite impressive,” says Sébastien Marcel, an expert on pattern recognition and machine learning at Idiap Research Institute in Martigny, Switzerland.

Chinese biometric applications are being put to use for security in areas such as banking and sporting events—as they are in countries like the United Kingdom, where surveillance cameras are also ubiquitous. But critics worry that concerns about an Orwellian state—often overblown in the past—are finally well-founded. They fear that the new technologies will help China's one-party state clamp down on dissent and monitor political opponents. Technologies that allow authorities to quickly sift through vast amounts of data “really raise the specter of a Big Brother society,” says Nicholas Bequelin, a researcher at Human Rights Watch in Hong Kong. In China, he adds, “there will be a sea change in the ability of the security agencies to monitor nationwide specific individuals—for law-enforcement purposes, but also for political motives.”


China's involvement with biometrics dates to the 1980s, when the Ministry of Public Security funded research at Tsinghua University and Peking University on fingerprint authentication, says Zhou Jie, a computer scientist at Tsinghua University in Beijing. Globally, work on biometric recognition was coming into its own. In 1987, Lawrence Sirovich and Michael Kirby, mathematicians at Brown University, discovered a way to analyze faces, essentially by turning images into vectors made up of pixels. Those processed images, called eigenfaces, paved the way for accurate face-recognition technology.

Sophisticated security equipment soon proved its worth to Chinese authorities. Footage from surveillance cameras helped security forces identify students and workers who had been in and around Tiananmen Square during the 1989 protests. The cameras had been manufactured by a U.K. company and their installation paid for by the World Bank, according to a report by the now-defunct Canadian government agency Rights & Democracy. In the wake of the crackdown, the U.S. Congress enacted export restrictions banning the transfer of crime-fighting equipment—including biometric technologies like fingerprint scanners—to China.

Those export controls sparked China's biometrics boom. Chinese researchers “had to start from scratch,” Jain says, by building homegrown versions of Western technologies. China began rolling out its technologies on a large scale in 2004, when checkpoints on the Hong Kong–mainland border installed face- and fingerprint-recognition systems for Hong Kong residents who frequently cross the border.

A series of ambitious projects followed. At the 2008 Beijing Olympics, face-recognition systems provided by the Beijing company AuthenMetric helped verify attendees at the opening and closing ceremonies, by comparing their faces to photos on their ticket application forms. The stadium hosting softball games used hand-vein recognition to control access to the stands. Today, research into China's third-generation identification cards, which will use fingerprinting, are giving the industry another boost, Zhou says.

Li sums up the advantages of working in China in one word: “People.” “It's not unusual for a professor to have 20 graduate students working on face recognition,” Jain says. That workforce came in handy in a venture overseen by Song-Chun Zhu, a computer scientist at the University of California, Los Angeles. Zhu developed a technology that converts images into text—another way to achieve Li's goal of making video searchable. Much of the grunt work was done at the Lotus Hill Institute for Computer Vision and Information Science, which Zhu established in his hometown of Ezhou, Hubei Province, in 2005. He hired graduates of local art colleges to annotate images, affixing them with labels from a predetermined group of categories. The employees focused on people and cars, and just those two tasks kept them busy for years. But by 2010, the lab had built a database of 2 million images containing objects in 500 categories.

Although Zhou says China still lags behind other countries in the development and application of biometric technologies, it is now tackling problems that vex researchers around the world. Take spoofing. Standard face-recognition systems can be fooled by a person holding up a photo of another individual's face. In response to that and other challenges, Li developed infrared face recognition. The application, which was used to confirm the identity of workers at Expo 2010 Shanghai China, “has a natural immunity to photos, because it uses different imaging-process principles,” Li says. Two spectra—one visual and one infrared—make it robust, adds Marcel, who coordinates Trusted Biometrics under Spoofing Attacks, an E.U. project to combat fraud in biometric technologies. CASIA is a collaborator on the project.

Chinese researchers are also tackling a more intractable problem: “noncooperative” face recognition. While technologies like Li's allow police to search for clothing and other identifiers, pinpointing specific faces in surveillance footage is still impossible. Existing recognition technologies work best in controlled, evenly lit environments like security checkpoints, where subjects are facing one direction and positioned a certain distance from the camera. That could change if researchers can develop technologies for recognizing faces from different angles, in variable lighting, and over time, as a person's appearance changes. “The most significant challenge is aging,” Li says. “That's an unsolved problem.”


In a presentation at the 2009 Asian Biometrics Consortium conference in Tokyo, Tan Tieniu, director of CASIA's National Laboratory of Pattern Recognition, said that among the sites targeted for iris-recognition systems and other biometric technologies in coming years are the country's thousands of prisons, detention houses, and labor camps—sites that house dissidents as well as prisoners. And the new flood of research follows on the development of an extensive surveillance system. In 1988, only 2% of local budgets went toward public security, according to Christian Göbel, a political scientist at Heidelberg University in Germany and Lund University in Sweden. By 2010, that figure had increased to 6%, Göbel says—and the rise in absolute terms was even starker.

Today, lampposts in Tiananmen Square are studded with cameras, as are the homes of political dissidents. After a series of protests in Tibet in 2008, the Chinese government installed cameras at monasteries throughout the autonomous area. Riots the following year in Xinjiang, a heavily Muslim region plagued by unrest, brought a similar response, Bequelin says: “There was a very clear policy of installing cameras to identify people and to monitor in a way that was intimidating.”

The lack of citizen protections in China allows for greater latitude in employing biometric technologies. In China “you are able to cross images gathered from a CCTV system with other databases,” Bequelin says: “The key here is interoperability.” The Shenzhen company China Information Security Technology Inc. is developing an identification card for migrant workers that its Web site says will be traceable through police geographical information systems. “The [privacy] issue doesn't exist in China,” Jain says.

To be sure, many of the technologies being developed in China do not have sinister uses. Fingerprinting is used in some provinces to verify the identities of students taking China's annual university entrance examination. Coal mines in northeastern China use an iris-recognition system that grew out of research done at CASIA to control access and keep track of workers during an accident. “These technologies help to sustain authoritarian rule in China, but they're not only there for that,” Göbel says. “They also address concerns of the population.” And yet, Bequelin says, such dual-use technologies should be monitored “by a strong civil society or a political system that is responsive to citizens' concerns.” In China, he adds, that “is just not there.”