Facial recognition surveillance in São Paulo could worsen racism

São Paulo, Brazil – As the city of São Paulo prepares to roll out thousands of surveillance cameras with facial recognition, experts are raising concerns on the indiscriminate use of this technology in the Brazilian megalopolis could exacerbate problems such as structural racism and inequality, while also posing risks to data privacy and cybersecurity.

The Smart Sampa project is the latest among a series of initiatives involving modern surveillance techniques in various Brazilian states. It is significant due to the sheer size of the population it will impact: São Paulo, the most populous city in the Southern Hemisphere, is home to 12 million people.

The project aims to roll out a single video surveillance platform that integrates and supports the operations of emergency and traffic services, the city’s public transport network, and police forces. By 2024, up to 20,000 cameras will be installed, and an equal number of third-party and private cameras will be integrated into the network.

The new cameras will enable the city to monitor schools, medical practices, public spaces such as squares and parks, as well as social media content relevant to public administration.

The combination of real-time analytics and facial recognition technology – which detects and compares faces in a given space using artificial intelligence (AI) algorithms – is meant to expedite the process of identifying wanted criminals, stolen cars, missing persons, lost objects, and managing public transport across the city.

“We hope to conclude the contracting process [for Smart Sampa] as soon as possible, as it will greatly enhance security and mobility in the city,” said São Paulo Mayor Ricardo Nunes during the relaunch of the tendering process in May.

Improvements are urgently needed in both areas and complaints about excessive waiting times from bus users to the city transport authority have risen by 42 percent in the first quarter of 2023 and data from the urban security department indicates a 35.7 percent rise in muggings in the city in 2023 in relation to 2021.

With real-time data from the cameras and algorithms, the city expects to predict and act upon occurrences faster. It also hopes to anticipate traffic patterns and potential congestion points and use insights to adjust bus schedules, for example.

Despite the appeal of remote surveillance technologies in solving the city’s problems, critics of Smart Sampa fear that the project will infringe upon citizens’ fundamental human rights, including privacy, freedom of expression, assembly and association.

These concerns led to the suspension of the tendering process on two occasions and prompted investigations by public prosecutors into potential project pitfalls in areas including citizen privacy.

The contracting process was allowed to continue after the São Paulo courts concluded there was insufficient evidence to prove the system is biased against Black individuals. PK9, a tech firm based in São Paulo, pitched a monthly bid of 9.2 million Brazilian real ($1.8m) to operate the system over a 60-month period.

The risks for Black Brazilians

One of the key issues highlighted by experts regarding Smart Sampa pertains to the negative consequences the system could generate, particularly for groups such as the Black community, which constitutes 56 percent of Brazil’s population, according to the Brazilian Institute of Geography and Statistics (IBGE). Experts argue the project could undermine the right to non-discrimination and challenge the principle of presumption of innocence.

According to Fernanda Rodrigues, a digital rights lawyer and research coordinator at the Institute for Research on Internet and Society (IRIS), facial recognition technology has the potential to lead to false positives – wrongly matching a person’s face with an image in the database –  and drive mass incarceration of Black individuals as a result.

“As well as the risks that the information fed to these platforms may not be accurate and the system itself might fail, there is a problem that precedes technology implications, which is racism”, Rodrigues said.

“We know the penal system in Brazil is selective, so we can conclude that [use of surveillance with facial recognition] is all about augmenting the risks and harms to this population”, Rodrigues added, referring to the high representation of Black individuals in Brazilian jails, who account for over 67 percent of the imprisoned population, according to 2022 data from the Brazilian Public Security Forum.

A study conducted by the Center for Studies on Public Security and Citizenship (CESeC), which monitors the impact of facial recognition use by police nationwide, revealed that more than 90 percent of the individuals arrested through decisions based on facial recognition in Brazil are Black. In the state of Rio de Janeiro, the percent of unjust arrests based on photographic recognition and involving Black individuals reached 81 percent in 2021, according to data from the Public Defenders Chamber of Rio de Janeiro.

Another problem relates to the lack of openness regarding the project. One of the organisations behind the national campaign “Take My Face Out of Your Aim” and other initiatives calling for a ban on facial recognition, CESeC and other nonprofit bodies are involved in a number of legal actions against governments that choose to roll out the technology for urban security and other purposes.

Last year the São Paulo City Hall held a virtual public consultation over the last two weeks of August, inviting experts to contribute their views, with a single day set aside for people to get their queries answered.

“Participation was limited and the suggestions made were largely ignored,” said Celina Bottino, project director at the Institute for Technology & Society of Rio de Janeiro.

In response to the public pushback, the Smart Sampa tender was updated with a study on the impact of the technology, which acknowledged shortcomings such as a high probability of biases in the facial recognition features, as well as the risk of unauthorised use and exposure of personal data, as well as the likelihood of privacy violations.

To address these risks, the report noted the platform would only consider detections with 90 percent parity, and all alerts issued would be analysed by trained personnel to mitigate injustices, as well as an advanced data protection and access control system.

Calls for suspension

The Public Defender’s Office of São Paulo launched a civil lawsuit in May, alongside the Laboratory for Public Policy and Internet (LAPIN) and Uneafro Brasil, supported by Rede Liberdade, a network of legal professionals working with human rights organisations. The ongoing case calls for the suspension of the project and a halt on the use of facial recognition systems for urban safety.

“Facial recognition is a complex technology with aspects that have not yet been fully mapped, which makes it incompatible with a tendering process, since [this format] does not allow for the required transparency and participation. This incurs a series of risks to the fundamental rights of vulnerable populations, such as Black and transgender individuals, as well as children,” the organisations said in a statement.

The civil action also argues that the impact study is lacking in opinions and detailed studies on the potential damages of the technology, as well as ways to overcome the risks: “The City Hall only presents specific aspects about the impacts of the Smart Sampa programme, which were rarely discussed with the civil society and other institutions,” says public defender Surrailly Youssef.

Even if a supplier has already been chosen, the service may be suspended at any time if violations of rights or principles of public administration are found, says public defender Cecilia Nascimento Ferreira.

“The judicial measure, however, sought to avoid spending public funds on a reckless technology contracted via an inadequate modality given the complexity of the service,” she adds.

The government of São Paulo did not respond to requests by Al Jazeera for an interview.

Elsewhere in the Americas, Brazil’s neighbour Argentina deemed the use of facial recognition in its capital Buenos Aires illegal last year. In the US, several states revised their stance to ban facial recognition to support public services, even though the technology remains mostly unregulated. On the other hand, the adoption of the technology has advanced in countries across the globe, such as China and India, albeit not without criticism.

“Factors including transparency and accountability in the application of these new technologies seem to have been forgotten by decision-makers worldwide,” says Pablo Nunes, a political scientist and coordinator at CESeC. “Even in a scenario where a ban doesn’t happen, I hope that we will have minimal safeguards and regulations in place for the use of facial recognition in Brazil.”