The New York Civil Liberties Union filed a lawsuit Monday against the state education department over the new security system in Lockport schools, which recognizes two things: guns and faces. WBFO’s Kyle Mackie reports on how the system works and how the same technology used by Scotland Yard and INTERPOL ended up in a school district in Niagara County.
The security system is only supposed to identify some people whose photos are stored in a database maintained by the Lockport City School District. That includes Level 2 and 3 sex offenders, unauthorized school staff, like a suspended teacher, and other adult visitors whom law enforcement determines pose a credible threat. The district’s privacy policy also states that local law enforcement officials may be entered into the database to avoid confusion should they have to enter a school building with a weapon drawn.
“It’s like looking through a hole in a fence,” said Dr. Robert LiPuma, district director of technology, data security and communications. “The software is scanning all these people going by and they’re just going by — nope, nope, nope, nope, nope, nope, nope, nope — till finally, ‘Hey, there goes Michelle. I’m looking for Michelle.’”
At that point, LiPuma said two levels of human verification are required to confirm the match. Just one is required to confirm a gun identification, which then triggers an automatic alert to Lockport police. Human-confirmed facial matches do not trigger an automatic law enforcement response, LiPuma said. But in a potential active shooter situation, “That time — minutes, seconds — can save lives.”
Some national experts disagree. In a recent issue brief, the Future of Privacy Forum, a nonprofit organization committed to exploring privacy challenges posed by emerging technologies, said there’s no evidence that facial recognition systems will actually make schools safer or give first responders time to intervene in the event of a school shooting. That’s partially because most shootings are perpetrated by current students and Lockport agreed not to include any students in its database at the direction of the New York State Education Department (NYSED). The anti-gun violence coalition Everytown for Gun Safety also pointed out in a February report that student school shooters are familiar with their school’s safety protocols and procedures.
“Unfortunately, we’re talking about a system that is not proven to work and a system that is not only incredibly expensive to purchase but also to maintain,” said Amelia Vance, director of youth and education privacy at the forum.
“There are many other research-based methods that could be adopted to protect student safety much better,” Vance said, citing hiring more school counselors to care for students’ mental health and implementing visitor management systems that scan driver’s licenses against sex offender registries as two examples.
There have also been multiple government and academic studies showing that most commercial facial recognition systems exhibit racial and gender biases and are more likely to misidentify women and people of color than white men, though the technology is improving, according to Craig Watson, image group manager at the National Institute of Standards and Technology (NIST). NIST is the leading government body studying the accuracy of facial recognition algorithms, and AEGIS has performed well in its past studies. A spokesperson told The Buffalo News last summer that the system’s accuracy rate was between 98 and 99%.
However, in Part 3 of NIST’s ongoing Face Recognition Vendor Test, the ID3 algorithm that power’s the Lockport system — which is itself called AEGIS and was developed by a Canadian company called SN Technologies — didn’t perform as well on every kind of face.
Watson said the study had about 200 facial recognition algorithms analyze more than 18 million images of 8 million people from around the world who were classified as white, Black, Asian or American Indian. In one test using mugshots, the baseline performance for identifying white males was 1 false positive, meaning the wrong match, for every 10,000 images. ID3 and most of the other algorithms performed worse for women across the board, misidentifying 1 in every 3,000 white women, about 1 in 1,000 Asian and Black women and 1 in 100 Native American women.
That means the algorithm is 10 times better at correctly identifying white men than Asian and Black women and 100 times better at identifying white men than Native American women. ID3 is also better at identifying white men than men of other races. Black and Asian men were misidentified in the test at the rate of about 1 in 3,000 and Native American men at the rate of 1 in 300.
Most of the other algorithms had similar results, and NIST didn’t rank the different developers on their performance. Watson also said it’s impossible to directly compare such test results with how an algorithm might work in the real world, though it may be a helpful indication.
“The test we do here is very detailed, and I do believe that it gives you a good measure of how that algorithm is likely to perform in that system,” he said.
SN Technologies told WBFO the company has done “more work” on the ID3 algorithm for AEGIS but declined an interview request for this story. LiPuma said any false positives in Lockport will be easily caught and dismissed by human monitors, like if a student or parent gets incorrectly flagged as a sex offender, for example. The NYCLU isn’t convinced, and the group has also raised concerns about the involvement of a man named Tony Olivo in Lockport’s purchase of AEGIS.
“We have the best technology in the world in a school in Western New York and everybody wants to shut it, not everybody, but there's a lot of people that want to shut it down,” Olivo told WBFO in January.
Olivo is a former U.S. Marshall and partner at a risk management company in Orchard Park. He’s also a paid consultant for SN Tech. Speaking at a Lockport school board meeting in 2018, Olivo said he was impressed when he saw a presentation by the company at an anti-terrorism conference in Toronto.
“I approached them and I asked them if they had a program like this for school districts. And quite frankly, they looked at me like I had three heads because they don’t have these [school safety] issues outside the United States, sadly.”
At the time, AEGIS was used mostly by European law enforcement agencies like Scotland Yard, INTERPOL and the French Ministry of Defense, according to Olivo. He helped SN Tech adapt the system for use in American schools. He also already had relationships with dozens of school districts across New York state, including Lockport, from offering free security screenings after the 2012 Sandy Hook Elementary School shooting in Newtown, Connecticut. But Olivo said that doesn’t mean he made any money off of Lockport’s eventual purchase of the AEGIS system.
“No,” Olivo said, asked by WBFO whether he made a commission on the sale. “I'm compensated as a consultant by SN Tech.”
Lockport paid for its five-year, $1.4 million AEGIS license using public funds allocated through the Smart Schools Bond Act. That 2014 program made $2 billion in taxpayer funds available for educational technology, like laptops or expanding Internet access. One of the six categories schools can also expense for is high-tech security. Johanna Miller, director of the NYCLU’s Education Policy Center, said SN Tech saw that as a business opportunity.
“We think that that public money could be serving our kids in a lot better ways than going to a private company,” Miller said during a February town hall meeting hosted by the NYCLU in Lockport.
Another concern is that some Lockport parents and community members, including the majority of about 40 people who attended the town hall, said they feel like they were in the dark about the whole sale. “That never happened, sorry,” one parent responded when Stefanie Coyle of the NYCLU explained to attendees that the district said parents were consulted.
“I’m just saying, they certified that they engaged with stakeholders, including parents and teachers and students, so if that didn’t happen, that seems like a real problem,” Coyle said.
The NYCLU said the state should have required more meaningful community engagement. A handful of Lockport high school students who have been speaking out also said they were never told that the new security system included facial recognition.
“We just kind of feel lied to [and] deceived,” said Eli Schrader, a graduating senior. “If [there’s] anyone that they should talk to about it, it should be us. And I kind of feel like they avoided the whole situation.”
WBFO asked Lockport Superintendent Michelle Bradley how — and whether — the district has been talking to students about AEGIS. She answered by saying officials have focused on the shared responsibility of students and staff to keep schools safe.
“As you know, we just got this up and running,” Bradley said. “So, what I will say is we comply with safety plan requirements, so that includes putting code of conducts in the language of students so that they understand expectations, what we expect them to follow to keep all of our schools safe.”
The district also said it followed all of the state’s rules for Smart Schools purchases and held a public meeting about its plan to buy AEGIS, as required, during a regular school board meeting in August 2016. Meeting minutes show there were no public speakers involved in that conversation.
“It doesn't take a rocket scientist to understand that nobody in their right mind, not even me, would go to a school board meeting in the afternoon in August,” said Jim Shultz, a Lockport parent and one of the plaintiffs in the NYCLU lawsuit. “It’s exactly how you'd do it if you don't want anybody to participate, and that's what happened.”
The NYCLU and Lockport residents like Shultz aren’t alone is criticizing the state’s review process: Monica Wallace, a Democrat representing Cheektowaga and Lancaster, is sponsoring a bill in the New York State Assembly that would impose a moratorium on all biometric technology in schools through 2022 or until NYSED completes a thorough study and makes recommendations for school districts on things like facial recognition technology.
“I thought, before we go ahead and start using our children as guinea pigs, we should take a deep look at this and decide from a statewide perspective, is this [a] technology that we think the benefits of the technology outweigh the risks?” the lawmaker said.
Wallace also told WBFO she had hoped her bill would pass the state Senate during the spring legislative session, but the coronavirus pandemic disrupted that schedule. In the meantime, at least one other local school district — Depew, which falls within Wallace’s district — has purchased a one-year AEGIS subscription but not yet activated the system.
The state education department did not respond to multiple interview requests or specific questions sent by email about the use of facial recognition technology in New York schools. A NYSED spokesperson told WBFO Tuesday that the department doesn’t comment on pending legislation. Wallace and her senate co-sponsor, Brian Kavanagh released a joint statement about the lawsuit that reads, in part:
“The lawsuit recently filed by the New York Civil Liberties Union against the New York State Education Department for permitting the use of facial recognition technology in the Lockport City School District illustrates, yet again, why it is urgent for NYSED to alter its course on this issue. We have proposed legislation to impose a moratorium on the use of facial recognition technology in schools, but NYSED and school districts themselves have the authority to take that step now, without waiting for either the Legislature or the Courts to require that.”
