Menu

Predictive Policing: When Algorithms Decide Who’s Guilty Before the Crime

Predictive Policing: When Algorithms Decide Who's Guilty Before the Crime
Predictive Policing: When Algorithms Decide Who's Guilty Before the Crime

In 2002, Steven Spielberg released Minority Report, a science fiction film in which a specialized police unit arrests people for murders they haven’t committed yet, based on psychic predictions. Audiences understood it as dystopian fiction. What they didn’t know was that the film would prove to be something closer to a documentary.

Across the United States and around the world, police departments are using algorithmic systems to predict crime before it happens — to generate risk scores for individuals who haven’t committed any crime, to direct police resources toward people identified by machine learning as likely future offenders, and to make decisions about who gets watched, who gets stopped, and who gets targeted, all based on data patterns that most people don’t know are being collected about them.

The technology is real. It’s operational. And the evidence about what it does — particularly to communities of color and low-income neighborhoods — is deeply troubling.

PredPol: The Algorithm That Claimed to See the Future

The company that put predictive policing on the map was PredPol (later rebranded as Geolitica). Founded in 2012 by UCLA professor Jeffrey Brantingham and former Santa Cruz police chief Zach Friend, PredPol offered law enforcement agencies a product that used historical crime data to generate 500-square-foot boxes on a map where crimes were predicted to occur during a given shift.

The algorithm was inspired by earthquake aftershock modeling — the observation that crimes, like earthquakes, tend to cluster in space and time. A burglary today makes a burglary in the same block more likely tomorrow. A car theft in a neighborhood elevates the probability of another car theft nearby. PredPol’s algorithm identified these spatial and temporal clusters and directed patrol officers to spend time in the highlighted zones.

By 2017, PredPol claimed to be used by more than 150 police departments across the US and UK. The pitch was compelling: data-driven policing, free from human bias, maximizing efficiency by deploying officers where they’d most reduce crime.

The problem? The data wasn’t free from human bias. It was saturated with it.

Garbage In, Injustice Out: The Bias Problem

Predictive policing algorithms learn from historical crime data. But historical crime data doesn’t reflect where crime actually occurs — it reflects where police have historically been deployed and where they’ve historically made arrests. These are not the same thing.

Decades of documented racial bias in American policing have concentrated enforcement resources in Black and Latino neighborhoods, regardless of actual crime rates. Drug offenses — which occur across all demographic groups at roughly similar rates — have historically been prosecuted far more frequently in communities of color, because that’s where police have been directed to look.

When you feed this historically biased data into a predictive algorithm, the algorithm doesn’t detect bias — it learns it, encodes it, and amplifies it. The system predicts more crime in areas where police have historically made more arrests. More police are then sent to those areas. More arrests are made. The data confirms the prediction. The loop closes.

A 2016 investigation by the Human Rights Data Analysis Group analyzed PredPol’s predictions in Oakland and found that the algorithm’s predictions correlated strongly with neighborhood demographics — specifically, with the racial composition of neighborhoods — independently of actual crime patterns. The algorithm wasn’t predicting crime. It was predicting police attention to historically over-policed areas.

The Heat List: Chicago’s Algorithmic Targeting

While PredPol focused on geographic prediction, Chicago developed something more personal: the Strategic Subject List, colloquially known as the “heat list” or “SSL.” Instead of predicting where crimes might occur, Chicago’s system predicted who might be involved in gun violence — assigning risk scores to over 400,000 individuals based on factors including prior arrests, age, gang affiliation, and whether they were the victim of a prior shooting.

People on the list received visits from police at their homes — not because they had done anything, but because an algorithm said they were at risk of being involved in future violence. These “custom notification” visits were presented as community outreach, but recipients reported them as threatening and intimidating.

The list’s contents — the very fact that a list existed — was kept from the people on it and from the public. City officials repeatedly denied its existence before a Guardian investigation in 2017 obtained the list through public records requests and confirmed its scope.

The investigation also found significant problems with the list’s composition. People who had been victims of violence appeared on it with high risk scores — an outcome that made sense algorithmically (being a shooting victim correlates with future involvement in gun violence) but was perverse in practice: people who had already suffered violence were being targeted as future threats. The algorithm couldn’t distinguish between victim and perpetrator because the data couldn’t either.

New Orleans and Palantir’s Secret Six Years

In 2018, The Verge revealed that the New Orleans Police Department had been secretly running a predictive policing program powered by Palantir Technologies for six years — a program that had been hidden not just from the public but from the city’s own elected officials.

The program, established through a series of philanthropic grants that obscured Palantir’s involvement, analyzed social media activity, criminal history, association networks, and other data sources to build profiles of individuals deemed at risk of involvement in future violence. Officers used the profiles to make decisions about whom to surveil, whom to stop, and how to allocate investigative resources.

The revelation that a major American city had been subjected to a six-year covert algorithmic surveillance and policing program — one that city council members and the public had no knowledge of — raised profound questions about democratic accountability. If elected officials don’t know what surveillance technologies are being deployed, how can citizens have any meaningful oversight?

New Orleans terminated the program following the revelation. The question of what happened to the data the program had collected on thousands of New Orleans residents was never fully resolved.

COMPAS and the Sentencing Algorithm

Predictive algorithms have also entered the courtroom itself. COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) is a commercial risk assessment tool used in criminal sentencing and parole decisions in numerous US states. The algorithm assigns defendants a “recidivism risk score” — a prediction of how likely they are to reoffend — and judges use these scores as inputs to sentencing decisions.

In 2016, ProPublica published a landmark investigation of COMPAS titled “Machine Bias” that found the algorithm made racially disparate errors. Black defendants were nearly twice as likely as white defendants to be falsely flagged as future criminals. White defendants were more likely to be incorrectly flagged as low risk and subsequently re-arrested.

Northpointe, the company that made COMPAS, disputed the methodology. Academic statisticians engaged in a lengthy debate about exactly what kind of fairness the algorithm should optimize for. But the core finding — that the algorithm produced different error rates for different racial groups, with Black defendants bearing the greater burden of false positive predictions — was robust to multiple analytical approaches.

The COMPAS case crystallized a fundamental problem with algorithmic decision-making in criminal justice: the algorithm’s inputs, outputs, and internal workings are proprietary trade secrets that defendants cannot access or challenge. Algorithmic opacity means algorithmic unaccountability.

Facial Recognition: When Prediction Becomes Misidentification

The predictive policing ecosystem has expanded to include facial recognition technology, adding a layer of error and bias that has already sent innocent people to jail. The cases of Robert Williams, Michael Oliver, and Nijeer Parks — all Black men wrongfully arrested based on facial recognition misidentification — illustrate the concrete human cost of algorithmic policing.

Studies by the National Institute of Standards and Technology (NIST) have documented significant racial disparities in facial recognition accuracy. The algorithms misidentify Black, Asian, and Indigenous faces at rates up to 100 times higher than they misidentify white faces — a bias with direct, demonstrable, documented origins in the racial composition of the training data used to build the systems.

When a facial recognition system misidentifies a person and flags them as a suspect, the police officers who receive that lead tend to trust it — to treat the algorithmic identification as evidence rather than as a hypothesis requiring verification. The consequences range from wrongful arrest to wrongful imprisonment.

Detroit — which uses facial recognition despite documented cases of misidentification — has declined to stop using the technology following wrongful arrest scandals, arguing that the tool is only one input among many. Critics respond that in practice, an algorithmic identification anchors subsequent investigation in ways that make exculpatory evidence harder to see.

The Scale of Deployment: What We Know and Don’t Know

One of the most troubling aspects of predictive policing is how little systematic information exists about its deployment. Law enforcement agencies are not generally required to disclose what surveillance technologies they use or how. The public often learns about programs through leaks, journalists’ public records requests, or after something goes wrong.

A 2020 report by the Electronic Frontier Foundation and related organizations found that predictive policing tools — broadly defined — were in use in police departments serving roughly one-third of the American population. The actual number may be higher; many smaller departments use commercial products without public disclosure.

The commercial market for predictive policing tools has been resilient despite controversies. PredPol/Geolitica ultimately shut down in 2023, but competitors including ShotSpotter (acoustic gunshot detection with predictive analytics), Axon, and numerous Palantir-based implementations continue to operate and expand.

The Civil Liberties Reckoning

The civil liberties implications of predictive policing are multidimensional. At the most basic level, there is the question of whether it is just to subject a person to heightened police scrutiny — stops, searches, surveillance, home visits — based on an algorithm’s calculation of their future behavior. The presumption of innocence is a cornerstone of Anglo-American legal tradition. Predictive policing inverts it.

There are also documented First Amendment concerns. Some predictive systems have incorporated social media activity into their risk calculations — who you follow, what you post, what you repost. When political speech and association affect your algorithmic risk score, the chilling effect on expression and assembly is direct and concrete.

The ACLU, the Center on Privacy and Technology, and numerous civil rights organizations have called for moratoria on predictive policing technologies pending independent audits and regulatory frameworks. Some cities — including Santa Cruz, Los Angeles, and New York — have implemented partial restrictions on specific predictive tools. Federal legislation on the subject remains absent.

Down the Rabbit Hole

  • Algorithmic Accountability Laws: Which jurisdictions have passed laws requiring disclosure, auditing, or community oversight of predictive policing tools, and have these laws been effective?
  • The School-to-Prison Pipeline: Predictive analytics are increasingly being used in school discipline systems — what does it mean when the same kind of algorithmic profiling that targets adults is applied to children?
  • Fusion Centers and Data Sharing: Law enforcement fusion centers aggregate data across agencies — how do predictive policing tools plug into these multi-agency intelligence sharing networks?
  • International Export: US predictive policing technology is being exported to authoritarian governments worldwide — what are the implications of deploying these tools in contexts without even the limited civil liberties protections that exist in the US?
  • AI Policing’s Next Frontier: As large language models and generative AI become integrated into law enforcement tools, what new forms of algorithmic bias and error are emerging, and who is watching?

Disclaimer: This article is intended for educational and entertainment purposes. All case studies, statistics, and program descriptions are drawn from publicly available journalism, academic research, and government reports. Readers are encouraged to consult primary sources and engage with this ongoing policy debate directly.

dive down the rabbit hole

Predictive Policing: When Algorithms Decide Who’s Guilty Before the Crime

Conspiracy Realist
Predictive Policing: When Algorithms Decide Who's Guilty Before the Crime

In 2002, Steven Spielberg released Minority Report, a science fiction film in which a specialized police unit arrests people for murders they haven’t committed yet, based on psychic predictions. Audiences understood it as dystopian fiction. What they didn’t know was that the film would prove to be something closer to a documentary.

Across the United States and around the world, police departments are using algorithmic systems to predict crime before it happens — to generate risk scores for individuals who haven’t committed any crime, to direct police resources toward people identified by machine learning as likely future offenders, and to make decisions about who gets watched, who gets stopped, and who gets targeted, all based on data patterns that most people don’t know are being collected about them.

The technology is real. It’s operational. And the evidence about what it does — particularly to communities of color and low-income neighborhoods — is deeply troubling.

PredPol: The Algorithm That Claimed to See the Future

The company that put predictive policing on the map was PredPol (later rebranded as Geolitica). Founded in 2012 by UCLA professor Jeffrey Brantingham and former Santa Cruz police chief Zach Friend, PredPol offered law enforcement agencies a product that used historical crime data to generate 500-square-foot boxes on a map where crimes were predicted to occur during a given shift.

The algorithm was inspired by earthquake aftershock modeling — the observation that crimes, like earthquakes, tend to cluster in space and time. A burglary today makes a burglary in the same block more likely tomorrow. A car theft in a neighborhood elevates the probability of another car theft nearby. PredPol’s algorithm identified these spatial and temporal clusters and directed patrol officers to spend time in the highlighted zones.

By 2017, PredPol claimed to be used by more than 150 police departments across the US and UK. The pitch was compelling: data-driven policing, free from human bias, maximizing efficiency by deploying officers where they’d most reduce crime.

The problem? The data wasn’t free from human bias. It was saturated with it.

Garbage In, Injustice Out: The Bias Problem

Predictive policing algorithms learn from historical crime data. But historical crime data doesn’t reflect where crime actually occurs — it reflects where police have historically been deployed and where they’ve historically made arrests. These are not the same thing.

Decades of documented racial bias in American policing have concentrated enforcement resources in Black and Latino neighborhoods, regardless of actual crime rates. Drug offenses — which occur across all demographic groups at roughly similar rates — have historically been prosecuted far more frequently in communities of color, because that’s where police have been directed to look.

When you feed this historically biased data into a predictive algorithm, the algorithm doesn’t detect bias — it learns it, encodes it, and amplifies it. The system predicts more crime in areas where police have historically made more arrests. More police are then sent to those areas. More arrests are made. The data confirms the prediction. The loop closes.

A 2016 investigation by the Human Rights Data Analysis Group analyzed PredPol’s predictions in Oakland and found that the algorithm’s predictions correlated strongly with neighborhood demographics — specifically, with the racial composition of neighborhoods — independently of actual crime patterns. The algorithm wasn’t predicting crime. It was predicting police attention to historically over-policed areas.

The Heat List: Chicago’s Algorithmic Targeting

While PredPol focused on geographic prediction, Chicago developed something more personal: the Strategic Subject List, colloquially known as the “heat list” or “SSL.” Instead of predicting where crimes might occur, Chicago’s system predicted who might be involved in gun violence — assigning risk scores to over 400,000 individuals based on factors including prior arrests, age, gang affiliation, and whether they were the victim of a prior shooting.

People on the list received visits from police at their homes — not because they had done anything, but because an algorithm said they were at risk of being involved in future violence. These “custom notification” visits were presented as community outreach, but recipients reported them as threatening and intimidating.

The list’s contents — the very fact that a list existed — was kept from the people on it and from the public. City officials repeatedly denied its existence before a Guardian investigation in 2017 obtained the list through public records requests and confirmed its scope.

The investigation also found significant problems with the list’s composition. People who had been victims of violence appeared on it with high risk scores — an outcome that made sense algorithmically (being a shooting victim correlates with future involvement in gun violence) but was perverse in practice: people who had already suffered violence were being targeted as future threats. The algorithm couldn’t distinguish between victim and perpetrator because the data couldn’t either.

New Orleans and Palantir’s Secret Six Years

In 2018, The Verge revealed that the New Orleans Police Department had been secretly running a predictive policing program powered by Palantir Technologies for six years — a program that had been hidden not just from the public but from the city’s own elected officials.

The program, established through a series of philanthropic grants that obscured Palantir’s involvement, analyzed social media activity, criminal history, association networks, and other data sources to build profiles of individuals deemed at risk of involvement in future violence. Officers used the profiles to make decisions about whom to surveil, whom to stop, and how to allocate investigative resources.

The revelation that a major American city had been subjected to a six-year covert algorithmic surveillance and policing program — one that city council members and the public had no knowledge of — raised profound questions about democratic accountability. If elected officials don’t know what surveillance technologies are being deployed, how can citizens have any meaningful oversight?

New Orleans terminated the program following the revelation. The question of what happened to the data the program had collected on thousands of New Orleans residents was never fully resolved.

COMPAS and the Sentencing Algorithm

Predictive algorithms have also entered the courtroom itself. COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) is a commercial risk assessment tool used in criminal sentencing and parole decisions in numerous US states. The algorithm assigns defendants a “recidivism risk score” — a prediction of how likely they are to reoffend — and judges use these scores as inputs to sentencing decisions.

In 2016, ProPublica published a landmark investigation of COMPAS titled “Machine Bias” that found the algorithm made racially disparate errors. Black defendants were nearly twice as likely as white defendants to be falsely flagged as future criminals. White defendants were more likely to be incorrectly flagged as low risk and subsequently re-arrested.

Northpointe, the company that made COMPAS, disputed the methodology. Academic statisticians engaged in a lengthy debate about exactly what kind of fairness the algorithm should optimize for. But the core finding — that the algorithm produced different error rates for different racial groups, with Black defendants bearing the greater burden of false positive predictions — was robust to multiple analytical approaches.

The COMPAS case crystallized a fundamental problem with algorithmic decision-making in criminal justice: the algorithm’s inputs, outputs, and internal workings are proprietary trade secrets that defendants cannot access or challenge. Algorithmic opacity means algorithmic unaccountability.

Facial Recognition: When Prediction Becomes Misidentification

The predictive policing ecosystem has expanded to include facial recognition technology, adding a layer of error and bias that has already sent innocent people to jail. The cases of Robert Williams, Michael Oliver, and Nijeer Parks — all Black men wrongfully arrested based on facial recognition misidentification — illustrate the concrete human cost of algorithmic policing.

Studies by the National Institute of Standards and Technology (NIST) have documented significant racial disparities in facial recognition accuracy. The algorithms misidentify Black, Asian, and Indigenous faces at rates up to 100 times higher than they misidentify white faces — a bias with direct, demonstrable, documented origins in the racial composition of the training data used to build the systems.

When a facial recognition system misidentifies a person and flags them as a suspect, the police officers who receive that lead tend to trust it — to treat the algorithmic identification as evidence rather than as a hypothesis requiring verification. The consequences range from wrongful arrest to wrongful imprisonment.

Detroit — which uses facial recognition despite documented cases of misidentification — has declined to stop using the technology following wrongful arrest scandals, arguing that the tool is only one input among many. Critics respond that in practice, an algorithmic identification anchors subsequent investigation in ways that make exculpatory evidence harder to see.

The Scale of Deployment: What We Know and Don’t Know

One of the most troubling aspects of predictive policing is how little systematic information exists about its deployment. Law enforcement agencies are not generally required to disclose what surveillance technologies they use or how. The public often learns about programs through leaks, journalists’ public records requests, or after something goes wrong.

A 2020 report by the Electronic Frontier Foundation and related organizations found that predictive policing tools — broadly defined — were in use in police departments serving roughly one-third of the American population. The actual number may be higher; many smaller departments use commercial products without public disclosure.

The commercial market for predictive policing tools has been resilient despite controversies. PredPol/Geolitica ultimately shut down in 2023, but competitors including ShotSpotter (acoustic gunshot detection with predictive analytics), Axon, and numerous Palantir-based implementations continue to operate and expand.

The Civil Liberties Reckoning

The civil liberties implications of predictive policing are multidimensional. At the most basic level, there is the question of whether it is just to subject a person to heightened police scrutiny — stops, searches, surveillance, home visits — based on an algorithm’s calculation of their future behavior. The presumption of innocence is a cornerstone of Anglo-American legal tradition. Predictive policing inverts it.

There are also documented First Amendment concerns. Some predictive systems have incorporated social media activity into their risk calculations — who you follow, what you post, what you repost. When political speech and association affect your algorithmic risk score, the chilling effect on expression and assembly is direct and concrete.

The ACLU, the Center on Privacy and Technology, and numerous civil rights organizations have called for moratoria on predictive policing technologies pending independent audits and regulatory frameworks. Some cities — including Santa Cruz, Los Angeles, and New York — have implemented partial restrictions on specific predictive tools. Federal legislation on the subject remains absent.

Down the Rabbit Hole

  • Algorithmic Accountability Laws: Which jurisdictions have passed laws requiring disclosure, auditing, or community oversight of predictive policing tools, and have these laws been effective?
  • The School-to-Prison Pipeline: Predictive analytics are increasingly being used in school discipline systems — what does it mean when the same kind of algorithmic profiling that targets adults is applied to children?
  • Fusion Centers and Data Sharing: Law enforcement fusion centers aggregate data across agencies — how do predictive policing tools plug into these multi-agency intelligence sharing networks?
  • International Export: US predictive policing technology is being exported to authoritarian governments worldwide — what are the implications of deploying these tools in contexts without even the limited civil liberties protections that exist in the US?
  • AI Policing’s Next Frontier: As large language models and generative AI become integrated into law enforcement tools, what new forms of algorithmic bias and error are emerging, and who is watching?

Disclaimer: This article is intended for educational and entertainment purposes. All case studies, statistics, and program descriptions are drawn from publicly available journalism, academic research, and government reports. Readers are encouraged to consult primary sources and engage with this ongoing policy debate directly.

Predictive Policing: When Algorithms Decide Who’s Guilty Before the Crime

Adam Smith
Predictive Policing: When Algorithms Decide Who's Guilty Before the Crime

In 2002, Steven Spielberg released Minority Report, a science fiction film in which a specialized police unit arrests people for murders they haven’t committed yet, based on psychic predictions. Audiences understood it as dystopian fiction. What they didn’t know was that the film would prove to be something closer to a documentary.

Across the United States and around the world, police departments are using algorithmic systems to predict crime before it happens — to generate risk scores for individuals who haven’t committed any crime, to direct police resources toward people identified by machine learning as likely future offenders, and to make decisions about who gets watched, who gets stopped, and who gets targeted, all based on data patterns that most people don’t know are being collected about them.

The technology is real. It’s operational. And the evidence about what it does — particularly to communities of color and low-income neighborhoods — is deeply troubling.

PredPol: The Algorithm That Claimed to See the Future

The company that put predictive policing on the map was PredPol (later rebranded as Geolitica). Founded in 2012 by UCLA professor Jeffrey Brantingham and former Santa Cruz police chief Zach Friend, PredPol offered law enforcement agencies a product that used historical crime data to generate 500-square-foot boxes on a map where crimes were predicted to occur during a given shift.

The algorithm was inspired by earthquake aftershock modeling — the observation that crimes, like earthquakes, tend to cluster in space and time. A burglary today makes a burglary in the same block more likely tomorrow. A car theft in a neighborhood elevates the probability of another car theft nearby. PredPol’s algorithm identified these spatial and temporal clusters and directed patrol officers to spend time in the highlighted zones.

By 2017, PredPol claimed to be used by more than 150 police departments across the US and UK. The pitch was compelling: data-driven policing, free from human bias, maximizing efficiency by deploying officers where they’d most reduce crime.

The problem? The data wasn’t free from human bias. It was saturated with it.

Garbage In, Injustice Out: The Bias Problem

Predictive policing algorithms learn from historical crime data. But historical crime data doesn’t reflect where crime actually occurs — it reflects where police have historically been deployed and where they’ve historically made arrests. These are not the same thing.

Decades of documented racial bias in American policing have concentrated enforcement resources in Black and Latino neighborhoods, regardless of actual crime rates. Drug offenses — which occur across all demographic groups at roughly similar rates — have historically been prosecuted far more frequently in communities of color, because that’s where police have been directed to look.

When you feed this historically biased data into a predictive algorithm, the algorithm doesn’t detect bias — it learns it, encodes it, and amplifies it. The system predicts more crime in areas where police have historically made more arrests. More police are then sent to those areas. More arrests are made. The data confirms the prediction. The loop closes.

A 2016 investigation by the Human Rights Data Analysis Group analyzed PredPol’s predictions in Oakland and found that the algorithm’s predictions correlated strongly with neighborhood demographics — specifically, with the racial composition of neighborhoods — independently of actual crime patterns. The algorithm wasn’t predicting crime. It was predicting police attention to historically over-policed areas.

The Heat List: Chicago’s Algorithmic Targeting

While PredPol focused on geographic prediction, Chicago developed something more personal: the Strategic Subject List, colloquially known as the “heat list” or “SSL.” Instead of predicting where crimes might occur, Chicago’s system predicted who might be involved in gun violence — assigning risk scores to over 400,000 individuals based on factors including prior arrests, age, gang affiliation, and whether they were the victim of a prior shooting.

People on the list received visits from police at their homes — not because they had done anything, but because an algorithm said they were at risk of being involved in future violence. These “custom notification” visits were presented as community outreach, but recipients reported them as threatening and intimidating.

The list’s contents — the very fact that a list existed — was kept from the people on it and from the public. City officials repeatedly denied its existence before a Guardian investigation in 2017 obtained the list through public records requests and confirmed its scope.

The investigation also found significant problems with the list’s composition. People who had been victims of violence appeared on it with high risk scores — an outcome that made sense algorithmically (being a shooting victim correlates with future involvement in gun violence) but was perverse in practice: people who had already suffered violence were being targeted as future threats. The algorithm couldn’t distinguish between victim and perpetrator because the data couldn’t either.

New Orleans and Palantir’s Secret Six Years

In 2018, The Verge revealed that the New Orleans Police Department had been secretly running a predictive policing program powered by Palantir Technologies for six years — a program that had been hidden not just from the public but from the city’s own elected officials.

The program, established through a series of philanthropic grants that obscured Palantir’s involvement, analyzed social media activity, criminal history, association networks, and other data sources to build profiles of individuals deemed at risk of involvement in future violence. Officers used the profiles to make decisions about whom to surveil, whom to stop, and how to allocate investigative resources.

The revelation that a major American city had been subjected to a six-year covert algorithmic surveillance and policing program — one that city council members and the public had no knowledge of — raised profound questions about democratic accountability. If elected officials don’t know what surveillance technologies are being deployed, how can citizens have any meaningful oversight?

New Orleans terminated the program following the revelation. The question of what happened to the data the program had collected on thousands of New Orleans residents was never fully resolved.

COMPAS and the Sentencing Algorithm

Predictive algorithms have also entered the courtroom itself. COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) is a commercial risk assessment tool used in criminal sentencing and parole decisions in numerous US states. The algorithm assigns defendants a “recidivism risk score” — a prediction of how likely they are to reoffend — and judges use these scores as inputs to sentencing decisions.

In 2016, ProPublica published a landmark investigation of COMPAS titled “Machine Bias” that found the algorithm made racially disparate errors. Black defendants were nearly twice as likely as white defendants to be falsely flagged as future criminals. White defendants were more likely to be incorrectly flagged as low risk and subsequently re-arrested.

Northpointe, the company that made COMPAS, disputed the methodology. Academic statisticians engaged in a lengthy debate about exactly what kind of fairness the algorithm should optimize for. But the core finding — that the algorithm produced different error rates for different racial groups, with Black defendants bearing the greater burden of false positive predictions — was robust to multiple analytical approaches.

The COMPAS case crystallized a fundamental problem with algorithmic decision-making in criminal justice: the algorithm’s inputs, outputs, and internal workings are proprietary trade secrets that defendants cannot access or challenge. Algorithmic opacity means algorithmic unaccountability.

Facial Recognition: When Prediction Becomes Misidentification

The predictive policing ecosystem has expanded to include facial recognition technology, adding a layer of error and bias that has already sent innocent people to jail. The cases of Robert Williams, Michael Oliver, and Nijeer Parks — all Black men wrongfully arrested based on facial recognition misidentification — illustrate the concrete human cost of algorithmic policing.

Studies by the National Institute of Standards and Technology (NIST) have documented significant racial disparities in facial recognition accuracy. The algorithms misidentify Black, Asian, and Indigenous faces at rates up to 100 times higher than they misidentify white faces — a bias with direct, demonstrable, documented origins in the racial composition of the training data used to build the systems.

When a facial recognition system misidentifies a person and flags them as a suspect, the police officers who receive that lead tend to trust it — to treat the algorithmic identification as evidence rather than as a hypothesis requiring verification. The consequences range from wrongful arrest to wrongful imprisonment.

Detroit — which uses facial recognition despite documented cases of misidentification — has declined to stop using the technology following wrongful arrest scandals, arguing that the tool is only one input among many. Critics respond that in practice, an algorithmic identification anchors subsequent investigation in ways that make exculpatory evidence harder to see.

The Scale of Deployment: What We Know and Don’t Know

One of the most troubling aspects of predictive policing is how little systematic information exists about its deployment. Law enforcement agencies are not generally required to disclose what surveillance technologies they use or how. The public often learns about programs through leaks, journalists’ public records requests, or after something goes wrong.

A 2020 report by the Electronic Frontier Foundation and related organizations found that predictive policing tools — broadly defined — were in use in police departments serving roughly one-third of the American population. The actual number may be higher; many smaller departments use commercial products without public disclosure.

The commercial market for predictive policing tools has been resilient despite controversies. PredPol/Geolitica ultimately shut down in 2023, but competitors including ShotSpotter (acoustic gunshot detection with predictive analytics), Axon, and numerous Palantir-based implementations continue to operate and expand.

The Civil Liberties Reckoning

The civil liberties implications of predictive policing are multidimensional. At the most basic level, there is the question of whether it is just to subject a person to heightened police scrutiny — stops, searches, surveillance, home visits — based on an algorithm’s calculation of their future behavior. The presumption of innocence is a cornerstone of Anglo-American legal tradition. Predictive policing inverts it.

There are also documented First Amendment concerns. Some predictive systems have incorporated social media activity into their risk calculations — who you follow, what you post, what you repost. When political speech and association affect your algorithmic risk score, the chilling effect on expression and assembly is direct and concrete.

The ACLU, the Center on Privacy and Technology, and numerous civil rights organizations have called for moratoria on predictive policing technologies pending independent audits and regulatory frameworks. Some cities — including Santa Cruz, Los Angeles, and New York — have implemented partial restrictions on specific predictive tools. Federal legislation on the subject remains absent.

Down the Rabbit Hole

  • Algorithmic Accountability Laws: Which jurisdictions have passed laws requiring disclosure, auditing, or community oversight of predictive policing tools, and have these laws been effective?
  • The School-to-Prison Pipeline: Predictive analytics are increasingly being used in school discipline systems — what does it mean when the same kind of algorithmic profiling that targets adults is applied to children?
  • Fusion Centers and Data Sharing: Law enforcement fusion centers aggregate data across agencies — how do predictive policing tools plug into these multi-agency intelligence sharing networks?
  • International Export: US predictive policing technology is being exported to authoritarian governments worldwide — what are the implications of deploying these tools in contexts without even the limited civil liberties protections that exist in the US?
  • AI Policing’s Next Frontier: As large language models and generative AI become integrated into law enforcement tools, what new forms of algorithmic bias and error are emerging, and who is watching?

Disclaimer: This article is intended for educational and entertainment purposes. All case studies, statistics, and program descriptions are drawn from publicly available journalism, academic research, and government reports. Readers are encouraged to consult primary sources and engage with this ongoing policy debate directly.

Table of contents