Contribution
My Lords, we have been debating Part 1, which gives substantial powers to the Cabinet Office when the Minister has reasonable grounds to suspect fraud, and we are about to kick off on Part 2, which gives substantial powers to the DWP. Those include police-style powers to enter private premises, search them and seize property, as well as powers to demand information. Those are potentially very intrusive powers, so it is essential that they can be exercised only when it is genuinely appropriate to do so.
The two amendments in this group cover both Parts 1 and 2, and they provide essential clarification as to how the DWP and PSFA should interpret the legal threshold for most of the investigative powers in the Bill, which is the requirement to have “reasonable grounds” of suspicion of fraud.
The amendments are intended to ensure that, when the DWP and PSFA are exercising their investigative powers under this Bill, reasonable grounds do not include generalisations or stereotypes of certain categories of people—for example, that members of a particular social group are more likely to be involved in fraudulent activity than others. Investment in data analytics and other emerging technologies, such as AI, for fraud risk detection is inevitably, and probably rightly, increasing. The Government have signalled their intention to turbocharge AI and to mainline AI into the veins of the nation, including the public sector.
The Government are, as we speak, trying to pass the Data (Use and Access) Bill, which would repeal the current ban on automated decision-making and profiling of individuals. The DWP has invested heavily in artificial intelligence, widening its scope last year to include use of a machine-learning tool to identify fraud in universal credit advances applications, and it intends to develop further models. This is despite a warning from the Auditor-General in 2023 of
“an inherent risk that the algorithms are biased towards selecting claims for review from certain vulnerable people or groups with protected characteristics”.
The DWP admitted that its,
“ability to test for unfair impacts across protected characteristics is currently limited”.
There are real concerns about the inaccuracy of algorithms, particularly when such inaccuracy is discriminatory, when mistakes disproportionately impact a certain group of people. It is well evidenced that machine-learning algorithms can learn to discriminate in a way that no democratic society would wish to incorporate into any reasonable decision-making process about individuals. An internal DWP fairness analysis of the universal credit payments algorithm, which was published only due to a freedom of information request, has revealed a “statistical significant outcome disparity” according to people’s age, disability, marital status and nationality.
This is not just a theoretical concern. Recent real-life experiences in both the Netherlands and Sweden should provide a real warning for us, and are clear evidence that we must have robust safeguards in place. Machine-learning algorithms used in the Netherlands’ child tax credit scandal learned to profile those with dual nationality and low income as being suspects for fraud. From 2015 to 2019, the authorities penalised families over suspicion of fraud based on the system’s risk indicators. Tens of thousands of families, often with lower incomes or belonging to ethnic minorities, were pushed into poverty. Some victims committed suicide. More than a thousand children were taken into foster care. The scandal ultimately led to the resignation of the then Prime Minister, Mark Rutte.
In Sweden in 2024, an investigation found that the machine-learning system used by the country’s social insurance agency is disproportionately flagging certain groups for further investigation over social benefits fraud, including women, individuals with foreign backgrounds, low-income earners and people without university degrees. Once cases are flagged, fraud investigators have the power to trawl through a person’s social media accounts, obtain data from institutions and even interview an individual’s neighbours as part of their investigations.
The two amendments that I have tabled are based on paragraph 2.2 of Code A to the Police and Criminal Evidence Act 1984, in relation to police stop and search powers, which states that:
“Reasonable suspicion cannot be based on generalisations or stereotypical images of certain groups or categories of people as more likely to be involved in criminal activity”.
These amendments would not reduce the ability of departments to go after fraud. Indeed, I argue that by ensuring that the reasonable suspicion is genuine, rather than based on stereotypes, they should improve the targeting of investigations and therefore make the investigations more effective, not less so.
The Bill extends substantial intrusive powers to the Cabinet Office, the PFSA and the DWP, and those powers must be subject to robust safeguards in the Bill. The use of “generalisations or stereotypes”, whether through automated systems or otherwise, should never be seen as grounds for reasonable suspicion. I hope the Minister will see the need for these safeguards in that context, just as they are needed and exist in relation to stop and search powers. I beg to move.