Baroness Fox of Buckley (Non-Afl)

My Lords, my Amendments 82 and 88 attempt to bring much-needed transparency and clarity to how and why banks are being asked to check their customers’ bank accounts via EVMs. I also support Amendment 89ZA by the noble Lord, Lord Vaux. Amendment 82 would require the Secretary of State to make eligibility indicators publicly available. As noble Lords will have gathered by now, I am opposed to eligibility verification notices in general and in principle. However, if they are to remain in the legislation, we need to maximise the transparency around them to guard against overreach. Amendment would 88 requires codes of practice to “to include scrutiny provisions about the algorithms used by banks and the effectiveness of the eligibility verification measure” in this clause. At present, there is insufficient oversight in the Bill. We know that algorithms are central to the Bill. That is in line with the Government’s commitment to turbocharge data analytics and AI into public services in general and fraud risk detection in particular. Under this Bill, thousands of decisions regarding the collection and review of the private financial information of people receiving benefits will be—de facto at least—automated. That is a high-risk way to facilitate making decisions; especially those of a sensitive nature. Yet there is no other way for banks, building societies and so on to conduct the benefits eligibility checks that the Bill compels them to do without an algorithmic system—we have already touched on that—but the Bill does not include the specific eligibility search criteria of the algorithm involved. These amendments seek to address this lack of oversight. The Explanatory Notes offer examples of search criteria, such as capital holdings or the legal limits of stays abroad, but there are no provisions to limit the criteria or provide transparency on them. That lack of transparency makes me question whether the Government are using the most appropriate mechanisms for their ends here, given the complexity of benefits eligibility per se, individuals varied circumstances and the sheer scale of the population’s financial accounts, joint accounts and so on. It is unclear, and certainly no evidence has yet been provided that I am convinced by, why the Government think that banks are better placed than the DWP to conduct these complex assessments, especially when it involves outsourcing unconsented automated surveillance to third parties such as banks. These are all things that we have already discussed, so what I am specifically looking at here are the difficulties in relation to what we are asking banks to do. There is no information in the Bill specifying who is responsible for supplying the algorithms required for this surveillance. Can the Minister clarify whether the DWP will provide third-party organisations such as banks with its existing search methodology? Will third parties be responsible for developing and deploying their own? I can understand that this might be being worked on. I have gathered from some of the things that the Minister has already mentioned that these technical issues might still be being resolved. However, it is not clear in the Bill who or what will decide on the algorithms, and there will be no accountability in relation to what we ask those algorithms to do. In both the cases that I have given, we need to be able to probe how the powers will be put into practice. Can the Minister tell us how much testing has been done on the systems the banks will use? If it has not been done so far, when will we have it? I do not understand why the Bill does not have provisions for quality assurance checks or a periodic review of these new automated systems. Without such quality checks, it seems inevitable that inaccurate information will be flagged and mistakes will occur, at great human cost. We heard similar concerns from the noble Lord, Lord Sikka, today. The noble Lord, Lord Vaux, referenced the Netherlands’ child tax credit scandal, which led to, for example, more than a thousand children being taken into foster care. That was because of algorithmic problems and a particular use of algorithms, with precisely the same ends of tackling fraud. Surely the Minister can see that the constant scanning of millions of accounts in relation to often complex queries and claims will make false positive matches for fraud highly likely. I was trying to listen to what the Minister said earlier about how no decisions will be taken. Maybe we can clarify all that finally here, in terms of my concerns. I am worried that, as a result, a significant number of false positives will lead to account holders’ personal details being wrongly flagged up to the Government for further investigation, which in turn may incur further privacy intrusion—let alone penalties. There have been problematic previous schemes that we should learn the lessons from. Take for example financial institutions’ suspicious activity reports, or SARs, which are used to combat money laundering—a laudable aim. But these SARs already have problems. A 2017 study of a sample of the largest banks found that, of approximately 16 million reviewed, 640,000 SARs were filed, yet only 4% of them resulted in law enforcement involvement. Then there is a DWP pilot: the housing benefit accuracy award initiative, which was used to produce a risk score for housing benefit claimants that was then used as the basis for review of housing benefits by local councils. The algorithm flagged approximately 400,000 cases a year, identifying most of them as high-risk cases. As a consequence, councils were required to conduct file case reviews of those flagged, which involved invasive checks of bank statements, payslips, rent, et cetera. I know someone who was a victim of this and can testify to how awful that experience was. Benefits were suspended where claimants were not compliant or able to produce evidence to support their claim. But data obtained from the DWP by Big Brother Watch, which has been absolutely heroic in alerting the public to the problems associated with the Bill, showed that only one in three people on housing benefits subject to review were in fact being paid the wrong amount. That meant that 200,000 people were placed under suspicion at the hands of an algorithm, despite having done nothing wrong. The algorithm risks are amplified tenfold in the Bill—we should be taking this much more seriously in terms of scrutiny. On recording how people’s data will be assessed and not relying solely on algorithms, we are given assurances in the Explanatory Notes—the Minister has been clear about this—that “a human will always be involved in any further inquiries and any decision taken afterwards that might affect eligibility or benefit awards”. But these assurances are not an adequate safeguard alone, as we have already touched on in earlier groups. On one hand, there is a tendency for human deference to algorithmic outputs—we have all heard the phrase, “The data does not lie”—and I fear that that is what will happen. On the other hand, courts are currently required to presume that computer systems operate correctly, placing the onus upon defendants to provide evidence that the systems they are implicated by are flawed. There is also the small matter of staffing and resources. With many thousands of accounts being flagged to the DWP under the proposed system, it is not clear what is feasible in terms of the scale and nature of human involvement, or whether it will be genuinely meaningful. The Minister only moments ago assured us that members of the DWP would always be involved. Maybe this is the kind of job creation scheme that the Government are involved in, but it seems that that is an awful lot of civil servants who will be required if fraud is happening everywhere, and so on and so forth. So I worry. Indeed, the impact assessment on the Bill acknowledges that the DWP may have to slow the volume of data requests to manage the potential volumes, because there will be so many. If a human decision-maker does not have enough time to properly review a decision—which is my fear—as may well be the case with the deluge of data that DWP will expect to receive from banks, the human input cannot be properly regarded as meaningful.