FactRank: new tool brings automatic fact-checking a step closer
FactRank is a new online tool that automatically detects ‘checkable’ claims made by politicians in parliamentary debates or tweets, and therefore enables fact-checkers to work much faster. Alexander Pleijter, a researcher and lecturer in Journalism and New Media, helped develop FactRank.
How does FactRank work?
‘FactRank is software designed to automatically detect sentences that may need to be fact-checked. It scans a text and identifies the sentences that are factual and relevant: that is to say, they contain a fact – which may or may not be true – and are relevant to a large group of people or are politically interesting, for example. We use FactRank to automatically scan all the transcripts of Dutch parliamentary debates, and it identifies claims that would be worth checking, either by our own Nieuwscheckers team or by other fact-checkers, to see if they’re true.
‘The way it works is based on artificial intelligence. We began by coding and entering thousands of these kinds of sentence ourselves. These enabled the software to learn how to recognise their features, and it can now scan texts automatically. And as we’re able to look at every sentence that FactRank detects and say whether it is indeed a factual and relevant claim, the software just keeps on improving.’
What kinds of text do you scan with FactRank?
‘FactRank automatically retrieves all transcripts of Dutch parliamentary debates as soon as they go online, and proceeds to detect the check-worthy claims that they contain. It also scans all the tweets of Dutch politicians. We created the tool in conjunction with Belgian fact-checkers, so the Dutch-language documents from their parliament are also scanned by FactRank. But in principle you can enter any kind of text at all, such as politicians’ speeches. And we’d really like to use it to instantly scan the claims made in talk shows or election debates on TV. So we’re currently talking to TV stations about the possibility of receiving transcripts, even though that’s a lot of work because we have to make separate agreements with each station.
‘The tool is already showing results: we’ve published our first fact-check using FactRank. In the coronavirus debate on 8 April, Lodewijk Asscher said that ‘since the coronavirus lockdown, the number of reports of domestic violence in Friesland has increased by 50 percent.’ That claim was detected by FactRank, and we decided to fact-check it. And we found that
: it wasn't true.’ (Link in Dutch.)
So it makes fact-checking much faster?
‘Absolutely! Of course it isn’t “live” fact-checking yet because the transcripts first have to be produced by the secretary general. But they’re usually online on the same day or the next. And then FactRank automatically scans them. That’s when the journalistic work starts, to actually check a claim: reading reports, finding or retrieving sources, and so on. This obviously takes some time. But we don’t have to monitor debates “live” anymore. I just take a look each morning to see whether any of the claims identified by FactRank are ones that we at Nieuwscheckers would like to work on. It makes a big difference.’
And will fact-checking soon be completely automatic?
‘Well, we haven’t reached that stage yet, but there are two English-language tools – which FactRank is partly based on – that are quite a lot more advanced. They can also produce instant transcripts, for example of what a politician says on TV, and scan them for claims that are worth fact-checking. These tools are also linked to a database of previous fact-checks, and new claims are instantly run through this. So if a claim has already been checked before, it’s found immediately. We aren’t able to do that – not yet, anyway!’
Scanning texts with FactRank
Anyone can view the results of FactRank at FactRank.org. Click on ‘Rank’ in the menu to see the tool’s results. In the dropdown menu under ‘All sources’, choose ‘Twitter’ to see the latest tweets by MP that make a claim that might need fact-checking.