claim-fact-checking

A cli, python based application to fact check a statement by comparing to relevant and recent news.

Installation

Local Environment with Conda

git clone https://github.com/yatshunlee/claim-fact-checking.git
conda env create -f env/environment.yml
conda activate claim-fact-check

Colab Environment

!pip install -U -q transformers
!pip install -U -q accelerate
!pip install -q news-please
!pip install -q google
!pip install -U -q sentence-transformers

Usage with OpenAI GPT 3.5

1. replace with your valid openai api key in fraud_detection_gpt_3_5.py
2. run python main_gpt_3_5.py

Usage with Google Gemma-2b-it

1. replace with your valid huggingface access token in fraud_detection_gemma.py
2. run python main_gemma.py

Architecture

1. Retrieve a statement / query that the user wants to fact-check.
2. Make it a better google search query: sites, keywords, time range.
3. Perform webscrapping to get relevant news content about the query: Google Search, NewsPlease.
4. Utilize text entailment to keep top relevent news by their titles. Then conduct sentence embedding and KNN to get the most relevant sentences in the news.
5. Design the prompt by retrieval augmented generation (RAG).
6. Fit into the GPT 3.5 model to generate a response.

Fact Checker

Check statement: Joe Biden told people not to vote

GPT 3.5

Gemma-2b-it

Check statement: Taylor Swift Promoting Le Creuset Cookware

GPT 3.5

Gemma-2b-it