jill/LLMsForDisinformationAnalysis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update readme. add human score calculation changes

Browse Source
This commit is contained in:
William Jeynes
2026-02-18 21:05:01 +00:00
parent dee9973c2a
commit b70b75bf28
2 changed files with 21 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+12 -3
View File
@@ -9,10 +9,19 @@ Final Dissertation Submission Repository
## Repository Structure ## Repository Structure
``` ```
├── run.sh # Bash script to run project elements from one place
├── data/ # Holder from project data, filled using scripts
├── literature/ ├── literature/
| └── report.pdf # Final submission report | └── report.pdf # Final submission report
├── agent/ # Code for main project pipeline ├── agent/ # Code for main project pipeline
| └── -- todo -- | ├── agent.ts # Graph definition file
└── supporting/ # Code for visualisation tools and more | ├── conditionals/ # Conditional translations
└── -- todo -- | ├── prompts/ # System promps, plus replacement code
| ├── tools/ # Internal and LLM facing tools
| └── utils/ # Logger
└── supporting/
├── dbkf/ # Tool to download claims from DBKF for use in wrapper
├── RAGAS_Service # Small python API to make RAGAS metrics available in the TS projects (required to run pipeline)
├── scorer # Frontend for labelling data, plus associated analysis
└── Wrapper # Bulk run pipeline on pre-downloaded claims
``` ```
supporting/scorer/display.py
+9 -1
View File
@@ -219,12 +219,20 @@ elif view == "Single Claim Random":
for rank_position, idx in enumerate(ordered_indices): for rank_position, idx in enumerate(ordered_indices):
claim_obj = claims[idx] claim_obj = claims[idx]
score = 0
if n == 1: if n == 1:
score = 1.0 score = 1.0
else: else:
score = 1 - (rank_position / (n - 1)) score = 1 - (rank_position / (n - 1))
if (claim_obj["extra_info"] != ""):
print(claim_obj["extra_info"])
if (claim_obj["extra_info"].find("PERFECT") != -1):
score = 1
else:
score *= 0.5
claim_obj["human_score"] = round(score, 3) claim_obj["human_score"] = round(score, 3)
save_data(DATA_FILE, st.session_state.data) save_data(DATA_FILE, st.session_state.data)