Commit Graph

36 Commits

Author SHA1 Message Date
William Jeynes c94812ed80 Prepare for mass data collection. Reduce concurrency as to not overwhelm scraper on long sessions. Remode duplicates from fetch script. Removing naming wierdness on scorer frontend. 2026-02-27 14:41:10 +00:00
William Jeynes 201176e71c Refactor scorer for future maintainabiliy 2026-02-26 10:25:49 +00:00
William Jeynes 6c3aa7343d Update how scoring works with two passes of the data for timesaving. Add section on edge case handling to rules. 2026-02-26 10:09:36 +00:00
William Jeynes 8317fd85df Add file logging for errors. Add exponential backoff retry to web search. On failed web search, do not crash pipeline, return placeholder text to loanguage model 2026-02-24 13:05:35 +00:00
William Jeynes 3d0cacd24e Redo rules a little bit. Update fetch to retreive only from some sources. Add statistics to display, fix rules display 2026-02-23 21:56:27 +00:00
William Jeynes cca3c42f5b Fix longstnading bug in wrapper. Add handling to allow for duplicate events to be handelled. Remove analysis script (will replace with more indepth work in main frontend) 2026-02-22 23:12:14 +00:00
William Jeynes 4d92f14527 Getting hits on the block list IMMEDIATLEY. Log to file, might be important later 2026-02-22 15:42:27 +00:00
William Jeynes 2f33338007 Do not enter existing data if it has no good trigger events 2026-02-22 15:29:48 +00:00
William Jeynes d1ab938c0b Add filtering from known disinformation sources 2026-02-22 15:14:58 +00:00
William Jeynes 8ffe8dec82 Use cleaned trigger events in input.jsonl 2026-02-19 12:23:38 +00:00
William Jeynes 5efce05821 Update REAME to include description of data files 2026-02-19 11:43:25 +00:00
William Jeynes 78a49e2843 Start writing cleaned jsonl output. Re-add sentence to trigger prompt. Fix recursion limit 2026-02-19 11:36:31 +00:00
William Jeynes 6f20ade780 Make open webpage more appealing 2026-02-19 10:07:39 +00:00
William Jeynes b70b75bf28 Update readme. add human score calculation changes 2026-02-18 21:05:01 +00:00
William Jeynes dee9973c2a More work on scorer 2026-02-18 20:44:14 +00:00
William Jeynes a2cb93b44e Start refining scorer. Filter data passed to trigger event agent 2026-02-18 15:03:13 +00:00
William Jeynes 3f14b61cd4 Move all data to own folder. Add run shell script. Experiment (unsuccessfully so far) with example retreival 2026-02-16 22:42:13 +00:00
William Jeynes 90894b2c10 Add some preliminary analysis 2026-02-16 14:42:47 +00:00
William Jeynes 6d478fe7ec Add multi claim runner. Add dbkf fetcher for automated testing. Add visualisation tool plus human score enterer. 2026-02-13 15:15:01 +00:00
William Jeynes fa6e7017b0 Add run scripts 2026-02-12 23:52:15 +00:00
William Jeynes 7fe63d6a98 Refactor calculating score. Add sort node for vanity 2026-02-12 23:46:00 +00:00
William Jeynes b06c08daab Add relation model. Add calculate score initial version 2026-02-12 23:26:59 +00:00
William Jeynes c89f73e138 Implement RAGAS metrics 2026-02-12 22:52:22 +00:00
William Jeynes 6dd6bf7eaf implement verification model 2026-02-12 22:32:24 +00:00
William Jeynes bef856d53a Refactor example retreiving, add option for dynamic data. Add hybrid reranking to tooling. Add parsing and loop infrastructure for trigger event processing 2026-02-12 14:33:12 +00:00
William Jeynes 06a302ec36 remove .keep 2026-02-09 21:46:28 +00:00
William Jeynes adccbd5740 cleanup requirements.txt for ragas service 2026-02-09 21:45:56 +00:00
William Jeynes eba5eb40a2 Add RAGAS initial version 2026-02-09 21:26:54 +00:00
William Jeynes cd2c8621e8 FEAT: implement temp version of main tooling feedback loop 2026-02-09 20:25:36 +00:00
William Jeynes 5841e8a922 add search query 2026-02-09 16:45:17 +00:00
William Jeynes 02eac0f553 Allow multiple source CSV files for normalisation. Implement real model node. Add normalizarion prompt. Implement normalization setup. Start on RAG retreival functions 2026-02-09 16:32:40 +00:00
William Jeynes 8eaa7bfbff Add initial code for retreival ranking for normalisation 2026-01-29 21:53:38 +00:00
William Jeynes a1373da891 create final nodes 2026-01-28 22:03:21 +00:00
William Jeynes c6416622e4 start adding dummy nodes 2026-01-28 21:26:34 +00:00
William Jeynes a3201d17a2 add initial testing 2026-01-27 22:57:49 +00:00
William Jeynes fdf8be2414 Repository Structure 2026-01-27 21:09:33 +00:00