 William Jeynes
|
2252a42466
|
Add database link to README
|
2026-04-09 15:46:18 +01:00 |
|
|
William Jeynes
|
75ca1032a6
|
Add offset and limit in pereparation for the large dataset
|
2026-04-05 22:47:25 +01:00 |
|
|
William Jeynes
|
00d129bd28
|
add % valid URLs for different model
|
2026-04-05 12:31:09 +01:00 |
|
|
William Jeynes
|
cf923d6e87
|
Add new accuracy results
|
2026-04-05 11:51:28 +01:00 |
|
|
William Jeynes
|
f821e9643d
|
Add url validity metrics
|
2026-04-04 20:02:25 +01:00 |
|
|
William Jeynes
|
43ecd04135
|
add multithreading
|
2026-04-04 19:42:02 +01:00 |
|
|
William Jeynes
|
8c0921057b
|
start on work to calculate % if valid URLS
|
2026-04-04 18:52:47 +01:00 |
|
|
William Jeynes
|
b610e8c989
|
Add sentence transformers to requirements for ensemble service
|
2026-03-31 15:52:14 +01:00 |
|
|
William Jeynes
|
f8d4155b7c
|
Add more robust parsing of LLM JSON output
|
2026-03-27 11:09:59 +00:00 |
|
|
William Jeynes
|
5e374a8bd6
|
Fix errors seen during longer runs: selenium exceptions, insecure certificates, recusrsion limit exceeded, BM25 document corpus too small
|
2026-03-26 12:22:13 +00:00 |
|
|
William Jeynes
|
fbc688b8f9
|
add date to returned data
|
2026-03-25 22:37:14 +00:00 |
|
|
William Jeynes
|
77cdd9a01c
|
Add statistics for model experiments. Fix dead link in documentation
|
2026-03-25 21:57:52 +00:00 |
|
|
William Jeynes
|
a7f5978f64
|
Update documentation. Stop storing context. Decide on final claims source
|
2026-03-25 14:24:55 +00:00 |
|
|
William Jeynes
|
872346c657
|
Update run.sh to match new evaluation service
|
2026-03-24 19:16:48 +00:00 |
|
|
William Jeynes
|
8f939d54c4
|
Implement ensemble into final model structure
|
2026-03-24 19:07:24 +00:00 |
|
|
William Jeynes
|
624d45bc53
|
Re-allow multithreading on service. Add results table
|
2026-03-24 18:29:40 +00:00 |
|
|
William Jeynes
|
80bc151379
|
add majority voting
|
2026-03-24 16:50:41 +00:00 |
|
|
William Jeynes
|
5ce64290ce
|
Make an ensemble model to combine scores together (very high accuracy)
|
2026-03-24 15:50:41 +00:00 |
|
|
William Jeynes
|
87fccb7e2b
|
Add downloading from hugging face
|
2026-03-24 13:23:08 +00:00 |
|
|
William Jeynes
|
8c1e35f66f
|
Increase dropout on regression model to cut down on overfitting
|
2026-03-24 13:16:18 +00:00 |
|
|
William Jeynes
|
44395bb251
|
add linear regression model initial version
|
2026-03-24 12:25:15 +00:00 |
|
|
William Jeynes
|
e368c50577
|
Add training scripts for distilled, flan. Add run service for flan
|
2026-03-23 22:43:59 +00:00 |
|
|
William Jeynes
|
c69730df6b
|
Refine scoring to allow for better iteration on frontend. Update generate_adversarial.py
|
2026-03-22 16:04:38 +00:00 |
|
|
William Jeynes
|
f4e84af272
|
Make the model less overfitting. Make it harder for an event to be classed as "perfect"
|
2026-03-18 01:05:24 +00:00 |
|
|
William Jeynes
|
886b9a7d5d
|
Ensire works on CUDA for extra speed
|
2026-03-17 23:14:50 +00:00 |
|
|
William Jeynes
|
8052d5c7ba
|
Working on making the classifier harsher on unseen data
|
2026-03-17 22:19:03 +00:00 |
|
|
William Jeynes
|
b08c1ada70
|
Small changes for the next set of human ranking
|
2026-03-17 00:18:32 +00:00 |
|
|
William Jeynes
|
c89c7054fe
|
Update agent to support new verification style. Update frontend to support new file format and remove redundant logic from old experiments.
|
2026-03-16 17:16:58 +00:00 |
|
|
William Jeynes
|
0a7bb114d2
|
Add removing of duplicates from pipeline. Add to sort step. Move score logic to robertaMetrics node.
|
2026-03-13 14:51:14 +00:00 |
|
|
William Jeynes
|
d5c6cb444d
|
Add better scoring, ignoring duplicates, catching under and over confidence. Showing difference between "FINE" and "PERFECT"
|
2026-03-13 12:18:52 +00:00 |
|
|
William Jeynes
|
8311556855
|
Add ROBERTA classifier ranking PoC, with 77pc off the bat
|
2026-03-13 11:24:51 +00:00 |
|
|
William Jeynes
|
f09e36e740
|
Add initial version of ROBERTA classifier, add ability for multi pi charts
|
2026-03-11 22:02:31 +00:00 |
|
|
William Jeynes
|
ef6330ec07
|
Add re-ranker mode to support re-ranking experiments, hopefully we can reduce the loss
|
2026-03-06 17:27:09 +00:00 |
|
|
William Jeynes
|
f14d112017
|
Add difference between auto scoring system and our own labels
|
2026-03-03 15:58:39 +00:00 |
|
|
William Jeynes
|
6ae551a93f
|
Ensure date is passed to pipeline. Fix woring
|
2026-03-02 14:58:26 +00:00 |
|
|
William Jeynes
|
c94812ed80
|
Prepare for mass data collection. Reduce concurrency as to not overwhelm scraper on long sessions. Remode duplicates from fetch script. Removing naming wierdness on scorer frontend.
|
2026-02-27 14:41:10 +00:00 |
|
|
William Jeynes
|
201176e71c
|
Refactor scorer for future maintainabiliy
|
2026-02-26 10:25:49 +00:00 |
|
|
William Jeynes
|
6c3aa7343d
|
Update how scoring works with two passes of the data for timesaving. Add section on edge case handling to rules.
|
2026-02-26 10:09:36 +00:00 |
|
|
William Jeynes
|
8317fd85df
|
Add file logging for errors. Add exponential backoff retry to web search. On failed web search, do not crash pipeline, return placeholder text to loanguage model
|
2026-02-24 13:05:35 +00:00 |
|
|
William Jeynes
|
3d0cacd24e
|
Redo rules a little bit. Update fetch to retreive only from some sources. Add statistics to display, fix rules display
|
2026-02-23 21:56:27 +00:00 |
|
|
William Jeynes
|
cca3c42f5b
|
Fix longstnading bug in wrapper. Add handling to allow for duplicate events to be handelled. Remove analysis script (will replace with more indepth work in main frontend)
|
2026-02-22 23:12:14 +00:00 |
|
|
William Jeynes
|
4d92f14527
|
Getting hits on the block list IMMEDIATLEY. Log to file, might be important later
|
2026-02-22 15:42:27 +00:00 |
|
|
William Jeynes
|
2f33338007
|
Do not enter existing data if it has no good trigger events
|
2026-02-22 15:29:48 +00:00 |
|
|
William Jeynes
|
d1ab938c0b
|
Add filtering from known disinformation sources
|
2026-02-22 15:14:58 +00:00 |
|
|
William Jeynes
|
8ffe8dec82
|
Use cleaned trigger events in input.jsonl
|
2026-02-19 12:23:38 +00:00 |
|
|
William Jeynes
|
5efce05821
|
Update REAME to include description of data files
|
2026-02-19 11:43:25 +00:00 |
|
|
William Jeynes
|
78a49e2843
|
Start writing cleaned jsonl output. Re-add sentence to trigger prompt. Fix recursion limit
|
2026-02-19 11:36:31 +00:00 |
|
|
William Jeynes
|
6f20ade780
|
Make open webpage more appealing
|
2026-02-19 10:07:39 +00:00 |
|
|
William Jeynes
|
b70b75bf28
|
Update readme. add human score calculation changes
|
2026-02-18 21:05:01 +00:00 |
|
|
William Jeynes
|
dee9973c2a
|
More work on scorer
|
2026-02-18 20:44:14 +00:00 |
|