Skip to content
This repository has been archived by the owner on Feb 15, 2025. It is now read-only.

(feat) Add Needle in a Haystack Evaluation Infrastructure #944

Closed
jalling97 opened this issue Aug 23, 2024 · 0 comments · Fixed by #945
Closed

(feat) Add Needle in a Haystack Evaluation Infrastructure #944

jalling97 opened this issue Aug 23, 2024 · 0 comments · Fixed by #945
Assignees
@jalling97
Labels
enhancement New feature or request python Pull requests that update Python code
Milestone
Current - RAG UX ...

Comments

@jalling97
Copy link
Contributor

We should add the underlying infrastructure that runs the Needle in a Haystack evaluations that will ultimately be a part of the LFAI RAG Evals framework.

This does not include the code to run it in a pipeline. Just to be able to run the evaluation on a deployed instance of LFAI.
@jalling97 jalling97 self-assigned this Aug 23, 2024
@jalling97 jalling97 added enhancement New feature or request python Pull requests that update Python code labels Aug 23, 2024
@jalling97 jalling97 mentioned this issue Aug 23, 2024
Merged
2 tasks
@jalling97 jalling97 linked a pull request Aug 23, 2024 that will close this issue
feat: add RAG Evals Runner and Needle in a Haystack Evaluation #945
Merged
2 tasks
@jalling97 jalling97 mentioned this issue Sep 3, 2024
Closed
11 tasks
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@jalling97 jalling97

Labels
enhancement New feature or request python Pull requests that update Python code
Projects
None yet
Milestone
Current - RAG UX Enhancements | Model...
Development

Successfully merging a pull request may close this issue.

feat: add RAG Evals Runner and Needle in a Haystack Evaluation defenseunicorns/leapfrogai
1 participant
@jalling97