Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add ContextRelevanceEvaluator to RAG eval harness #15

Merged

Conversation

shadeMe
Copy link
Contributor

@shadeMe shadeMe commented Jun 18, 2024

Proposed Changes:

This PR adds support for using the ContextRelevanceEvaluator component with the RAG evaluation harness.

How did you test it?

Unit tests

Checklist

@shadeMe shadeMe requested a review from a team as a code owner June 18, 2024 12:46
@shadeMe shadeMe requested review from anakin87 and davidsbatista and removed request for a team and anakin87 June 18, 2024 12:46
@shadeMe shadeMe force-pushed the feat/rag-eval-harness-ctx-relevance-metric branch from 6d76d21 to 93013de Compare June 18, 2024 13:00
@coveralls
Copy link

coveralls commented Jun 18, 2024

Pull Request Test Coverage Report for Build 9565496957

Details

  • 98 of 99 (98.99%) changed or added relevant lines in 3 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.2%) to 97.696%

Changes Missing Coverage Covered Lines Changed/Added Lines %
test/evaluation/harness/rag/test_harness.py 95 96 98.96%
Totals Coverage Status
Change from base Build 9564589848: 0.2%
Covered Lines: 721
Relevant Lines: 738

💛 - Coveralls

@shadeMe shadeMe force-pushed the feat/rag-eval-harness-ctx-relevance-metric branch from 93013de to d4553a6 Compare June 18, 2024 13:09
@coveralls
Copy link

coveralls commented Jun 18, 2024

Pull Request Test Coverage Report for Build 9565655224

Details

  • 104 of 105 (99.05%) changed or added relevant lines in 3 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.7%) to 98.118%

Changes Missing Coverage Covered Lines Changed/Added Lines %
test/evaluation/harness/rag/test_harness.py 101 102 99.02%
Totals Coverage Status
Change from base Build 9564589848: 0.7%
Covered Lines: 730
Relevant Lines: 744

💛 - Coveralls

@shadeMe shadeMe force-pushed the feat/rag-eval-harness-ctx-relevance-metric branch from d4553a6 to 0c96477 Compare June 18, 2024 13:20
@coveralls
Copy link

coveralls commented Jun 18, 2024

Pull Request Test Coverage Report for Build 9565831835

Details

  • 99 of 99 (100.0%) changed or added relevant lines in 3 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.8%) to 98.238%

Totals Coverage Status
Change from base Build 9564589848: 0.8%
Covered Lines: 725
Relevant Lines: 738

💛 - Coveralls

@shadeMe shadeMe merged commit e208798 into deepset-ai:main Jun 18, 2024
5 checks passed
@shadeMe shadeMe deleted the feat/rag-eval-harness-ctx-relevance-metric branch June 18, 2024 14:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants