Skip to content

Issues: dpaleka/consistency-forecasting

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

34 Open 106 Closed
34 Open 106 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Make a standard evaluation setup for consistency evaluation and ground truth scoring
#278 opened Oct 22, 2024 by dpaleka
@dpaleka
evaluation.py recalc no longer works on stuff that never had any stats to begin with
#270 opened Sep 29, 2024 by abhimanyupallavisudhir
Redundant LFS objects should be removed low-priority
#268 opened Sep 29, 2024 by abhimanyupallavisudhir
@Gusanidas
generated_related_questions should be more consistent with the project's overall structure low-priority
#258 opened Sep 25, 2024 by abhimanyupallavisudhir
violation should have more tests low-priority
#256 opened Sep 25, 2024 by abhimanyupallavisudhir
@abhimanyupallavisudhir
Recompute calibration bins in ground truth eval
#252 opened Sep 24, 2024 by dpaleka
1
Make evaluation.py robust to None violation values
#237 opened Sep 23, 2024 by dpaleka
Approaching Human-Level Forecasting with Language Models and Gnews retrieved articles bug Something isn't working
#235 opened Sep 23, 2024 by dpaleka
@dpaleka
6
Perscache ignore param does not really work enhancement New feature or request
#231 opened Sep 23, 2024 by dpaleka
1
OpenAI's new structured output mode does not work with Pydantic models that include datetime
#219 opened Sep 17, 2024 by dpaleka
by July 2024 in NewsAPI generated qs would be better as during the interval
#213 opened Sep 15, 2024 by dpaleka
@FlightVin
4
Caching is not aware of the LLM client setup
#206 opened Sep 12, 2024 by dpaleka
1
FQ Verification overzealous on instantiated questions not-urgent-for-paper
#199 opened Sep 9, 2024 by abhimanyupallavisudhir
1
Add CAIS' FiveThirtyNine forecaster enhancement New feature or request
#198 opened Sep 9, 2024 by dpaleka
1
Spurious answer and Pydantic BaseSentenceFormat_stripped validate_python calls
#190 opened Sep 3, 2024 by dpaleka
ForecastingQuestion.__str__() should not be non-obvious and used in a nontrivial way in the codebase
#185 opened Sep 1, 2024 by dpaleka
Verification is discarding real questions that are understandable, but maybe less well-formed enhancement New feature or request
#182 opened Aug 29, 2024 by dpaleka
@evan-wang-13
1
Convert structured data events from ForecastBench to our FQ format enhancement New feature or request
#181 opened Aug 29, 2024 by dpaleka
@dpaleka
1
AntecedentChecker and instantiator enhancement New feature or request
#175 opened Aug 25, 2024 by dpaleka
@abhimanyupallavisudhir
1
Improve FQ verification by going through questions that have passed verification at some point, but are bad not-urgent-for-paper
#174 opened Aug 24, 2024 by dpaleka
7 tasks
1
question_created should be allowed to be present in the ground truth dataset, and Forecaster should take it if present
#173 opened Aug 23, 2024 by dpaleka
3 of 6 tasks
@dpaleka
1
Improve FQ verification and corresponding tests not-urgent-for-paper
#171 opened Aug 22, 2024 by dpaleka
FQ IDs not Persisting Through Codebase
#170 opened Aug 21, 2024 by evan-wang-13
1
Describe a standard for ForecastingQuestion format for future papers in both forecasting and evaluating forecasting. paper-material
#168 opened Aug 18, 2024 by dpaleka
FQ verification should reject questions that define something by referring to the "current" state of the world and do not have question_created not-urgent-for-paper
#154 opened Aug 11, 2024 by dpaleka
ProTip! Add no:assignee to see everything that’s not assigned.