-
Notifications
You must be signed in to change notification settings - Fork 138
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Full experiment with the new C-specific prompt #370
Conversation
@DavidKorczynski: Are there any other changes required to use the C-specific prompt? |
/gcbrun exp -n dg |
No
Yeah, I don't think it makes too much sense to try on C++ unless we're testing that it doesn't break the workflow or so -- I wouldn't expect the generated prompts to generate good C++ harnesses. We can use these benchmarks for pure C projects only: #371 |
Adds a set of benchmarks that have been extracted for the projects - clib - htslib - croaring - kamailio - opensips - unit - ntpsec - bind9 - libyang - miniz - mdbtools - libiec61850 and using the oracles - far-reach-low-coverage - low-cov-with-fuzz-keyword - easy-params-far-reach Ref: #370 Signed-off-by: David Korczynski <david@adalogics.com>
4339083
to
9add280
Compare
/gcbrun exp -n dk-9000 -b c-specific |
ded4eb8
to
6dde591
Compare
The experiment report with c-benchmarks only:
We are trying a different solution to this. Missing function implementationHere is an example. @DavidKorczynski will you address these? |
LLM |
Outdated. |
Running a full experiment with the new C-specific prompt.