Benchmark Runner for C Benchmarks

[mattchorlian]

Opening this draft PR to track progress of converting all C benchmarks to use the benchmark runner:

• Port current C++ benchmark runner to C
• Big.lf
• Chameneos.lf
• Counting.lf
• PingPong.lf
• ThreadRing.lf
• Throughput.lf
• BoundedBuffer.lf
• CigaretteSmoker.lf
• Dictionary.lf
• LogisticMap.lf
• Philosophers.lf
• SleepingBarber.lf
• SortedLinkList.lf
• Apsp.lf
• FilterBank.lf
• GuidedSearch.lf
• MatMul.lf
• NQueens.lf
• PiPrecision.lf
• RadixSort.lf
• Trapezoidal.lf

[mattchorlian]

I'm noticing that we have BoundedBuffer.lf and ProdCons.lf both here even though they are the same benchmark. I suppose this is (in part) the Savina naming convention causing confusion again. In any case, we should decide which one to keep around before merging this in!

[lhstrh]

This looks great, @mattchorlian and @LDeng0205! I've only requested minor changes. Let's merge this into master ASAP.

[housengw]

Seems like running MatMul.lf with more than 1 thread produces a random validation error. I'm guessing it's a memory issue..

[cmnrd]

This looks great! Looks like we are close to completing the C benchmarks.

Should we also include the Banking Benchmark #624 here and fix it up?

Seems like running MatMul.lf with more than 1 thread produces a random validation error. I'm guessing it's a memory issue.

Could you describe what the issue seems to be? Is it a segfault that occurs sometimes? Or is the result invalid (since you mention a validation error)? The MatMul benchmark has a built-in race condition and hence the result is expected to be invalid for parallel execution. This is a bug in the original Savina benchmark...

[housengw]

the validation error in multithreaded run of MatMul.lf is likely a race condition.

[lhstrh]

the validation error in multithreaded run of MatMul.lf is likely a race condition.

Could you elaborate?

[housengw]

The MatMul benchmark has random validation errors when running with more than 1 thread. Since there is no segfault and the validation error happens at random indices in the matrix, it is most likely a race condition.
As @cmnrd mentioned, the MatMul benchmark has a built-in race condition, so running it with multiple threads would cause such an error.

[lhstrh]

Looks ready to me! This is a big feat and will be very useful for the evaluation of the new schedulers in reactor-c 🚀🚀🚀 Thanks!

[mattchorlian]

Should we also include the Banking Benchmark #624 here and fix it up?

We decided to get this merged first and deal with Banking separately @cmnrd

[lhstrh]

Should we also include the Banking Benchmark #624 here and fix it up?

We decided to get this merged first and deal with Banking separately @cmnrd

Come to thing of it, there is one distinct advantage to adding the other stuff to this PR: we'll have a cleaner history and release notes (if we automatically generate them). Doesn't seem like it would be much work?

[cmnrd]

Looks great! I am happy with both options, merging as is or including banking as well.

[LDeng0205]

I just added benchmark runner to Banking as well so we could include it with this PR. However, I still have a java.lang.OutOfMemoryError when I compile it with NumAccounts = 1000.

[edwardalee]

Is this branch merged with master? It looks like it is still generating in-line code, which suggests it is using a version of master that predates scalability-banks.