Main interface implementation and first benchmark problems

[BatyLeo]

This PR is an attempt at a common interface for all Benchmarks:

• AbstractBenchmark abstract type
• A benchmark is generator for everything needed to build a pipeline
• Dedicated DataSample struct for staoring sample
• generate_dataset allows generating a dataset (of type Vector{<:DataSample}) for training/testing of requested size
• generate_maximizer generates a callable optimizer function
• generate_statistical_model generates an untrained Flux model specific for the considered benchmark
• plot_data can visualize isnatnces/solutions, depending on the given inputs
• compute_gap computes the performance metric of given pipeline on given dataset

And some implemented examples;

• WarcraftBenchmark: path finding on image maps
• PortfolioOptimizationBenchmark: portfolio optimization with uncertain prices
• FixedSizeShortestPathBenchmark: shortest path on a square graph of fixed size
• SubsetSelectionBenchmark: minimalist pipeline with top-k layer

Questions:

• Do we need a pipeline struct?
• Is the interface generic enough?
• What tests can we perform beside training a pipeline?

Note: we should rename the package as DecisionFocusedLearningBenchmarks, and also rename all names mentioning InferOpt

[BatyLeo]

@gdalle JET doesn't like my code, do you know what's wrong? I don't understand its error message.

[gdalle]

I think the key line is this one:

no matching method found `rand(::Random.MersenneTwister, ::Type{Float32}, ::Distributions.Uniform{Float64}, ::Int64)`

You probably cannot force the sampling type of a Distribution

[BatyLeo]

You're right, I need to specify the type in the distribution constructor, not in the sampling. Thanks!

[codecov]

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

ℹ️ You can also turn on project coverage checks and project coverage reporting on Pull Request comment

Thanks for integrating Codecov - We've got you covered ☂️

[gdalle]

incroyable !!!!

[BatyLeo]

Just came back from holidays, starting a fresh PR!