CI: Simplifying benchmarks call

[datapythonista]

Just removing the two files used for calling the benchmarks, and implementing what they do directly in the github actions config.

[jreback]

i don't think we should do this just in the github action, rather call the script as otherwise its very hard to debug things.

[datapythonista]

i don't think we should do this just in the github action, rather call the script as otherwise its very hard to debug things.

My experience is exactly the opposite. There were some problems with the benchmarks in one PR, and it was quite complicated to understand what we were executing, with 3 files involved. I think this makes it very clear what it's being executed, that's why I made that change.

Not sure what you mean by debug. The only advantage of having a separate script (I don't see any advantage for having two scripts as we do now) would be if we wanted to run the script locally. But if we wanted to do that (I don't think we will in practice), locally just asv is needed.

Can you expand on what is hard to debug with this change for you?

[jreback]

Not sure what you mean by debug. The only advantage of having a separate script (I don't see any advantage for having two scripts as we do now) would be if we wanted to run the script locally. But if we wanted to do that (I don't think we will in practice), locally just asv is needed.

so how does one debug the errors on the benchmarks now? it is totally non-obvious how to run a github action (impossible in fact) locally.

if its 'just' asv.....then that should be the github action. is this the case?

you have to always be able to repro locally, the same way as the CI otherwise debugging is impossible.

[jreback]

kk thanks @datapythonista this looks good. ideally should have a small section in the docs about this, but i don't think really a big deal.

[datapythonista]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).