You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
While the tool doesn't really care about what the data is about, it actually cares a lot about improvement/regression, but for this, it assumes that a lower score is better.
This assumption is wrong if we feed data like throughput; so it would be a great addition if the tool wouldn't report improvements as regressions and vice versa.
Maybe a command line flag setting a boolean higher-is-better would be a good starting point.
The text was updated successfully, but these errors were encountered:
I think that we have quite a few places where the assumption is that lower is better. A horrible temporary hack is for users to invert their y-axis scales when inputting data. However, we have one additional feature which would defeat even that: because we currently assume that times are in seconds, we have the "times less than 0.001s are irrelevant" tweak, which is clearly going to do unexpected things if the units aren't in seconds.
While the tool doesn't really care about what the data is about, it actually cares a lot about improvement/regression, but for this, it assumes that a lower score is better.
This assumption is wrong if we feed data like throughput; so it would be a great addition if the tool wouldn't report improvements as regressions and vice versa.
Maybe a command line flag setting a boolean
higher-is-better
would be a good starting point.The text was updated successfully, but these errors were encountered: