first_over_time and last_over_time

[cyriltovena]

This is an attempt to allow users to select the first or last discrete sample values from a range vector.
This way we can create gauge, the only drawback being the log need to exists at least in every range.

May be we could transfer the last or first value to the next range, but it will still be a problem for small ranges.

Still wondering how useful this is compare to min/max_over_time.

WDYT @slim-bean and @chancez you both wanted this feature, I'm still not sure.

[cyriltovena]

@grafana/loki-team WDYT ?

[kavirajk]

May be we could transfer the last or first value to the next range, but it will still be a problem for small ranges.

what kind of problems we talking about for small ranges? you mean in terms of performance?

[cyriltovena]

May be we could transfer the last or first value to the next range, but it will still be a problem for small ranges.

what kind of problems we talking about for small ranges? you mean in terms of performance?

the log need to exists at least in every range

If you want to build some sort of gauge with logs, then you need to have the log exist in every range, you're selecting otherwise you'll have gaps if not no value at all.

[owen-d]

I'm open to it. Part of me wants us to wait until we see real use case requests for this but my suspicion is this will be a helpful addition.

[chancez]

I don't recall having a use-case for this, but perhaps it was within another related topic.

[stale]

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[cyriltovena]

@slim-bean I guess we're going forward with this ? To me it seems to resolve a problem.

[cyriltovena]

Alright the use-case we found is to show a gauge based of a log of stream. Currently it's almost possible with max or min but that doesn't give really the last seen value.

[codecov-io]

Codecov Report

Merging #3050 (1f5c84c) into master (322e4bc) will decrease coverage by 0.03%.
The diff coverage is 66.66%.

@@            Coverage Diff             @@
##           master    #3050      +/-   ##
==========================================
- Coverage   63.25%   63.21%   -0.04%     
==========================================
  Files         188      188              
  Lines       16304    16314      +10     
==========================================
  Hits        10313    10313              
- Misses       5051     5062      +11     
+ Partials      940      939       -1     

Impacted Files	Coverage Δ
pkg/logql/lex.go	91.07% <ø> (ø)
pkg/logql/functions.go	44.20% <60.00%> (+1.23%)	⬆️
pkg/logql/ast.go	88.12% <100.00%> (ø)
pkg/promtail/targets/file/filetarget.go	62.23% <0.00%> (-4.20%)	⬇️
pkg/canary/comparator/comparator.go	76.36% <0.00%> (-1.82%)	⬇️
pkg/logql/evaluator.go	90.23% <0.00%> (+0.35%)	⬆️
pkg/querier/queryrange/downstreamer.go	97.64% <0.00%> (+2.35%)	⬆️

[stale]

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[chancez]

How does this overlap with the feature of the @ modifier in PromQL? Eg: prometheus/prometheus#8425 adds first/last functions for a time range in a range query. I wonder if that concept could be used for something like this.

[stale]

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 66.66667% with 4 lines in your changes missing coverage. Please review.

Project coverage is 63.07%. Comparing base (ecca5d3) to head (203ea92).

Files with missing lines	Patch %	Lines
pkg/logql/functions.go	60.00%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3050      +/-   ##
==========================================
- Coverage   63.14%   63.07%   -0.07%     
==========================================
  Files         206      206              
  Lines       17738    17748      +10     
==========================================
- Hits        11201    11195       -6     
- Misses       5537     5548      +11     
- Partials     1000     1005       +5     

Files with missing lines	Coverage Δ
pkg/logql/ast.go	88.64% <100.00%> (ø)
pkg/logql/lex.go	91.37% <ø> (ø)
pkg/logql/functions.go	45.00% <60.00%> (+1.15%)	⬆️

... and 3 files with indirect coverage changes