Current Lineage Metrics

[phixMe]

Problem

We currently in our materialized view lineage tables don't provide the current hour and day respectively, which is the timeframe that our materialized view did not get

Solution

• Separate tables for current_hour_lineage_metrics and current_day_lineage_metrics
• Deletes all table data at hourly and daily intervals as part of the cleanup. (Only retains current hour and current day)
• Inserts to simple table on OL events.

One-line summary: Includes current events in the lineage metrics aggregations.

Checklist

• You've signed-off your work
• Your changes are accompanied by tests (if relevant)
• Your change contains a small diff and is self-contained
• You've updated any relevant documentation (if relevant)
• You've included a one-line summary of your change for the CHANGELOG.md (Depending on the change, this may not be necessary).
• You've versioned your .sql database schema migration according to Flyway's naming convention (if relevant)
• You've included a header in any source code files (if relevant)

[netlify]

✅ Deploy Preview for peppy-sprite-186812 canceled.

Name	Link
🔨 Latest commit	6ae44db
🔍 Latest deploy log	https://app.netlify.com/sites/peppy-sprite-186812/deploys/670587b82060760007e29ea5

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 6 lines in your changes missing coverage. Please review.

Project coverage is 82.92%. Comparing base (9450410) to head (6ae44db).
Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
api/src/main/java/marquez/api/StatsResource.java	0.00%	5 Missing ⚠️
...pi/src/main/java/marquez/service/StatsService.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #2900      +/-   ##
============================================
- Coverage     82.97%   82.92%   -0.05%     
  Complexity     1485     1485              
============================================
  Files           264      264              
  Lines          6849     6853       +4     
  Branches        315      316       +1     
============================================
  Hits           5683     5683              
- Misses         1011     1015       +4     
  Partials        155      155              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[wslulciuc]

💯 💯 🥇