Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[APP-6612] Add Filters for Downloading Logs #4673

Merged
merged 31 commits into from
Jan 15, 2025
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
386db8c
Support downloading logs
JosephBorodach Dec 31, 2024
232f143
Restructure RobotsLogsAction to extend functionality for outputing lo…
JosephBorodach Jan 2, 2025
6a02466
Refactor saveLogsToDisk to handle large log files efficiently
JosephBorodach Jan 2, 2025
e14e9be
Restructure fetchAndSaveLogs to write logs to the file while retrievi…
JosephBorodach Jan 2, 2025
761d6fe
Add nolint for not checking the error returned when closing the file
JosephBorodach Jan 2, 2025
05dc2c2
add headers to log output for file downloads
JosephBorodach Jan 2, 2025
921a908
Add constant variable for FormatJSON to appease linter. Add error che…
JosephBorodach Jan 2, 2025
690b887
Make the json format consecutive json object instead of one large jso…
JosephBorodach Jan 3, 2025
e2786a5
Make error handling more comprehensive for requiring Output and Forma…
JosephBorodach Jan 3, 2025
225b5cb
Make fetchAndSaveLogs, streamLogsForPart and printLogsToConsole metho…
JosephBorodach Jan 3, 2025
b0698af
Add filters including keyword, levels, and start and end times to the…
JosephBorodach Jan 3, 2025
b3e00d3
Merge remote-tracking branch 'upstream/main' into APP-6612-Add-Filter…
JosephBorodach Jan 8, 2025
bf0df5f
Replace dataFlagStart, dataFlagEnd, logsFlagStartTime and logsFlagEnd…
JosephBorodach Jan 8, 2025
7abd091
Add logic to streamLogsForPart for the filters
JosephBorodach Jan 8, 2025
0c6c6f9
Remove validation checks
JosephBorodach Jan 8, 2025
b292477
Remove constant variables for formatJSON and formatText
JosephBorodach Jan 8, 2025
0d65512
Correct cli argument types to strings from timestamps
JosephBorodach Jan 8, 2025
791cda4
Fix start issue
JosephBorodach Jan 13, 2025
774614b
Add parseTimeString helper method and refactor streamLogsForPart and …
JosephBorodach Jan 13, 2025
6a8138e
Remove logsFlagErrors flag from logs because it is redundant since th…
JosephBorodach Jan 13, 2025
410bad0
Permit the combination of start + count + end in RobotsLogsAction
JosephBorodach Jan 13, 2025
f281e27
Make error for 'args.Start != && args.Count > 0 && args.End == ' in …
JosephBorodach Jan 13, 2025
f35b373
Merge branch 'main' into APP-6612-Add-Filters-For-Downloading-Logs
JosephBorodach Jan 13, 2025
575f98e
Remove logsFlagErrors flag from logs because it is redundant since th…
JosephBorodach Jan 13, 2025
c164165
Fix bug to allow for more than 100 logs to be retrieved from streamLo…
JosephBorodach Jan 13, 2025
2f273e2
Fix useless print with f usage found Errorf
JosephBorodach Jan 14, 2025
ef5db1a
Fix parseTimeString - timeLayout always receives same timelayout
JosephBorodach Jan 14, 2025
b8a5d77
Cleanup streamLogsForPart logic
JosephBorodach Jan 14, 2025
21d27c6
Remove redundant comments from streamLogsForPart and add clarifying c…
JosephBorodach Jan 15, 2025
8a30121
Add examples for using the start and end time options for the logs co…
JosephBorodach Jan 15, 2025
0c77e33
Add todo comments to RobotsLogsAction
JosephBorodach Jan 15, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions cli/app.go
Original file line number Diff line number Diff line change
Expand Up @@ -1805,11 +1805,11 @@ var app = &cli.App{
},
&cli.StringFlag{
Name: generalFlagStart,
Usage: "ISO-8601 timestamp in RFC3339 format indicating the start of the interval filter",
Usage: "ISO-8601 timestamp in RFC3339 format indicating the start of the interval filter (e.g., 2025-01-15T14:00:00Z)",
},
&cli.StringFlag{
Name: generalFlagEnd,
Usage: "ISO-8601 timestamp in RFC3339 format indicating the end of the interval filter",
Usage: "ISO-8601 timestamp in RFC3339 format indicating the end of the interval filter (e.g., 2025-01-15T15:00:00Z)",
},
&cli.IntFlag{
Name: logsFlagCount,
Expand Down
7 changes: 5 additions & 2 deletions cli/client.go
Copy link
Contributor Author

@JosephBorodach JosephBorodach Jan 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jr22 - Is there a reason we don't have unit tests for this function's helpers? Should I add unit tests?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not a reason that im aware of, feel free to add test coverage if you want!

Original file line number Diff line number Diff line change
Expand Up @@ -852,6 +852,11 @@ func (c *viamClient) streamLogsForPart(part *apppb.RobotPart, args robotsLogsArg

// Fetch logs in batches and write them to the output.
for fetchedLogCount := 0; fetchedLogCount < maxLogsToFetch; {
// We do not request the exact limit specified by the user in the `count` argument because the API enforces a maximum
// limit of 100 logs per batch fetch. To keep the RDK independent of specific limits imposed by the app API,
// we always request the next full batch of logs as allowed by the API (currently 100). This approach
// ensures that if the API limit changes in the future, only the app API logic needs to be updated without requiring
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏼 much better

// changes in the RDK.
resp, err := c.client.GetRobotPartLogs(c.c.Context, &apppb.GetRobotPartLogsRequest{
Id: part.Id,
Filter: keyword,
Expand Down Expand Up @@ -882,7 +887,6 @@ func (c *viamClient) streamLogsForPart(part *apppb.RobotPart, args robotsLogsArg
resp.Logs = resp.Logs[:remainingLogsNeeded]
}

// Write the logs to the output
for _, log := range resp.Logs {
formattedLog, err := formatLog(log, part.Name, args.Format)
if err != nil {
Expand All @@ -894,7 +898,6 @@ func (c *viamClient) streamLogsForPart(part *apppb.RobotPart, args robotsLogsArg
}
}

// Increment the total number of logs fetched so far by the count of logs retrieved in this batch.
fetchedLogCount += len(resp.Logs)

// End of pagination if there is no next page token.
Expand Down
Loading