Skip to content

Commit

Permalink
fix: Resolve bug in stringify logic for cli object cmds
Browse files Browse the repository at this point in the history
The stringify logic(which may not be needed?) that enables users
to use the object store as a simple key/value string store had a
bug in the logic which checked out large the bytes were before
inserting it into the buffer.

This changes resolves that error and also adds some more documentation
around these commands.
  • Loading branch information
clintjedwards committed Jan 26, 2025
1 parent aa47933 commit df4175d
Show file tree
Hide file tree
Showing 3 changed files with 24 additions and 5 deletions.
1 change: 0 additions & 1 deletion TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ thought into this feature that it can become a game changer for Gofer as a whole

# Small things I want to keep track of that I definitely need to do.

* Revisit making it so that object and secret stores can icrementally upload
* When the user tries to get a run and the status if failed, we should automaticallly print the failed reasonings and
then tell the user how to go about finding more information.
* When the user runs the pipeline config on it's own we should pretty print the json so that it's debuggable easily.
Expand Down
14 changes: 12 additions & 2 deletions gofer/src/cli/pipeline/object.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use crate::cli::Cli;
use anyhow::{bail, Context, Result};
use bytes::{Buf, BufMut};
use bytes::BufMut;
use clap::{Args, Subcommand};
use comfy_table::{Cell, CellAlignment, Color, ContentArrangement};
use futures::StreamExt;
Expand All @@ -26,6 +26,10 @@ pub enum ObjectCommands {
},

/// Read an object from the pipeline store
#[command(after_help = r#"Examples:
gofer pipeline object get simple test > /tmp/gofer_test
gofer pipeline object get simple test --stringify
"#)]
Get {
/// Pipeline Identifier.
id: String,
Expand All @@ -44,6 +48,10 @@ pub enum ObjectCommands {
/// on configuration). Once this limit is reached the _oldest_ object will be removed to make space for the new object.
///
/// You can store both regular text values or read in entire files using the '@' symbol and piping.
#[command(after_help = r#"Examples:
gofer pipeline object put simple test ~/.bin/gofer
echo "hello" > gofer pipeline object put simple test @
"#)]
Put {
/// Pipeline Identifier.
id: String,
Expand Down Expand Up @@ -163,7 +171,9 @@ impl Cli {
while let Some(chunk) = object.next().await {
let chunk = chunk?;

if buffer.remaining() > chunk.len() {
let remaining_space = buffer.capacity() - buffer.len();

if remaining_space > chunk.len() {
buffer.put(chunk);
} else {
bail!("Could not stringify object; object larger than 1KB");
Expand Down
14 changes: 12 additions & 2 deletions gofer/src/cli/run/object.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use crate::cli::Cli;
use anyhow::{bail, Context, Result};
use bytes::{Buf, BufMut};
use bytes::BufMut;
use clap::{Args, Subcommand};
use comfy_table::{Cell, CellAlignment, Color, ContentArrangement};
use futures::StreamExt;
Expand Down Expand Up @@ -29,6 +29,10 @@ pub enum ObjectCommands {
},

/// Read an object from the run store
#[command(after_help = r#"Examples:
gofer run object get simple 1 test > /tmp/gofer_test
gofer run object get simple 1 test --stringify
"#)]
Get {
/// Pipeline Identifier.
pipeline_id: String,
Expand All @@ -53,6 +57,10 @@ pub enum ObjectCommands {
/// Run objects are kept individual to each run and removed after a certain run limit. This means that after a certain
/// amount of runs for a particular pipeline a run's objects will be discarded. The limit of amount of objects you can
/// store per run is of a much higher limit.
#[command(after_help = r#"Examples:
gofer run object put simple 1 test ~/.bin/gofer
echo "hello" > gofer run object put simple 1 test @
"#)]
Put {
/// Pipeline Identifier.
pipeline_id: String,
Expand Down Expand Up @@ -197,7 +205,9 @@ impl Cli {
while let Some(chunk) = object.next().await {
let chunk = chunk?;

if buffer.remaining() > chunk.len() {
let remaining_space = buffer.capacity() - buffer.len();

if remaining_space > chunk.len() {
buffer.put(chunk);
} else {
bail!("Could not stringify object; object larger than 1KB");
Expand Down

0 comments on commit df4175d

Please sign in to comment.