Updated demo that converts images to tensors #48
Conversation
| @@ -7,6 +7,7 @@ edition = "2018" | |||
| publish = false | |||
|
|
|||
| [dependencies] | |||
| image = { version = "0.23.14", default-features = false, features = ["gif", "jpeg", "ico", "png", "pnm", "tga", "tiff", "webp", "bmp", "hdr", "dxt", "dds", "farbfeld"] } | |||
There was a problem hiding this comment.
We probably only need to indicate which features get used:
| image = { version = "0.23.14", default-features = false, features = ["gif", "jpeg", "ico", "png", "pnm", "tga", "tiff", "webp", "bmp", "hdr", "dxt", "dds", "farbfeld"] } | |
| image = { version = "0.23.14", default-features = false, features = ["jpeg"] } |
There was a problem hiding this comment.
I don't see any reason not to include other file types? It will decode .png, .gif etc into tensors currently.
There was a problem hiding this comment.
Well, I'm suggesting the reason would be that we don't need to include features that don't get used here. We could add a comment for users taking a look at this library that more decoding features are available, but not compiling unused features might slightly speed up compile time and make it more clear to users that they also only need to include features that they use.
| let filename: String = format!("{}{}{}", "fixture/images/", i, ".bgr"); | ||
| let tensor_data = fs::read(filename).unwrap(); | ||
| for i in 0..5 { | ||
| let filename: String = format!("{}{}{}", "fixture/images/", i, ".jpg"); |
There was a problem hiding this comment.
| let filename: String = format!("{}{}{}", "fixture/images/", i, ".jpg"); | |
| let filename = format!("fixture/images/{}.jpg", i); |
| @@ -83,6 +85,27 @@ fn sort_results(buffer: &[f32]) -> Vec<InferenceResult> { | |||
| results | |||
| } | |||
|
|
|||
| fn image_to_tensor(path: String, height: u32, width: u32) -> Vec<u8> { | |||
| let pixels = Reader::open(path).unwrap().decode().unwrap(); | |||
| let dyn_img: DynamicImage = pixels.resize_exact(width, height, image::imageops::Triangle); | |||
There was a problem hiding this comment.
| let dyn_img: DynamicImage = pixels.resize_exact(width, height, image::imageops::Triangle); | |
| let dyn_img = pixels.resize_exact(width, height, image::imageops::Triangle); |
| for i in 0..raw_u8_arr.len() { | ||
| // Read the number as a f32 and break it into u8 bytes | ||
| let u8_f32: f32 = raw_u8_arr[i] as f32; | ||
| let u8_bytes = u8_f32.to_ne_bytes(); |
There was a problem hiding this comment.
With all of this re-arranging of bytes, I think we should be checking the top classification results in the main function above to make sure the classification is still correct.
There was a problem hiding this comment.
As in comparing to the original pre-tensored images? I could add .bgr versions of each of the .jpg to check that they yield the same result.
There was a problem hiding this comment.
I guess that could work, but I meant that you could figure out the top result for each of the .bgr images, put those in an array, and then in this change make sure that the .jpg images are classified with those results.
| @@ -83,6 +85,27 @@ fn sort_results(buffer: &[f32]) -> Vec<InferenceResult> { | |||
| results | |||
| } | |||
|
|
|||
| fn image_to_tensor(path: String, height: u32, width: u32) -> Vec<u8> { | |||
There was a problem hiding this comment.
This needs some documentation explaining that the file at path is resized to height x width and then the pixel precision is converted to FP32.
Demo now uses the crate 'image' to open and convert images into tensors at runtime, instead of requiring they be converted before the app is run. This is the first step to creating a stand alone library that can be used by any app that imports wasi-nn.
a52cb01 to
43aa2e6
Compare
Demo now uses the crate 'image' to open and convert images
into tensors at runtime, instead of requiring they be converted
before the app is run. This is the first step to creating a stand
alone library that can be used by any app that imports wasi-nn.