Change IR scale factor to 1.0 if tile size is too big

[eugene123tw]

Summary

• Set scale factor to 1.0 if tile size is larger than average image size
• Change adaptive avgpool to avgpool to due ONNX tracing error:

TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. 
We can't record the data flow of Python values, so this value will be treated as a constant in the future. 
This means that the trace might not generalize to other inputs!
  k = [int(size[i] / output_size[i]) for i in range(0, len(size))]

How to test

Checklist

• I have added unit tests to cover my changes.​
• I have added integration tests to cover my changes.​
• I have added e2e tests for validation.
• I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
• I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
• I have linked related issues.

License

• I submit my code changes under the same Apache License that covers the project.
  Feel free to contact the maintainers if that's a concern.
• I have updated the license header for each file (see an example below).

# Copyright (C) 2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

[jaegukhyun]

It seems that this PR contents have changed from set ir_scale_factor to 1 if tile size is large to set default ir_scale_factor as 1.
Could you explain the background of this? @goodsong81 @sungmanc Is it okay to change default ir_scale_factor value?

[sungmanc]

It seems that this PR contents have changed from set ir_scale_factor to 1 if tile size is large to set default ir_scale_factor as 1. Could you explain the background of this? @goodsong81 @sungmanc Is it okay to change default ir_scale_factor value?

I have a concern to change the default value. In the table below, we can easily check the Elapsed CPU time will be larger if we select the IR scale factor as 1. (5.28 -->2.87). Although IR FPS seems different, actually they are same since the number of tiles are different.

[sungmanc]

@eugene123tw , @GalyaZalesskaya what do you think about the table above?

[goodsong81]

From @GalyaZalesskaya's latest experiments, the effect of IR scale factor seems to have decreased after @sovrasov's async optimization, which essentially parallelize tile computation.

So now the ir_scale_factor is

• error prone
• no much speed up
• make ModelAPI hard to generalize on tiling inference

In my opinion, it's OK to set 1 as default. Our, actually my original intention of "spatial concat" for tiles is to compensate the 1-size batch inference, and now async call is doing that.
@GalyaZalesskaya could you attach some summarized perf table here?

[eugene123tw]

From @GalyaZalesskaya's latest experiments, the effect of IR scale factor seems to have decreased after @sovrasov's async optimization, which essentially parallelize tile computation.

So now the ir_scale_factor is

• error prone
• no much speed up
• make ModelAPI hard to generalize on tiling inference

In my opinion, it's OK to set 1 as default. Our, actually my original intention of "spatial concat" for tiles is to compensate the 1-size batch inference, and now async call is doing that. @GalyaZalesskaya could you attach some summarized perf table here?

@sungmanc Songki summarised all the points. There's one point to add, we won't remove this parameter yet as it might have an impact on backward compatibility. And we can always switch it back if something unforeseen happens.

[GalyaZalesskaya]

From @GalyaZalesskaya's latest experiments, the effect of IR scale factor seems to have decreased after @sovrasov's async optimization, which essentially parallelize tile computation.

So now the ir_scale_factor is

• error prone
• no much speed up
• make ModelAPI hard to generalize on tiling inference

In my opinion, it's OK to set 1 as default. Our, actually my original intention of "spatial concat" for tiles is to compensate the 1-size batch inference, and now async call is doing that. @GalyaZalesskaya could you attach some summarized perf table here?

Sure, here are the experiments on small data objects datasets and Vitens-Kiegetal datasets:

Yes, setting ir_scale_factor to 1 will increase the performance time, but for now the tiling accuracy is low and doesn't work well.

[sungmanc]

Thanks for the detailed explanation