Tiling Speed & Accuracy Enhancement

[eugene123tw]

Summary

• Optimization and Bug Fix:

  • Implemented various optimizations in tiling.
  • Addressed a memory cache bug.

• RLE Encoder Integration:

  • Added an RLE encoder to compute RLE directly from torch.Tensor.
  • Reduced processing time by eliminating the need for torch -> numpy -> mask_utils.encode.

• Customized mAP Calculation:

  • Introduced a customized OTXInstSegMeanAveragePrecision.
  • Streamlined mAP calculation directly from RLE, avoiding the slower torch -> numpy -> mask_utils.encode process.

• Tile Merge Enhancement:

  • Modified the tile merge process from Datumaro, transitioning from np array-based merging to direct merging of predictions from torch.Tensor.

• Tile Recipes for Detection and Instance Segmentation:

  • Added various tile recipes to enhance performance in both detection and instance segmentation tasks.

How to test

Checklist

• I have added unit tests to cover my changes.​
• I have added integration tests to cover my changes.​
• I have added e2e tests for validation.
• I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
• I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
• I have linked related issues.

License

• I submit my code changes under the same Apache License that covers the project.
  Feel free to contact the maintainers if that's a concern.
• I have updated the license header for each file (see an example below).

# Copyright (C) 2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

[eugene123tw]

@eugene123tw thanks for your effort in optimizing the tiling pipeline. I have a question related to backward compatibility: perfectly if we can achieve the same metrics when loading a snapshot from 1.x (which would be quite common scenario in Geti), so my question is do we have such backward compatibility (at least up to input preprocessing)? The second point what's already waiting around the corner is compatibility with MAPI. We have to provide it as we hit integration to Geti milestone. Tiler from 1.x was implemented in python/c++ versions, and de-facto became a standard for Geti's inference server based on MediaPipe. For now, I haven't heard about postrpocessing versioning there, which means we have to make 100% compatible tile merging in OTX 2.0 torch implementation.

Hi @sovrasov, appreciate your feedback. Regarding the first question, the straightforward answer is currently not, but I'm working on a new PR to address this gap. To tackle your concern, I'll focus on bridging the inconsistency between 1.5 tiling and 2.x tiling. I plan to conduct a test following your suggestion, which involves acquiring a tiling model, evaluating its performance in 1.5, attempting to load it into 2.x, and verifying if the accuracy aligns.

As for the second question, the objective remains consistent with OTX 1.5. We will implement a regression test in 2.0 to ensure that OV tiling achieves the same accuracy as PyTorch tiling. However, before implementing the test, I'll work on narrowing the gap between versions 1.5 and 2.0.

Currently, there are three tickets associated with this matter, tracked under https://jira.devtools.intel.com/browse/CVS-131551.

[vinnamkim]

LGTM.