Add PIMO (#2329)

* PIMO (#1726)

* update

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* test binclf curves numpy and numba and fixes

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* correct som docstrings

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* torch interface and tests

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* torch interface and tests

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* constants regrouped in dataclass as class vars

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* result class was unneccesary for per_image_binclf_curve

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* factorize function _get_threshs_minmax_linspace

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* small docs fixes

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* add pimo numpy version and test

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* move validation

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* add `shared_fpr_metric` option

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* add pimo torch functional version and test

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* add torchmetrics interface and test

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* renames and put things in init

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* validate inputs in result objects

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* result objects to from dict and tests

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* add save and load methods to result objects and test

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* refactor validations and minor changes

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* test result objects' properties

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* minor refactors

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* add missing docstrings

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* minore vocabulary fix for consistency

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* add per image scores statistics and test it

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* refactor constants notation

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* add stats tests and test it

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* change the meaning of AUPIMO.num_thresh

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* interface to format pairwise test results

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* improve doc

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* add optional `paths` to result objects and some minor fixes and refactors

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* remove frozen from dataclasses and some done todos

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* review headers

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* doc modifs

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* refactor `score_less_than_thresh` in `_binclf_one_curve_python`

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* correct license comments

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* fix doc

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* numba as extra requirement

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* refactor copyrights from jpcbertoldo

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* remove from __future__ import annotations

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* refactor validations names

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* dedupe file path validation

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* fix tests

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* Add todo

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* refactor enums

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* only logger.warning

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* refactor test imports

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* refactor docs

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* refactor some docs

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* correct pre commit errors

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* remove author tag

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* add thrid party program

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* Update src/anomalib/metrics/per_image/pimo.py

* move HAS_NUMBA

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* remove PIMOSharedFPRMetric

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* make torchmetrics compute avg by dft

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* pre-commit hooks corrections

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* correct numpy.trapezoid

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

---------

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>
Co-authored-by: Samet Akcay <samet.akcay@intel.com>

* 🗑️ Remove numba (#2313)

* remove numba

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* fix pre-commit checks

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* add third-party-programs.txt

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

---------

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* 🗑️ Remove unused methods (#2315)

* remove numba

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* fix pre-commit checks

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* remove all unused methods

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

---------

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* PIMO: Port Numpy → Torch (#2316)

* remove numba

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* fix pre-commit checks

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* remove all unused methods

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* replace numpy with torch

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

---------

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* 🔨Refactor methods across files (#2321)

* remove numba

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* fix pre-commit checks

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* remove all unused methods

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* replace numpy with torch

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* refactor code

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* refactor
move functional inside update
remove path from the metric

* Add changes from comments

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

---------

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* Remove model to model comparison (#2325)

* rename to pimo

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* minor refactor

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* remove model to model comparison

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* fix test

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* PR comments

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* Minor refactor

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

---------

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* PR comments

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* Remove unused enums

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* update doc strings

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* update param names

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* add aupimo basic usage tutorial notebook (#2330)

* add aupimo basic usage tutorial notebook

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* update scipy import

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* add cite us

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* minor

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* modify texts and add illustration

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

* udpate working dir

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

---------

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>

---------

Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com>
Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>
Co-authored-by: Joao P C Bertoldo <24547377+jpcbertoldo@users.noreply.github.com>
Co-authored-by: Samet Akcay <samet.akcay@intel.com>

src/anomalib/data/utils/path.py

@@ -142,13 +142,20 @@ def contains_non_printable_characters(path: str | Path) -> bool:
     return not printable_pattern.match(str(path))
 
 
-def validate_path(path: str | Path, base_dir: str | Path | None = None, should_exist: bool = True) -> Path:
+def validate_path(
+    path: str | Path,
+    base_dir: str | Path | None = None,
+    should_exist: bool = True,
+    extensions: tuple[str, ...] | None = None,
+) -> Path:
     """Validate the path.
 
     Args:
         path (str | Path): Path to validate.
         base_dir (str | Path): Base directory to restrict file access.
         should_exist (bool): If True, do not raise an exception if the path does not exist.
+        extensions (tuple[str, ...] | None): Accepted extensions for the path. An exception is raised if the
+            path does not have one of the accepted extensions. If None, no check is performed. Defaults to None.
 
     Returns:
         Path: Validated path.
@@ -213,6 +220,11 @@ def validate_path(path: str | Path, base_dir: str | Path | None = None, should_e
             msg = f"Read or execute permissions denied for the path: {path}"
             raise PermissionError(msg)
 
+    # Check if the path has one of the accepted extensions
+    if extensions is not None and path.suffix not in extensions:
+        msg = f"Path extension is not accepted. Accepted extensions: {extensions}. Path: {path}"
+        raise ValueError(msg)
+
     return path
 
 

src/anomalib/metrics/__init__.py

@@ -19,6 +19,7 @@
 from .f1_max import F1Max
 from .f1_score import F1Score
 from .min_max import MinMax
+from .pimo import AUPIMO, PIMO
 from .precision_recall_curve import BinaryPrecisionRecallCurve
 from .pro import PRO
 from .threshold import F1AdaptiveThreshold, ManualThreshold
@@ -35,6 +36,8 @@
     "ManualThreshold",
     "MinMax",
     "PRO",
+    "PIMO",
+    "AUPIMO",
 ]
 
 logger = logging.getLogger(__name__)

src/anomalib/metrics/pimo/__init__.py

@@ -0,0 +1,23 @@
+"""Per-Image Metrics."""
+
+# Original Code
+# https://github.com/jpcbertoldo/aupimo
+#
+# Modified
+# Copyright (C) 2024 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+
+from .binary_classification_curve import ThresholdMethod
+from .pimo import AUPIMO, PIMO, AUPIMOResult, PIMOResult
+
+__all__ = [
+    # constants
+    "ThresholdMethod",
+    # result classes
+    "PIMOResult",
+    "AUPIMOResult",
+    # torchmetrics interfaces
+    "PIMO",
+    "AUPIMO",
+    "StatsOutliersPolicy",
+]