🚀Add SuperSimpleNet model #2428

blaz-r · 2024-11-21T08:52:13Z

📝 Description

This PR adds the SuperSimpleNet model implementation.
I also verified the performance on MVTec AD and the results are very close tho the ones reported in the paper.

✨ Changes

Select what type of change your PR is:

🐞 Bug fix (non-breaking change which fixes an issue)
🔨 Refactor (non-breaking change which refactors the code base)
🚀 New feature (non-breaking change which adds functionality)
💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
📚 Documentation update
🔒 Security update

✅ Checklist

Before you submit your pull request, please make sure you have completed the following steps:

📋 I have summarized my changes in the CHANGELOG and followed the guidelines for my type of change (skip for minor changes, documentation updates, and test enhancements).
📚 I have made the necessary updates to the documentation (if applicable).
🧪 I have written tests that support my changes and prove that my fix is effective or my feature works (if applicable).

For more information about code review checklists, see the Code Review Checklist.

Signed-off-by: Blaz Rolih <blaz.rolih@gmail.com> Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Signed-off-by: blaz-r <blaz.rolih@gmail.com> Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Signed-off-by: Blaz Rolih <blaz.rolih@gmail.com> Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Signed-off-by: blaz-r <blaz.rolih@gmail.com> Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

samet-akcay · 2024-11-21T08:59:44Z

This is amazing @blaz-r! Thanks a lot! I'll review it shortly

Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

blaz-r · 2024-11-21T09:48:01Z

I have fixed the linter error and formated the files. The tests for SuperSimpleNet are still failing on onnx and openvino export locally. I'm not entirely sure how to fix this, but I'll take a lot when I get the time.

blaz-r · 2024-11-21T14:14:08Z

Tests are failing on imgaug, somethign with numpy version incompatiblity it seems.

samet-akcay · 2024-11-22T11:43:17Z

@blaz-r, we will need to remove imgaug dependency to bump the numpy version.

samet-akcay · 2024-11-22T12:06:31Z

src/anomalib/models/image/supersimplenet/README.md

+directly predicts the anomaly map and score. The predicted anomaly map is upscaled to match the input image size
+and refined with a Gaussian filter.
+
+This implementation supports both unsupervised and supervised setting, but Anomalib currently supports only unsupervised learning.


What are the missing points in Anomalib to support supervised setting

Right now I believe there are no standard supervised datasets. Another problem is the Folder dataset as it assumes that abnormal samples are always in test set:

anomalib/src/anomalib/data/image/folder.py

Lines 174 to 175 in bcc0b43

samples.loc[(samples.label == DirType.NORMAL), "split"] = Split.TRAIN

samples.loc[(samples.label == DirType.ABNORMAL) | (samples.label == DirType.NORMAL_TEST), "split"] = Split.TEST

Another thing for full reproduction of SuperSimpleNet results is the fixed flipping augmentation and frequency sampling. This is however not necessary, but needed for best results. It's also not SuperSimpleNet specific, so might be worth considering if other supervised model will be supported.

yeah, we would like to diversify the model pool, and include more learning types than one-class models. Thanks for the feedback.
@abc-125, you might want to be aware of this discussion as you have recently worked on this stuff

Thank you for adding me, it would be great to have supervised models and datasets in Anomalib. Recently, I looked at how to add a supervised dataset, and it certainly would require changing some base structures, such as paths to folders (I guess we will need abnormal_train_dir and maybe renaming the rest to make it easier to understand, normal_train_dir, etc.):

anomalib/src/anomalib/data/image/folder.py

Lines 195 to 202 in bcc0b43

normal_dir (str | Path | Sequence): Path to the directory containing normal images.

root (str | Path | None): Root folder of the dataset.

Defaults to ``None``.

abnormal_dir (str | Path | Sequence | None, optional): Path to the directory containing abnormal images.

Defaults to ``None``.

normal_test_dir (str | Path | Sequence | None, optional): Path to the directory containing

normal images for the test dataset.

Defaults to ``None``.

samet-akcay · 2024-11-25T16:48:44Z

@blaz-r, I've created #2436 to remove imgaug stuff and write our own perlin noise generation stuff. Would it be an idea to try this in supersimplenet ?

samet-akcay · 2024-12-02T11:47:27Z

@blaz-r can you pull the latest changes again? The tests should pass after that

blaz-r · 2024-12-02T16:58:36Z

Okay, I am currently at the ICPR. I will try to sort this out when I get some time.

samet-akcay · 2024-12-02T18:02:47Z

Okay, I am currently at the ICPR. I will try to sort this out when I get some time.

ah nice, enjoy the conference!

…ature/supersimplenet # Conflicts: # src/anomalib/models/components/feature_extractors/torchfx.py

Signed-off-by: blaz-r <blaz.rolih@gmail.com>

blaz-r · 2024-12-23T23:31:23Z

This is now using the update perlin noise code and includes the latest v2 changes.

Signed-off-by: blaz-r <blaz.rolih@gmail.com>

…ature/supersimplenet

blaz-r · 2025-01-03T19:47:24Z

Can you run the checks for this please @djdameln @samet-akcay

codecov · 2025-01-03T23:58:18Z

Codecov Report

Attention: Patch coverage is 97.81659% with 5 lines in your changes missing coverage. Please review.

Project coverage is 78.93%. Comparing base (8a82dc7) to head (abc71ac).
Report is 3 commits behind head on release/v2.0.0.

Files with missing lines	Patch %	Lines
...nomalib/models/image/supersimplenet/torch_model.py	97.32%	3 Missing ⚠️
...lib/models/image/supersimplenet/lightning_model.py	95.91%	2 Missing ⚠️

Additional details and impacted files

@@                Coverage Diff                 @@
##           release/v2.0.0    #2428      +/-   ##
==================================================
+ Coverage           78.53%   78.93%   +0.39%     
==================================================
  Files                 303      311       +8     
  Lines               12934    13197     +263     
==================================================
+ Hits                10158    10417     +259     
- Misses               2776     2780       +4

Flag	Coverage Δ
integration_py3.10	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

samet-akcay · 2025-01-04T08:04:50Z

Can you run the checks for this please @djdameln @samet-akcay

@blaz-r, the tests seem to pass. I'll give another review and we could merge. Thanks for this great contribution!

samet-akcay

I only have a single comment...

Anomalib's model naming convention is usually snake-case to camel case for model directory vs model class implementation. If this is followed, supersimplenet would be super_simple_net as the class name is SuperSimpleNet. Which one do you think is better in this case?

blaz-r · 2025-01-04T10:17:02Z

I think that supersimplenet would be better, since this is a single word. So I will rename the class to Supersimplenet.

Signed-off-by: blaz-r <blaz.rolih@gmail.com>

blaz-r · 2025-01-04T11:40:16Z

I have now renamed the class to follow the naming convention.

djdameln

Very nice and clean implementation, thanks a lot Blaž!

I only have some minor comments, mainly some naming issues.

djdameln · 2025-01-06T13:09:36Z

src/anomalib/models/image/supersimplenet/lightning_model.py

+        optim = AdamW(
+            [
+                {
+                    "params": self.model.adaptor.parameters(),
+                    "lr": 0.0001,
+                },
+                {
+                    "params": self.model.segdec.parameters(),
+                    "lr": 0.0002,
+                    "weight_decay": 0.00001,
+                },
+            ],
+        )
+        sched = MultiStepLR(
+            optim,
+            milestones=[int(self.trainer.max_epochs * 0.8), int(self.trainer.max_epochs * 0.9)],
+            gamma=0.4,
+        )
+        return [optim], [sched]


We prefer to use full variables names, i.e. optimizer and scheduler

I update this and some other names as well.

djdameln · 2025-01-06T13:11:36Z

src/anomalib/models/image/supersimplenet/lightning_model.py

+    @property
+    def learning_type(self) -> LearningType:
+        """Return the learning type of the model.
+
+        Returns:
+            LearningType: Learning type of the model.
+        """
+        return LearningType.ONE_CLASS


I guess this would change when we add support for the supervised mode to the overall Anomalib pipeline. Maybe you could add a comment about this, as it might confuse users. Just a brief mention in the docstring would be sufficient.

I added a comment clarifying this.

djdameln · 2025-01-06T13:16:58Z

src/anomalib/models/image/supersimplenet/torch_model.py

+def init_weights(m: nn.Module) -> None:
+    """Init weight of the model.
+
+    Args:
+        m (nn.Module): torch module.
+    """
+    if isinstance(m, nn.Linear | nn.Conv2d):
+        nn.init.xavier_normal_(m.weight)
+    elif isinstance(m, nn.BatchNorm1d | nn.BatchNorm2d):
+        nn.init.constant_(m.weight, 1)


Again, I would prefer a full variable name here for readability.

Suggested change

def init_weights(m: nn.Module) -> None:

"""Init weight of the model.

Args:

m (nn.Module): torch module.

"""

if isinstance(m, nn.Linear | nn.Conv2d):

nn.init.xavier_normal_(m.weight)

elif isinstance(m, nn.BatchNorm1d | nn.BatchNorm2d):

nn.init.constant_(m.weight, 1)

def init_weights(module: nn.Module) -> None:

"""Init weight of the model.

Args:

module (nn.Module): torch module.

"""

if isinstance(module, nn.Linear | nn.Conv2d):

nn.init.xavier_normal_(module.weight)

elif isinstance(module, nn.BatchNorm1d | nn.BatchNorm2d):

nn.init.constant_(module.weight, 1)

This is now changed.

djdameln · 2025-01-06T13:21:57Z

src/anomalib/models/image/supersimplenet/torch_model.py

+        return sum(feature.shape[1] for feature in features.values())
+
+
+class FeatureAdaptor(nn.Module):


I would prefer FeatureAdapter, since we try to use American English throughout the code base. but I guess it would cause a discrepancy with the official implementation in your own repo.

@samet-akcay @ashwinvaidya17 thoughts?

as someone based in the UK, not sure what to say here 😁

I changed this to FeatureAdapter, I think it's clear enough when compared with the official code 😄

djdameln · 2025-01-06T13:25:55Z

src/anomalib/models/image/supersimplenet/anomaly_generator.py

+from anomalib.data.utils.generators import generate_perlin_noise
+
+
+class SSNAnomalyGenerator(nn.Module):


Not sure about the naming here. Why not simply use AnomalyGenerator? You don't use the SSN prefix for the other submodules.

This is now just AnomalyGenerator.

src/anomalib/models/image/supersimplenet/torch_model.py

Signed-off-by: blaz-r <blaz.rolih@gmail.com>

blaz-r · 2025-01-06T21:43:01Z

Thank you for the review. The comments are now addressed. Let me know if anything else needs to be sorted.

blaz-r · 2025-01-07T08:35:30Z

Seems like all checks pass but one that is failing due to some dead links. I think this is unrelated to my PR, or did I break something in docs?

samet-akcay · 2025-01-07T08:57:36Z

Seems like all checks pass but one that is failing due to some dead links. I think this is unrelated to my PR, or did I break something in docs?

no worries, it is related to some new documentation URL, which is not on the readthedocs yet

djdameln

Thanks for the awesome work!

blaz-r requested review from ashwinvaidya17 and samet-akcay as code owners November 21, 2024 08:52

blaz-r and others added 19 commits November 21, 2024 09:53

Initial code structure

90625bb

Signed-off-by: Blaz Rolih <blaz.rolih@gmail.com> Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Add feature extraction

4d5e480

Signed-off-by: Blaz Rolih <blaz.rolih@gmail.com> Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Add adaptor and segdec

742e0ca

Signed-off-by: blaz-r <blaz.rolih@gmail.com> Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Format files

fd95793

Signed-off-by: blaz-r <blaz.rolih@gmail.com> Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Add anomaly generation

26e6f5a

Signed-off-by: blaz-r <blaz.rolih@gmail.com> Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Add configurable params and docstrings

eeddb15

Signed-off-by: blaz-r <blaz.rolih@gmail.com> Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Add dosctring and fix lint issue

090120d

Signed-off-by: blaz-r <blaz.rolih@gmail.com> Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Implement loss and training step

2be15bd

Signed-off-by: blaz-r <blaz.rolih@gmail.com> Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Update torchFX to also support str weights

f986c8e

Signed-off-by: blaz-r <blaz.rolih@gmail.com> Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Update names

e08366d

Signed-off-by: blaz-r <blaz.rolih@gmail.com> Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Add supervision based param settings and validation step

e45b6d4

Signed-off-by: blaz-r <blaz.rolih@gmail.com> Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Add optimizer configs

8768607

Signed-off-by: blaz-r <blaz.rolih@gmail.com> Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Fix loss and types

0e5a6c7

Signed-off-by: blaz-r <blaz.rolih@gmail.com> Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Add SSN to init

1238512

Signed-off-by: blaz-r <blaz.rolih@gmail.com> Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Add SSN description

28006f1

Signed-off-by: Blaz Rolih <blaz.rolih@gmail.com> Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Update supersimplenet README.md

40e7475

Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Update readme with arch and results

ce6c7f5

Signed-off-by: blaz-r <blaz.rolih@gmail.com> Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Update SuperSimpleNet README.md

a47fc50

Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Update architecture image

3c65da6

Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

blaz-r force-pushed the feature/supersimplenet branch from 6acde98 to 3c65da6 Compare November 21, 2024 08:53

blaz-r added 2 commits November 21, 2024 10:13

Add SuperSimpleNet to init

07e02fb

Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

Fix copyright location and format

fda2c4e

Signed-off-by: blaz.rolih <blaz.rolih@fri.uni-lj.si>

samet-akcay reviewed Nov 22, 2024

View reviewed changes

blaz-r added 4 commits December 23, 2024 23:28

Merge remote-tracking branch 'refs/remotes/og/release/v2.0.0' into fe…

fd499ad

…ature/supersimplenet # Conflicts: # src/anomalib/models/components/feature_extractors/torchfx.py

Update SSN ano. gen. with new perlin method

521aee7

Signed-off-by: blaz-r <blaz.rolih@gmail.com>

Add latest v2 structure to SSN

6687abc

Signed-off-by: blaz-r <blaz.rolih@gmail.com>

Add docstring at the beginning of SSN files.

50032c6

Signed-off-by: blaz-r <blaz.rolih@gmail.com>

blaz-r added 5 commits December 24, 2024 00:44

Add SSN to init files

5bcb533

Signed-off-by: blaz-r <blaz.rolih@gmail.com>

Include SSN in docs

a569e03

Signed-off-by: blaz-r <blaz.rolih@gmail.com>

Verify and update ssn res for refactored perlin

bb142dd

Signed-off-by: blaz-r <blaz.rolih@gmail.com>

Update readme with res for refactored perlin

64fba6c

Signed-off-by: blaz-r <blaz.rolih@gmail.com>

Merge remote-tracking branch 'refs/remotes/og/release/v2.0.0' into fe…

528d594

…ature/supersimplenet

samet-akcay self-requested a review January 3, 2025 22:48

samet-akcay reviewed Jan 4, 2025

View reviewed changes

Rename models to match naming convention

ebc16ba

Signed-off-by: blaz-r <blaz.rolih@gmail.com>

samet-akcay approved these changes Jan 4, 2025

View reviewed changes

djdameln requested changes Jan 6, 2025

View reviewed changes

blaz-r added 2 commits January 6, 2025 22:18

Rename variables to fit the naming convention

d1ded74

Signed-off-by: blaz-r <blaz.rolih@gmail.com>

Clarify learning type docstring and fix some var names.

abc71ac

Signed-off-by: blaz-r <blaz.rolih@gmail.com>

djdameln approved these changes Jan 7, 2025

View reviewed changes

samet-akcay merged commit a304221 into openvinotoolkit:release/v2.0.0 Jan 7, 2025
6 of 7 checks passed

	samples.loc[(samples.label == DirType.NORMAL), "split"] = Split.TRAIN
	samples.loc[(samples.label == DirType.ABNORMAL) \| (samples.label == DirType.NORMAL_TEST), "split"] = Split.TEST

	normal_dir (str \| Path \| Sequence): Path to the directory containing normal images.
	root (str \| Path \| None): Root folder of the dataset.
	Defaults to ``None``.
	abnormal_dir (str \| Path \| Sequence \| None, optional): Path to the directory containing abnormal images.
	Defaults to ``None``.
	normal_test_dir (str \| Path \| Sequence \| None, optional): Path to the directory containing
	normal images for the test dataset.
	Defaults to ``None``.

		return sum(feature.shape[1] for feature in features.values())


		class FeatureAdaptor(nn.Module):

		from anomalib.data.utils.generators import generate_perlin_noise


		class SSNAnomalyGenerator(nn.Module):

🚀Add SuperSimpleNet model #2428

🚀Add SuperSimpleNet model #2428

Conversation

blaz-r commented Nov 21, 2024

📝 Description

✨ Changes

✅ Checklist

samet-akcay commented Nov 21, 2024

blaz-r commented Nov 21, 2024 • edited Loading

blaz-r commented Nov 21, 2024

samet-akcay commented Nov 22, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

samet-akcay commented Nov 25, 2024

samet-akcay commented Dec 2, 2024

blaz-r commented Dec 2, 2024

samet-akcay commented Dec 2, 2024

blaz-r commented Dec 23, 2024

blaz-r commented Jan 3, 2025

codecov bot commented Jan 3, 2025 • edited Loading

Codecov Report

samet-akcay commented Jan 4, 2025

samet-akcay left a comment

Choose a reason for hiding this comment

blaz-r commented Jan 4, 2025

blaz-r commented Jan 4, 2025

djdameln left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

blaz-r commented Jan 6, 2025

blaz-r commented Jan 7, 2025

samet-akcay commented Jan 7, 2025

djdameln left a comment

Choose a reason for hiding this comment

blaz-r commented Nov 21, 2024 •

edited

Loading

samet-akcay commented Nov 22, 2024 •

edited

Loading

codecov bot commented Jan 3, 2025 •

edited

Loading