Example-based #165

AntoninPoche · 2024-09-25T09:52:33Z

Introduce the Example-based module

This pull request introduces example-based methods from four families of methods. Namely:

Similar Examples
Semi-factuals
Counterfactuals
Prototypes

The API

The API is common for the four families, and it has three steps:

Construct a projection function with the model.
Initialize the method with a dataset, projection, and other parameters.
Explain given samples with a local explanation.

projection = ProjectionMethod(model)

explainer = ExampleMethod(
    cases_dataset=cases_dataset, 
    k=k,
    projection=projection,
    case_returns=case_returns,
    distance=distance,
)

examples = explainer.explain(inputs, targets)

…-ai/xplique into antonin/example-based-merge

AntoninPoche

First part of my review. I will wait to review all to make a final comment.

AntoninPoche · 2024-09-30T09:04:46Z

README.md

+| `KLEORSimMiss` | Semi Factuals | [KLEOR](../semifactuals/kleor/) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1gA7mhWhWzdKholZWkTvAg4FzFnzS8NHF) |
+| `KLEORGlobalSim` | Semi Factuals | [KLEOR](../semifactuals/kleor/) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1gA7mhWhWzdKholZWkTvAg4FzFnzS8NHF) |
+||||
+| `ProtoGreedy` | Prototypes | [ProtoGreedy](../prototypes/proto_greedy/) | **TODO** |


Add link when tutorial is ready.

AntoninPoche · 2024-09-30T09:05:24Z

TUTORIALS.md

@@ -20,6 +20,8 @@ Here is the lists of the available tutorial for now:
 | Metrics                         | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1WEpVpFSq-oL1Ejugr8Ojb3tcbqXIOPBg)            |
 | Concept Activation Vectors      | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1iuEz46ZjgG97vTBH8p-vod3y14UETvVE)            |
 | Feature Visualization           | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1st43K9AH-UL4eZM1S4QdyrOi7Epa5K8v)            |
+| Example-Based Methods           | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1gA7mhWhWzdKholZWkTvAg4FzFnzS8NHF)            |
+| Prototypes                      | **TODO**            |


Add a link when the tutorial is ready.

AntoninPoche · 2024-09-30T09:05:53Z

TUTORIALS.md

+| **Tutorial Name**                      | Notebook                                                                                                                                                |
+| :------------------------------------- | :-----------------------------------------------------------------------------------------------------------------------------------------------------: |
+| Example-Based Methods: Getting started | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1gA7mhWhWzdKholZWkTvAg4FzFnzS8NHF) |
+| Prototypes: Getting started            | **TODO**                                                                                     |


Add a link when the tutorial is ready.

docs/api/example_based/api_example_based.md

xplique/example_based/base_example_method.py

lucashervier

I think we have done a nice first draft. I have made some comments on the points I do not want to forget to discuss with you. I have made very few comments on prototypes as I am not familiar with the methods.

lucashervier · 2024-09-30T12:45:36Z

.pylintrc

We removed it to have all configurations in the setup.cfg file instead. Why do we add it back?

I will try to remove it and see it works again.

lucashervier · 2024-09-30T12:46:46Z

TUTORIALS.md

@@ -20,6 +20,8 @@ Here is the lists of the available tutorial for now:
 | Metrics                         | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1WEpVpFSq-oL1Ejugr8Ojb3tcbqXIOPBg)            |
 | Concept Activation Vectors      | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1iuEz46ZjgG97vTBH8p-vod3y14UETvVE)            |
 | Feature Visualization           | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1st43K9AH-UL4eZM1S4QdyrOi7Epa5K8v)            |
+| Example-Based Methods           | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1gA7mhWhWzdKholZWkTvAg4FzFnzS8NHF)            |
+| Prototypes                      | **TODO**            |


We need to actually add the link of the notebook here

lucashervier · 2024-09-30T12:46:54Z

TUTORIALS.md

+| **Tutorial Name**                      | Notebook                                                                                                                                                |
+| :------------------------------------- | :-----------------------------------------------------------------------------------------------------------------------------------------------------: |
+| Example-Based Methods: Getting started | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1gA7mhWhWzdKholZWkTvAg4FzFnzS8NHF) |
+| Prototypes: Getting started            | **TODO**                                                                                     |


docs/api/example_based/api_example_based.md

lucashervier · 2024-09-30T14:00:36Z

xplique/commons/data_conversion.py



 def tensor_sanitize(inputs: Union[tf.data.Dataset, tf.Tensor, np.ndarray],
-                    targets: Optional[Union[tf.Tensor, np.ndarray]]) -> Tuple[tf.Tensor, tf.Tensor]:
+                    targets: Union[tf.Tensor, np.ndarray]) -> Tuple[tf.Tensor, tf.Tensor]:


We can't use None anymore?

It was never the case for classic attribution methods. The typing did not correspond to what the method supported.

xplique/example_based/datasets_operations/convert_torch_to_tf.py

lucashervier · 2024-09-30T14:07:51Z

xplique/example_based/datasets_operations/tf_dataset_operations.py

+    return False
+
+
+def is_not_shuffled(dataset: Optional[tf.data.Dataset]) -> bool:


The name is not really well suited. In addition, I would rather use an iterator to check that is two instantiations of the same dataset..

I agree on the name, but what do you mean on the iterator?

xplique/example_based/projections/base.py

xplique/example_based/projections/commons.py

AntoninPoche

Second part of my review.

xplique/example_based/counterfactuals.py

xplique/example_based/projections/commons.py

xplique/example_based/projections/hadamard.py

xplique/example_based/projections/latent_space.py

AntoninPoche

Third part of the review, there are many small modifications in docstrings.

xplique/example_based/prototypes.py

xplique/example_based/similar_examples.py

xplique/plots/image.py

AntoninPoche

To summarize remaining comments:

No link or wrong link for prototypes tutorial.
Some questions on example based test, for pytorch ones and the ones regarding datasets operations.
Questions on the FilterKNN on how the mask is applied.
A pylint problem too-many-positional-arguments. If resolved, this could lead to loose retro-compatibility. See pylint doc on this warning for more informations.

…ompatibility

AntoninPoche · 2024-10-07T16:12:46Z

I closed this PR; the commit history was a mess. The new PR for the Example-Based module is #166.

AdelinCO and others added 30 commits February 14, 2024 16:26

example-based: add cole method

c8df34e

tests: unit testing cole

c331801

example based: introduce projections

e1e4d82

example based: introduce search methods

eda3317

example based: add base class

4b0a340

cole: update and improve

0daf281

example based tests: update and complete

62a621d

plots: add image visualization for example based

2aa133d

commons: add operations for tf dataset

bed4747

pylint: disable similarities for signatures

9f9acfb

example based: introduce base example method abstraction

d29f92f

example based: adapt similar examples

69fef12

example based: adapt cole

539c387

example based: adapt tests

7a58c9f

base example method: dataset projections in projections

2255896

base projection: dataset projections in projections

ead19ea

latent space projection: dataset projections in projections

48bc853

attribution projection: dataset projections in projections

fcacfd1

example based: introduce hadamard projection

4dbb6b6

base search method: remove projection from search

6e8ddb1

knn search method: remove projection from search

5abb298

cole: add hadamard product possibility

92056f1

projections: factorize model splitting

3f69e7b

example based projections: fuse custom projection with base class

5cdf1c1

projections tests: adapt to changes and complete

0c141bb

similar examples tests: adapt to changes

56ddd00

cole tests: adapt to changes and add hadamard

c44d256

tf operations: add get device for dataset mapping

445fce1

feat: add a new KNN object and improve distance computation efficiency

7cfa2f3

tests: add tests for KNNs

52d1c6a

AntoninPoche added 7 commits September 24, 2024 11:30

base example methods: projection can now be none

f8bc136

example based: small pylint fixes

f8bb9f8

search methods: small pylint fixes

cb8eee9

prototypes: optimize and harmonize

f436713

tests prototypes: adapt to prototypes changes and complete

145f0d5

Merge branch 'antonin/example-based-merge' of https://github.com/deel…

222527a

…-ai/xplique into antonin/example-based-merge

prototypes: small fix

c8a0abe

AntoninPoche requested review from lucashervier and ChafikBak September 25, 2024 09:53

AntoninPoche added 2 commits September 25, 2024 16:05

ci: restrict numpy version and specify torch test

97fd2cf

test example based: small fix

a4414dd

AntoninPoche self-assigned this Sep 26, 2024

AntoninPoche commented Sep 30, 2024

View reviewed changes

rise: fix pylint

4debfd6

lucashervier requested changes Sep 30, 2024

View reviewed changes

AntoninPoche commented Oct 1, 2024

View reviewed changes

AntoninPoche commented Oct 2, 2024

View reviewed changes

AntoninPoche added 3 commits October 3, 2024 12:28

all: resolve part of example based pr comments

ef7db46

example based: small fix

b684ad3

example based: small fix

a876d7e

AntoninPoche commented Oct 3, 2024

View reviewed changes

AntoninPoche added 6 commits October 4, 2024 15:52

example based: small fix

3bbf406

example based docs: add prototypes tuto link

f316e9f

example based: fix pylint issues

9c7e3ee

setup: ignore pylint too-many-positionnal-arguments because of retroc…

eaa9d49

…ompatibility

example based: small fixes

aac96c2

example based tests: small fix

3f4b406

AntoninPoche mentioned this pull request Oct 7, 2024

Example based #166

Merged

AntoninPoche closed this Oct 7, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Example-based #165

Example-based #165

AntoninPoche commented Sep 25, 2024 •

edited

Loading

AntoninPoche left a comment

AntoninPoche Sep 30, 2024

AntoninPoche Sep 30, 2024

AntoninPoche Sep 30, 2024

lucashervier left a comment

lucashervier Sep 30, 2024

AntoninPoche Oct 2, 2024

lucashervier Sep 30, 2024

lucashervier Sep 30, 2024

lucashervier Sep 30, 2024

AntoninPoche Oct 2, 2024

lucashervier Sep 30, 2024

AntoninPoche Oct 1, 2024

AntoninPoche left a comment

AntoninPoche left a comment

AntoninPoche left a comment

AntoninPoche commented Oct 7, 2024

		return False


		def is_not_shuffled(dataset: Optional[tf.data.Dataset]) -> bool:

Example-based #165

Example-based #165

Conversation

AntoninPoche commented Sep 25, 2024 • edited Loading

Introduce the Example-based module

The API

AntoninPoche left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lucashervier left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AntoninPoche left a comment

Choose a reason for hiding this comment

AntoninPoche left a comment

Choose a reason for hiding this comment

AntoninPoche left a comment

Choose a reason for hiding this comment

AntoninPoche commented Oct 7, 2024

AntoninPoche commented Sep 25, 2024 •

edited

Loading