TFX Iris sample

[numerology]

Illustrative purpose

DO NOT SUBMIT Changed my mind. I think we should include this sample :)

Verified at https://3ed47013582d1554-dot-us-central2.pipelines.googleusercontent.com/#/runs/details/a17f06a2-7799-4ad0-9bde-bc9c275b6355

Environment:
SDK: TFX 0.21.0 + KFP 0.2.4
Runtime: Hosted pipeline 0.2.3

• Sample test coverage
• Prebuilt sample config

---

This change is 

[numerology]

/cc @jingzhang36
FYI

[Ark-kun]

I wonder what would be the best option when we want to import executors from multiple components.

[numerology]

Ummm. That's a good question. I don't see any shortcut to that.

I guess partly that's because we don't often explicitly interact with TFX component executor when authoring the pipeline.

[Ark-kun]

/lgtm

[jingzhang36]

I have a question regarding the Evaluator output of this pipeline. I did a run of this pipeline and the Evaluator's output in my public GCS bucket gs://jingzhangjz-project-outputs/tfx_iris/8d8d58b1-2826-4533-ae0d-c04a65b2864d/Evaluator/evaluation/30.

While I used the following to parse the metric file under that bucket

import tensorflow as tf
from tensorflow_model_analysis.proto import metrics_for_slice_pb2
from typing import Any
for record in tf.compat.v1.python_io.tf_record_iterator('gs://jingzhangjz-project-outputs/tfx_iris/8d8d58b1-2826-4533-ae0d-c04a65b2864d/Evaluator/evaluation/30/metrics'):
metrics_for_slice = metrics_for_slice_pb2.MetricsForSlice.FromString(record)
print(metrics_for_slice)

The parse data seems missing metrics field. They are like
slice_key {
single_slice_keys {
column: "sepal_length"
float_value: 4.900000095367432
}
}
slice_key {
single_slice_keys {
column: "sepal_length"
float_value: 5.699999809265137
}
}
slice_key {
...
}
....

There are all slice_key fields, but no metrics field. As a reference, the taxi example in KFP has parsed data that looks like:
slice_key{
...
}
metrics {
key: "precision"
value {
double_value {
}
}
}
metrics {
}

[numerology]

There are all slice_key fields, but no metrics field.

It seems like a Keras problem IIUC. Switching to vanilla TF solved this.

[numerology]

There are all slice_key fields, but no metrics field.

It seems like a Keras problem IIUC. Switching to vanilla TF solved this.

And if you indeed want to use Keras, you'll need to handcraft the eval_config like following:

eval_config = tfma.EvalConfig(
      model_specs=[
          tfma.ModelSpec(name='candidate', label_key='variety'),
          tfma.ModelSpec(
              name='baseline', label_key='variety', is_baseline=True)
      ],
      slicing_specs=[tfma.SlicingSpec()],
      metrics_specs=[
          tfma.MetricsSpec(metrics=[
              tfma.MetricConfig(
                  class_name='SparseCategoricalAccuracy',
                  threshold=tfma.config.MetricThreshold(
                      value_threshold=tfma.GenericValueThreshold(
                          lower_bound={'value': 0.9}),
                      change_threshold=tfma.GenericChangeThreshold(
                          direction=tfma.MetricDirection.HIGHER_IS_BETTER,
                          absolute={'value': -1e-10})))
          ])])

[jingzhang36]

/lgtm

[k8s-ci-robot]

New changes are detected. LGTM label has been removed.

[numerology]

/assign @rmgogogo

[numerology]

@rmgogogo , this is the forked native Keras sample.

[neuromage]

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: neuromage

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• backend/OWNERS [neuromage]
• frontend/OWNERS [neuromage]
• samples/OWNERS [neuromage]
• test/OWNERS [neuromage]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[numerology]

/retest

[numerology]

/retest

[numerology]

/retest

[k8s-ci-robot]

New changes are detected. LGTM label has been removed.

[numerology]

/retest