Add a streaming_op() to the pr_curves plugin. #587
Conversation
|
We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for the commit author(s). If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google. |
This op produces the same result (a Tensor Summary containing the PR curve data) as the existing op() function, but it does so by aggregating the result over multiple batches of data. The function also returns an update_op that should be used to update the summary for each batch of data. With these changes, the function provides a tf.metrics-like function for generating PR curve data.
|
CLAs look good, thanks! |
There was a problem hiding this comment.
Also add an entry for pr_curve_streaming_op in summary.py after this line:
https://github.com/tensorflow/tensorboard/blob/master/tensorboard/summary.py#L42
| positives, true negatives, false negatives, precision, recall. | ||
| update_op: An operation that updates the summary with the latest data. | ||
| """ | ||
| thresholds = [i * 1.0 / float(num_thresholds - 1) |
There was a problem hiding this comment.
Does i / float( ... suffice? Without the * 1.0?
| self.assertEqual(expected_proto.value[0].tag, 'pr_curve_1/pr_curves') | ||
| expected_proto.value[0].tag = 'pr_curve/pr_curves' | ||
|
|
||
| self.assertProtoEquals(expected_proto, proto) |
There was a problem hiding this comment.
Oh wow, I should use assertProtoEquals more often.
|
Updated tensorboard/summary.py. Also verified all tests pass (locally) with latest tf-nightly. |
| @@ -164,6 +164,98 @@ def op( | |||
| description, | |||
| collections) | |||
|
|
|||
|
|
|||
| def streaming_op(tag, labels, predictions, num_thresholds=200, weights=None, | |||
There was a problem hiding this comment.
nit: Could we reorganize the arguments to be one per row?
This op produces the same result (a Tensor Summary containing the PR
curve data) as the existing op() function, but it does so by
aggregating the result over multiple batches of data. The function
also returns an update_op that should be used to update the summary
for each batch of data.
With these changes, the function provides a tf.metrics-like function
for generating PR curve data.