Projector: Perturb t-SNE solution

[francoisluus]

A user-defined perturb factor determines the extent of random position shifts upon pressing of Perturb button, which is useful to move out of a local optimum or to alter the solution for a fresh perspective. Perturb T-SNE by independently scaling point positions when Perturb button is clicked, which is useful to move out of a local optima.

Demo here: http://tensorserve.com:6014

git clone -b projector-tsne-perturb https://github.com/francoisluus/tensorboard-supervise.git
cd tensorboard-supervise
git checkout 5b6a71b30f8f5b68896bef4354c90fd4d8bd73e1
bazel run tensorboard -- --logdir /home/emnist-2000 --host 0.0.0.0 --port 6014

Design and behavior:

1. Behavior remains exactly the same when the perturb button is never clicked.
2. Only one change is made in the t-SNE step() function, with a random independent downscaling of point coordinates of which the scaling magnitude is influenced by the perturb factor.
3. A small perturb factor of 0.1 scales a coordinate randomly in range [0.9, 1], and a large perturb factor of 0.8 scales in range [0.2, 1].
4. A coordinate scaling larger than 1 appears to cause unwanted effects where points on the extrema can be pushed out further, which squeezes the majority of the points in as the embedding is normally scaled to fit in the extrema. So we only randomly scale down coordinates, which appears to render the desired effect.
5. The perturb button can be pressed, when activated, at any time and repetitively to produce an even greater cumulative perturbation.

t-SNE panel before:

t-SNE panel with Perturb slider/button:

[dsmilkov]

This is an amazing work. Thank you for the detailed PR description, and sincere apologies for a long-overdue review!

[francoisluus]

I do appreciate the consideration, thanks for merging. There's no rush, I'd rather you mull over some of the ideas.

I have found that t-SNE local optima are actually harder to get out of than expected with scaling-only, so allow me to propose some improvements later based on affine perturbation. There's also going to be some ideas to streamline this t-SNE option as we're going to run out of room in the t-SNE projections panel. We could capture perturbation intensity from the user without the use of a slider, button press-duration is one possibility though at the risk of this optionality being too obscure.

[dsmilkov]

Yeah, for now I like the button approach. Some of these decisions might be better made once we get other user's feedback. Again, great work!