[docs] Reorganize the tensor data support docs; general editing

[ericl]

Why are these changes needed?

Editing pass over the tensor support docs for clarity:

• Make heavy use of tabbed guides to condense the content
• Rewrite examples to be more organized around creating vs reading tensors
• Use doc_code for testing

[clarkzinzow]

Awesome, massive improvement! Mostly nits and the like, but also a few other tentative suggestions:

1. I mentioned this in a comment in the review, but I think that a framing of "tensor datasets" (single-tensor-column table that presents a collection-of-tensors concept and API to the user) and "tensors in tabular datasets" (multi-column table that contains one or more tensor columns) makes more sense than the current framing of "single-column" and "multi-column".
2. Should we have a "Transforming Tensor Datasets" section that demonstrates batch transformations on tensor data? I know that it will be similar to the "Consuming Tensor Datasets" section and I know there's a call-out in that section, but not having a "how to transform tensor data" section will seem like a glaring omission when a user is scanning the docs. It would also be a nice section to add examples of how the tensor extension can be manipulated in a Pandas DataFrame as if its a native type (e.g. support arithmetic and aggregation operations and the like).
3. If we are showing how tensor datasets are formatted in batch transformations and consumption, I think that we should also have a section describing how rows for tensor datasets are presented, in the row-based transformation and consumption APIs (i.e. that tensor datasets are transparently converted to NumPy ndarrays in row-based APIs).

[clarkzinzow]

Ah nice I was just thinking about us needing to do this.

[ericl]

@clarkzinzow changes addressed (except the tensor dataset naming); ptal

[bveeramani]

This PR will conflict with #27295 because #27295 contains changes to dataset-tensor-support.rst.

[clarkzinzow]

LGTM, thanks for making those changes!

[scv119]

hmm seems this PR slows down many_tasks for unknown reasons. #27606

[ericl]

Maybe it's the eager Arrow import? Clark might know how to defer it to the right place.

…

On Sun, Aug 7, 2022, 12:02 AM Chen Shen ***@***.***> wrote: hmm seems this PR slows down many_tasks for unknown reasons. #27606 <#27606> — Reply to this email directly, view it on GitHub <#26952 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAADUSSXRGY34M72UT5UV7TVX5NQRANCNFSM54QMCUTQ> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[clarkzinzow]

@ericl Yep this was fixed in this PR: #27653

I also implemented a more generic fix for this issue, where eagerly importing ray.data, ray.workflow, and ray.autoscaler can lead to performance degradation of workloads that don't use those libraries, by delaying expensive subpackage imports until attribute access. But upon benchmarking, I didn't see a compelling enough speedup to warrant the added complexity, so I closed the PR.