Revert "[AIR][Predictor] Enable numpy based predictor"

[fishbone]

Reverts #28917

Executed 1 out of 1 test: 1 fails locally.
(13:58:01) INFO: Build completed, 1 test FAILED, 4 total actions
(base) root@9947804ba968:/ray# cat   /root/.cache/bazel/_bazel_root/5fe90af4e7d1ed9fcf52f59e39e126f5/execroot/com_github_ray_project_ray/bazel-out/k8-opt/testlogs/python/ray/tests/test_basic/test_attempts/attempt_2.log
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //python/ray/tests:test_basic
-----------------------------------------------------------------------------
Traceback (most recent call last):
  File "/root/.cache/bazel/_bazel_root/5fe90af4e7d1ed9fcf52f59e39e126f5/execroot/com_github_ray_project_ray/bazel-out/k8-opt/bin/python/ray/tests/test_basic.runfiles/com_github_ray_project_ray/python/ray/tests/test_basic.py", line 11, in <module>
    import ray
  File "/ray/python/ray/__init__.py", line 171, in <module>
    from ray import data  # noqa: E402,F401
  File "/ray/python/ray/data/__init__.py", line 3, in <module>
    from ray.data.dataset import Dataset
  File "/ray/python/ray/data/dataset.py", line 30, in <module>
    from ray.air.util.data_batch_conversion import BlockFormat
  File "/ray/python/ray/air/util/data_batch_conversion.py", line 6, in <module>
    import pandas as pd
ModuleNotFoundError: No module named 'pandas'

It seems that unnecessary libs are imported with minimal install.

[fishbone]

Just noticed it's being fixed here #30352

I'll hold this PR.