Update embeddings tools for 2025-01-30 LTS

[mlin]

Variety of minor changes & readme updates for our embedding preparation tooling, as we worked through them for the new LTS.

[mlin]

NOTE: the Python unit test failures are expected to be fixed by releasing the new embeddings.

[ivirshup]

This is very minor, but numpy has comparison + assertion functions that give helpful errors messages under np.testing. E.g.:

In [3]: np.testing.assert_array_equal([1, 2, 3], [1, 3, 3])
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In[3], line 1
----> 1 np.testing.assert_array_equal([1, 2, 3], [1, 3, 3])

File ~/miniforge3/envs/ingestion/lib/python3.11/site-packages/numpy/_utils/__init__.py:85, in _rename_parameter.<locals>.decorator.<locals>.wrapper(*args, **kwargs)
     83             raise TypeError(msg)
     84         kwargs[new_name] = kwargs.pop(old_name)
---> 85 return fun(*args, **kwargs)

    [... skipping hidden 1 frame]

File ~/miniforge3/envs/ingestion/lib/python3.11/site-packages/numpy/testing/_private/utils.py:885, in assert_array_compare(comparison, x, y, err_msg, verbose, header, precision, equal_nan, equal_inf, strict, names)
    880         err_msg += '\n' + '\n'.join(remarks)
    881         msg = build_err_msg([ox, oy], err_msg,
    882                             verbose=verbose, header=header,
    883                             names=names,
    884                             precision=precision)
--> 885         raise AssertionError(msg)
    886 except ValueError:
    887     import traceback

AssertionError: 
Arrays are not equal

Mismatched elements: 1 / 3 (33.3%)
Max absolute difference among violations: 1
Max relative difference among violations: 0.33333333
 ACTUAL: array([1, 2, 3])
 DESIRED: array([1, 3, 3])