vaex-y way of getting dt components in bulk

[v1gnesh]

pandas has dt.components that returns a df.

I'm sure the following can be done concurrently to finish the whole op faster.

df2['ms'] = df.start.dt.minute.astype('uint8')
df2['me'] = df.end.dt.dt.minute.astype('uint8')
df2['hs'] = df.start.dt.hour.astype('uint8')
df2['he'] = df.end.dt.dt.hour.astype('uint8')
df2['ss'] = df.start.dt.second.astype('uint8')
df2['se'] = df.end.dt.second.astype('uint8')

Can this be done in a more elegant manner, with a list comprehension or something?

[JovanVeljanoski]

Hey!

So the way vaex works, due to its expression system, these expressions will be in fact evaluated simultaneously, depending on what is needed. If you use all of them at once, they will be evaluated concurrently. If you use only some of then, than only those will be evaluated. Remember, we are working with expressions, the underlying data is immutable.

The only thing I can think of, is that if you do this a lot, you may define your own extension function to save you a bit of typing. You can the examples in the tutorial.

Also, it is not a very good idea to do "cross-dataframe" expression definition, since for the time being, this is not possible (exception being joins).

So this like
df2['ms'] = df.start.dt.minute.astype('uint8')
should probably be

df2['ms'] = df.start.dt.minute.astype('uint8')
although your example will also work if df2 exists and has the same columns as df.

I hope this helps a bit.