[BUG]: Hard-coded keys in Production DFP

[efajardo-nv]

Version

23.01

Which installation method(s) does this occur on?

Docker

Describe the bug.

Production DFP examples fail with KeyError when changing config.ae.userid_column_name and config.ae.timestamp_column_name from defaults ("username", "timestamp").

Minimum reproducible example

Change "timestamp" to "ts" in line 187 of `dfp_azure_pipeline.py`:


config.ae.timestamp_column_name = "timestamp"

Relevant log output

Traceback (most recent call last):
  File "dfp_azure_pipeline.py", line 318, in <module>
    run_pipeline(obj={}, auto_envvar_prefix='DFP', show_default=True, prog_name="dfp")
  File "/opt/conda/envs/morpheus/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/opt/conda/envs/morpheus/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/opt/conda/envs/morpheus/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/conda/envs/morpheus/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "dfp_azure_pipeline.py", line 314, in run_pipeline
    pipeline.run()
  File "/my_data/gitrepos/efajardo-nv/Morpheus/morpheus/pipeline/pipeline.py", line 517, in run
    asyncio.run(self._do_run())
  File "/opt/conda/envs/morpheus/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/opt/conda/envs/morpheus/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/my_data/gitrepos/efajardo-nv/Morpheus/morpheus/pipeline/pipeline.py", line 495, in _do_run
    await self.join()
  File "/my_data/gitrepos/efajardo-nv/Morpheus/morpheus/pipeline/pipeline.py", line 251, in join
    await self._mrc_executor.join_async()
  File "/my_data/gitrepos/efajardo-nv/Morpheus/examples/digital_fingerprinting/production/morpheus/dfp/stages/dfp_split_users_stage.py", line 121, in extract_users
    message[self._config.ae.timestamp_column_name].min(),
  File "/opt/conda/envs/morpheus/lib/python3.8/site-packages/pandas/core/frame.py", line 3458, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/opt/conda/envs/morpheus/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3363, in get_loc
    raise KeyError(key) from err
KeyError: 'ts'

Code of Conduct

• I agree to follow Morpheus' Code of Conduct
• I have searched the open bugs and have found no duplicates for this bug report