feat: added text summarization to pgRag

[synapse]

Fixes #7

• added summarization function (this requires setting up a chat model and to be passed in along side the embeddings, although optional)
• added configurable summarization options but comes with pre built-in configs
• did a couple of linting fixes

[marceloFerreira90]

code wise looks good. But lets clean it up a bit. Can you move all numbers and and all text to a constants file.

I would recommend loading the prompts from a txt file as these will change with more requirements

[marceloFerreira90]

code wise looks good. But lets clean it up a bit. Can you move all numbers and and all text to a constants file.

I would recommend loading the prompts from a txt file as these will change with more requirements

[nherment]

Looks good overall:

• I would not necessarily expose the summarize function. If we think Nearform should provide one it may be good to put it in its own library.
• The summarization should be automated and be run on each added document as part of the processDocument job. The summary should then be added to the vectorized chunks for that document (metadata: { parentDocumentId: <sourc edoc id> }

[synapse]

Looks good overall:

• I would not necessarily expose the summarize function. If we think Nearform should provide one it may be good to put it in its own library.
• The summarization should be automated and be run on each added document as part of the processDocument job. The summary should then be added to the vectorized chunks for that document (metadata: { parentDocumentId: <sourc edoc id> }

• I know we've talked about not exposing it, but I did it to prevent the CI from failing because of unused variable. As soon as we start using it inside the project I'll remove it from the export.
• adding the auto summarization next