Ability to add comments to versions of prompts ("commit message")

[ajram23]

Describe the feature or potential improvement

As you make new versions you completely lose track. Sooner you folks add this the better it is going to be, otherwise will have to use git offline and just your feature to tag and serve prompts which would be disappointing. Hope you can prioritize this soon. Thank you!

Additional information

No response

[marcklingen]

Thanks for your plus one on this. This is high on our list, among other things unfortunately. See https://langfuse.com/roadmap for an overview of what we are currently working on. Very much open for contributions on this by the way.

[ajram23]

ok no big deal. The chatPrompt issue is a bigger deal than this. I can work around this issue for now.

[marliessophie]

We've released a first version of comments across objects in the Langfuse UI. Please see our changelog for details. As always - we'd love to hear feedback on the feature!

Additionally, we believe that 'commit messages' for each prompt version are another avenue we'd like to additionally explore in the future. Excited to hear thoughts/input on this as well

[ajram23]

@marliessophie trying it out, while not related one of the huge improvements would you be to link to a LLM to provide feedback to prompts. Right now I am having to copy and paste and get feedback from a LLM. Most versions end up brief drafts because of typos! This would also allow the check in comments to be drafted, which is what I am doing right now. Feedback coming on the other stuff soon!

[ajram23]

@marliessophie looks like you folks added a comments section for the prompt instead of versioning comment :( not what I asked at all. Not sure how this request got translated to that.

[ajram23]

@marliessophie may be its confusing because you have the comments not while updating the prompt but after updating the prompt. Thats the one change I will make. Sorry for the confusion

[marcklingen]

may be its confusing because you have the comments not while updating the prompt but after updating the prompt. Thats the one change I will make. Sorry for the confusion

thanks for the feedback! yes, we plan to add basically a commit message to prompts specifically. as we rolled out comments across the product this was a simple addition to make this avaialble today already and facilitate more of a conversation on each version in larger teams

[ajram23]

Thanks for getting it done. At the very least I have some context now!

[emieldatalytica]

To add on this subject: I'm proposing to add flexible tags to prompt versions.

Currently I just have one prompt collection, where I tie model parameters to the prompt itself:

Now imagine that a new model is released by openAI that we’d like to A/B test. My idea was that product managers (no-coders) could potentially register the same prompt, with a different config and label that for instance as the test variant. Adding tags in addition to labels to prompt versions would be helpful in identifying which prompt versions for instance are configured to which model.

The tags would be different from prompt labels, in the sense that labels are unique and tags are not. Tags would be treated more like metadata of a prompt version.

[marcklingen]

thanks for sharing!