Historical Configuration Management and Evaluation

[wlbksy]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to file this report (我已阅读并同意 Language Policy).

1. Is this request related to a challenge you're experiencing?

In the current dify code, each time the publish button is clicked on the app configuration page, the current app configuration is saved in the datatable app_model_configs, which can be considered as a commit of the configuration.

However, it does not meet the following needs:

1. Sometimes we want to revert to an old configuration commit, but currently dify lacks tools for this.
2. Sometimes we want to evaluate two historical configuration commits, and need to be able to call an API for each of the configurations of two different commits.

2. Describe the feature you'd like to see

1. Have a webpage for managing historical commits

   • Display all historical commits/tags/release
   • Tag or release a historical commit, fill in a description
   • Show diff between two commits like git diff

2. backend API provide a way to call to make AI task for a specific configuration commit/tag/release, defaulting to the latest configuration commit.

3. How will this feature improve your workflow or experience?

It would make Historical Configuration useful both for version control and evaluation

4. Additional context or comments

Proposal

1. Add tag or release fields to the app_model_configs datatable, and add a description field.
2. Add an API interface for adding tag/release/description to a configuration commit.
3. Add an API, which returns all related configurations of provided configuration commit/tag/release ID, along with the datasets/documents/upload_files/tools/tool_api_providers used by this commit (obtained by joining through id across datatables)
4. Add a webpage to display a historical configuration
5. Add a tag/release management page
6. Add a diff page between two configurations.

5. Can you help us with this feature?

• I am interested in contributing to this feature.

[dosubot[bot]]

🤖

Thank you for your detailed feature request. Your proposal for managing historical commits of the app configuration in dify is well thought out and addresses a real need for version control and evaluation of different configurations.

Let's break down your proposal and discuss how each part could be implemented:

1. Add tag or release fields to the app_model_configs datatable, and add a description field.

   • This would involve modifying the AppModelConfig model in the models.py file to include these new fields. The tag and release fields could be strings, and the description field could be a text field. After adding these fields, a database migration would need to be created to apply these changes to the database.

2. Add an API interface for adding tag/release/description to a configuration commit.

   • This could be done by adding a new method to the ModelConfigResource class in the api/controllers/console/app/model_config.py file. This method would handle a PUT request to update the tag, release, and description fields of a specific AppModelConfig instance.

3. Add an API, which returns all related configurations of provided configuration commit/tag/release ID, along with the datasets/documents/upload_files/tools/tool_api_providers used by this commit (obtained by joining through id across datatables)

   • This would involve creating a new API endpoint that takes a commit/tag/release ID as a parameter and returns the corresponding AppModelConfig instance and all related data. The related data could be retrieved by performing joins on the relevant datatables.

4. Add a webpage to display a historical configuration

   • This would involve creating a new page in the web app that displays the details of a specific AppModelConfig instance. This page could be accessed by clicking on a commit/tag/release ID on the historical commits page.

5. Add a tag/release management page

   • This would involve creating a new page in the web app that allows users to manage tags and releases. This page could provide functionality for adding, editing, and deleting tags and releases.

6. Add a diff page between two configurations.

   • This would involve creating a new page in the web app that displays the differences between two AppModelConfig instances. This could be implemented by comparing the fields of the two instances and highlighting the differences.

As for your question about contributing to this feature, I'm afraid I can't answer that as I don't have the ability to accept or reject contributions. However, I would recommend reaching out to the repository maintainers directly to discuss your proposal and how you might be able to contribute.

I hope this helps! If you have any further questions or need clarification on anything, please don't hesitate to ask.

Sources

• web/app/components/app/configuration/index.tsx
• api/controllers/console/app/model_config.py
• api/migrations/versions/ab23c11305d4_add_dataset_query_variable_at_app_model_.py
• api/services/app_model_config_service.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[VincePotato]

Good suggestion! We are considering adding version control to our app builder(studio). Thanks for your useful feedback.

[wlbksy]

Good suggestion! We are considering adding version control to our app builder(studio). Thanks for your useful feedback.

What's your opinion about the idea of a call for a specific version, so that we users could evaluate and compare two versions, helping us make decision about which version is good to go?

[github-actions[bot]]

Close due to it's no longer active, if you have any questions, you can reopen it.

[wlbksy]

bot being fooling around as if it's blind on the ref from the other PR

[github-actions[bot]]

Close due to it's no longer active, if you have any questions, you can reopen it.

[wlbksy]

bad bot

[github-actions[bot]]

Close due to it's no longer active, if you have any questions, you can reopen it.

[wlbksy]

bad bot