fix: tongyi json output

[lxm]

Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes #5397

Type of Change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

api change

Accroding to tongyi api docs. There's no different between chat models and completaion models.

original code: in api/core/model_runtime/model_providers/tongyi/llm/llm.py line 245, 249 will cause prompt missing when using json response format

        else:
            if mode == LLMMode.CHAT:
                params['messages'] = self._convert_prompt_messages_to_tongyi_messages(prompt_messages)
            else:
                params['prompt'] = prompt_messages[0].content.rstrip()

json parse change

as comment in file api/core/model_runtime/model_providers/__base/large_language_model.py, some LLM with directly output json data without markdown block

test result with same prompt

请按以下格式输出json数据 `{"data": "test", "code": 200, "msg": "success"}`

before change

after change

• TODO raise a error when json output did not satisfy

Suggested Checklist:

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• My changes generate no new warnings
• I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods
• optional I have made corresponding changes to the documentation
• optional I have added tests that prove my fix is effective or that my feature works
• optional New and existing unit tests pass locally with my changes

[laipz8200]

Thanks for your contribution, please open an issue first, then link it to this PR. It will helpful for us to manage our community. 😇

[laipz8200]

The _code_block_mode_stream_processor_with_backtick method in LargeLanguageModel is designed to parse markdown code block content.

If the model doesn't return the expected format, we shouldn't parse it. Alternatively, you can implement it within the Tongyi LLM.

[lxm]

But no markdown requires in any system prompt, maybe we should change system prompt?

[laipz8200]

But no markdown requires in any system prompt, maybe we should change system prompt?

Yeah, maybe we should rewrite the system prompt and test it on more models. Let's discuss this in the related issue.

[lxm]

So, I changed the block_prompts in tongyi model and add test cases for json ouput.
Tests that check response_format should be add to more models.

[laipz8200]

Thanks a lot for doing that. We'll evaluate the default system prompt later.