[Epic] AI Insights + Assistant - Add "Other" option to the existing OpenAI Connector dropdown list (#8936)

[e40pud]

Summary

Epic https://github.com/elastic/security-team/issues/8936

At the moment, our users are able to host local LLMs via an inference server that follows OpenAI's API spec using our OpenAI connector by changing the URL to their own server. Whilst this works just fine, it's not evident that this is something we support.

With these changes we add "Other" option to the existing OpenAI Connector dropdown list to provide a way for users to know that this is possible. Having an "Other" option in the current drop down list, with no pre-populated text in the URL, Model and Auth input sections.

All the underlying requests and data are identical for the existing OpenAI option in the dropdown.

Recording

Screen.Recording.2024-10-03.at.14.35.26.mov

Testing

To test the workflow follow these steps:

1. Create a new OpenAI connector
2. Select Other (OpenAI Compatible Service) provider
3. Fill out required fields
4. Save new connector
5. Now you can use it as a service in AI Assistance to send requests to.

Credentials
Use these credentials for testing LLama 3.1 70B and Mistral Large 2407:

https://p.elstc.co/paste/ofqxUGiW#RR1Pedserj9hWKm3vOw5oVEHWuwbX7i9Jl2rS7q7MRP

Checklist

Delete any items that are not applicable to this PR.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
  • [Request] Update setup local LLM page security-docs#5965
• Unit or functional tests were updated or added to match the most common scenarios

[afharo]

Telemetry changes LGTM

[e40pud]

@elasticmachine merge upstream

[kibana-ci]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 997982c

Failed CI Steps

• FTR Configs #4
• FTR Configs #103

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
searchPlayground	165.2KB	165.8KB	+627.0B
securitySolution	20.6MB	20.6MB	+104.0B
stackConnectors	580.4KB	584.1KB	+3.7KB
total			+4.4KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
stackConnectors	56.1KB	56.1KB	+16.0B

History

• 💔 Build #239496 failed f64fb9b
• 💛 Build #239336 was flaky fe07e40
• 💛 Build #239178 was flaky 56b5de8
• 💔 Build #239153 failed b49311e
• 💔 Build #239121 failed 8dba2f7

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @e40pud

[pmuellr]

ResponseOps changes LGTM

Standard warning about config/secrets schema changes in serverless: if we ship this code in serverless, and someone creates a connector using the new schema, and then we have to roll back the serverless release, this connector will fail to validate, and won't be able to execute. It should be deletable and editable in the UX, so could be edited to match the older schema, and would then start working again. Low risk in that happening. A safer approach is to just release the schema changes without anything else, wait a week, then the rest of the PR. That way on a rollback, the schema will be valid ...

[stephmilovic]

LGTM!

[joemcelroy]

Works really well! Thank you.

[stephmilovic]

@elasticmachine merge upstream

[stephmilovic]

@elasticmachine merge upstream

[kibanamachine]

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11263805561

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 91b795c

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
searchPlayground	165.4KB	165.9KB	+591.0B
securitySolution	20.6MB	20.6MB	+104.0B
stackConnectors	580.4KB	584.1KB	+3.7KB
total			+4.4KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
stackConnectors	56.1KB	56.1KB	+16.0B

History

• 💔 Build #240776 failed decc7b3
• 💔 Build #240595 failed 709c0bd
• 💛 Build #240059 was flaky 1008079

cc @e40pud

[kibanamachine]

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

[leemthompo]

@e40pud was there plans to update documentation for this change?

[e40pud]

@leemthompo I think we should update this section https://www.elastic.co/guide/en/security/current/connect-to-byo-llm.html#_configure_the_connector_in_your_elastic_deployment and mention that user should select Other (OpenAI Compatible Service) as OpenAI provider. Let me create a doc ticket for that.

cc @jamesspi

[e40pud]

@leemthompo here is the docs ticket elastic/security-docs#5965, please let me know if there is any information needed.

[jamesspi]

@Charelzard to track docs and blog update.