[Obs AI Assistant] Error when using ollama model locally

[neptunian]

Similar to the bug described here, #204014, the model (llama 3.2) isn't being passed to chat completions and fails with an unexpected error

[2024-12-12T14:02:39.743-05:00][WARN ][plugins.actions.gen-ai] action execution failure: .gen-ai:8608ccb6-4e2a-4045-a729-ab4b556ea5ad: Llama: an error occurred while running the action: Status code: 400. Message: API Error: Bad Request - model is required; retry: true
[2024-12-12T14:02:39.744-05:00][ERROR][plugins.observabilityAIAssistant.service] Error: Unexpected error
at createInferenceInternalError (elastic/kibana/x-pack/platform/packages/shared/ai-infra/inference-common/src/errors.ts:49:10)
at elastic/kibana/x-pack/platform/plugins/shared/inference/server/chat_complete/adapters/openai/openai_adapter.ts:68:92

[elasticmachine]

Pinging @elastic/appex-ai-infra (Team:AI Infra)

[elasticmachine]

Pinging @elastic/obs-ai-assistant (Team:Obs AI Assistant)

[pgayvallet]

the model isn't being passed to chat completions

So this part is working as expected. The model is not a parameter of the inference APIs, and it not sent to the stack connector.

The stack connector is in charge of passing the information from the config to the provider during the remote call:

kibana/x-pack/plugins/stack_connectors/server/connector_types/openai/openai.ts

Lines 204 to 210 in 9372027

	const executeBody = getRequestWithStreamOption(
	this.provider,
	this.url,
	body,
	stream,
	...('defaultModel' in this.config ? [this.config.defaultModel] : [])
	);

However looking at the code forging the request, it seems that the defaultModel is only being passed for openAI provider type and not for other:

kibana/x-pack/plugins/stack_connectors/server/connector_types/openai/lib/utils.ts

Lines 65 to 81 in 83a701e

	export function getRequestWithStreamOption(
	provider: string,
	url: string,
	body: string,
	stream: boolean,
	defaultModel?: string
	): string {
	switch (provider) {
	case OpenAiProviderType.OpenAi:
	return openAiGetRequestWithStreamOption(url, body, stream, defaultModel!);
	case OpenAiProviderType.AzureAi:
	return azureAiGetRequestWithStreamOption(url, body, stream);
	case OpenAiProviderType.Other:
	return otherOpenAiGetRequestWithStreamOption(body, stream);
	default:
	return body;
	}

The right approach for the fix seems to be to adapt getRequestWithStreamOption and otherOpenAiGetRequestWithStreamOption

[pgayvallet]

So, I opened #204934 that will take care of the issue in the connector.

That fixes the general error and allows to properly communicates with ollama.

However, there seems to be something wrong with what the o11y assistant is doing, as the stream get closed in the middle and fails with something that seems related to the title generation:

Screen.Recording.2024-12-19.at.14.04.05.mov

The server error is, as always with RXJS, incredibly usable and totally shows where the issue is coming from:

[2024-12-19T14:04:13.554+01:00][ERROR][plugins.observabilityAIAssistant.service] Error
    at _super (/kibana/node_modules/rxjs/dist/cjs/internal/util/createErrorClass.js:7:26)
    at new EmptyErrorImpl (/kibana/node_modules/rxjs/dist/cjs/internal/util/EmptyError.js:6:5)
    at /kibana/node_modules/rxjs/dist/cjs/internal/operators/last.js:13:271
    at /kibana/node_modules/rxjs/dist/cjs/internal/operators/throwIfEmpty.js:14:86
    at OperatorSubscriber._this._complete (/kibana/node_modules/rxjs/dist/cjs/internal/operators/OperatorSubscriber.js:56:21)
    at Subscriber.complete (/kibana/node_modules/rxjs/dist/cjs/internal/Subscriber.js:69:18)

After some debug logs, this is being thrown here:

kibana/x-pack/platform/plugins/shared/observability_solution/observability_ai_assistant/server/service/client/index.ts

Lines 428 to 434 in 4455087

	return output$.pipe(
	instrumentAndCountTokens('complete'),
	withoutTokenCountEvents(),
	catchError((error) => {
	this.dependencies.logger.error(error);
	return throwError(() => error);
	}),

That observable is massive and we're getting out of my area of ownership though, so I'll let the @elastic/obs-ai-assistant team take a look if they so want.