Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[KB] create @kbn/product-doc-artifact-builder package #193847

Merged
merged 16 commits into from
Oct 7, 2024

Conversation

pgayvallet
Copy link
Contributor

@pgayvallet pgayvallet commented Sep 24, 2024

Summary

Related #193473

Add initial implementation of the knowledge base artifact builder. This PR only introduces the builder script, it doesn't do anything about automation.

@pgayvallet pgayvallet added the Team:AI Infra AppEx AI Infrastructure Team label Sep 24, 2024
@pgayvallet pgayvallet changed the title KB packaging: POC of documentation extraction [KB] create @kbn/kb-artifact-builder package Oct 2, 2024
@pgayvallet
Copy link
Contributor Author

/ci

@pgayvallet
Copy link
Contributor Author

/ci

@pgayvallet
Copy link
Contributor Author

/ci

@pgayvallet pgayvallet marked this pull request as ready for review October 2, 2024 13:27
@pgayvallet pgayvallet requested a review from a team as a code owner October 2, 2024 13:27
@elasticmachine
Copy link
Contributor

Pinging @elastic/appex-ai-infra (Team:AI Infra)

@@ -116,6 +116,7 @@ export const IGNORE_DIRECTORY_GLOBS = [
'src/babel-*',
'packages/*',
'packages/core/*/*',
'x-pack/packages/ai-infra/*',
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@elastic/kibana-operations the ai-infra packages will live in that folder, and I kinda like my package folders to follow their names (e.g ai-infra/kb-artifact-builder instead of ai-infra/kb_artifact_builder), so I took the liberty to add that pattern.

Please tell me if you'd prefer me to just go with underscores.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues with it

@pgayvallet pgayvallet added release_note:skip Skip the PR/issue when compiling release notes v9.0.0 backport:prev-minor Backport to (9.0) the previous minor version (i.e. one version back from main) v8.16.0 labels Oct 2, 2024
@pgayvallet
Copy link
Contributor Author

@elasticmachine merge upstream

@jbudz
Copy link
Member

jbudz commented Oct 2, 2024

We're CODEOWNERS on one file here, but I don't see any other reviewers. Would it help it review the entire PR?

@pgayvallet pgayvallet requested a review from a team October 2, 2024 14:54
Copy link
Member

@jbudz jbudz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

src/dev/precommit_hook/casing_check_config.js

Copy link
Contributor Author

@pgayvallet pgayvallet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self-review

Comment on lines 63 to 64
// log.info('Checking connectivity against clusters');
// await checkConnectivity({ sourceClient, embeddingClient });
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't have ping permission on the source cluster, had to disable that for now...

.option('stackVersion', {
describe: 'The stack version to generate documentation for',
string: true,
default: '8.16', // TODO: master is on 9.0 now, not sure we can default to version in package.json?
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to figure that out at some point. version will be passed by the CI when we have this automated, so maybe using the version from the package is fine (but atm we have 0 docs for 9.0, so...)

Comment on lines 28 to 34
const searchRes = await client.search({
index,
size: 10000,
query: {
bool: {
must: [{ term: { product_name: productName } }],
},
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should use PIT instead of size: 10000 search, but we don't have products with more than 5k documents atm, and memory usage is fine given the resources we will running the task with

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add a check to assert this, until we update this to use PIT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah would make sense to fail in that case rather than silently ignoring. Will do in extract_documentation as the task is done first.

Comment on lines 68 to 71
const response = await client.search({
index,
size: 10000,
query: {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as previous - it's bad, but it should be okay for now.

Comment on lines 12 to 15
export const performSemanticSearch = async ({
searchQuery,
index,
client,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused, but keeping it for now as it will be used elsewhere in a follow-up

Copy link
Member

@legrego legrego left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall structure and flow LGTM. My comments are primarily around logging, to help our future selves debug this.

I'm sure some of the CLI options will change as we learn more (e.g. authentication mechanism), but it's too early to think about that.

}) => {
log.info(`Starting building artifact for product ${productName} and version ${stackVersion}`);

const targetIndex = `kb-test-${pseudoRandSuffix()}`;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Followup/non-blocking: what do you think about a more descriptive index name? Something that references any of [current date, CI agent, product name, version] would help us keep track of these indices, and perhaps help to trace issues in the event of a job failure.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems better yeah, and will do, even if I'm not sure of the concrete gains if we delete the index as a finally step

Comment on lines 28 to 34
const searchRes = await client.search({
index,
size: 10000,
query: {
bool: {
must: [{ term: { product_name: productName } }],
},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add a check to assert this, until we update this to use PIT?

});
log.info(`Extracted ${documents.length} documents from the source cluster`);

await createTargetIndex({
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it intentional that we don't cleanup the target index once we're done with it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. TBH I was keeping the index because I was then using it for my semantic search tests in the POC. And in practice everything should run on a disposable cluster. But you're right, we should (and we should also do it in case of failure as cleanup). Will adapt

@pgayvallet
Copy link
Contributor Author

@legrego I implemented all the requested changes, PTAL!

To get an idea how what logging looks like, an example on a single artifact run:

<< kibana % node scripts/build_kb_artifacts.js --stack-version 8.15 --product-name Kibana
 info Starting building artifacts for version=[8.15] and products=[Kibana]
 info Ensuring ELSER is installed on the embedding cluster
 info Starting building artifact for product [Kibana] and version [8.15]
 info Starting to extract documents from source cluster
 info Finished extracting documents from source. 589 documents were extracted
 info Starting indexing process
 info Indexed 1 of 59 chunks (took 53133ms)
[...]
 info Indexed 59 of 59 chunks (took 3057ms)
 info Finished indexing process
 info Starting to create chunk files in directory [.../kibana/build/temp-kb-artifacts/Kibana]
 info Writing chunk file content-1.ndjson containing 5 docs
[...]
 info Writing chunk file content-95.ndjson containing 6 docs
 info Finished creating chunk files
 info Starting to create artifact from build folder [.../kibana/build/temp-kb-artifacts/Kibana] into target [.../kibana/build/kb-artifacts]
 info Finished creating artifact [kibana-kb-kibana-8.15.zip]
 info Deleting index kb-artifact-builder-kibana-8.15
 info Finished building artifact for product [Kibana] and version [8.15]

@pgayvallet pgayvallet disabled auto-merge October 4, 2024 13:55
@pgayvallet pgayvallet changed the title [KB] create @kbn/kb-artifact-builder package [KB] create @kbn/product-doc-artifact-builder package Oct 7, 2024
@pgayvallet pgayvallet enabled auto-merge (squash) October 7, 2024 09:59
@kibana-ci
Copy link
Collaborator

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/product-doc-artifact-builder - 1 +1
Unknown metric groups

API count

id before after diff
@kbn/product-doc-artifact-builder - 1 +1

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@pgayvallet pgayvallet merged commit 1ab1add into elastic:main Oct 7, 2024
41 checks passed
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11215372223

@kibanamachine kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Oct 9, 2024
@kibanamachine
Copy link
Contributor

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 193847 locally

2 similar comments
@kibanamachine
Copy link
Contributor

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 193847 locally

@kibanamachine
Copy link
Contributor

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 193847 locally

@kibanamachine
Copy link
Contributor

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 193847 locally

@pgayvallet
Copy link
Contributor Author

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

@pgayvallet pgayvallet removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label Oct 14, 2024
pgayvallet added a commit to pgayvallet/kibana that referenced this pull request Oct 14, 2024
## Summary

Related elastic#193473

Add initial implementation of the knowledge base artifact builder. This
PR only introduces the builder script, it doesn't do anything about
automation.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
(cherry picked from commit 1ab1add)

# Conflicts:
#	.github/CODEOWNERS
pgayvallet added a commit that referenced this pull request Oct 15, 2024
… (#196206)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[KB] create `@kbn/product-doc-artifact-builder` package
(#193847)](#193847)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Pierre
Gayvallet","email":"pierre.gayvallet@elastic.co"},"sourceCommit":{"committedDate":"2024-10-07T12:21:09Z","message":"[KB]
create `@kbn/product-doc-artifact-builder` package (#193847)\n\n##
Summary\r\n\r\nRelated
https://github.com/elastic/kibana/issues/193473\r\n\r\nAdd initial
implementation of the knowledge base artifact builder. This\r\nPR only
introduces the builder script, it doesn't do anything
about\r\nautomation.\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"1ab1add68e15b4a227209952c6ab69b0f96ca456","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport
missing","v9.0.0","backport:prev-minor","v8.16.0","Team:AI
Infra"],"number":193847,"url":"https://github.com/elastic/kibana/pull/193847","mergeCommit":{"message":"[KB]
create `@kbn/product-doc-artifact-builder` package (#193847)\n\n##
Summary\r\n\r\nRelated
https://github.com/elastic/kibana/issues/193473\r\n\r\nAdd initial
implementation of the knowledge base artifact builder. This\r\nPR only
introduces the builder script, it doesn't do anything
about\r\nautomation.\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"1ab1add68e15b4a227209952c6ab69b0f96ca456"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/193847","number":193847,"mergeCommit":{"message":"[KB]
create `@kbn/product-doc-artifact-builder` package (#193847)\n\n##
Summary\r\n\r\nRelated
https://github.com/elastic/kibana/issues/193473\r\n\r\nAdd initial
implementation of the knowledge base artifact builder. This\r\nPR only
introduces the builder script, it doesn't do anything
about\r\nautomation.\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"1ab1add68e15b4a227209952c6ab69b0f96ca456"}},{"branch":"8.x","label":"v8.16.0","labelRegex":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
pgayvallet added a commit that referenced this pull request Nov 19, 2024
## Summary

Close #193473
Close #193474

This PR utilize the documentation packages that are build via the tool
introduced by #193847, allowing to
install them in Kibana and expose documentation retrieval as an LLM task
that AI assistants (or other consumers) can call.

Users can now decide to install the Elastic documentation from the
assistant's config screen, which will expose a new tool for the
assistant, `retrieve_documentation` (only implemented for the o11y
assistant in the current PR, shall be done for security as a follow up).

For more information, please refer to the self-review.

## General architecture

<img width="1118" alt="Screenshot 2024-10-17 at 09 22 32"
src="https://github.com/user-attachments/assets/3df8c30a-9ccc-49ab-92ce-c204b96d6fc4">

## What this PR does

Adds two plugin:
- `productDocBase`: contains all the logic related to product
documentation installation, status, and search. This is meant to be a
"low level" components only responsible for this specific part.
- `llmTasks`: an higher level plugin that will contain various LLM tasks
to be used by assistants and genAI consumers. The intent is not to have
a single place to put all llm tasks, but more to have a default place
where we can introduce new tasks from. (fwiw, the `nlToEsql` task will
probably be moved to that plugin).

- Add a `retrieve_documentation` tool registration for the o11y
assistant
- Add a component on the o11y assistant configuration page to install
the product doc

(wiring the feature to the o11y assistant was done for testing purposes
mostly, any addition / changes / enhancement should be done by the
owning team - either in this PR or as a follow-up)

## What is NOT included in this PR:

- Wire product base feature to the security assistant (should be done by
the owning team as a follow-up)
  - installation
  - utilization as tool

- FTR tests: this is somewhat blocked by the same things we need to
figure out for elastic/kibana-team#1271

## Screenshots 

### Installation from o11y assistant configuration page

<img width="1476" alt="Screenshot 2024-10-17 at 09 41 24"
src="https://github.com/user-attachments/assets/31daa585-9fb2-400a-a2d1-5917a262367a">

### Example of output

#### Without product documentation installed 

<img width="739" alt="Screenshot 2024-10-10 at 09 59 41"
src="https://github.com/user-attachments/assets/993fb216-6c9a-433f-bf44-f6e383d20d9d">

#### With product documentation installed

<img width="718" alt="Screenshot 2024-10-10 at 09 55 38"
src="https://github.com/user-attachments/assets/805ea4ca-8bc9-4355-a434-0ba81f8228a9">

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Alex Szabo <alex.szabo@elastic.co>
Co-authored-by: Matthias Wilhelm <matthias.wilhelm@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
pgayvallet added a commit to pgayvallet/kibana that referenced this pull request Nov 19, 2024
## Summary

Close elastic#193473
Close elastic#193474

This PR utilize the documentation packages that are build via the tool
introduced by elastic#193847, allowing to
install them in Kibana and expose documentation retrieval as an LLM task
that AI assistants (or other consumers) can call.

Users can now decide to install the Elastic documentation from the
assistant's config screen, which will expose a new tool for the
assistant, `retrieve_documentation` (only implemented for the o11y
assistant in the current PR, shall be done for security as a follow up).

For more information, please refer to the self-review.

## General architecture

<img width="1118" alt="Screenshot 2024-10-17 at 09 22 32"
src="https://github.com/user-attachments/assets/3df8c30a-9ccc-49ab-92ce-c204b96d6fc4">

## What this PR does

Adds two plugin:
- `productDocBase`: contains all the logic related to product
documentation installation, status, and search. This is meant to be a
"low level" components only responsible for this specific part.
- `llmTasks`: an higher level plugin that will contain various LLM tasks
to be used by assistants and genAI consumers. The intent is not to have
a single place to put all llm tasks, but more to have a default place
where we can introduce new tasks from. (fwiw, the `nlToEsql` task will
probably be moved to that plugin).

- Add a `retrieve_documentation` tool registration for the o11y
assistant
- Add a component on the o11y assistant configuration page to install
the product doc

(wiring the feature to the o11y assistant was done for testing purposes
mostly, any addition / changes / enhancement should be done by the
owning team - either in this PR or as a follow-up)

## What is NOT included in this PR:

- Wire product base feature to the security assistant (should be done by
the owning team as a follow-up)
  - installation
  - utilization as tool

- FTR tests: this is somewhat blocked by the same things we need to
figure out for elastic/kibana-team#1271

## Screenshots

### Installation from o11y assistant configuration page

<img width="1476" alt="Screenshot 2024-10-17 at 09 41 24"
src="https://github.com/user-attachments/assets/31daa585-9fb2-400a-a2d1-5917a262367a">

### Example of output

#### Without product documentation installed

<img width="739" alt="Screenshot 2024-10-10 at 09 59 41"
src="https://github.com/user-attachments/assets/993fb216-6c9a-433f-bf44-f6e383d20d9d">

#### With product documentation installed

<img width="718" alt="Screenshot 2024-10-10 at 09 55 38"
src="https://github.com/user-attachments/assets/805ea4ca-8bc9-4355-a434-0ba81f8228a9">

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Alex Szabo <alex.szabo@elastic.co>
Co-authored-by: Matthias Wilhelm <matthias.wilhelm@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
(cherry picked from commit 455c781)

# Conflicts:
#	.github/CODEOWNERS
pgayvallet added a commit that referenced this pull request Nov 19, 2024
…200754)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[LLM tasks] Add product documentation retrieval task
(#194379)](#194379)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Pierre
Gayvallet","email":"pierre.gayvallet@elastic.co"},"sourceCommit":{"committedDate":"2024-11-19T14:28:26Z","message":"[LLM
tasks] Add product documentation retrieval task (#194379)\n\n##
Summary\r\n\r\nClose
https://github.com/elastic/kibana/issues/193473\r\nClose
https://github.com/elastic/kibana/issues/193474\r\n\r\nThis PR utilize
the documentation packages that are build via the tool\r\nintroduced by
#193847, allowing to\r\ninstall
them in Kibana and expose documentation retrieval as an LLM task\r\nthat
AI assistants (or other consumers) can call.\r\n\r\nUsers can now decide
to install the Elastic documentation from the\r\nassistant's config
screen, which will expose a new tool for the\r\nassistant,
`retrieve_documentation` (only implemented for the o11y\r\nassistant in
the current PR, shall be done for security as a follow up).\r\n\r\nFor
more information, please refer to the self-review.\r\n\r\n## General
architecture\r\n\r\n<img width=\"1118\" alt=\"Screenshot 2024-10-17 at
09 22
32\"\r\nsrc=\"https://github.com/user-attachments/assets/3df8c30a-9ccc-49ab-92ce-c204b96d6fc4\">\r\n\r\n##
What this PR does\r\n\r\nAdds two plugin:\r\n- `productDocBase`:
contains all the logic related to product\r\ndocumentation installation,
status, and search. This is meant to be a\r\n\"low level\" components
only responsible for this specific part.\r\n- `llmTasks`: an higher
level plugin that will contain various LLM tasks\r\nto be used by
assistants and genAI consumers. The intent is not to have\r\na single
place to put all llm tasks, but more to have a default place\r\nwhere we
can introduce new tasks from. (fwiw, the `nlToEsql` task
will\r\nprobably be moved to that plugin).\r\n\r\n- Add a
`retrieve_documentation` tool registration for the
o11y\r\nassistant\r\n- Add a component on the o11y assistant
configuration page to install\r\nthe product doc\r\n\r\n(wiring the
feature to the o11y assistant was done for testing purposes\r\nmostly,
any addition / changes / enhancement should be done by the\r\nowning
team - either in this PR or as a follow-up)\r\n\r\n## What is NOT
included in this PR:\r\n\r\n- Wire product base feature to the security
assistant (should be done by\r\nthe owning team as a follow-up)\r\n -
installation\r\n - utilization as tool\r\n\r\n- FTR tests: this is
somewhat blocked by the same things we need to\r\nfigure out for
https://github.com/elastic/kibana-team/issues/1271\r\n\r\n## Screenshots
\r\n\r\n### Installation from o11y assistant configuration
page\r\n\r\n<img width=\"1476\" alt=\"Screenshot 2024-10-17 at 09 41
24\"\r\nsrc=\"https://github.com/user-attachments/assets/31daa585-9fb2-400a-a2d1-5917a262367a\">\r\n\r\n###
Example of output\r\n\r\n#### Without product documentation installed
\r\n\r\n<img width=\"739\" alt=\"Screenshot 2024-10-10 at 09 59
41\"\r\nsrc=\"https://github.com/user-attachments/assets/993fb216-6c9a-433f-bf44-f6e383d20d9d\">\r\n\r\n####
With product documentation installed\r\n\r\n<img width=\"718\"
alt=\"Screenshot 2024-10-10 at 09 55
38\"\r\nsrc=\"https://github.com/user-attachments/assets/805ea4ca-8bc9-4355-a434-0ba81f8228a9\">\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Alex Szabo <alex.szabo@elastic.co>\r\nCo-authored-by: Matthias Wilhelm
<matthias.wilhelm@elastic.co>\r\nCo-authored-by: Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"455c781c6d1e1161f66e275299cf06064a0ffde2","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","backport:prev-minor","ci:cloud-deploy","Team:Obs
AI Assistant","ci:project-deploy-observability","Team:AI
Infra","v8.17.0"],"number":194379,"url":"https://github.com/elastic/kibana/pull/194379","mergeCommit":{"message":"[LLM
tasks] Add product documentation retrieval task (#194379)\n\n##
Summary\r\n\r\nClose
https://github.com/elastic/kibana/issues/193473\r\nClose
https://github.com/elastic/kibana/issues/193474\r\n\r\nThis PR utilize
the documentation packages that are build via the tool\r\nintroduced by
#193847, allowing to\r\ninstall
them in Kibana and expose documentation retrieval as an LLM task\r\nthat
AI assistants (or other consumers) can call.\r\n\r\nUsers can now decide
to install the Elastic documentation from the\r\nassistant's config
screen, which will expose a new tool for the\r\nassistant,
`retrieve_documentation` (only implemented for the o11y\r\nassistant in
the current PR, shall be done for security as a follow up).\r\n\r\nFor
more information, please refer to the self-review.\r\n\r\n## General
architecture\r\n\r\n<img width=\"1118\" alt=\"Screenshot 2024-10-17 at
09 22
32\"\r\nsrc=\"https://github.com/user-attachments/assets/3df8c30a-9ccc-49ab-92ce-c204b96d6fc4\">\r\n\r\n##
What this PR does\r\n\r\nAdds two plugin:\r\n- `productDocBase`:
contains all the logic related to product\r\ndocumentation installation,
status, and search. This is meant to be a\r\n\"low level\" components
only responsible for this specific part.\r\n- `llmTasks`: an higher
level plugin that will contain various LLM tasks\r\nto be used by
assistants and genAI consumers. The intent is not to have\r\na single
place to put all llm tasks, but more to have a default place\r\nwhere we
can introduce new tasks from. (fwiw, the `nlToEsql` task
will\r\nprobably be moved to that plugin).\r\n\r\n- Add a
`retrieve_documentation` tool registration for the
o11y\r\nassistant\r\n- Add a component on the o11y assistant
configuration page to install\r\nthe product doc\r\n\r\n(wiring the
feature to the o11y assistant was done for testing purposes\r\nmostly,
any addition / changes / enhancement should be done by the\r\nowning
team - either in this PR or as a follow-up)\r\n\r\n## What is NOT
included in this PR:\r\n\r\n- Wire product base feature to the security
assistant (should be done by\r\nthe owning team as a follow-up)\r\n -
installation\r\n - utilization as tool\r\n\r\n- FTR tests: this is
somewhat blocked by the same things we need to\r\nfigure out for
https://github.com/elastic/kibana-team/issues/1271\r\n\r\n## Screenshots
\r\n\r\n### Installation from o11y assistant configuration
page\r\n\r\n<img width=\"1476\" alt=\"Screenshot 2024-10-17 at 09 41
24\"\r\nsrc=\"https://github.com/user-attachments/assets/31daa585-9fb2-400a-a2d1-5917a262367a\">\r\n\r\n###
Example of output\r\n\r\n#### Without product documentation installed
\r\n\r\n<img width=\"739\" alt=\"Screenshot 2024-10-10 at 09 59
41\"\r\nsrc=\"https://github.com/user-attachments/assets/993fb216-6c9a-433f-bf44-f6e383d20d9d\">\r\n\r\n####
With product documentation installed\r\n\r\n<img width=\"718\"
alt=\"Screenshot 2024-10-10 at 09 55
38\"\r\nsrc=\"https://github.com/user-attachments/assets/805ea4ca-8bc9-4355-a434-0ba81f8228a9\">\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Alex Szabo <alex.szabo@elastic.co>\r\nCo-authored-by: Matthias Wilhelm
<matthias.wilhelm@elastic.co>\r\nCo-authored-by: Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"455c781c6d1e1161f66e275299cf06064a0ffde2"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/194379","number":194379,"mergeCommit":{"message":"[LLM
tasks] Add product documentation retrieval task (#194379)\n\n##
Summary\r\n\r\nClose
https://github.com/elastic/kibana/issues/193473\r\nClose
https://github.com/elastic/kibana/issues/193474\r\n\r\nThis PR utilize
the documentation packages that are build via the tool\r\nintroduced by
#193847, allowing to\r\ninstall
them in Kibana and expose documentation retrieval as an LLM task\r\nthat
AI assistants (or other consumers) can call.\r\n\r\nUsers can now decide
to install the Elastic documentation from the\r\nassistant's config
screen, which will expose a new tool for the\r\nassistant,
`retrieve_documentation` (only implemented for the o11y\r\nassistant in
the current PR, shall be done for security as a follow up).\r\n\r\nFor
more information, please refer to the self-review.\r\n\r\n## General
architecture\r\n\r\n<img width=\"1118\" alt=\"Screenshot 2024-10-17 at
09 22
32\"\r\nsrc=\"https://github.com/user-attachments/assets/3df8c30a-9ccc-49ab-92ce-c204b96d6fc4\">\r\n\r\n##
What this PR does\r\n\r\nAdds two plugin:\r\n- `productDocBase`:
contains all the logic related to product\r\ndocumentation installation,
status, and search. This is meant to be a\r\n\"low level\" components
only responsible for this specific part.\r\n- `llmTasks`: an higher
level plugin that will contain various LLM tasks\r\nto be used by
assistants and genAI consumers. The intent is not to have\r\na single
place to put all llm tasks, but more to have a default place\r\nwhere we
can introduce new tasks from. (fwiw, the `nlToEsql` task
will\r\nprobably be moved to that plugin).\r\n\r\n- Add a
`retrieve_documentation` tool registration for the
o11y\r\nassistant\r\n- Add a component on the o11y assistant
configuration page to install\r\nthe product doc\r\n\r\n(wiring the
feature to the o11y assistant was done for testing purposes\r\nmostly,
any addition / changes / enhancement should be done by the\r\nowning
team - either in this PR or as a follow-up)\r\n\r\n## What is NOT
included in this PR:\r\n\r\n- Wire product base feature to the security
assistant (should be done by\r\nthe owning team as a follow-up)\r\n -
installation\r\n - utilization as tool\r\n\r\n- FTR tests: this is
somewhat blocked by the same things we need to\r\nfigure out for
https://github.com/elastic/kibana-team/issues/1271\r\n\r\n## Screenshots
\r\n\r\n### Installation from o11y assistant configuration
page\r\n\r\n<img width=\"1476\" alt=\"Screenshot 2024-10-17 at 09 41
24\"\r\nsrc=\"https://github.com/user-attachments/assets/31daa585-9fb2-400a-a2d1-5917a262367a\">\r\n\r\n###
Example of output\r\n\r\n#### Without product documentation installed
\r\n\r\n<img width=\"739\" alt=\"Screenshot 2024-10-10 at 09 59
41\"\r\nsrc=\"https://github.com/user-attachments/assets/993fb216-6c9a-433f-bf44-f6e383d20d9d\">\r\n\r\n####
With product documentation installed\r\n\r\n<img width=\"718\"
alt=\"Screenshot 2024-10-10 at 09 55
38\"\r\nsrc=\"https://github.com/user-attachments/assets/805ea4ca-8bc9-4355-a434-0ba81f8228a9\">\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Alex Szabo <alex.szabo@elastic.co>\r\nCo-authored-by: Matthias Wilhelm
<matthias.wilhelm@elastic.co>\r\nCo-authored-by: Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"455c781c6d1e1161f66e275299cf06064a0ffde2"}},{"branch":"8.x","label":"v8.17.0","labelRegex":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
paulinashakirova pushed a commit to paulinashakirova/kibana that referenced this pull request Nov 26, 2024
## Summary

Close elastic#193473
Close elastic#193474

This PR utilize the documentation packages that are build via the tool
introduced by elastic#193847, allowing to
install them in Kibana and expose documentation retrieval as an LLM task
that AI assistants (or other consumers) can call.

Users can now decide to install the Elastic documentation from the
assistant's config screen, which will expose a new tool for the
assistant, `retrieve_documentation` (only implemented for the o11y
assistant in the current PR, shall be done for security as a follow up).

For more information, please refer to the self-review.

## General architecture

<img width="1118" alt="Screenshot 2024-10-17 at 09 22 32"
src="https://github.com/user-attachments/assets/3df8c30a-9ccc-49ab-92ce-c204b96d6fc4">

## What this PR does

Adds two plugin:
- `productDocBase`: contains all the logic related to product
documentation installation, status, and search. This is meant to be a
"low level" components only responsible for this specific part.
- `llmTasks`: an higher level plugin that will contain various LLM tasks
to be used by assistants and genAI consumers. The intent is not to have
a single place to put all llm tasks, but more to have a default place
where we can introduce new tasks from. (fwiw, the `nlToEsql` task will
probably be moved to that plugin).

- Add a `retrieve_documentation` tool registration for the o11y
assistant
- Add a component on the o11y assistant configuration page to install
the product doc

(wiring the feature to the o11y assistant was done for testing purposes
mostly, any addition / changes / enhancement should be done by the
owning team - either in this PR or as a follow-up)

## What is NOT included in this PR:

- Wire product base feature to the security assistant (should be done by
the owning team as a follow-up)
  - installation
  - utilization as tool

- FTR tests: this is somewhat blocked by the same things we need to
figure out for elastic/kibana-team#1271

## Screenshots 

### Installation from o11y assistant configuration page

<img width="1476" alt="Screenshot 2024-10-17 at 09 41 24"
src="https://github.com/user-attachments/assets/31daa585-9fb2-400a-a2d1-5917a262367a">

### Example of output

#### Without product documentation installed 

<img width="739" alt="Screenshot 2024-10-10 at 09 59 41"
src="https://github.com/user-attachments/assets/993fb216-6c9a-433f-bf44-f6e383d20d9d">

#### With product documentation installed

<img width="718" alt="Screenshot 2024-10-10 at 09 55 38"
src="https://github.com/user-attachments/assets/805ea4ca-8bc9-4355-a434-0ba81f8228a9">

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Alex Szabo <alex.szabo@elastic.co>
Co-authored-by: Matthias Wilhelm <matthias.wilhelm@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this pull request Dec 12, 2024
## Summary

Close elastic#193473
Close elastic#193474

This PR utilize the documentation packages that are build via the tool
introduced by elastic#193847, allowing to
install them in Kibana and expose documentation retrieval as an LLM task
that AI assistants (or other consumers) can call.

Users can now decide to install the Elastic documentation from the
assistant's config screen, which will expose a new tool for the
assistant, `retrieve_documentation` (only implemented for the o11y
assistant in the current PR, shall be done for security as a follow up).

For more information, please refer to the self-review.

## General architecture

<img width="1118" alt="Screenshot 2024-10-17 at 09 22 32"
src="https://github.com/user-attachments/assets/3df8c30a-9ccc-49ab-92ce-c204b96d6fc4">

## What this PR does

Adds two plugin:
- `productDocBase`: contains all the logic related to product
documentation installation, status, and search. This is meant to be a
"low level" components only responsible for this specific part.
- `llmTasks`: an higher level plugin that will contain various LLM tasks
to be used by assistants and genAI consumers. The intent is not to have
a single place to put all llm tasks, but more to have a default place
where we can introduce new tasks from. (fwiw, the `nlToEsql` task will
probably be moved to that plugin).

- Add a `retrieve_documentation` tool registration for the o11y
assistant
- Add a component on the o11y assistant configuration page to install
the product doc

(wiring the feature to the o11y assistant was done for testing purposes
mostly, any addition / changes / enhancement should be done by the
owning team - either in this PR or as a follow-up)

## What is NOT included in this PR:

- Wire product base feature to the security assistant (should be done by
the owning team as a follow-up)
  - installation
  - utilization as tool

- FTR tests: this is somewhat blocked by the same things we need to
figure out for elastic/kibana-team#1271

## Screenshots 

### Installation from o11y assistant configuration page

<img width="1476" alt="Screenshot 2024-10-17 at 09 41 24"
src="https://github.com/user-attachments/assets/31daa585-9fb2-400a-a2d1-5917a262367a">

### Example of output

#### Without product documentation installed 

<img width="739" alt="Screenshot 2024-10-10 at 09 59 41"
src="https://github.com/user-attachments/assets/993fb216-6c9a-433f-bf44-f6e383d20d9d">

#### With product documentation installed

<img width="718" alt="Screenshot 2024-10-10 at 09 55 38"
src="https://github.com/user-attachments/assets/805ea4ca-8bc9-4355-a434-0ba81f8228a9">

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Alex Szabo <alex.szabo@elastic.co>
Co-authored-by: Matthias Wilhelm <matthias.wilhelm@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:prev-minor Backport to (9.0) the previous minor version (i.e. one version back from main) release_note:skip Skip the PR/issue when compiling release notes Team:AI Infra AppEx AI Infrastructure Team v8.16.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants