Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add simulate.ingest specification #3400

Merged
merged 4 commits into from
Jan 8, 2025
Merged

Add simulate.ingest specification #3400

merged 4 commits into from
Jan 8, 2025

Conversation

lcawl
Copy link
Contributor

@lcawl lcawl commented Jan 1, 2025

Relates to #3377

This PR adds a missing specification based on https://www.elastic.co/guide/en/elasticsearch/reference/master/simulate-ingest-api.html
It also moves the existing simulate ingest example files to appropriate subfolders

Copy link
Contributor

github-actions bot commented Jan 1, 2025

Following you can find the validation results for the APIs you have changed.

API Status Request Response
ingest.delete_geoip_database 🟢 1/1 1/1
ingest.delete_ip_location_database 🟢 1/1 1/1
ingest.delete_pipeline 🟢 15/15 15/15
ingest.geo_ip_stats 🟢 1/1 1/1
ingest.get_geoip_database 🟢 6/6 6/6
ingest.get_ip_location_database 🟢 7/7 7/7
ingest.get_pipeline 🟢 22/22 22/22
ingest.processor_grok 🟢 1/1 1/1
ingest.put_geoip_database 🟢 3/3 3/3
ingest.put_ip_location_database 🟢 4/4 4/4
ingest.put_pipeline 🟢 60/60 60/60
ingest.simulate 🟢 10/10 10/10
simulate.ingest 🔴 11/12 12/12

You can validate these APIs yourself by using the make validate target.

@lcawl lcawl marked this pull request as ready for review January 1, 2025 08:12
@l-trotta l-trotta assigned l-trotta and unassigned l-trotta Jan 7, 2025
Copy link
Contributor

github-actions bot commented Jan 7, 2025

Following you can find the validation results for the APIs you have changed.

API Status Request Response
ingest.delete_geoip_database 🟢 1/1 1/1
ingest.delete_ip_location_database 🟢 1/1 1/1
ingest.delete_pipeline 🟢 15/15 15/15
ingest.geo_ip_stats 🟢 1/1 1/1
ingest.get_geoip_database 🟢 6/6 6/6
ingest.get_ip_location_database 🟢 7/7 7/7
ingest.get_pipeline 🟢 22/22 22/22
ingest.processor_grok 🟢 1/1 1/1
ingest.put_geoip_database 🟢 3/3 3/3
ingest.put_ip_location_database 🟢 4/4 4/4
ingest.put_pipeline 🟢 60/60 60/60
ingest.simulate 🟢 10/10 10/10
simulate.ingest 🔴 11/12 12/12

You can validate these APIs yourself by using the make validate target.

@JoshMock JoshMock self-requested a review January 7, 2025 18:13
@l-trotta
Copy link
Contributor

l-trotta commented Jan 7, 2025

I have 2 main doubts I need to clear with this one:

  • Request: while everything looks good, it seems like the server processes all maps as Map<String, Map<String, Object>> rather then parsing the specific objects, leading to validation fails. we need to decide if we want to be generic in the same way.
  • Response: the response does looks very similar to the SimulateDocumentResult class we have in the spec, but it slightly differs, so we should map it from scratch based on what the server outputs. I see that in the documentation there's an executed_pipelines field which I cannot find anywhere in the server code except for SimulateIndexResponse, which should be another case, could it be wrong?

Copy link
Contributor

github-actions bot commented Jan 8, 2025

Following you can find the validation results for the APIs you have changed.

API Status Request Response
ingest.delete_geoip_database 🟢 1/1 1/1
ingest.delete_ip_location_database 🟢 1/1 1/1
ingest.delete_pipeline 🟢 15/15 15/15
ingest.geo_ip_stats 🟢 1/1 1/1
ingest.get_geoip_database 🟢 6/6 6/6
ingest.get_ip_location_database 🟢 7/7 7/7
ingest.get_pipeline 🟢 22/22 22/22
ingest.processor_grok 🟢 1/1 1/1
ingest.put_geoip_database 🟢 3/3 3/3
ingest.put_ip_location_database 🟢 4/4 4/4
ingest.put_pipeline 🟢 60/60 60/60
ingest.simulate 🟢 10/10 10/10
simulate.ingest 🔴 11/12 12/12

You can validate these APIs yourself by using the make validate target.

1 similar comment
Copy link
Contributor

github-actions bot commented Jan 8, 2025

Following you can find the validation results for the APIs you have changed.

API Status Request Response
ingest.delete_geoip_database 🟢 1/1 1/1
ingest.delete_ip_location_database 🟢 1/1 1/1
ingest.delete_pipeline 🟢 15/15 15/15
ingest.geo_ip_stats 🟢 1/1 1/1
ingest.get_geoip_database 🟢 6/6 6/6
ingest.get_ip_location_database 🟢 7/7 7/7
ingest.get_pipeline 🟢 22/22 22/22
ingest.processor_grok 🟢 1/1 1/1
ingest.put_geoip_database 🟢 3/3 3/3
ingest.put_ip_location_database 🟢 4/4 4/4
ingest.put_pipeline 🟢 60/60 60/60
ingest.simulate 🟢 10/10 10/10
simulate.ingest 🔴 11/12 12/12

You can validate these APIs yourself by using the make validate target.

Copy link
Member

@JoshMock JoshMock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my concerns are all addressed now. looks like you have a couple conflicts to work out, but otherwise LGTM.

Copy link
Contributor

github-actions bot commented Jan 8, 2025

Following you can find the validation results for the APIs you have changed.

API Status Request Response
ingest.delete_geoip_database 🟢 1/1 1/1
ingest.delete_ip_location_database 🟢 1/1 1/1
ingest.delete_pipeline 🟢 15/15 15/15
ingest.geo_ip_stats 🟢 1/1 1/1
ingest.get_geoip_database 🟢 6/6 6/6
ingest.get_ip_location_database 🟢 7/7 7/7
ingest.get_pipeline 🟢 22/22 22/22
ingest.processor_grok 🟢 1/1 1/1
ingest.put_geoip_database 🟢 3/3 3/3
ingest.put_ip_location_database 🟢 4/4 4/4
ingest.put_pipeline 🟢 60/60 60/60
ingest.simulate 🟢 10/10 10/10
simulate.ingest 🔴 11/12 12/12

You can validate these APIs yourself by using the make validate target.

@lcawl lcawl merged commit 46b99ee into main Jan 8, 2025
8 checks passed
@lcawl lcawl deleted the simulate-ingest branch January 8, 2025 21:04
Copy link
Contributor

github-actions bot commented Jan 8, 2025

The backport to 8.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-8.x 8.x
# Navigate to the new working tree
cd .worktrees/backport-8.x
# Create a new branch
git switch --create backport-3400-to-8.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 46b99ee0c07b9aabd0fa6f3dcdcc63738cb219d9
# Push it to GitHub
git push --set-upstream origin backport-3400-to-8.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-8.x

Then, create a pull request where the base branch is 8.x and the compare/head branch is backport-3400-to-8.x.

Copy link
Contributor

github-actions bot commented Jan 8, 2025

The backport to 8.17 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-8.17 8.17
# Navigate to the new working tree
cd .worktrees/backport-8.17
# Create a new branch
git switch --create backport-3400-to-8.17
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 46b99ee0c07b9aabd0fa6f3dcdcc63738cb219d9
# Push it to GitHub
git push --set-upstream origin backport-3400-to-8.17
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-8.17

Then, create a pull request where the base branch is 8.17 and the compare/head branch is backport-3400-to-8.17.

lcawl added a commit that referenced this pull request Jan 8, 2025
@lcawl
Copy link
Contributor Author

lcawl commented Jan 8, 2025

💚 All backports created successfully

Status Branch Result
8.x

Questions ?

Please refer to the Backport tool documentation

lcawl added a commit that referenced this pull request Jan 8, 2025
@lcawl
Copy link
Contributor Author

lcawl commented Jan 8, 2025

💚 All backports created successfully

Status Branch Result
8.17

Questions ?

Please refer to the Backport tool documentation

lcawl added a commit that referenced this pull request Jan 8, 2025
lcawl added a commit that referenced this pull request Jan 9, 2025
lcawl added a commit that referenced this pull request Jan 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants