Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Script: Update Metadata #4

Open
wants to merge 28 commits into
base: pain_wfield-ingest-doc-meta
Choose a base branch
from

Conversation

stu-elastic
Copy link
Owner

No description provided.

stu-elastic and others added 28 commits June 7, 2022 11:26
Adds the `meta()` API call and a Metadata class for the Update context

There are different metadata available in the update context depending
on whether it is an update or an insert (via upsert).

For update, scripts can read `index`, `id`, `routing`, `version`, `timestamp` and `type`.

For insert, scripts can read `index`, `id` and `timestamp`.

Scripts can always read and write the op but the available ops are different.
Updates allow `NOOP`, `INDEX` and `DELETE`.
Inserts allow `NOOP` and `CREATE`.

Refs: elastic#86472
…stic/elasticsearch into pain_wfield-update-only-metadata
…elasticsearch into pain_wfield-update-only-metadata
…elasticsearch into pain_wfield-update-only-metadata
Adds the `Metadata` class and `metadata()` method to the ingest context.

Metadata has getters and setters for index, id, routing, version and versionType.
It also has a getter for timestamp.

Refs: elastic#86472
This commit changes the `/_nodes/stats` API to return the pipelines in
descending order by the time spent executing. If a pipeline ties another
pipeline, they are sorted by ingest count.

This ensures that the most used and most expensive pipelines show up
first in the JSON response (to help tracking down which pipelines may be
most prevalent).

For example:

```
{
  "nodes" : {
    "yiv09_ETRKSv4q8kz0d59A" : {
      ...
      "ingest" : {
        "total" : {...},
        "pipelines" : {
          "set-name" : {
            "count" : 4, // <-- tie-breaker
            "time_in_millis" : 12, // <-- sorted first by this
            ...
          },
          "set-face" : {
            "count" : 2,
            "time_in_millis" : 8,  // <-- less than 12
            ...
          }
```
…87763)

This commit only introduces the storage classes, unused for now.
Relates to elastic#86224
This commit bumps the version of randomized runner that tests use to
2.8.0.
Test methods tend to have descriptive names for the cases that they are
testing. Sometimes they require javadoc, for sure, but not often enough
to add a little "WEAK WARNING" marker on every single test method in my
IDE.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Generate release notes for v8.3.0 (elastic#87294)

* Generate release notes for v8.3.0

* [DOCS] Add 8.3 migration file to index

* Fixed version number

* Fix formatting of deprecation in 85326

* Use asciidoc format for deprecations

* Extract static content from migration/index

* This was just an enhancement

* Nope, it was an upgrade

* Added migration/index.asciidoc generation support (elastic#87318)

Including extracting static content from migration/index, so the template would be as light as possible.

The reason for this work is because the gradle task `generateReleaseNotes` was not correctly adding new links and imports to the migrations/index and that caused documentation to fail building for 8.3.0.

* [DOCS] Add ml-cpp PRs to release notes

* Added back incorrectly deleted changlog

* Added missing highlight

* Fixed spelling of StackOverflowError

Co-authored-by: lcawl <lcawley@elastic.co>

* Brought back missing changelog for 87235 (elastic#87370)

* Brought back missing changelog for 87235

* Regenerate release notes

* Regenerate release notes for BC3 (elastic#87449)

* Regenerate release notes for BC3

* Re-applied manual fixes that Lisa Cawley used

* Fixed breaking changes generation

To match the manual edits done by Lisa Cawley

* Fixed failing test

Since the manual edits by Lisa removed the `-SNAPSHOT` from the docs
we remove it from the tests too.

* Remove changelogs for intermediate lucene upgrades

* Update release notes for BC4 (elastic#87635)

* Update release notes for BC4

* Adding missing changelog for geo_grid

* Added missing highlight notes for 84250

* Update docs/reference/release-notes/highlights.asciidoc

Co-authored-by: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com>

* Update docs/reference/release-notes/highlights.asciidoc

Co-authored-by: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com>

* Backported fixes to original yaml

* Control sort order of release highlights

So that small changes to an individual highlight don't completely shuffle the entire document.

We also added links to the PRs from the highlight titles, for convenience.
Otherwise readers need to search the release notes for the changelog entry and click there,
which is a lot more work.

* Fixed failing test after new ordered highlights

Also made test verify re-ordering of highlights

Co-authored-by: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com>

* Update release notes for BC5 (elastic#87808)

* Update release notes for BC6 (elastic#87912)

* Update release notes for BC9 (elastic#88011)

* Regenerate release notes after version bump 8.4.0

And after forward porting 8.3.0 release notes changes.

* Regenerate after some changelog entries removed

* Re-prune the changelogs

* Removed three changelog entries

Co-authored-by: lcawl <lcawley@elastic.co>
Co-authored-by: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com>
…ult (elastic#88148)

This changes AbstractGradleFuncTest to run with configuration cache enabled by default. That gives us better test coverage on how our build logic works with configuration cache enabled and makes it explicit which parts are not yet working.
We fixed some minor configuration cache related issues in our code.
S3 sometimes enters a state where blob downloads repeatedly fail but
with nontrivial progress between failures. Often each attempt yields 10s
or 100s of MBs of data. Today we abort a download after three (by
default) such failures, but this may not be enough to completely
retrieve a large blob during one of these flaky patches.

With this commit we start to avoid counting download attempts that
retrieved at least 1% of the configured `buffer_size` (typically 1MB)
towards the maximum number of retries.

Closes elastic#87243
Block the cluster applier before disrupting the cluster so the victim
node doesn't try and rejoin too soon.

Closes elastic#86974
…8193)

Add a section on running failed tests as indicated by CI.
User profile is behind a feature flag in 8.3. When 8.3 is still a
snapshot build, the feature flag is enabled automatically. But now 8.3
is released, the feature flag is by default off. This PR adjust the node
selector to only select node version 8.4 or higher.

Resolves: elastic#88202
For fully-formed single-node clusters, emit a periodic warning if seed_hosts has been set to a non-empty list.

Closes elastic#85222
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.