Skip to content
This repository has been archived by the owner on Jan 9, 2025. It is now read-only.

feat: add elasticsearch component #211

Merged
merged 27 commits into from
Aug 8, 2024

Conversation

zidanehakim
Copy link
Collaborator

@zidanehakim zidanehakim commented Jul 11, 2024

Because

  • We need to integrate Elasticsearch into our data pipeline.
  • This will allow us to cover data movement user stories

This commit

  • Allow client to connect to their Elasticsearch database through VDP

  • Auth using passwordless API Key and cloud ID through elasticsearch go sdk

  • Create execution function to handle these following tasks

  • TASK_INDEX (INSERT)

  • TASK_UPDATE

  • TASK_SEARCH

  • TASK_DELETE

  • TASK_CREATE_INDEX

  • TASK_DELETE_INDEX

Copy link

codecov bot commented Jul 21, 2024

Codecov Report

Attention: Patch coverage is 51.00154% with 318 lines in your changes missing coverage. Please review.

Project coverage is 40.55%. Comparing base (219c77e) to head (f300438).
Report is 6 commits behind head on main.

Files Patch % Lines
data/elasticsearch/v0/tasks.go 55.00% 177 Missing and 84 partials ⚠️
data/elasticsearch/v0/main.go 26.08% 32 Missing and 2 partials ⚠️
data/elasticsearch/v0/client.go 0.00% 22 Missing ⚠️
store/store.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #211      +/-   ##
==========================================
+ Coverage   39.75%   40.55%   +0.80%     
==========================================
  Files         123      128       +5     
  Lines       12351    13490    +1139     
==========================================
+ Hits         4910     5471     +561     
- Misses       6808     7228     +420     
- Partials      633      791     +158     
Flag Coverage Δ
unittests 40.55% <51.00%> (+0.80%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

data/elasticsearch/v0/tasks.go Outdated Show resolved Hide resolved
data/elasticsearch/v0/config/tasks.json Outdated Show resolved Hide resolved
data/elasticsearch/v0/config/tasks.json Outdated Show resolved Hide resolved
@zidanehakim zidanehakim requested a review from donch1989 August 8, 2024 08:38
@donch1989 donch1989 merged commit eb492ca into instill-ai:main Aug 8, 2024
8 checks passed
donch1989 pushed a commit that referenced this pull request Aug 13, 2024
🤖 I have created a release *beep* *boop*
---


##
[0.25.0-beta](v0.24.0-beta...v0.25.0-beta)
(2024-08-13)


### Features

* add a hook to avoid we miss make document
([#244](#244))
([4c4531d](4c4531d))
* add elasticsearch component
([#211](#211))
([eb492ca](eb492ca))
* add Fireworks AI component
([#237](#237))
([0c40652](0c40652))
* add Groq component
([#269](#269))
([1401220](1401220))
* add mongodb component
([#198](#198))
([2cb550f](2cb550f))
* add qdrant component
([#271](#271))
([bd2b9e6](bd2b9e6))
* add weaviate component
([#246](#246))
([cb3e667](cb3e667))
* add WhatsApp component
([#226](#226))
([28d0de8](28d0de8))
* **artifact:** add artifact component
([#268](#268))
([dabf472](dabf472))
* **artifact:** add artifact component
([#275](#275))
([15fc0d2](15fc0d2))
* **document:** integrate pdf2md in document operator
([#277](#277))
([07360d1](07360d1))
* **groq, fireworksai:** take out the unsupported models from instill
credit ([#283](#283))
([8978acd](8978acd))
* make component ID accessible on IExecution
([#257](#257))
([dd63656](dd63656))
* **openai:** support `gpt-4o-2024-08-06` and structured output
([#280](#280))
([8bdaef7](8bdaef7))
* **sql:** add TASK_INSERT_MANY and fix sql query validation
([#252](#252))
([3a93cea](3a93cea))
* **text:** add tokenizer for cohere & new gpt-4o
([#276](#276))
([5d8cec3](5d8cec3))
* **text:** revert "add tokenizer for cohere & new gpt-4o
([#276](#276))"
([910a330](910a330))


### Bug Fixes

* **artifact:** add the description to remind users to add file
extension ([#281](#281))
([5ff5d7a](5ff5d7a))
* ignore bold case and add all line to result
([#272](#272))
([219c77e](219c77e))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
No open projects
Status: 👋 Done
Development

Successfully merging this pull request may close these issues.

3 participants