Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Job fixes #573

Merged
merged 4 commits into from
Sep 29, 2017
Merged

Job fixes #573

merged 4 commits into from
Sep 29, 2017

Conversation

jsnoble
Copy link
Member

@jsnoble jsnoble commented Sep 28, 2017

the issue with updating the job was that it would save the job without validating it. Validating it would also give the job all the missing fields that it needs to correctly run hence the interval error that you have listed in the ticket. I choose to validate at the update to make sure any attempted update changes are correct instead of waiting at startJob to do so.

@godber
Copy link
Member

godber commented Sep 29, 2017

I don't know that the fix for #568 works right.

I created a reindex job.

Stopped that job.

Tweaked it so that the time_resolution was invalid.

Submitted that updated definition to the existing job_id as follows

curl -XPUT -Ss localhost:5678/jobs/a59ad582-4f5b-43f2-af7e-a8844fa581a6 -d@/Users/godber/Workspace/terascope/opensource/teraslice/examples/jobs/reindex.json

There was no output from curl but the following error showed up in the teraslice logs:

[2017-09-29T00:57:32.177Z] ERROR: teraslice/8351 on stretch.local: (module=jobs_service)
    could not updateJob Error validating job: Error: Error: time_resolution: time_resolution for elasticsearch_reader must be set in either "s"[seconds] or "ms"[milliseconds]: value was "clowns"
        at Object.rv.validate (/Users/godber/Workspace/terascope/opensource/teraslice/node_modules/convict/lib/convict.js:501:15)
        at Object.validateConfig (/Users/godber/Workspace/terascope/opensource/teraslice/lib/config/validators/config.js:26:16)
        at /Users/godber/Workspace/terascope/opensource/teraslice/lib/config/validators/job.js:41:45
        at Array.map (native)
        at Object.validate (/Users/godber/Workspace/terascope/opensource/teraslice/lib/config/validators/job.js:37:46)
        at /Users/godber/Workspace/terascope/opensource/teraslice/lib/cluster/services/jobs.js:586:46
        at tryCatcher (/Users/godber/Workspace/terascope/opensource/teraslice/node_modules/bluebird/js/main/util.js:26:23)
        at Promise._resolveFromResolver (/Users/godber/Workspace/terascope/opensource/teraslice/node_modules/bluebird/js/main/promise.js:483:31)
        at new Promise (/Users/godber/Workspace/terascope/opensource/teraslice/node_modules/bluebird/js/main/promise.js:71:37)
        at _validateJob (/Users/godber/Workspace/terascope/opensource/teraslice/lib/cluster/services/jobs.js:583:16)
        at /Users/godber/Workspace/terascope/opensource/teraslice/lib/cluster/services/jobs.js:347:24
        at tryCatcher (/Users/godber/Workspace/terascope/opensource/teraslice/node_modules/elasticsearch_api/node_modules/bluebird/js/release/util.js:16:23)
        at Promise._settlePromiseFromHandler (/Users/godber/Workspace/terascope/opensource/teraslice/node_modules/elasticsearch_api/node_modules/bluebird/js/release/promise.js:512:31)
        at Promise._settlePromise (/Users/godber/Workspace/terascope/opensource/teraslice/node_modules/elasticsearch_api/node_modules/bluebird/js/release/promise.js:569:18)
        at Promise._settlePromise0 (/Users/godber/Workspace/terascope/opensource/teraslice/node_modules/elasticsearch_api/node_modules/bluebird/js/release/promise.js:614:10)
        at Promise._settlePromises (/Users/godber/Workspace/terascope/opensource/teraslice/node_modules/elasticsearch_api/node_modules/bluebird/js/release/promise.js:693:18)
        at Async._drainQueue (/Users/godber/Workspace/terascope/opensource/teraslice/node_modules/elasticsearch_api/node_modules/bluebird/js/release/async.js:133:16)
        at Async._drainQueues (/Users/godber/Workspace/terascope/opensource/teraslice/node_modules/elasticsearch_api/node_modules/bluebird/js/release/async.js:143:10)
        at Immediate.Async.drainQueues [as _onImmediate] (/Users/godber/Workspace/terascope/opensource/teraslice/node_modules/elasticsearch_api/node_modules/bluebird/js/release/async.js:17:14)
        at processImmediate [as _immediateCallback] (timers.js:396:17)

It does look like I got a 200 response from Express (when using curl -v)

curl -v -XPUT -Ss localhost:5678/jobs/a59ad582-4f5b-43f2-af7e-a8844fa581a6 -d@/Users/godber/Workspace/terascope/opensource/teraslice/examples/jobs/reindex.json
*   Trying ::1...
* Connected to localhost (::1) port 5678 (#0)
> PUT /jobs/a59ad582-4f5b-43f2-af7e-a8844fa581a6 HTTP/1.1
> Host: localhost:5678
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Length: 586
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 586 out of 586 bytes
< HTTP/1.1 200 OK
< X-Powered-By: Express
< Content-Type: application/json
< Date: Fri, 29 Sep 2017 01:14:21 GMT
< Connection: keep-alive
< Content-Length: 0
<
* Connection #0 to host localhost left intact

I was expecting an error returned to me when updating the job.

@godber
Copy link
Member

godber commented Sep 29, 2017

The latest commit fixes it:

curl -XPUT -Ss localhost:5678/jobs/a59ad582-4f5b-43f2-af7e-a8844fa581a6 -d@/Users/godber/Workspace/terascope/opensource/teraslice/examples/jobs/reindex.json
{
    "error": 500,
    "message": "Could not update job"
}

@godber
Copy link
Member

godber commented Sep 29, 2017

The fix for #562 appears to work:

curl -Ss localhost:5678/jobs/347eefed-54c5-4128-aa46-b7a880705aaa/_stop -XPOST
{
    "error": 500,
    "message": "no execution context was found for job_id: 347eefed-54c5-4128-aa46-b7a880705aaa"
}

@godber
Copy link
Member

godber commented Sep 29, 2017

job_id appears added to logs, #572 wasn't really explicit enough to be sure.

@godber
Copy link
Member

godber commented Sep 29, 2017

Integration tests pass on latest commit.

@godber godber merged commit a76761b into terascope:master Sep 29, 2017
@jsnoble jsnoble deleted the job_fixes branch November 21, 2017 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants