feat: use new batched queries in task replacement (bug 1840830) #496
Conversation
6a7f2a0 to
a8f5f09
Compare
|
I have no idea what's up with those task failures, never seen that before.. |
|
Wow this decision task generated quite a messed up graph. Things end up using the index-task docker image instead of whatever docker image they wanted. full-task-graph.json looks sane, but task-graph.json has PebYGdYSQgu0_iTKTCg_fQ as dependency where it shouldn't be. |
|
I think I didn't have this problem when I hadn't yet changed the tests (to check if the local failures were the same). Does that give any hints as to what I should do @jcristau ? |
3e84498 to
74d994f
Compare
9b4dce5 to
16c441e
Compare
|
Tests are fixed, I hadn't fixed a problem in both places it was and the result was confusing to me. Everything seems good now. |
|
Tests are fixed, I hadn't fixed a problem in both places it was and the result was confusing to me. Everything seems good now. |
16c441e to
b079932
Compare
|
Thanks for the quick review and the very pertinent comments. |
b079932 to
6434015
Compare
Batching queries to get from task index to status reduces the number of (sometimes trans-continental) queries from 2*900+ to ~2. This reduces the time spent replacing tasks from 20% to 75% depending on the use-case. The 20% improvement in wall time was observed when running `mach taskgraph morphed` in a CI worker, while the 75% improvement was observed in a developer machine in France running `mach taskgraph full`. More information in taskcluster/taskcluster-rfcs#189.
|
@ahal Thanks, all of your comments should be addressed now. |
Batching queries to get from task index to status reduces the number of (sometimes trans-continental) queries from 2*900+ to ~2. This reduces the time spent replacing tasks from 20% to 75% depending on the use-case.
The 20% improvement in wall time was observed when running
mach taskgraph morphedin a CI worker, while the 75% improvement was observed in a developer machine in France runningmach taskgraph full.More information in taskcluster/taskcluster-rfcs#189.