Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add DocumentDB support #2

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Add DocumentDB support #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
d0ff49a
Typo
slayer1551 Jun 1, 2022
b936d3b
Fix new job window modal from not closing
slayer1551 Jun 1, 2022
9e14bd6
AWS Document DB Support
slayer1551 Jun 1, 2022
d90ea5f
Fix totalPage count
slayer1551 Jun 1, 2022
32b3687
Update agendash.js
slayer1551 Jun 1, 2022
a76b9e5
Temp: Remove the attempt to do a $facet query
jaywink Sep 15, 2022
53e2739
Temp: Apply a 1000 document limit to the jobs query
jaywink Sep 15, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 84 additions & 58 deletions lib/controllers/agendash.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ module.exports = function (agenda, options) {
} else {
preMatch[options.property] = { $regex: options.query, $options: "i" };
}

}

const postMatch = {};
Expand All @@ -56,69 +57,94 @@ module.exports = function (agenda, options) {
}

const collection = agenda._collection.collection || agenda._collection;
return collection
.aggregate([
{ $match: preMatch },
{
$sort: {
nextRunAt: -1,
lastRunAt: -1,
lastFinishedAt: -1,
},
const aggregateQuery = [
{ $match: preMatch },
{
$sort: {
nextRunAt: -1,
lastRunAt: -1,
lastFinishedAt: -1,
},
{
$project: {
job: "$$ROOT",
_id: "$$ROOT._id",
running: {
$and: ["$lastRunAt", { $gt: ["$lastRunAt", "$lastFinishedAt"] }],
},
scheduled: {
$and: ["$nextRunAt", { $gte: ["$nextRunAt", new Date()] }],
},
queued: {
$and: [
"$nextRunAt",
{ $gte: [new Date(), "$nextRunAt"] },
{ $gte: ["$nextRunAt", "$lastFinishedAt"] },
],
},
completed: {
$and: [
"$lastFinishedAt",
{ $gt: ["$lastFinishedAt", "$failedAt"] },
],
},
failed: {
$and: [
"$lastFinishedAt",
"$failedAt",
{ $eq: ["$lastFinishedAt", "$failedAt"] },
],
},
repeating: {
$and: ["$repeatInterval", { $ne: ["$repeatInterval", null] }],
},
},
{
$project: {
job: "$$ROOT",
_id: "$$ROOT._id",
running: {
$and: ["$lastRunAt", { $gt: ["$lastRunAt", "$lastFinishedAt"] }],
},
},
{ $match: postMatch },
{
$facet: {
pages: [
{ $count: "totalMatchs" },
{
$project: {
totalPages: {
$ceil: { $divide: ["$totalMatchs", options.limit] },
},
},
},
scheduled: {
$and: ["$nextRunAt", { $gte: ["$nextRunAt", new Date()] }],
},
queued: {
$and: [
"$nextRunAt",
{ $gte: [new Date(), "$nextRunAt"] },
{ $gte: ["$nextRunAt", "$lastFinishedAt"] },
],
filtered: [{ $skip: options.skip }, { $limit: options.limit }],
},
completed: {
$and: [
"$lastFinishedAt",
{ $gt: ["$lastFinishedAt", "$failedAt"] },
],
},
failed: {
$and: [
"$lastFinishedAt",
"$failedAt",
{ $eq: ["$lastFinishedAt", "$failedAt"] },
],
},
repeating: {
$and: ["$repeatInterval", { $ne: ["$repeatInterval", null] }],
},
},
])
.toArray();
},
{ $match: postMatch }
];

const awsDocumentDBQuery = [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we explain the limitation with this query, for some reason we're trying to run a query, failing and then trying to run another query that's docDB compatible.

// Limit to 1000 documents because this crashes on larger data sets
// FIXME fix properly
{
$limit: 1000
},
{
$group: {
_id: null,
totalMatchs: {
$sum: 1
},
results: {
$push: '$$ROOT'
}
}
},
{
$project: {
pages: [{totalPages: '$totalMatchs'}],
filtered: {
$slice: [
'$results',
options.skip,
options.limit
]
}
}
}
]
return collection.aggregate([...aggregateQuery,...awsDocumentDBQuery])
.toArray()
.then((result) => {
result[0].pages[0].totalPages = Math.ceil(result[0].pages[0].totalPages / options.limit);
return result;
})
.catch((error) => {
console.error('docdb query error', error);
console.debug(JSON.stringify([...aggregateQuery,...awsDocumentDBQuery]));
return error;
})
};

const getOverview = async () => {
Expand Down
6 changes: 3 additions & 3 deletions public/app/js/confirms.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
const popupmessage = Vue.component("popup-message", {
props: ["job", "deletec", "requeuec", "createc"],
template: `
<div v-if="deletec" class="alert alert-success popupmessage">Job Deleted successfull</div>
<div v-else-if="requeuec" class="alert alert-success popupmessage">Job Requeue successfull</div>
<div v-else-if="createc" class="alert alert-success popupmessage">Job Created successfull</div>
<div v-if="deletec" class="alert alert-success popupmessage">Job Deleted successful</div>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Job Deleted successfully, surely?

<div v-else-if="requeuec" class="alert alert-success popupmessage">Job Requeue successful</div>
<div v-else-if="createc" class="alert alert-success popupmessage">Job Created successful</div>
`,
});
3 changes: 1 addition & 2 deletions public/app/js/newJob.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,6 @@ const newJob = Vue.component("new-job", {
.then((data) => {
this.$emit("popup-message");
this.$emit("refresh-data");
this.$refs.Close.click();
this.clear();
})
.catch(console.log);
Expand Down Expand Up @@ -79,7 +78,7 @@ const newJob = Vue.component("new-job", {
</form>
</div>
<div class="modal-footer">
<button type="button" class="btn btn-info" @click="create()">Create Job</button>
<button type="button" class="btn btn-info" data-dismiss="modal" @click="create()">Create Job</button>
<button type="button" class="btn btn-secondary" data-dismiss="modal">Cancel</button>
</div>
</div>
Expand Down