Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate from callbacks to async/await #3559

Merged
merged 58 commits into from
Mar 7, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
8d85ae5
pluginfw/hooks.js: allow returning a Promise in aCallFirst(), aCallAll()
raybellis Jan 18, 2019
80b3019
pluginfw/plugins.js: converted to Promise API
raybellis Jan 18, 2019
a579dfc
pluginfw/installer.js: use Promise version of hooks.aCallAll() in ins…
raybellis Jan 18, 2019
4877ec3
server.js: rewritten to use Promises
raybellis Jan 18, 2019
ec5baa2
PadMessageHandler.js: convert two remaining API calls to thenify
raybellis Jan 22, 2019
c4f1f83
APIHandler.js: use promises
raybellis Jan 22, 2019
5d7162a
utils/ImportHtml.js: migrate to thenify
raybellis Jan 22, 2019
584e481
PadMessageHandler.js: migrate to thenify
raybellis Jan 22, 2019
5ef4a2d
more thenify in node/utils/*
raybellis Jan 22, 2019
0c2d662
plugins download and search: converted to Promises
raybellis Jan 23, 2019
23a3a07
tests.js: remove use of async.js
raybellis Jan 23, 2019
34fdaa4
db/SecurityManager.js: convert checkAccess() to thenify
raybellis Jan 23, 2019
d5d2871
access controls: promisification
raybellis Jan 23, 2019
29e9f86
db/DB.js: add Promise-only API methods
raybellis Jan 23, 2019
70a045a
db/GroupManager.js: mostly converted to Promises / async
raybellis Jan 25, 2019
16c4c33
db/AuthorManager.js: renamed doesAuthorExists() -> doesAuthorExist()
raybellis Jan 25, 2019
a875ca6
db/SessionManager.js: mostly converted to Promises
raybellis Jan 25, 2019
bf9e3f9
db/PadManager.js: renamed doesPadExists() -> doesPadExist()
raybellis Jan 25, 2019
eedae98
db/PadManager.js: convert sanitizePadId() to Promises
raybellis Jan 25, 2019
8f53e44
db/AuthorManager.js: partial conversion to Promises
raybellis Jan 25, 2019
1b6430a
db/PadMessageHandler.js: partial conversion to Promises
raybellis Jan 25, 2019
e7dc076
db/API.js: complete conversion to Promises
raybellis Jan 25, 2019
7709fd4
utility scripts: converted to use the Promise interface
raybellis Jan 26, 2019
e58da69
db/SecurityManager.js: converted checkAccess() to pure Promises
raybellis Jan 28, 2019
005c0af
db/SessionManager.js: completely converted to Promises/async
raybellis Jan 28, 2019
8108964
db/AuthorManager.js: further conversion
raybellis Jan 28, 2019
bbe4a5f
db/PadManager.js: more conversion to Promises/async
raybellis Jan 28, 2019
7f19033
SocketIORouter: code formatting cleanups
raybellis Jan 30, 2019
982d4f3
db/Pad.js: start use of promise DB methods
raybellis Jan 30, 2019
58d0e6c
APIHandler.js: further cleanup
raybellis Jan 30, 2019
c499a08
bin/repairPad.js: conversion to promise/async
raybellis Jan 30, 2019
d543d5a
PadMessageHandler.js: convert handleUserChanges() to Promises
raybellis Jan 30, 2019
9246a1d
PadMessageHandler.js: further conversion
raybellis Jan 30, 2019
bb80325
PadMessageHandler.js: completed conversion
raybellis Jan 30, 2019
5192a0c
db/ReadOnlyManager.js: completed conversion
raybellis Jan 30, 2019
62345ac
import/export: conversion to Promises/async
raybellis Jan 31, 2019
6d1b6b2
db/Pad.js: convert to promises/async
raybellis Jan 31, 2019
4622309
TidyHtml.js: convert to promises
raybellis Jan 31, 2019
ccb49dc
padDiff.js: convert to Promises/async
raybellis Jan 31, 2019
b1c5024
remove thenify use - no longer required
raybellis Jan 31, 2019
07ae44d
PadMessageHandler.js: cope better with session disconnects
raybellis Jan 31, 2019
e7c2fad
convert some async loops into parallel loops
raybellis Feb 1, 2019
7699337
allow some operations to proceed in parallel
raybellis Feb 1, 2019
b664eb4
ImportHandler.js: ensure import connection closing happens at the rig…
raybellis Feb 8, 2019
ebb8a64
errorhandling.js: use promise db.doShutdown interface
raybellis Jan 31, 2019
9497ee7
prepare to async: trivial reformatting
muxator Feb 8, 2019
b699621
padurlsanitize.js: invert a condition prior to refactoring
muxator Feb 8, 2019
96d875b
padurlsanitize.js: rewritten to consume promises
raybellis Jan 23, 2019
630af9a
db/SessionStore.js: call nextTick() only if there is something to do
muxator Feb 8, 2019
583ea92
db/SessionStore.js: do not migrate to Promises. Make optional all(), …
raybellis Jan 23, 2019
40c4507
db/GroupManager.js: factored out a variable
muxator Feb 9, 2019
17fe32e
start using "thenify" to support callback and promises
raybellis Jan 21, 2019
b0846de
db/SessionManager.js: "authorMangager" -> "authorManager"
muxator Feb 9, 2019
3802073
db/DB.js: allow a Promise return instead of callbacks in init()
raybellis Jan 18, 2019
e841798
prepare to async: typos in error messages
muxator Feb 15, 2019
11453d5
prepare to async: stricter checks
muxator Mar 1, 2019
98993fe
db/SessionManager.js: "groupMangager" -> "groupManager"
muxator Feb 9, 2019
ac7663c
db/DB.js: prevent DB layer from returning undefined
raybellis Mar 5, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
223 changes: 86 additions & 137 deletions bin/checkAllPads.js
Original file line number Diff line number Diff line change
@@ -1,145 +1,94 @@
/*
This is a debug tool. It checks all revisions for data corruption
*/
* This is a debug tool. It checks all revisions for data corruption
*/

if(process.argv.length != 2)
{
if (process.argv.length != 2) {
console.error("Use: node bin/checkAllPads.js");
process.exit(1);
}

//initialize the variables
var db, settings, padManager;
var npm = require("../src/node_modules/npm");
var async = require("../src/node_modules/async");

var Changeset = require("../src/static/js/Changeset");

async.series([
//load npm
function(callback) {
npm.load({}, callback);
},
//load modules
function(callback) {
settings = require('../src/node/utils/Settings');
db = require('../src/node/db/DB');

//initialize the database
db.init(callback);
},
//load pads
function (callback)
{
padManager = require('../src/node/db/PadManager');

padManager.listAllPads(function(err, res)
{
padIds = res.padIDs;
callback(err);
});
},
function (callback)
{
async.forEach(padIds, function(padId, callback)
{
padManager.getPad(padId, function(err, pad) {
if (err) {
callback(err);
}

//check if the pad has a pool
if(pad.pool === undefined )
{
console.error("[" + pad.id + "] Missing attribute pool");
callback();
return;
}

//create an array with key kevisions
//key revisions always save the full pad atext
var head = pad.getHeadRevisionNumber();
var keyRevisions = [];
for(var i=0;i<head;i+=100)
{
keyRevisions.push(i);
}

//run trough all key revisions
async.forEachSeries(keyRevisions, function(keyRev, callback)
{
//create an array of revisions we need till the next keyRevision or the End
var revisionsNeeded = [];
for(var i=keyRev;i<=keyRev+100 && i<=head; i++)
{
revisionsNeeded.push(i);
}

//this array will hold all revision changesets
var revisions = [];

//run trough all needed revisions and get them from the database
async.forEach(revisionsNeeded, function(revNum, callback)
{
db.db.get("pad:"+pad.id+":revs:" + revNum, function(err, revision)
{
revisions[revNum] = revision;
callback(err);
});
}, function(err)
{
if(err)
{
callback(err);
return;
}

//check if the revision exists
if (revisions[keyRev] == null) {
console.error("[" + pad.id + "] Missing revision " + keyRev);
callback();
return;
}

//check if there is a atext in the keyRevisions
if(revisions[keyRev].meta === undefined || revisions[keyRev].meta.atext === undefined)
{
console.error("[" + pad.id + "] Missing atext in revision " + keyRev);
callback();
return;
}

var apool = pad.pool;
var atext = revisions[keyRev].meta.atext;

for(var i=keyRev+1;i<=keyRev+100 && i<=head; i++)
{
try
{
//console.log("[" + pad.id + "] check revision " + i);
var cs = revisions[i].changeset;
atext = Changeset.applyToAText(cs, atext, apool);
}
catch(e)
{
console.error("[" + pad.id + "] Bad changeset at revision " + i + " - " + e.message);
callback();
return;
}
}

callback();
});
}, callback);
});
}, callback);
}
], function (err)
{
if(err) throw err;
else
{
console.log("finished");
process.exit(0);
// load and initialize NPM
let npm = require('../src/node_modules/npm');
npm.load({}, async function() {

try {
// initialize the database
let settings = require('../src/node/utils/Settings');
let db = require('../src/node/db/DB');
await db.init();

// load modules
let Changeset = require('../src/static/js/Changeset');
let padManager = require('../src/node/db/PadManager');

// get all pads
let res = await padManager.listAllPads();

for (let padId of res.padIDs) {

let pad = await padManager.getPad(padId);

// check if the pad has a pool
if (pad.pool === undefined) {
console.error("[" + pad.id + "] Missing attribute pool");
continue;
}

// create an array with key kevisions
// key revisions always save the full pad atext
let head = pad.getHeadRevisionNumber();
let keyRevisions = [];
for (let rev = 0; rev < head; rev += 100) {
keyRevisions.push(rev);
}

// run through all key revisions
for (let keyRev of keyRevisions) {

// create an array of revisions we need till the next keyRevision or the End
var revisionsNeeded = [];
for (let rev = keyRev ; rev <= keyRev + 100 && rev <= head; rev++) {
revisionsNeeded.push(rev);
}

// this array will hold all revision changesets
var revisions = [];

// run through all needed revisions and get them from the database
for (let revNum of revisionsNeeded) {
let revision = await db.get("pad:" + pad.id + ":revs:" + revNum);
revisions[revNum] = revision;
}

// check if the revision exists
if (revisions[keyRev] == null) {
console.error("[" + pad.id + "] Missing revision " + keyRev);
continue;
}

// check if there is a atext in the keyRevisions
if (revisions[keyRev].meta === undefined || revisions[keyRev].meta.atext === undefined) {
console.error("[" + pad.id + "] Missing atext in revision " + keyRev);
continue;
}

let apool = pad.pool;
let atext = revisions[keyRev].meta.atext;

for (let rev = keyRev + 1; rev <= keyRev + 100 && rev <= head; rev++) {
try {
let cs = revisions[rev].changeset;
atext = Changeset.applyToAText(cs, atext, apool);
} catch (e) {
console.error("[" + pad.id + "] Bad changeset at revision " + i + " - " + e.message);
}
}
}
console.log("finished");
process.exit(0);
}
} catch (err) {
console.trace(err);
process.exit(1);
}
});
Loading