Merge pull request #201 from Tencent/dev

Version 1.2.0
Tencent · Jul 26, 2018 · ac8d719 · ac8d719
2 parents 285de00 + b91e064
commit ac8d719
Show file tree

Hide file tree

Showing 35 changed files with 676 additions and 368 deletions.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -1,28 +1,27 @@
-# Contributing to TSW
-Welcome to [report Issues](https://github.com/Tencent/TSW/issues) or [pull requests](https://github.com/Tencent/TSW/pulls). It's recommended to read the following Contributing Guide first before contributing. 
+# 为TSW做出贡献
+欢迎您 [提出问题](https://github.com/Tencent/TSW/issues) 或 [pull requests](https://github.com/Tencent/TSW/pulls)， 建议您在为TSW做出贡献前先阅读以下TSW贡献指南。 
 
 ## Issues
-We use Github Issues to track public bugs and feature requests.
+我们通过Github Issues来收集问题和功能相关的需求。
 
-### Search Known Issues First
-Please search the existing issues to see if any similar issue or feature request has already been filed. You should make sure your issue isn't redundant.
+### 首先查看已知的问题
+在您准备提出问题以前，请先查看现有的Github Issues是否已有其他人提出过相似的功能或问题，以确保您提出的问题是有效的。
 
-### Reporting New Issues
-If you open an issue, the more information the better. Such as detailed description, code blocks of your problem.
+### 提交问题
+问题的表述应当尽可能的详细，可以包含相关的代码块。
 
 ## Pull Requests
-We strongly welcome your pull request to make TSW better. 
+我们期待您通过PR（Pull Requests）让TSW变的更加完善。
 
-### Branch Management
-There are two main branches here:
+### 分支管理
+仓库一共包含两个分支:
 
-1. `master` branch.
-	1. **Don't submit any PR on `master` branch.**
-2. `dev` branch. 
-	1. It is our stable developing branch. After full testing, `dev` will be merged to `master` branch for the next release.
-	2. **You are recommended to submit bugfix or feature PR on `dev` branch.**
+1. `master` 分支
+	1. **请勿在master分支上提交任何PR。**
+2. `dev` 分支
+	1. `dev`分支作为稳定的开发分支，经过测试后会在下一个版本合并到`master`分支。
+	2. **PR应该在`dev`分支上提交。**
 
-Normal bugfix or feature request should be submitted to `dev` branch. After full testing, we will merge them to `master` branch for the next release. 
 
 ```
 master
@@ -32,18 +31,17 @@ dev
 feature/bugfix PR
 ```  
 
-### Make Pull Requests
-The code team will monitor all pull request, we run some code check and test on it. After all tests passed, we will accecpt this PR. But it won't merge to `master` branch at once, which have some delay.
+### PR流程
+TSW团队会查看所有的PR，我们会运行一些代码检查和测试，一经测试通过，我们会接受这次PR，但不会立即将代码合并到master分支上，会有一些延迟。
 
-Before submitting a pull request, please make sure the followings are done:
+当您准备PR时，请确保已经完成以下几个步骤:
 
-1. Fork the repo and create your branch from `master`.
-2. Update code or documentation if you have changed APIs.
-3. Add the copyright notice to the top of any new files you've added.
-4. Check your code lints and checkstyles.
-5. Test and test again your code.
-6. Now, you can submit your pull request on `dev`.
+1. 将仓库fork下来并且基于`dev`分支创建您的开发分支。
+2. 如果您更改了APIs请更新代码及文档。
+3. 在您添加的每一个新文件头部加上版权声明。
+4. 检查您的代码语法及格式。
+5. 反复测试。
+6. 现在，您可以开始在`dev`分支上PR了。
 
-## License
-By contributing to TSW, you agree that your contributions will be licensed
-under its [MIT LICENSE](https://github.com/Tencent/TSW/blob/master/LICENSE)
+## 许可证
+通过为TSW做出贡献，代表您同意将其版权归为TSW所有，TSW的开源协议为[MIT LICENSE](https://github.com/Tencent/TSW/blob/master/LICENSE)
diff --git a/CONTRIBUTING_en.md b/CONTRIBUTING_en.md
@@ -0,0 +1,49 @@
+# Contributing to TSW
+Welcome to [report Issues](https://github.com/Tencent/TSW/issues) or [pull requests](https://github.com/Tencent/TSW/pulls). It's recommended to read the following Contributing Guide first before contributing. 
+
+## Issues
+We use Github Issues to track public bugs and feature requests.
+
+### Search Known Issues First
+Please search the existing issues to see if any similar issue or feature request has already been filed. You should make sure your issue isn't redundant.
+
+### Reporting New Issues
+If you open an issue, the more information the better. Such as detailed description, code blocks of your problem.
+
+## Pull Requests
+We strongly welcome your pull request to make TSW better. 
+
+### Branch Management
+There are two main branches here:
+
+1. `master` branch.
+	1. **Don't submit any PR on `master` branch.**
+2. `dev` branch. 
+	1. It is our stable developing branch. After full testing, `dev` will be merged to `master` branch for the next release.
+	2. **You are recommended to submit bugfix or feature PR on `dev` branch.**
+
+Normal bugfix or feature request should be submitted to `dev` branch. After full testing, we will merge them to `master` branch for the next release. 
+
+```
+master
+ ↑
+dev   
+ ↑ 
+feature/bugfix PR
+```  
+
+### Make Pull Requests
+The code team will monitor all pull request, we run some code check and test on it. After all tests passed, we will accecpt this PR,But it won't merge to `master` branch at once, which have some delay.
+
+Before submitting a pull request, please make sure the followings are done:
+
+1. Fork the repo and create your branch from `dev`.
+2. Update code or documentation if you have changed APIs.
+3. Add the copyright notice to the top of any new files you've added.
+4. Check your code lints and checkstyles.
+5. Test and test again your code.
+6. Now, you can submit your pull request on `dev`.
+
+## License
+By contributing to TSW, you agree that your contributions will be licensed
+under its [MIT LICENSE](https://github.com/Tencent/TSW/blob/master/LICENSE)
diff --git a/README_en.md b/README_en.md
@@ -69,7 +69,7 @@ docker run -v configure_dir:/data/release/node_modules -p 8080:80 tsw
 
 Find a bug or have a feature request? Please read the [issues](https://github.com/Tencent/TSW/issues) guidelines and search for existing and closed issues.
 
-If your problem or idea is not addressed yet, please read through our [contributing guidelines](./CONTRIBUTING.md) and open a new [issues](https://github.com/Tencent/TSW/issues).
+If your problem or idea is not addressed yet, please read through our [contributing guidelines](./CONTRIBUTING_en.md) and open a new [issues](https://github.com/Tencent/TSW/issues).
 
 ## License
 

diff --git a/bin/proxy/admin.actions.js b/bin/proxy/admin.actions.js
@@ -26,6 +26,14 @@ module.exports = {
         res.end('done!\r\n');
     },
 
+    '/heapdump': function (req, res) {
+        process.emit('sendCmd2workerOnce', {
+            CMD: 'heapdump',
+            GET: req.GET
+        });
+        res.writeHead(200, { 'Content-Type': 'text/plain; charset=UTF-8' });
+        res.end('done!\r\n');
+    },
 
     '/profiler': function (req, res) {
         process.emit('sendCmd2workerOnce', {

diff --git a/bin/proxy/config.js b/bin/proxy/config.js
@@ -46,7 +46,7 @@ if (isFirstLoad) {
 
 
 if (fs.existsSync('/etc/tsw.config.js')) {
-    cache.config = require('/usr/local/node_modules/config.js');
+    cache.config = require('/etc/tsw.config.js');
 } else if (fs.existsSync('/usr/local/node_modules/config.js')) {
     cache.config = require('/usr/local/node_modules/config.js');
 } else if (fs.existsSync('/data/release/node_modules/config.js')) {

diff --git a/bin/proxy/dump.heap.sh b/bin/proxy/dump.heap.sh
@@ -0,0 +1,7 @@
+#!/bin/bash
+
+echo "begin heapdump"
+
+curl 127.0.0.1:12701/heapdump
+
+
diff --git a/bin/proxy/http.proxy.js b/bin/proxy/http.proxy.js
@@ -101,6 +101,11 @@ methodMap.reload = function() {
     process.emit('reload');
 };
 
+// heapdump
+methodMap.heapdump = function(message) {
+    process.emit('heapdump', message.GET);
+};
+
 // profiler
 methodMap.profiler = function(message) {
     process.emit('profiler', message.GET);
@@ -125,6 +130,24 @@ process.on('top100', function(e) {
     global.top100 = [];
 });
 
+
+process.on('heapdump', function(e) {
+    if (isWin32Like) {
+        return;
+    }
+
+    require('heapdump').writeSnapshot(__dirname + '/cpu' + serverInfo.cpu + '.' + Date.now() + '.heapsnapshot', function(err, filename) {
+        if (err) {
+            logger.error(`dump heap error ${err.message}`);
+            return;
+        }
+        logger.info('dump written to ${filename}', {
+            filename: filename
+        });
+    });
+});
+
+
 process.on('profiler', function(data = {}) {
     logger.info('profiler time: ${time}', data);
     if (isWin32Like) {
@@ -177,10 +200,6 @@ function requestHandler(req, res) {
         // 发者模式清除缓存
         cleanCache();
     }
-    if (req.headers.connection === 'upgrade' && req.headers.upgrade === 'websocket') {
-        // websocket
-        return;
-    }
     res.flush = res.flush || empty;
     parseGet(req);  // 解析get参数
     doRoute(req, res); // HTTP路由
@@ -212,7 +231,7 @@ function cleanCache() {
 }
 
 function listen(cpu) {
-    const user_00 = config.workerUid || 'user_00';
+    const wokerUid = config.workerUid || 'user_00';
     serverInfo.cpu = cpu || 0;
     global.cpuUsed = cpuUtil.getCpuUsed(serverInfo.cpu);
 
@@ -257,14 +276,14 @@ function listen(cpu) {
 
             if (!isWin32Like) {
                 try {
-                    process.setuid(user_00);
+                    process.setuid(wokerUid);
                 } catch (err) {
-                    logger.error(`switch to uid: ${user_00} fail!`);
+                    logger.error(`switch to uid: ${wokerUid} fail!`);
                     logger.error(err.stack);
                 }
 
                 logger.info('switch to uid: ${uid}', {
-                    uid: user_00
+                    uid: wokerUid
                 });
             }
 
@@ -407,6 +426,7 @@ function afterCpu80(cpuUsed) {
                 mail.SendMail(key, 'js', 600, {
                     'to': config.mailTo,
                     'cc': config.mailCC,
+                    'runtimeType': 'CPU',
                     'msgInfo': `${business.module}[CPU]${serverInfo.intranetIp}单核CPU${serverInfo.cpu}使用率为：${cpuUsed}，超过80%`,
                     'title': `${business.module}[CPU]${serverInfo.intranetIp}单核CPU${serverInfo.cpu}使用率为：${cpuUsed}，超过80%`,
                     'content': content,

diff --git a/bin/proxy/http.route.js b/bin/proxy/http.route.js
@@ -82,13 +82,13 @@ module.exports = function(req, res) {
                 d.remove(res);
 
                 if (d.currentContext) {
-                    d.currentContext.window.request = null;
-                    d.currentContext.window.response = null;
-                    d.currentContext.window.onerror = null;
-                    d.currentContext.window = null;
-                }
+                    if (d.currentContext.window) {
+                        d.currentContext.window.request = null;
+                        d.currentContext.window.response = null;
+                        d.currentContext.window.onerror = null;
+                        d.currentContext.window = null;
+                    }
 
-                if (d.currentContext) {
                     d.currentContext.log = null;
                     d.currentContext = null;
                 }
@@ -707,6 +707,7 @@ function onerror(req, res, err) {
 
 const errorIgnore = {
     'socket hang up': 'ignore',
+    'Cannot call write after a stream was destroyed': 'ignore',
     'Cannot read property \'asyncReset\' of null': 'ignore',
     'Cannot read property \'resume\' of null': 'ignore',
     'write ECONNRESET': 'ignore',

diff --git a/bin/proxy/index.js b/bin/proxy/index.js
@@ -10,6 +10,8 @@
 
 const plug = require('../tsw/plug.js');
 
+require('./version.js');
+
 plug('runtime/Console.hack.js');
 plug('runtime/fs.hack.js');
 plug('runtime/Dns.hack.js');

diff --git a/bin/proxy/master.js b/bin/proxy/master.js
@@ -14,12 +14,14 @@ const cluster = require('cluster');
 const cpuUtil = require('util/cpu.js');
 const fs = require('fs');
 const serverOS = require('util/isWindows.js');
+const mail = require('util/mail/mail.js');
 const { debugOptions } = process.binding('config');
 const methodMap = {};
 const workerMap = {};
 const cpuMap = [];
 const tnm2 = require('api/tnm2');
 const network = require('util/network.js');
+const serverInfo = require('serverInfo.js');
 
 process.on('uncaughtException', function(e) {
 
@@ -262,11 +264,9 @@ function checkWorkerAlive() {
             // 内存超限进程处理
             if (worker.lastMessage) {
                 const currMemory = worker.lastMessage.memoryUsage;
-
-                // logger.debug(currMemory);
+                const key = `memoryLimit.v1:${serverInfo.intranetIp}`;
 
                 if (currMemory && currMemory.rss > config.memoryLimit) {
-
                     logger.error('worker${cpu} pid=${pid} memoryUsage ${memoryUsage}, hit memoryLimit: ${memoryLimit}, kill it', {
                         memoryUsage: currMemory.rss,
                         memoryLimit: config.memoryLimit,
@@ -276,6 +276,15 @@ function checkWorkerAlive() {
 
                     logger.error(worker.lastMessage);
 
+                    mail.SendMail(key, 'js', 600, {
+                        'to': config.mailTo,
+                        'cc': config.mailCC,
+                        'runtimeType':'Memory',
+                        'msgInfo': `${serverInfo.intranetIp} 内存超限，服务已重启。请开发人员关注是否存在内存泄露`,
+                        'title': `${serverInfo.intranetIp} 内存超限告警`,
+                        'content': `<p><strong>${serverInfo.intranetIp} 内存超限，服务已重启。请开发人员关注是否存在内存泄露</strong></p>`
+                    });
+
                     restartWorker(worker);
                 }
             }
@@ -368,6 +377,8 @@ function masterEventHandler() {
             cpu: cpu
         });
 
+        tnm2.Attr_API('SUM_TSW_WORKER_FORK', 1);
+
         // 绑定cpu
         cpuUtil.taskset(cpu, currWorker.process.pid);
 
@@ -430,6 +441,7 @@ function masterEventHandler() {
 
     process.on('reload', function(GET) {
         logger.info('reload');
+        tnm2.Attr_API('SUM_TSW_WORKER_RELOAD', 1);
 
         for (const key in workerMap) {
             const worker = workerMap[key];

diff --git a/bin/tsw/util/mail/src/_config.js → bin/proxy/version.js b/bin/tsw/util/mail/src/_config.js → bin/proxy/version.js
@@ -8,17 +8,15 @@
 'use strict';
 
 
-define(function(require, exports, module) {
+const arr = process.versions.node.split('.');
 
-    return {
-
-        tmpl: {
-            create: true
-        },
-        all: {
-            create: false
-        }
-
-    };
-
-});
+if (arr[0] < 8) {
+    console.error('The nodejs version you installed is ' + process.versions.node);
+    console.error('Please update the nodejs version to 8.0.0');
+    process.exit(1);
+}
+if (arr[0] == 10 && (arr[1] >= 0 && arr[1] <= 3)) {
+    console.error('The nodejs version you installed is ' + process.versions.node);
+    console.error('Please update to the nodejs version which greater than 10.4.0');
+    process.exit(1);
+}