Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

karma breaks when singleRun === false and javascript code is invalid #1783

Closed
cafesanu opened this issue Jan 6, 2016 · 6 comments
Closed

Comments

@cafesanu
Copy link

cafesanu commented Jan 6, 2016

I'm in the process of upgrading karma form 0.10.x to 0.13.x. Everything is working except for one thing (that used to work before the upgrade), when I want to continuously run my tests for any code change(singleRun === false), whenever my code is invalid (lets say I delete a closing parentheses in an if statement something like... if(x{ return false} and I save the file ), karma breaks with the following error :

05 01 2016 18:59:42.589:ERROR [karma]: [TypeError: Not a string or buffer]
TypeError: Not a string or buffer
    at TypeError (native)
    at Hash.update (crypto.js:70:16)
    at sha1 (/Users/carlos/hb/hbng/node_modules/karma/lib/preprocessor.js:11:8)
    at nextPreprocessor (/Users/carlos/hb/hbng/node_modules/karma/lib/preprocessor.js:74:20)
    at /Users/carlos/hb/hbng/node_modules/karma-coverage/lib/preprocessor.js:140:7
    at Object.Instrumenter.instrument (/Users/carlos/hb/hbng/node_modules/istanbul/lib/instrumenter.js:596:17)
    at /Users/carlos/hb/hbng/node_modules/karma-coverage/lib/preprocessor.js:110:18
    at nextPreprocessor (/Users/carlos/hb/hbng/node_modules/karma/lib/preprocessor.js:78:28)
    at /Users/carlos/hb/hbng/node_modules/karma/lib/preprocessor.js:112:7
    at /Users/carlos/hb/hbng/node_modules/graceful-fs/graceful-fs.js:76:16
    at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:404:3)

After digging the whole day in this issue, I discovered that the break happens from v0.11.5 to v0.11.6. Specifically this change: 6e31cb2#diff-1006b0ebc64f4a0530e73029eed686b0R56 is the one responsible for the break.

I'm not familiar with the karma codebase, but after some debugging I noticed that, when my javascript code is invalid and the coverage preprocessor processes my js code, the error parameter changes from null to undefined (null when there is no errors, undefined when there is errors). Changing https://github.com/karma-runner/karma/blob/master/lib/preprocessor.js#L64 from if (error) { to if (error || typeof error === "undefined") { seems to solve the error and I'm able to watch continuously for changes.

Not sure if this is the right solution, but if it is, I could create a PR with this simple change if you want me to.

This is related with #520 and might be related with the issue closed recently due to inactivity #847

@cafesanu
Copy link
Author

Hi @dignifiedquire

Ok so digged even more into the issue and this is what I found:

When there is an invalid code (like the if without closed parenthesis ...if(x{ return false} ), istanbul catches the invalid code error(Code 1) in here https://github.com/gotwarlost/istanbul/blob/master/lib/instrumenter.js#L596.

Code 1

instrument: function (code, filename, callback) {
    if (!callback && typeof filename === 'function') {
        callback = filename;
        filename = null;
    }
    try {
        callback(null, this.instrumentSync(code, filename));
    } catch (ex) {
        callback(ex);//CODE REACHES THIS CATCH WHEN THERE IS INVALID CODE
   }
}

The callback function is in reality this parameter anonymous function (code 2) in here https://github.com/karma-runner/karma-coverage/blob/master/lib/preprocessor.js#L112...

Code 2

instrumenter.instrument(content, jsPath, function (err, instrumentedCode) {
  if (err) {
    log.error('%s\n  at %s', err.message, file.originalPath)
  }

  if (file.sourceMap && instrumenter.lastSourceMap()) {
    log.debug('Adding source map to instrumented file for "%s".', file.originalPath)
    var generator = SourceMapGenerator.fromSourceMap(new SourceMapConsumer(instrumenter.lastSourceMap().toString()))
    generator.applySourceMap(new SourceMapConsumer(file.sourceMap))
    file.sourceMap = JSON.parse(generator.toString())
    instrumentedCode += '\n//# sourceMappingURL=data:application/json;charset=utf-8;base64,'
    instrumentedCode += new Buffer(JSON.stringify(file.sourceMap)).toString('base64') + '\n'
  }

  // remember the actual immediate instrumented JS for given original path
  sourceCache[jsPath] = content

  if (includeAllSources) {
    // reset stateful regex
    coverageObjRegex.lastIndex = 0

    var coverageObjMatch = coverageObjRegex.exec(instrumentedCode)

    if (coverageObjMatch !== null) {
      var coverageObj = JSON.parse(coverageObjMatch[0])

      coverageMap.add(coverageObj)
    }
  }

  // RequireJS expects JavaScript files to end with `.js`
  if (useJSExtensionForCoffeeScript && instrumenterLiteral === 'ibrik') {
    file.path = file.path.replace(/\.coffee$/, '.js')
  }

  done(instrumentedCode)
})

If you notice, when the catch happens in the code 1, the exception is being passed via callback(ex), which is the errparameter in the anonymous function instrumenter.instrument(content, jsPath, function (err, instrumentedCode) { in Code 2

now, since no second parameter is being passed in callback(ex);, this mean the second parameter instrumentedCode in code 2 in undefined. At the end of code 2, done(instrumentedCode) is being called, and of course instrumentedCode is undefined as explained before.

this done callback function in Code 2 is in reality the function nextPreprocessor mentioned in my first comment(code 3) some days ago https://github.com/karma-runner/karma/blob/master/lib/preprocessor.js#L57

Code 3

var nextPreprocessor = function (error, content) {
  // normalize B-C
  if (arguments.length === 1 && typeof error === 'string') {
    content = error
    error = null
  }

  if (error) {
    file.content = null
    file.contentPath = null
    return done(error)
  }

  if (!preprocessors.length) {
    file.contentPath = null
    file.content = content
    file.sha = sha1(content)
    return done()
  }

  preprocessors.shift()(content, file, nextPreprocessor)
}

Now, if you notice, when done(instrumentedCode) in code 2 is called with instrumentedCode being undefined, nextPreprocessor in code 3 is of course going to have err as undefined!!! since neither if (arguments.length === 1 && typeof error === 'string') { nor if (error) { will catch there was an error (arguments.length is 1, but not a string)

So, I think my first solution still works, but another solution would be to modify the instrumenter.instrument anonymous function (code 2) in https://github.com/karma-runner/karma-coverage (which I noticed you @dignifiedquire are a contributor as well) to call the callback function done with err.message and not execute the instrumentedCode part, in other words, this instead

Proposed solution for https://github.com/karma-runner/karma-coverage/blob/master/lib/preprocessor.js#L112

    instrumenter.instrument(content, jsPath, function (err, instrumentedCode) {
      if (err) {
        log.error('%s\n  at %s', err.message, file.originalPath);
        done(err.message)
      } else {
          if (file.sourceMap && instrumenter.lastSourceMap()) {
            log.debug('Adding source map to instrumented file for "%s".', file.originalPath)
            var generator = SourceMapGenerator.fromSourceMap(new SourceMapConsumer(instrumenter.lastSourceMap().toString()))
            generator.applySourceMap(new SourceMapConsumer(file.sourceMap))
            file.sourceMap = JSON.parse(generator.toString())
            instrumentedCode += '\n//# sourceMappingURL=data:application/json;charset=utf-8;base64,'
            instrumentedCode += new Buffer(JSON.stringify(file.sourceMap)).toString('base64') + '\n'
          }

          // remember the actual immediate instrumented JS for given original path
          sourceCache[jsPath] = content

          if (includeAllSources) {
            // reset stateful regex
            coverageObjRegex.lastIndex = 0

            var coverageObjMatch = coverageObjRegex.exec(instrumentedCode)

            if (coverageObjMatch !== null) {
              var coverageObj = JSON.parse(coverageObjMatch[0])

              coverageMap.add(coverageObj)
            }
          }

          done(instrumentedCode)
      }
    })

I know the proposed soltion is not part of this repo, but I think my first solution is still valid. Will also open an isue in the karma-coverage repo.

@dignifiedquire or anyone else, let me know if what I say makes sense. I think I will create a PR in karma-coverage.

Thanks!!

@dignifiedquire
Copy link
Member

Thanks for the detailed analysis and debugging and report! I really appreciate it (please tell everybody else to report more issues like this 😄)
I will try to get to reviewing this in detail in the next days.

@dignifiedquire
Copy link
Member

Yes please open a PR to karma-coverage-reporter. (Yes I'm maintaining pretty much everything under the karma-runner organization) Should get this merged and fixed pretty quick I think.

@cafesanu
Copy link
Author

I'll try to tell people but wont promise any results 😄 😄 😄

@cafesanu
Copy link
Author

PR Created . karma-runner/karma-coverage#204

@hpurmann
Copy link
Contributor

hpurmann commented Jun 30, 2016

Sounds like this issue can be closed then?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants