Using monkey patching/mocking for testing mutations

[ElianHugh]

I've been interested in assessing the validity of unit tests in R packages and came across your mutant repo. Not sure if this is being worked on but I thought I would float an idea if so:

Would mocking/monkey patching a package's namespace be a simpler alternative to writing a mutated package for testing? i.e. using assignInNamespace rather than working with I/O. I think it would potentially make it easier to map changed lines to tests too.

An obvious downside to me is that you can't modify anything that doesn't end up directly in the package namespace, though I think that realistically speaking the majority of unit tests aren't looking at testing outside of functions anyway. I suppose that does limit the scope of the mutations, and it isn't really in the "spirit" of other mutation testing libs (which seem to typically rely on bytecode or source changes). Another downside is that you are creating a less predictable testing environment because you are relying on in-memory changes rather than source code modification.

Anyway, thank you for your work on the package, I'm curious to see if it picks up attention. It makes me a little uneasy with the 100% coverage ethos -- some form of test validation would definitely ease my conscience.

[sckott]

👋🏽 @ElianHugh

Thsi repo is not being worked on. I should mark it more officially to indicate it

Yeah, one could monkey patch, but last I checked CRAN would not allow use of monkey patching in packages on CRAN. Do you know otherwise? So it'd be fine if it's for code not pushed to CRAN

Yeah I think you're right that probably most tests folks write in R pkgs are for exported fxns, but surely some tests are for unexported fxns.

Yep, I think it's better to re-write a package after a mutation (rather than monkeypatch) b/c you have actual files you can manipulate, point to, etc.

It makes me a little uneasy with the 100% coverage ethos

do we say something about that in this repo?

[ElianHugh]

Sorry, I didn't mean to suggest this repo espoused 100% test coverage, more that mutation testing support would make the ethos more generally palatable :)

Fair enough wrt handling the files afterwards.

I wasn't aware that CRAN denies monkey patching but it seems like you're generally correct with some quick googling of the assignInMyNamespace documentation. Then again, there is a package called mockthat on CRAN that basically just does this anyway? Not totally sure what the deal is.

Sorry if this bumped things needlessly, I appreciate that the repo isn't worked on currently. Thanks again for taking the time to reply!

[sckott]

there is a package called mockthat

cool, hadn't seen it. it might be worth a try then. though if anything is true about CRAN it's inconsistent enforcement of rules b/c humans

I was happy to chat! Thanks for dropping in 😄

Note in another thread someone just mentioned a mutation tool that can be used for R #11 (comment)