AEMTransformRulesGenerator

When using an API within a project, it can happen that the API developer introduces some breaking changes. This means that your project wouldn't run properly anymore until you updated these changes within your code manually.

The aim of this project is to automatically generate transformation rules, which can be loaded into Pharo and propose you code changes in order to stay up to date with any API.

• Project description
• How to setup
  • Automatic/script setup
  • Manual setup
• Possible errors
• Future work
• Weblinks

Project description

APIEvolutionMiner is an existing tool that mines a project history to detect API changes. This tool is also capable of producing static analyzer rules that detect an old API usage in source code.

Although the tool was proven to work well it was never used for two reasons:

• At the time when it was developed Pharo didn't have a nice UI for reporting critics generated by the rules.
• You had to setup APIEvolutionMiner by yourself and manually recover the rules.

While the first issue is mitigated by QualityAssistant, the second one is solved by this project.

This project uses the APIEvolutionMiner for mining the revision history of any SmalltalkHub repository. Furthermore, it uses the rules generated by the APIEvolutionMiner and converts them into useful rewrite rules.

This conversion is done by generating sub-classes of RBTransformationRule. It respects also rules containing wildcards for objects or containing keyword messages.

However, there is a limitation if the number of arguments doesn't match. In this case the AEMTransformRulesGenerator cannot generate a rewrite rule, because there is a lack of information. It will simply skip these rules.

The resulting Smalltalk package file (AEMTransformRules.st) can be filed-in into your Pharo VM. Then, you can open the built-in Critic Browser for viewing the API changes. Below you can find an example of viewing the rules for Roassal2 within Critic Browser:

How to setup

Automatic/script setup

1. Clone this repository into your preferred directory.

2. Run the script Get-AEMRules.sh with any SmalltalkHub repository as argument (replace the link of GraphET2 with our desired project):

   $ ./Get-AEMRules.sh http://www.smalltalkhub.com/mc/ObjectProfile/GraphET2/main

   

3. The script will generate all the files in the following directory: ~/AEMTransformRules/AEMRules-{your_project_name}

Manual setup

1. Create a new folder for the APIEvolutionMiner and project files.

2. Download the APIEvolutionMiner-Jet Pharo image from here: https://ci.inria.fr/rmod/view/MinedRules/job/APIEvolutionMiner-Jet/lastSuccessfulBuild/artifact/APIEvolutionMiner-Jet.zip

   curl --silent --insecure -o APIEvolutionMiner-Jet.zip https://ci.inria.fr/rmod/view/MinedRules/job/APIEvolutionMiner-Jet/lastSuccessfulBuild/artifact/APIEvolutionMiner-Jet.zip
   unzip -o APIEvolutionMiner-Jet.zip

3. Download latest Pharo VM here: http://get.pharo.org/vm or just use your existing Pharo VM (be aware, the APIEvolutionMiner needs the PharoV20.sources).

   curl --silent get.pharo.org/vm | bash

4. Create the following sub-folders and files:

   mkdir latestMczFiles
   mkdir latestSourceCodeChanges
   touch ourMczFiles

5. Open the Pharo-UI with the APIEvolutionMiner-Jet.image and download the latest MCZ-files for your desired SmalltalkHub project (replace the links and names of GraphET2 with our desired project). This step should write into the previously created file ourMczFiles and folder latestMczFiles. Attention: For a large project, this step can take a long time and downloads several MBs data.

   AEMRepositoryDownloader downloaLatestMczsFor: 'http://www.smalltalkhub.com/mc/ObjectProfile/GraphET2/main'.

6. Now import the ring history. This will close the UI, but it's okay. This step should generate the file latestDataset and write into the previously created folder latestSourceCodeChanges. Depending on the size of the project, this step needs more than the usual memory size. If it crashes due to memory, just add the following line to the VM Pharo.ini: AddressSpaceLimit=1536 (update the number of memory according to your project).

   AEMRepositoryDownloader importToRingHistoryAndExportAssociations.

7. Copy the file latestDataset and rename the copy to dataset:

   cp -u latestDataset dataset

8. Copy the folder latestSourceCodeChanges and rename the copy to sourceCodeChanges:

   mkdir sourceCodeChanges
   cp -r latestSourceCodeChanges/* sourceCodeChanges

9. Open the Pharo-UI with the APIEvolutionMiner-Jet.image again and file-in the following file:

   • AEMTransformRulesGenerator.st

10. Then execute the following code within the Workspace:

```smalltalk
AEMSystemHistoryImporter loadSystemHistory: 'GraphET2'.
AEMTransformRulesGenerator new generateRBTransformationRules: ((AEMEvidenceRanking new systemHistory: ((AEMReport exportOneOneRules) systemHistory)) oneOneRules).
(RPackageOrganizer default packageNamed: #AEMTransformRules) fileOut.
```

11. The code above should have created the following files in your directory:

    • rules.csv (containing the APIEvolutionMiner rules)
    • AEMTransformRules.st (containing the RBTransformationRules, which can be directly used with Critic-Browser)

12. Finally the folder structure should look like this:

Possible errors

• PharoDebug.log: Error: can't find EOCD position
  • Solution:
    1. Go into folder latestMczFiles
    2. Sort .mcz (actually ZIP archives) by file-size
    3. Delete (remember the names) all files with file-size of 0 or 1 KB (corrupted, you can try to open them with your ZIP application to test them)
    4. Go back to one folder above. Remove the deleted packages from the list in file ourMczFiles.
    5. Finally, just run the Bash script with the same repository as input again.

Future work

• Integrate the code into a CI service
• Debug the command AEMRepositoryDownloader importToRingHistoryAndExportAssociations. since it crashes very often (especially for bigger projects/APIs)
• Update AEM in order to run at least on Pharo 4 (currently it needs still Pharo 2)

Weblinks

APIEvolutionMiner

• http://homepages.dcc.ufmg.br/~mtov/pub/2014_csmrwcre_apievolutionminer.pdf
• http://rmod.lille.inria.fr/web/software/APIEvolutionMiner
• http://smalltalkhub.com/#!/~andrehora/APIEvolutionMiner

Jenkins Jobs Inria RMoD Team

• https://ci.inria.fr/rmod/view/MinedRules/job/APIEvolutionMiner/lastSuccessfulBuild/consoleFull
• https://ci.inria.fr/rmod/view/MinedRules/job/Roassal-Monitoring/lastSuccessfulBuild/consoleFull
• https://ci.inria.fr/rmod/view/MinedRules/job/Roassal-APIEvolutionMiner/lastSuccessfulBuild/consoleFull
• https://ci.inria.fr/rmod/view/MinedRules/job/Moose-Report/lastSuccessfulBuild/consoleFull
• https://ci.inria.fr/pharo-contribution/view/Helper/job/JobTemplate/

Tested projects

• http://www.smalltalkhub.com/#!/~ObjectProfile/GraphET2
• http://www.smalltalkhub.com/mc/ObjectProfile/GraphET2/main
• http://www.smalltalkhub.com/#!/~ObjectProfile/Roassal2
• http://www.smalltalkhub.com/mc/ObjectProfile/Roassal2/main

PharoByExample

• http://pharobyexample.org/
• https://github.com/SquareBracketAssociates/UpdatedPharoByExample
• http://pharobooks.gforge.inria.fr/PharoByExampleTwo-Eng/latest/ZeroConf.pdf

Yuriy Tymchuk

• http://scg.unibe.ch/staff/YuriyTymchuk
• http://scg.unibe.ch/wiki/projects/mastersbachelorsprojects/apiEvolutionMiningService
• https://github.com/Uko