Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VS Code: Validate generated code #828

Merged
merged 110 commits into from
Jan 18, 2022
Merged
Show file tree
Hide file tree
Changes from 37 commits
Commits
Show all changes
110 commits
Select commit Hold shift + click to select a range
95e571f
Do not fail silently with uncaught exceptions.
petervdonovan Dec 29, 2021
8139acf
Add Jackson as dependency.
petervdonovan Dec 29, 2021
27a34f7
Minor cleanups.
petervdonovan Dec 29, 2021
bd720b5
Rust: Report errors on correct lines.
petervdonovan Dec 30, 2021
ae862f3
Discourage wildcard imports.
petervdonovan Dec 31, 2021
1faa529
C++, Rust: Increase specificity of diagnostic reports.
petervdonovan Dec 31, 2021
d1a36e1
Discourage wildcard imports in Kotlin files.
petervdonovan Dec 31, 2021
f5852ef
Tag all verbatim code blocks without breaking Python.
petervdonovan Dec 31, 2021
3096930
TS: Get ESLint up and running.
petervdonovan Jan 1, 2022
465a12f
TS: Combine linting and more thorough type checks.
petervdonovan Jan 1, 2022
f629b6e
Do not print possibly non-human-readable command output.
petervdonovan Jan 1, 2022
d3bd76c
Merge branch 'vscode-handle-build-result' into vscode-validate-genera…
petervdonovan Jan 1, 2022
f67c608
TS: Do not lint in standalone mode.
petervdonovan Jan 1, 2022
3466cf9
TS: Skip `pnpm install` if node_modules exists already.
petervdonovan Jan 1, 2022
d90f221
Bugfix.
petervdonovan Jan 2, 2022
a96b700
Bugfix.
petervdonovan Jan 2, 2022
7a65f5b
TS: Refactor TSGenerator.
petervdonovan Jan 2, 2022
6142f9f
Port base Validator to Java; refactor accordingly.
petervdonovan Jan 2, 2022
4a5bad4
Python: Basic, minimal syntax checking.
petervdonovan Jan 3, 2022
c5a0071
ErrorReporter: Make API more consistent.
petervdonovan Jan 3, 2022
9bb9df5
Python: The default validator is now Pylint.
petervdonovan Jan 3, 2022
dda288a
Python: Address false positives from Pylint.
petervdonovan Jan 5, 2022
0da08f2
Python: Fix some tests.
petervdonovan Jan 5, 2022
d3f3e06
C: Tiny progress report change.
petervdonovan Jan 5, 2022
01a69c2
C++: Report MSVC errors in correct locations.
petervdonovan Jan 6, 2022
da731e6
C++ validator: Stylistic cleanups.
petervdonovan Jan 7, 2022
d30b0cb
Python: Silence a false positive (and some true positives).
petervdonovan Jan 7, 2022
5e22fe2
Merge branch 'master' into vscode-validate-generated-code
petervdonovan Jan 7, 2022
7509f86
Add comments.
petervdonovan Jan 7, 2022
ecbdd04
Style: Prefer imperatives to third person.
petervdonovan Jan 7, 2022
71c0240
Comments only.
petervdonovan Jan 7, 2022
7cc45f3
Style: Prefer imperatives to third person.
petervdonovan Jan 7, 2022
3397032
Rust: Address test failure caused by CodeMap tags.
petervdonovan Jan 7, 2022
98e46cd
GitHub actions: bump actions/cache version.
petervdonovan Jan 7, 2022
a7c7eef
C: Address test failure.
petervdonovan Jan 7, 2022
c7011df
TypeScript CI: Delete cache step.
petervdonovan Jan 8, 2022
4839e8e
Apply suggestions from code review
petervdonovan Jan 8, 2022
8d3d7e4
Partially roll back changes to IntelliJ settings.
petervdonovan Jan 8, 2022
36cfae4
TypeScript CI: Hash by package.json, not lock files.
petervdonovan Jan 8, 2022
ae43ff6
Comment only.
petervdonovan Jan 8, 2022
6cd3e09
Python tests: Pass syntax checks in a less confusing way.
petervdonovan Jan 8, 2022
4be1d73
Revert "TypeScript CI: Hash by package.json, not lock files."
lhstrh Jan 8, 2022
1580ddc
Update org.lflang/src/org/lflang/util/LFCommand.java
lhstrh Jan 8, 2022
a057c0f
Update ci.yml
lhstrh Jan 8, 2022
535e328
LSP Tests: Initial test for "Build and Run."
petervdonovan Jan 9, 2022
d47baf7
TypeScript: Make ESLint more permissive and lint always.
petervdonovan Jan 9, 2022
6abe1fe
TypeScript tests: Make consistent with default ESLint guidelines.
petervdonovan Jan 9, 2022
15fffaa
LSP: Minor tweaks to progress reporting.
petervdonovan Jan 9, 2022
2db4f92
TypeScript: Fail just a little faster.
petervdonovan Jan 9, 2022
da6a227
CodeCov reports: Refactor.
petervdonovan Jan 9, 2022
591b520
Tests: Check if unit tests should have been failing.
petervdonovan Jan 9, 2022
812592c
Revert "Tests: Check if unit tests should have been failing."
petervdonovan Jan 9, 2022
2fecd83
Tests: Remove mentions of code-coverage-only tests.
petervdonovan Jan 9, 2022
898005f
CI: Delete a few possibly unnecesary lines.
petervdonovan Jan 9, 2022
be9e603
Update ci.yml
lhstrh Jan 9, 2022
3fb48ff
Tests: Reorganize LSP tests that require dependencies.
petervdonovan Jan 10, 2022
d7128a8
Merge master so that CI can run.
petervdonovan Jan 10, 2022
26d914f
CI: Repair syntax in YAML.
petervdonovan Jan 10, 2022
b798192
CI: Spelling correction.
petervdonovan Jan 10, 2022
ead1501
Tests: Exclude examples from LSP tests.
petervdonovan Jan 10, 2022
1ec3720
LSP: Try again to pass "Build and Run" test.
petervdonovan Jan 10, 2022
c8821ba
LSP: Respond to test failures.
petervdonovan Jan 10, 2022
862db03
Tests: Bugfix.
petervdonovan Jan 10, 2022
dc8cb30
Tests: Handle missing libprotoc.
petervdonovan Jan 10, 2022
07734e2
Tests: Handle special case more robustly.
petervdonovan Jan 11, 2022
041198b
Tests: Address comment from code review.
petervdonovan Jan 11, 2022
1d4dc8a
Tests: Initial draft for target language validation test.
petervdonovan Jan 11, 2022
6ebe594
Tests: Minor cleanup.
petervdonovan Jan 12, 2022
212fe79
LSP: Generate code for uninstantiated reactors.
petervdonovan Jan 12, 2022
40be2a2
LSP: Patch line adjustment edge case.
petervdonovan Jan 12, 2022
e8c4f9c
LSP: Prefer to run syntax checkers from srcGenPkgPath.
petervdonovan Jan 12, 2022
909e433
LSP: Make C++ line adjustment more comprehensive.
petervdonovan Jan 12, 2022
613b9f9
Tests: Use swap file (not temp dir) for generated tests.
petervdonovan Jan 12, 2022
c4ca40b
LSP: Break #line directives a little bit.
petervdonovan Jan 12, 2022
ca212f2
Rust: Ask cargo what the package root is.
petervdonovan Jan 12, 2022
01d924d
Rust: Repair broken schema, do not print JSON.
petervdonovan Jan 12, 2022
15b8d40
Tests: Patch over edge case where heuristic fails.
petervdonovan Jan 13, 2022
69c6e98
Tests: Refactor LSP tests a bit.
petervdonovan Jan 13, 2022
13217fa
CI: Add target language validation tests.
petervdonovan Jan 13, 2022
faca628
CI: LSP-related bugfix.
petervdonovan Jan 13, 2022
fe9e214
CI: Add protobuf dependencies to LSP workflow.
petervdonovan Jan 13, 2022
4a74ef0
LSP: Make error markers account for imports.
petervdonovan Jan 13, 2022
efbf662
LSP: Add Clang as a syntax checker.
petervdonovan Jan 13, 2022
bed7569
Tests: Bugfix corresponding to 4a74ef07.
petervdonovan Jan 13, 2022
b856e82
CI: Try again to use Clang on MacOS.
petervdonovan Jan 13, 2022
f831ac3
Attempt to address MacOS test failures.
petervdonovan Jan 14, 2022
80eabbc
CI: Test Python validation without Pylint.
petervdonovan Jan 16, 2022
b3168b5
Tests: Permit missing dependencies to abort validation.
petervdonovan Jan 16, 2022
f492794
Merge branch 'windows-ts' into vscode-validate-generated-code
petervdonovan Jan 16, 2022
ad17836
Tests: Address failing Python validation tests.
petervdonovan Jan 16, 2022
b411e17
TS: Address federated Windows test failure.
petervdonovan Jan 16, 2022
71911e2
Tests: Fix NPE.
petervdonovan Jan 16, 2022
e3871a0
Python: Repair no-pylint validation.
petervdonovan Jan 16, 2022
b2ec699
Tests: Permit validation to fail due to lack of support.
petervdonovan Jan 16, 2022
c17701b
Tests: Attempt to uninstall.
petervdonovan Jan 16, 2022
b5b25ab
CI: Attempt to repair holes in CodeCov reports.
petervdonovan Jan 17, 2022
b0196a4
Revert "CI: Attempt to repair holes in CodeCov reports."
petervdonovan Jan 17, 2022
8485495
CI: Add missing CodeCov report.
petervdonovan Jan 17, 2022
acf9859
TS: Repair handling of user-provided config files.
petervdonovan Jan 17, 2022
441a11a
Revert "CI: Add missing CodeCov report."
petervdonovan Jan 17, 2022
4e2fd35
CI: Attempt to debug failure that cannot be reproduced locally.
petervdonovan Jan 17, 2022
49806b3
LSP: Repair off-by-one error in GeneratorBase.
petervdonovan Jan 18, 2022
945cd80
LSP: Minor cleanups.
petervdonovan Jan 18, 2022
97d9354
Revert "Revert "CI: Add missing CodeCov report.""
petervdonovan Jan 18, 2022
5966476
CI: Update workflow refs.
petervdonovan Jan 18, 2022
7ae66a0
Comments only.
petervdonovan Jan 18, 2022
72d793f
Revert "CI: Attempt to debug failure that cannot be reproduced locally."
petervdonovan Jan 18, 2022
fe4c64d
Revert "CI: Update workflow refs."
petervdonovan Jan 18, 2022
c9939f6
Merge branch 'master' into vscode-validate-generated-code
petervdonovan Jan 18, 2022
b65f376
Revert "Revert "CI: Update workflow refs.""
petervdonovan Jan 18, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions .github/workflows/ts-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,6 @@ jobs:
java-version: 11
- name: Setup Node.js environment
uses: actions/setup-node@v2.1.2
- name: Cache .pnpm-store
uses: actions/cache@v1
with:
path: ~/.pnpm-store
key: ${{ runner.os }}-node${{ matrix.node-version }}-${{ hashFiles('**/pnpm-lock.yaml') }}
- name: Install pnpm
run: sudo npm i -g pnpm
- name: Install Dependencies Ubuntu
Expand Down
10 changes: 6 additions & 4 deletions .idea/codeStyles/Project.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,14 @@ private GeneratorResult buildWithProgress(LanguageClient client, String uri, boo
}
Progress progress = new Progress(client, "Build \"" + parsedUri.lastSegment() + "\"", mustComplete);
progress.begin();
GeneratorResult result = builder.run(
parsedUri, mustComplete, progress::report, progress.getCancelIndicator()
);
progress.end(result.getUserMessage());
GeneratorResult result = null;
try {
result = builder.run(
parsedUri, mustComplete, progress::report, progress.getCancelIndicator()
);
} finally {
progress.end(result == null ? "An internal error occurred." : result.getUserMessage());
}
return result;
}

Expand Down Expand Up @@ -81,7 +85,13 @@ public CompletableFuture<String> build(String uri) {
"Please wait for the Lingua Franca language server to be fully initialized."
);
return CompletableFuture.supplyAsync(
() -> buildWithProgress(client, uri, true).getUserMessage()
() -> {
try {
return buildWithProgress(client, uri, true).getUserMessage();
} catch (Exception e) {
return "An internal error occurred:\n" + e;
}
}
);
}

Expand Down
3 changes: 3 additions & 0 deletions org.lflang.targetplatform/org.lflang.targetplatform.target
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@
<unit id="org.aopalliance" version="0.0.0"/>
<unit id="org.eclipse.xtext.sdk.feature.group" version="0.0.0"/>
<unit id="org.eclipse.tm.terminal.feature.feature.group" version="0.0.0"/>
<unit id="com.fasterxml.jackson.core.jackson-core" version="0.0.0"/>
<unit id="com.fasterxml.jackson.core.jackson-annotations" version="0.0.0"/>
<unit id="com.fasterxml.jackson.core.jackson-databind" version="0.0.0"/>
<repository location="http://download.eclipse.org/releases/2021-06"/>
</location>
<location includeAllPlatforms="false" includeConfigurePhase="true" includeMode="planner" includeSource="false" type="InstallableUnit">
Expand Down
5 changes: 4 additions & 1 deletion org.lflang/META-INF/MANIFEST.MF
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,10 @@ Require-Bundle: org.eclipse.xtext,
org.eclipse.xtend.lib.macro,
org.apache.commons.cli;bundle-version="1.4",
org.jetbrains.kotlin.bundled-compiler;resolution:=optional,
org.eclipse.lsp4j;bundle-version="0.10.0"
org.eclipse.lsp4j;bundle-version="0.10.0",
com.fasterxml.jackson.core.jackson-core,
com.fasterxml.jackson.core.jackson-annotations,
com.fasterxml.jackson.core.jackson-databind
Bundle-RequiredExecutionEnvironment: JavaSE-11
Export-Package: org.lflang,
org.lflang.generator,
Expand Down
4 changes: 3 additions & 1 deletion org.lflang/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ dependencies {
implementation group: 'org.eclipse.emf', name: 'org.eclipse.emf.mwe2.launch', version: '2.12.1'
// https://mvnrepository.com/artifact/org.eclipse.lsp4j/org.eclipse.lsp4j
implementation group: 'org.eclipse.lsp4j', name: 'org.eclipse.lsp4j', version: '0.10.0'

implementation group: 'com.fasterxml.jackson.core', name: 'jackson-core', version: '2.12.4'
implementation group: 'com.fasterxml.jackson.core', name: 'jackson-annotations', version: '2.12.4'
implementation group: 'com.fasterxml.jackson.core', name: 'jackson-databind', version: '2.12.4'
}

configurations {
Expand Down
11 changes: 11 additions & 0 deletions org.lflang/src/lib/ts/.eslintrc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
lhstrh marked this conversation as resolved.
Show resolved Hide resolved
"parser": "@typescript-eslint/parser",
"plugins": [
"@typescript-eslint"
],
"extends": [
"eslint:recommended",
"plugin:@typescript-eslint/eslint-recommended",
"plugin:@typescript-eslint/recommended"
]
}
3 changes: 3 additions & 0 deletions org.lflang/src/lib/ts/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,9 @@
"@babel/preset-typescript": "^7.8.3",
"@types/google-protobuf": "^3.7.4",
"@types/node": "^13.9.2",
"@typescript-eslint/eslint-plugin": "^5.8.1",
"@typescript-eslint/parser": "^5.8.1",
"eslint": "^8.5.0",
"typescript": "^3.8.3",
"ts-protoc-gen": "^0.12.0"
},
Expand Down
45 changes: 20 additions & 25 deletions org.lflang/src/org/lflang/ASTUtils.xtend
Original file line number Diff line number Diff line change
Expand Up @@ -575,6 +575,19 @@ class ASTUtils {
* @return Textual representation of the given argument.
*/
def static String toText(Code code) {
return CodeMap.Correspondence.tag(code, toUntaggedText(code), true)
}

/**
* Translate the given code into its textual representation
* without any tags inserted.
petervdonovan marked this conversation as resolved.
Show resolved Hide resolved
* @param code AST node to render as string.
* @return Textual representation of the given argument.
*/
def private static String toUntaggedText(Code code) {
// FIXME: This function should not be necessary, but it is because we currently
petervdonovan marked this conversation as resolved.
Show resolved Hide resolved
// parse the content of code blocks in the validator and generator. See #810, #657.
var text = ""
if (code !== null) {
val node = NodeModelUtils.getNode(code)
if (node !== null) {
Expand All @@ -591,35 +604,17 @@ class ASTUtils {
str = str.substring(start + 2, end)
if (str.split('\n').length > 1) {
// multi line code
return str.trimCodeBlock
text = str.trimCodeBlock
} else {
// single line code
return str.trim
}
text = str.trim
}
} else if (code.body !== null) {
// Code must have been added as a simple string.
return code.body.toString
text = code.body.toString
}
}
return ""
}

/**
* Translate the given code into its textual representation,
* with a tag inserted to establish this representation's
* correspondence to the generated code. This tag is an
* implementation detail that is internal to the code
* generator.
* @param code the AST node to render as a string
* @return a textual representation of {@code code}
*/
def static String toTaggedText(Code code) {
// FIXME: Duplicates work already done in
// GeneratorBase::prSourceLineNumber. It does not
// make sense for both methods to persist in the
// code base at once.
val text = toText(code)
return CodeMap.Correspondence.tag(code, text, true)
return text
}

def static toText(TypeParm t) {
Expand Down Expand Up @@ -910,7 +905,7 @@ class ASTUtils {
}

def static boolean isZero(Code code) {
if (code !== null && code.toText.isZero) {
if (code !== null && code.toUntaggedText.isZero) {
return true
}
return false
Expand Down Expand Up @@ -951,7 +946,7 @@ class ASTUtils {
* @return True if the given code is an integer, false otherwise.
*/
def static boolean isInteger(Code code) {
return code.toText.isInteger
return code.toUntaggedText.isInteger
}

/**
Expand Down
7 changes: 0 additions & 7 deletions org.lflang/src/org/lflang/AstExtensions.kt
Original file line number Diff line number Diff line change
Expand Up @@ -144,13 +144,6 @@ val StateVar.isOfTimeType: Boolean get() = JavaAstUtils.isOfTimeType(this)
*/
fun Code.toText(): String = ASTUtils.toText(this)

/**
* Translate this code element into its tagged textual
* representation.
* @see ASTUtils.toTaggedText
*/
fun Code.toTaggedText(): String = ASTUtils.toTaggedText(this)

/**
* Translate this code element into its textual representation.
* @see ASTUtils.toText
Expand Down
54 changes: 52 additions & 2 deletions org.lflang/src/org/lflang/ErrorReporter.java
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@
import java.nio.file.Path;

import org.eclipse.emf.ecore.EObject;
import org.eclipse.lsp4j.DiagnosticSeverity;

import org.lflang.generator.Position;

/**
* Interface for reporting errors.
Expand Down Expand Up @@ -55,7 +58,7 @@ public interface ErrorReporter {
* Report an error at the specified line within a file.
*
* @param message The error message.
* @param line The line number to report at.
* @param line The one-based line number to report at.
* @param file The file to report at.
* @return a string that describes the error.
*/
Expand All @@ -66,12 +69,59 @@ public interface ErrorReporter {
* Report a warning at the specified line within a file.
*
* @param message The error message.
* @param line The line number to report at.
* @param line The one-based line number to report at.
* @param file The file to report at.
* @return a string that describes the warning.
*/
String reportWarning(Path file, Integer line, String message);

/**
* Report a message of severity {@code severity}.
* @param severity the severity of the message
* @param message the message to send to the IDE
* @return a string that describes the diagnostic
*/
default String report(DiagnosticSeverity severity, String message) {
switch (severity) {
case Error:
return reportError(message);
case Warning:
case Hint:
case Information: // FIXME: Information -> warning?? If this results in false alarms from LFC, we should expand API.
default:
return reportWarning(message);
}
}

/**
* Report a message of severity {@code severity} that
* pertains to line {@code line} of an LF source file.
* @param severity the severity of the message
* @param message the message to send to the IDE
* @param line the one-based line number associated
* with the message
* @return a string that describes the diagnostic
*/
default String report(DiagnosticSeverity severity, String message, int line) {
return report(severity, message);
}

/**
* Report a message of severity {@code severity} that
* pertains to the range [{@code startPos}, {@code endPos})
* of an LF source file.
* @param severity the severity of the message
* @param message the message to send to the IDE
* @param startPos the position of the first character
* of the range of interest
* @param endPos the position immediately AFTER the
* final character of the range of
* interest
* @return a string that describes the diagnostic
*/
default String report(DiagnosticSeverity severity, String message, Position startPos, Position endPos) {
return report(severity, message);
}

/**
* Check if errors where reported.
Expand Down
2 changes: 0 additions & 2 deletions org.lflang/src/org/lflang/generator/ActionInstance.java
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,9 @@ STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY

package org.lflang.generator;

import org.lflang.JavaAstUtils;
import org.lflang.TimeValue;
import org.lflang.lf.Action;
import org.lflang.lf.ActionOrigin;
import org.lflang.lf.Parameter;

/**
* Instance of an action.
Expand Down
17 changes: 4 additions & 13 deletions org.lflang/src/org/lflang/generator/CodeMap.java
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ public static class Correspondence {
"/\\*Correspondence: (?<lfRange>%s) \\-> (?<generatedRange>%s) \\(src=(?<path>%s)\\)\\*/",
Position.removeNamedCapturingGroups(Range.PATTERN),
Position.removeNamedCapturingGroups(Range.PATTERN),
".*"
".*?"
));

// TODO(peter): Add "private final boolean verbatim;" and make corresponding enhancements
Expand Down Expand Up @@ -153,21 +153,12 @@ public static String tag(EObject astNode, String representation, boolean verbati
Position lfStart = Position.fromOneBased(
oneBasedLfLineAndColumn.getLine(), oneBasedLfLineAndColumn.getColumn()
);
Position lfDisplacement;
final Position generatedCodeDisplacement = Position.displacementOf(representation);
final Path lfPath = Path.of(astNode.eResource().getURI().path());
if (verbatim) {
lfStart = lfStart.plus(Position.displacementOf(
node.getText().substring(0, indexOf(node.getText(), representation))
));
lfDisplacement = generatedCodeDisplacement;
} else {
lfDisplacement = Position.displacementOf(node.getText());
}
if (verbatim) lfStart = lfStart.plus(node.getText().substring(0, indexOf(node.getText(), representation)));
return new Correspondence(
lfPath,
new Range(lfStart, lfStart.plus(lfDisplacement)),
new Range(Position.ORIGIN, generatedCodeDisplacement)
new Range(lfStart, lfStart.plus(verbatim ? representation : node.getText())),
new Range(Position.ORIGIN, Position.displacementOf(representation))
) + representation;
}

Expand Down

This file was deleted.

Loading