Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GR-44222] [GR-47557] Separate SVM and Truffle (Unchained Part 6). #7102

Merged
merged 48 commits into from
Aug 2, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
e15a921
Move Truffle SVM code into separate jars.
chumer Jul 16, 2023
abce31a
Improve error handling when list modules fails.
chumer Jul 27, 2023
1bcf361
Add in module configuration for Truffle; Add new truffle-svm macro fo…
chumer Jul 17, 2023
62f8490
Reduce Truffle GraalVM component dependencies.
chumer Jul 17, 2023
57ae7cd
Fix TruffleToLibgraalEntryPoints validation no longer works if Truffl…
chumer Jul 18, 2023
976f725
Add ce-unchained environment configuration and fix problems associate…
chumer Jul 18, 2023
8954a2e
Enable useModulePath for Truffle in order to be able to run unittests…
chumer Jul 18, 2023
8c21c5e
Fix subprocess utils must also handle module path vm arguments.
chumer Jul 19, 2023
385b537
Move open packages to the unit test participant that needs declares t…
chumer Jul 19, 2023
40be3be
Add missing qualified export for jargraal accessing the truffle compi…
chumer Jul 19, 2023
10704a8
Fix missing exports.
chumer Jul 19, 2023
0039e0f
Support Requires in native-image.properties embedded jar files.
chumer Jul 21, 2023
45de1bf
Fix TruffleBaseFeature cannot depend on SubstrateTruffleRuntime direc…
chumer Jul 21, 2023
d6062ae
Make dependencies to truffle-runtime optional in the module-descripto…
chumer Jul 21, 2023
cf7c685
Fix do not use automatic modules as they do not work on the module-path.
chumer Jul 21, 2023
1db65f3
Modify mx sl command to use the previously built GraalVM. Add slimage…
chumer Jul 21, 2023
1bb349a
Remove Graal SDK dependency from libgraal.
chumer Jul 21, 2023
6a7fade
Fix Truffle instrumentation suite must not load Truffle types using r…
chumer Jul 24, 2023
6faaf6e
Fix this-escape error in OptimizedTruffleRuntime.
chumer Jul 24, 2023
db3d375
Fix mx python warnings.
chumer Jul 24, 2023
91ac86b
Fix missing qualified exports to Truffle in svm suite.
chumer Jul 24, 2023
3ca1057
Export SL to unittests and truffle to SL in case it is not used via T…
chumer Jul 24, 2023
a62e802
Fix initializeEngineCacheSupport was never initializing the engine ca…
chumer Jul 24, 2023
6fcd64f
Fixes and cleanup for separating Truffle from SVM.
chumer Jul 31, 2023
77cab69
Disable tests that are too slow for isolates. (GR-47643)
chumer Jul 31, 2023
9b6ab23
Fix minor problem after merge.
chumer Jul 31, 2023
3c721a3
Fix missing add-exports in legacy macro.
chumer Jul 31, 2023
d5a0fc7
Fix native image driver should exclude macro options in builds with -…
chumer Aug 1, 2023
2af5164
Use new feature syntax for Truffle features.
chumer Aug 1, 2023
028fac5
Fix misleading comment.
chumer Aug 1, 2023
0d4c05d
Cleanup native driver line parsing.
chumer Aug 1, 2023
0dad57a
Remove requires support; Instead use a regular arg.
chumer Aug 1, 2023
ef75121
Add comment for truffle library exclusion.
chumer Aug 1, 2023
7ef9b41
Fix missing exe and command suffixes on Windows.
chumer Aug 1, 2023
44e16b8
Remove commented UseModulePath.
chumer Aug 1, 2023
36fea38
Add comment about short-name usage in substrate vm component.
chumer Aug 1, 2023
8043560
Fix use explicit target_dir for sl native image builds.
chumer Aug 1, 2023
bc29989
Update common.json.
chumer Aug 1, 2023
20b0a61
Cleanup useless comment.
chumer Aug 1, 2023
6080f0b
Fix JFR feature.
chumer Aug 1, 2023
5b6bac9
Use mx.rmtree instead of shutil.rmtree.
chumer Aug 1, 2023
4223185
Fix slnative.
chumer Aug 1, 2023
dc73dde
Add a note about GR-47669.
chumer Aug 1, 2023
b53cf52
Fix trailing \.
chumer Aug 1, 2023
d9b1dc9
Move rebuild-images to the legacy Truffle macro to remove it from unc…
chumer Aug 1, 2023
ab294f8
Cleanup legacy truffel macro.
chumer Aug 1, 2023
892f331
Restore old behavior where rebuild-images.sh is still part of ni.
chumer Aug 1, 2023
818a8fd
Ignore truffle-svm macro if used on class-path only builds.
chumer Aug 1, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion common.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"Jsonnet files should not include this file directly but use ci/common.jsonnet instead."
],

"mx_version": "6.34.0",
"mx_version": "6.35.1",

"COMMENT.jdks": "When adding or removing JDKs keep in sync with JDKs in ci/common.jsonnet",
"jdks": {
Expand Down
6 changes: 1 addition & 5 deletions compiler/mx.compiler/mx_compiler.py
Original file line number Diff line number Diff line change
Expand Up @@ -704,11 +704,7 @@ def _unittest_config_participant(config):
# for the junit harness which uses reflection to find @Test methods. In addition, the
# tests widely use JVMCI classes so JVMCI needs to also export all its packages to
# ALL-UNNAMED.

mainClassArgs.extend(['-JUnitOpenPackages', 'jdk.internal.vm.ci/*=org.graalvm.truffle.runtime,jdk.internal.vm.compiler,ALL-UNNAMED'])
mainClassArgs.extend(['-JUnitOpenPackages', 'org.graalvm.truffle/*=ALL-UNNAMED'])
mainClassArgs.extend(['-JUnitOpenPackages', 'org.graalvm.truffle.compiler/*=ALL-UNNAMED'])
mainClassArgs.extend(['-JUnitOpenPackages', 'org.graalvm.truffle.runtime/*=ALL-UNNAMED'])

limited_modules = None
for arg in vmArgs:
Expand Down Expand Up @@ -1341,7 +1337,7 @@ def _jvmci_jars():
dir_name='graal',
license_files=[],
third_party_license_files=[],
dependencies=['Truffle API'],
dependencies=['Truffle Compiler'],
jar_distributions=[ # Dev jars (annotation processors)
'compiler:GRAAL_PROCESSOR',
],
Expand Down
6 changes: 4 additions & 2 deletions compiler/mx.compiler/suite.py
Original file line number Diff line number Diff line change
Expand Up @@ -452,7 +452,9 @@
"truffle:TRUFFLE_TEST",
"truffle:TRUFFLE_COMPILER",
"truffle:TRUFFLE_RUNTIME",
"regex:TREGEX"
"regex:TREGEX",
"ASM_TREE_9.5",
"ASM_UTIL_9.5",
],
"exclude" : [
"mx:JUNIT",
Expand Down Expand Up @@ -493,7 +495,7 @@
],
"exports" : [
"""* to com.oracle.graal.graal_enterprise,org.graalvm.nativeimage.pointsto,org.graalvm.nativeimage.builder,org.graalvm.nativeimage.llvm,com.oracle.svm.svm_enterprise,com.oracle.svm_enterprise.ml_dataset,org.graalvm.nativeimage.base,
org.graalvm.extraimage.builder,com.oracle.svm.extraimage_enterprise""",
org.graalvm.extraimage.builder,com.oracle.svm.extraimage_enterprise,org.graalvm.truffle.runtime.svm,com.oracle.truffle.enterprise.svm""",
"org.graalvm.compiler.java to org.graalvm.nativeimage.agent.tracing,org.graalvm.nativeimage.configure",
"org.graalvm.compiler.core.common to org.graalvm.nativeimage.agent.tracing,org.graalvm.nativeimage.objectfile",
"org.graalvm.compiler.debug to org.graalvm.nativeimage.objectfile",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -445,7 +445,7 @@ public static Subprocess process(List<String> command, Map<String, String> env,
}

private static boolean hasArg(String optionName) {
if (optionName.equals("-cp") || optionName.equals("-classpath")) {
if (optionName.equals("-cp") || optionName.equals("-classpath") || optionName.equals("-p")) {
return true;
}
if (optionName.equals("--version") ||
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,7 @@ public class KnownTruffleTypes extends AbstractKnownTruffleTypes {
public final ResolvedJavaType CompilationState = lookupType("com.oracle.truffle.runtime.CompilationState");

public final ResolvedJavaType OptimizedCallTarget = lookupTypeCached("com.oracle.truffle.runtime.OptimizedCallTarget");
public final ResolvedJavaMethod OptimizedCallTarget_call = findMethod(OptimizedCallTarget, "call", Object_Array);
public final ResolvedJavaMethod OptimizedCallTarget_callDirect = findMethod(OptimizedCallTarget, "callDirect", Node, Object_Array);
public final ResolvedJavaMethod OptimizedCallTarget_callInlined = findMethod(OptimizedCallTarget, "callInlined", Node, Object_Array);
public final ResolvedJavaMethod OptimizedCallTarget_callIndirect = findMethod(OptimizedCallTarget, "callIndirect", Node, Object_Array);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@ public InstrumentPhase.Instrumentation getInstrumentation() {
throw new IllegalStateException("PartialEvaluator is not yet initialized");
}
long[] accessTable = new long[instrumentationCfg.instrumentationTableSize];
instrumentation = new InstrumentPhase.Instrumentation(accessTable);
instrumentation = new InstrumentPhase.Instrumentation(types, accessTable);
}
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@
import static org.graalvm.compiler.truffle.compiler.TruffleCompilerOptions.InstrumentBranchesPerInlineSite;
import static org.graalvm.compiler.truffle.compiler.TruffleCompilerOptions.InstrumentationTableSize;

import java.lang.reflect.Method;
import java.util.AbstractMap;
import java.util.ArrayList;
import java.util.Collections;
Expand All @@ -55,47 +54,24 @@
import org.graalvm.compiler.nodes.java.StoreIndexedNode;
import org.graalvm.compiler.options.OptionValues;
import org.graalvm.compiler.phases.BasePhase;
import org.graalvm.compiler.truffle.compiler.KnownTruffleTypes;
import org.graalvm.compiler.truffle.compiler.TruffleCompilerOptions;
import org.graalvm.compiler.truffle.compiler.TruffleTierContext;

import jdk.vm.ci.code.CodeUtil;
import jdk.vm.ci.meta.JavaConstant;
import jdk.vm.ci.meta.JavaKind;
import jdk.vm.ci.meta.MetaUtil;
import jdk.vm.ci.meta.ResolvedJavaMethod;

public abstract class InstrumentPhase extends BasePhase<TruffleTierContext> {

private static boolean checkMethodExists(String declaringClassName, String methodName) {
try {
Class<?> declaringClass = Class.forName(declaringClassName);
for (Method m : declaringClass.getDeclaredMethods()) {
if (m.getName().equals(methodName)) {
return true;
}
}
} catch (ClassNotFoundException e) {
throw new NoClassDefFoundError(declaringClassName);
}
throw new NoSuchMethodError(declaringClassName + "." + methodName);
}

private static String asStackPattern(String declaringClassName, String methodName) {
assert checkMethodExists(declaringClassName, methodName);
return declaringClassName + "." + methodName;
}

private static final String[] OMITTED_STACK_PATTERNS = new String[]{
asStackPattern("com.oracle.truffle.runtime.OptimizedCallTarget", "executeRootNode"),
asStackPattern("com.oracle.truffle.runtime.OptimizedCallTarget", "profiledPERoot"),
asStackPattern("com.oracle.truffle.runtime.OptimizedCallTarget", "callDirect"),
asStackPattern("com.oracle.truffle.runtime.OptimizedDirectCallNode", "call"),
};
private final Instrumentation instrumentation;
protected final SnippetReflectionProvider snippetReflection;

public InstrumentPhase(SnippetReflectionProvider snippetReflection, Instrumentation instrumentation) {
this.snippetReflection = snippetReflection;
this.instrumentation = instrumentation;

}

public Instrumentation getInstrumentation() {
Expand Down Expand Up @@ -190,9 +166,20 @@ public int compare(Map.Entry<String, Point> x, Map.Entry<String, Point> y) {
public Map<String, Point> pointMap = new LinkedHashMap<>();
public int tableIdCount;
public int tableStartIndex;
private final String[] omittedStackPatterns;

public Instrumentation(long[] accessTable) {
public Instrumentation(KnownTruffleTypes types, long[] accessTable) {
this.accessTable = accessTable;
this.omittedStackPatterns = new String[]{
asStackPattern(types.OptimizedCallTarget_executeRootNode),
asStackPattern(types.OptimizedCallTarget_profiledPERoot),
asStackPattern(types.OptimizedCallTarget_callDirect),
asStackPattern(types.OptimizedCallTarget_call),
};
}

private static String asStackPattern(ResolvedJavaMethod method) {
return method.getDeclaringClass().toJavaName(true) + "." + method.getName();
}

/*
Expand Down Expand Up @@ -226,7 +213,7 @@ private static String filterAndEncode(MethodFilter methodFilter, Node node, Inst
}
}

private static String prettify(String key, Point p) {
private String prettify(String key, Point p) {
if (p.isPrettified()) {
StringBuilder sb = new StringBuilder();
NodeSourcePosition pos = p.getPosition();
Expand All @@ -235,7 +222,7 @@ private static String prettify(String key, Point p) {

callerChainLoop: while (pos != null) {
// Skip stack frame if it is a known pattern.
for (String pattern : OMITTED_STACK_PATTERNS) {
for (String pattern : omittedStackPatterns) {
if (pos.getMethod().format("%H.%n(%p)").contains(pattern)) {
pos = pos.getCaller();
continue callerChainLoop;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,8 @@

public class InstrumentationSuite extends PhaseSuite<TruffleTierContext> {
@SuppressWarnings("this-escape")
public InstrumentationSuite(InstrumentPhase.InstrumentationConfiguration instrumentationCfg, SnippetReflectionProvider snippetReflection, InstrumentPhase.Instrumentation instrumentation) {
public InstrumentationSuite(InstrumentPhase.InstrumentationConfiguration instrumentationCfg, SnippetReflectionProvider snippetReflection,
InstrumentPhase.Instrumentation instrumentation) {
if (instrumentationCfg.instrumentBranches) {
appendPhase(new InstrumentBranchesPhase(snippetReflection, instrumentation, instrumentationCfg.instrumentBranchesPerInlineSite));
}
Expand Down
2 changes: 1 addition & 1 deletion espresso/mx.espresso/jvm-ce
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# mx --dynamicimports=/vm,/substratevm --components="Espresso Launcher,LibGraal,SubstrateVM,suite:tools" --native-images=lib:jvmcicompiler --disable-installables=true graalvm-show

DYNAMIC_IMPORTS=/vm,/substratevm
COMPONENTS=Espresso Launcher,LibGraal,SubstrateVM,suite:tools
COMPONENTS=Espresso Launcher,LibGraal,SubstrateVM,suite:tools,tflm
NATIVE_IMAGES=lib:jvmcicompiler
DISABLE_INSTALLABLES=true
2 changes: 1 addition & 1 deletion espresso/mx.espresso/jvm-ce-llvm
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# mx --dynamicimports=/vm,/substratevm --components="Espresso Launcher,LibGraal,SubstrateVM,Java on Truffle LLVM Java libraries,suite:tools" --native-images=lib:jvmcicompiler --disable-installables=true graalvm-show

DYNAMIC_IMPORTS=/vm,/substratevm
COMPONENTS=Espresso Launcher,LibGraal,SubstrateVM,Java on Truffle LLVM Java libraries,suite:tools
COMPONENTS=Espresso Launcher,LibGraal,SubstrateVM,Java on Truffle LLVM Java libraries,suite:tools,tflm
NATIVE_IMAGES=lib:jvmcicompiler
DISABLE_INSTALLABLES=true
2 changes: 1 addition & 1 deletion espresso/mx.espresso/jvm-ee
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# mx --dynamicimports=/vm-enterprise,/substratevm-enterprise,/tools-enterprise --components="Espresso Launcher,LibGraal Enterprise,SubstrateVM Enterprise,suite:tools,suite:tools-enterprise" --native-images=lib:jvmcicompiler --disable-installables=true graalvm-show

DYNAMIC_IMPORTS=/vm-enterprise,/substratevm-enterprise,/tools-enterprise
COMPONENTS=Espresso Launcher,LibGraal Enterprise,SubstrateVM Enterprise,suite:tools,suite:tools-enterprise
COMPONENTS=Espresso Launcher,LibGraal Enterprise,SubstrateVM Enterprise,suite:tools,suite:tools-enterprise,tflm
NATIVE_IMAGES=lib:jvmcicompiler
DISABLE_INSTALLABLES=true
EXCLUDE_COMPONENTS=hprf
2 changes: 1 addition & 1 deletion espresso/mx.espresso/jvm-ee-llvm
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# mx --dynamicimports=/vm-enterprise,/substratevm-enterprise,/tools-enterprise --components="Espresso Launcher,LibGraal Enterprise,SubstrateVM Enterprise,Java on Truffle LLVM Java libraries,suite:tools,suite:tools-enterprise" --native-images=lib:jvmcicompiler --disable-installables=true graalvm-show

DYNAMIC_IMPORTS=/vm-enterprise,/substratevm-enterprise,/tools-enterprise
COMPONENTS=Espresso Launcher,LibGraal Enterprise,SubstrateVM Enterprise,Java on Truffle LLVM Java libraries,suite:tools,suite:tools-enterprise
COMPONENTS=Espresso Launcher,LibGraal Enterprise,SubstrateVM Enterprise,Java on Truffle LLVM Java libraries,suite:tools,suite:tools-enterprise,tflm
NATIVE_IMAGES=lib:jvmcicompiler
DISABLE_INSTALLABLES=true
EXCLUDE_COMPONENTS=hprf
Loading