Replies: 6 comments 10 replies
-
|
I'm a bit unclear what the request is. From a technical standpoint, native-image must know about a resource at buildtime to be able to package it into the binary for runtime. Otherwise, it's up to the program to read the data from an external source (e.g. file, database). In previous versions of native-image, it was possible to manually specify sets of resources using a regex. I think that went away with the https://www.graalvm.org/latest/reference-manual/native-image/metadata/ also mentions |
Beta Was this translation helpful? Give feedback.
-
Correct, the issue manifests when the program tries to access some resource at runtime that we don't necessarily need and thus didn't register. One could argue that the right thing to do in such cases is to still register the resource and just don't provide it at build-time (in order to not include it in the binary). However, this becomes pretty tedious when a library tries to access a resource by trying different names and/or different paths. E.g. a library might try to load files matching a pattern like
Maybe. A regex/wildcard/glob would work-around the issue by essentially registering all FWIW
Correct, but the intention is to ultimately make |
Beta Was this translation helpful? Give feedback.
-
|
@zakkak I am also considering to relax the current setting. I feel we have two major culprits here:
I am not in favor of adding a special flag to make things work (for debugging it is OK), as this moves Native Image away from the Java semantics. My plan is to experiment with these three approaches in the next 2 months and then we can decide what is easiest to support and use. @zakkak what are your thoughts? |
Beta Was this translation helpful? Give feedback.
-
That indeed works well for cases where we expect the various resources to reside on a single folder (which is known at build-time), however there are cases where different folders are visited as well and to make matters worse these folders might be unknown at build-time. cc @radcortez
+1, I have also seen the pattern described in jobrunr/jobrunr#1101 (comment) used in more than one places as well. Users interested in a method, instead of explicitly trying to reflectively get the specific method, load the class and invoke
I guess there is a typo here and the exception would be thrown when the
Packages without wildcards might do the trick as well. It's not clear to me what this would do though? Would it allow us to reflectively access any Class in What we are more interested in is allowing any Class/Method under
Don't we already have that with
I was thinking about extending the {
"reflection": [
{
"type": "io.smallrye.config.MyClass",
"uncheckedReflectiveAccesses": true
}
],
"resources": [
{
"type": "io.smallrye.config.MyClass"
"uncheckedReflectiveAccesses": true
}
]
}which would result in ignoring unregistered accesses from
I am afraid 1 and 2 are going to be too expensive (in terms of bloat). |
Beta Was this translation helpful? Give feedback.
-
|
How about To load configuration, we use this API to search for optional resources, since it either returns an enum with the found resources or an empty one if no resources are found. Right now, if you try with In Quarkus, we discover these resources during build time, and byte code generate a programmatic representation of the files, so we don't need the resources. We leave the method call from the library to return an empty enumeration, which works fine. We could register the resources, but the library has semantics where additional resources can be added at runtime, which are searched of both classpath, and file system. Of course, during native image, you can't add resources in the classpath, but the library is not aware of that, and it searches both paths. In this case, we cannot register the resources, because they are unknown to us during the build. I understand the decision for APIs as |
Beta Was this translation helpful? Give feedback.
-
|
The awkwardness comes with a benefit that you know that the native execution will behave exactly the same as the JVM. I would always trade a few more entries in the JSON file (that are explicitly mentioned in the error), for hours spent in debugging with How many resource entries do you need to add in this case? Could you please share a link to the pattern that is used here? Have you tried using the resource patterns with |
Beta Was this translation helpful? Give feedback.
-
|
In our case, we support the resource as a runtime configuration: For example, you can add Obviously, the classpath lookup will not discover any resources in native mode, but this is fine. I can work around the error by providing a substitution, but I feel that |
Beta Was this translation helpful? Give feedback.
-
Regarding bloat, I find that the biggest problem here are not the application classes, but the JDK classes. In a simple
Yes it would include all of these types to be available for
We do, exactly, just the default exception for
When you boil it down, this is a workaround for a lack of Maybe we could throw This way libraries would be able to handle this and we don't need extra JSON. In the future if a
|
Beta Was this translation helpful? Give feedback.
-
In theory I agree, but this approach assumes that libraries will be willing to include code specifically for handling GraalVM native-image compilation, which is not always the case. Metadata (like substitutions) allow us to work around this and support libraries that are not willing to include GraalVM specific code. |
Beta Was this translation helpful? Give feedback.
-
We could make a flag for this, but I would leave it as experimental for the time the third-party libraries don't adopt the new semantics. |
Beta Was this translation helpful? Give feedback.
-
|
Have you considered branching the logic of the original code on the We created an enum around the possible values for convenience. |
Beta Was this translation helpful? Give feedback.
-
|
@nreid260 if I get this right, you suggest adding a check to guard resource accesses, e.g. That's indeed an option when you have write access (or can convince the maintainers to merge this kind of change), i.e. this approach assumes that libraries will be willing to include code specifically for handling GraalVM native-image compilation, which is not always the case. As a result, in some cases we would need to introduce substitutions to apply this. |
Beta Was this translation helpful? Give feedback.
-
|
Yeah this is the basic idea. It's fairly natural to me that some codepaths which work for JVM and need to be changed for native. For libraries you don't control, you could do a little bytecode rewriting (e.g. with ASM) to insert the needed logic. Of course this takes effort, but possibly less effort than other options. I also wonder if library authors would be sympathetic to simpler resource handling logic even outside the context of graalification. They might support args like "skip loading external config files" or whatever. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, I full agree with the argument here. I would always strive to have a simple solution that is the same both with native and on the JVM. For example, in radcortez's example above it would be nice to look only on the file path and skip resources lookup altogether. But if this is not possible for some reason, we can always make a condition on |
Beta Was this translation helpful? Give feedback.
-
|
We discussed this with @radcortez and it's still problematic. Imagine the following scenario: The library allows users to load arbitrary files either from the classpath or the filesystem. The user defines the file name through a property called In JVM mode everything works as expected. The user sets In native mode the value of Substituting the logic and avoiding the classpath lookup works around the issue, but behaves differently than the JVM mode, since in native mode the classpath is never searched. That sounds fine since even in the current state that allows searching in the classpath the behavior would be different if the resource was not registered. The problem arises if an application developer using the library registers some resources, e.g. So far the only backwards compatible approach we could come up is to substitute the library code to introduce a try-catch that will handle the |
Beta Was this translation helpful? Give feedback.
-
|
In general, to make an informed decision it would be good to give examples of how many entries relative to all metadata entries need to be added for the project to work. If the number of entries is relatively small compared to the size of the project (e.g. <1-2% of all entries), I feel the benefits of strict semantics outweigh the cost of maintaining the extra entries (especially because the missing entries are pretty easy to find). |
Beta Was this translation helpful? Give feedback.
-
Hello,
At the moment,
--exact-reachability-metadatarequires all resources being accessed at run-time to be registered at build-time. This is a great way to detect if the code is trying to access some resource that has been mistakenly left out and is not available at run-time. However, it also results in errors when using libraries that try to load resources based on "trial and error", e.g. to find configuration files under different folders like the user's home directory, the current directory, etc.Having to register all the potential resources seems too tedious. I wonder if an option to exclude specific code-paths, classes, or packages from the
exact-reachability-metadatachecks would make sense. E.g., for resources we could allow unregistered accesses from packageio.smallrye.config.cc @vjovanov
Beta Was this translation helpful? Give feedback.
All reactions