Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kotlin anonymous classes provoke errors #420

Open
bric3 opened this issue Feb 10, 2022 · 10 comments
Open

Kotlin anonymous classes provoke errors #420

bric3 opened this issue Feb 10, 2022 · 10 comments

Comments

@bric3
Copy link

bric3 commented Feb 10, 2022

package com.github.bric3.ha

import java.util.concurrent.Executors
import java.util.concurrent.TimeUnit

fun main(args: Array<String>) {
    println(Reproducer().o)


    Executors.newSingleThreadScheduledExecutor()
        .scheduleAtFixedRate(
            Runnable {
                println(Reproducer().o)

            },
            2,
            2,
            TimeUnit.SECONDS
        )
}


class Reproducer {
    val o = object {
        override fun toString(): String {
            return "foo-bar"
        }
    }
}
/Users/brice.dutheil/.asdf/installs/java/trava-11.0.11+1/bin/java \
  -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:62176,suspend=y,server=n \
  -XX:HotswapAgent=external \
  -javaagent:hotswap-agent-1.4.2-SNAPSHOT.jar \
  -javaagent:/Users/brice.dutheil/Library/Caches/JetBrains/IntelliJIdea2021.3/captureAgent/debugger-agent.jar=file:/private/var/folders/vr/gc7zwl7x1kvcykpbl0j4zdb00000gq/T/capture.props \
  -Dfile.encoding=UTF-8 \
  -classpath 
/Users/brice.dutheil/ha-kotlin/build/classes/kotlin/main:/Users/brice.dutheil/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk8/1.5.31/ff5d99aecd328872494e8921b72bf6e3af97af3e/kotlin-stdlib-jdk8-1.5.31.jar:/Users/brice.dutheil/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-reflect/1.5.31/1523fcd842a47da0820cea772b19c51056fec8a9/kotlin-reflect-1.5.31.jar:/Users/brice.dutheil/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk7/1.5.31/77e0f2568912e45d26c31fd417a332458508acdf/kotlin-stdlib-jdk7-1.5.31.jar:/Users/brice.dutheil/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/1.5.31/6628d61d0f5603568e72d2d5915d2c034b4f1c55/kotlin-stdlib-1.5.31.jar:/Users/brice.dutheil/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-common/1.5.31/43331609c7de811fed085e0dfd150874b157c32/kotlin-stdlib-common-1.5.31.jar com.github.bric3.ha.ReproducerKt

This gives the following logs

HOTSWAP AGENT: 17:31:26.524 INFO (org.hotswap.agent.HotswapAgent) - Loading Hotswap agent {1.4.2-SNAPSHOT} - unlimited runtime class redefinition.
HOTSWAP AGENT: 17:31:27.539 INFO (org.hotswap.agent.config.PluginRegistry) - Discovered plugins: [JdkPlugin, ClassInitPlugin, AnonymousClassPatch, WatchResources, Hotswapper, Hibernate, Hibernate3JPA, Hibernate3, Spring, Jersey1, Jersey2, Jetty, Tomcat, ZK, Logback, Log4j2, MyFaces, Mojarra, Omnifaces, ELResolver, WildFlyELResolver, OsgiEquinox, Owb, Proxy, WebObjects, Weld, JBossModules, ResteasyRegistry, Deltaspike, GlassFish, Weblogic, Vaadin, Wicket, CxfJAXRS, FreeMarker, Undertow, MyBatis, IBatis, JacksonPlugin, Idea]
foo-bar-qux
foo-bar-qux
foo-bar-qux
foo-bar-qux
foo-bar-qux
foo-bar-qux
foo-bar-qux
foo-bar-qux
foo-bar-qux
foo-bar-qux
HOTSWAP AGENT: 17:32:02.727 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
HOTSWAP AGENT: 17:32:02.744 ERROR (org.hotswap.agent.annotation.handler.PluginClassFileTransformer) - InvocationTargetException in transform method on plugin 'class org.hotswap.agent.plugin.jvm.AnonymousClassPatchPlugin' class 'com/github/bric3/ha/Reproducer$o$1'.
java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.hotswap.agent.annotation.handler.PluginClassFileTransformer.transform(PluginClassFileTransformer.java:218)
	at org.hotswap.agent.annotation.handler.PluginClassFileTransformer.transform(PluginClassFileTransformer.java:112)
	at org.hotswap.agent.util.HotswapTransformer.transform(HotswapTransformer.java:246)
	at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:246)
	at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
	at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:563)
Caused by: java.lang.NullPointerException
	at org.hotswap.agent.plugin.jvm.AnonymousClassInfos.lastModified(AnonymousClassInfos.java:275)
	at org.hotswap.agent.plugin.jvm.AnonymousClassInfos.<init>(AnonymousClassInfos.java:109)
	at org.hotswap.agent.plugin.jvm.AnonymousClassPatchPlugin.getStateInfo(AnonymousClassPatchPlugin.java:244)
	at org.hotswap.agent.plugin.jvm.AnonymousClassPatchPlugin.patchAnonymousClass(AnonymousClassPatchPlugin.java:104)
	... 10 more

HOTSWAP AGENT: 17:32:02.746 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
HOTSWAP AGENT: 17:32:02.749 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
foo-bar
foo-bar
foo-bar
foo-bar
foo-bar
foo-bar
Disconnected from the target VM, address: '127.0.0.1:62575', transport: 'socket'

Process finished with exit code 130 (interrupted by signal 2: SIGINT)

The classes are here :

.rw-r--r--   820 brice.dutheil 10 Feb 17:33  Reproducer$o$1.class
.rw-r--r--   862 brice.dutheil 10 Feb 17:33  Reproducer.class
.rw-r--r--  1.9k brice.dutheil 10 Feb 17:33  ReproducerKt.class
@bric3
Copy link
Author

bric3 commented Feb 10, 2022

I'm not able to debug the agent, but I was able to modify the code and see what was inside the pool.

And on this very reduced reproducer, the class is not in the pool

lastModified called for: com.github.bric3.ha.Reproducer$o, classpool content

  • char:org.hotswap.agent.javassist.CtPrimitiveType@1e8ef55c[char]
  • long:org.hotswap.agent.javassist.CtPrimitiveType@32ac524b[long]
  • float:org.hotswap.agent.javassist.CtPrimitiveType@78f7c35a[float]
  • double:org.hotswap.agent.javassist.CtPrimitiveType@6dc64d1a[double]
  • short:org.hotswap.agent.javassist.CtPrimitiveType@48b71eb[short]
  • byte:org.hotswap.agent.javassist.CtPrimitiveType@5465453b[byte]
  • int:org.hotswap.agent.javassist.CtPrimitiveType@124ea1fe[int]
  • boolean:org.hotswap.agent.javassist.CtPrimitiveType@4dbf9030[boolean]
  • void:org.hotswap.agent.javassist.CtPrimitiveType@12e4b445[void]

@bric3
Copy link
Author

bric3 commented Feb 10, 2022

Also when developing within IJ for some reasons I had an issue with the AgentLogger, I am not quite why it got wrong.

HOTSWAP AGENT: 16:50:26.254 ERROR (org.hotswap.agent.annotation.handler.PluginClassFileTransformer) - InvocationTargetException in transform method on plugin 'class org.hotswap.agent.plugin.proxy.ProxyPlugin' class 'com/company/intellij/toolpane/SimpleToolWindow$TopList$BackGroundRatioRenderer$jLabel$1'.
java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.hotswap.agent.annotation.handler.PluginClassFileTransformer.transform(PluginClassFileTransformer.java:218)
	at org.hotswap.agent.annotation.handler.PluginClassFileTransformer.transform(PluginClassFileTransformer.java:112)
	at org.hotswap.agent.util.HotswapTransformer.transform(HotswapTransformer.java:246)
	at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:246)
	at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
	at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:563)
Caused by: java.lang.NoClassDefFoundError: org/hotswap/agent/logging/AgentLogger
	at org.hotswap.agent.plugin.proxy.hscglib.GeneratorParametersRecorder.<clinit>(GeneratorParametersRecorder.java:38)
Caused by: java.lang.NoClassDefFoundError: org/hotswap/agent/logging/AgentLogger

	at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized0(Native Method)
	at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized(Unsafe.java:1042)
	at java.base/jdk.internal.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43)
	at java.base/jdk.internal.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:186)
	at java.base/java.lang.reflect.Field.acquireFieldAccessor(Field.java:1105)
	at java.base/java.lang.reflect.Field.getFieldAccessor(Field.java:1086)
	at java.base/java.lang.reflect.Field.get(Field.java:418)
	at org.hotswap.agent.plugin.proxy.hscglib.GeneratorParametersTransformer.getGeneratorParamsMap(GeneratorParametersTransformer.java:103)
	at org.hotswap.agent.plugin.proxy.hscglib.GeneratorParametersTransformer.getGeneratorParams(GeneratorParametersTransformer.java:130)
	at org.hotswap.agent.plugin.proxy.ProxyPlugin.transformCglibProxy(ProxyPlugin.java:105)
	... 10 more
Caused by: java.lang.ClassNotFoundException: org.hotswap.agent.logging.AgentLogger PluginClassLoader(plugin=PluginDescriptor(name=PluginName, id=com.company.intellij, descriptorPath=plugin.xml, path=~/company/intellij-plugin/build/idea-sandbox/plugins/plugin-name, version=2.5.6-SNAPSHOT, package=null, isBundled=false), packagePrefix=null, instanceId=120, state=active)
Caused by: java.lang.ClassNotFoundException: org.hotswap.agent.logging.AgentLogger PluginClassLoader(plugin=PluginDescriptor(name=PluginName, id=com.company.intellij, descriptorPath=plugin.xml, path=~/company/intellij-plugin/build/idea-sandbox/plugins/plugin-name, version=2.5.6-SNAPSHOT, package=null, isBundled=false), packagePrefix=null, instanceId=120, state=active)

	at com.intellij.ide.plugins.cl.PluginClassLoader.loadClass(PluginClassLoader.java:235)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	... 21 more

@skybber
Copy link
Contributor

skybber commented Feb 10, 2022

From anonymous class plugin's point of view all code must published to class path before redefinition is called. I have noticed a problem with it, when IDE called redefinition before all compilation is synchronized to class path, may be it is the same problem. In my case there was a new class, that did not exist before.

@bric3
Copy link
Author

bric3 commented Feb 11, 2022

I don't think this is the case here. The patchAnonymousClass methods tries to identify the enclosing class, and it assumes that anonymous classes are suffixed $d (where d is a number) ; this is indeed what javac does, and what Eclipse compiler does (with slight differences in the numbering scheme).

String javaClass = className.replaceAll("/", ".");
String mainClass = javaClass.replaceAll("\\$\\d+$", "");

However it seems that kotlinc at this time names the class with additional $identifier it seems (probably to locate the anonymous class).

Hence it seems that indeed the class com.github.bric3.ha.Reproducer$o doesn't exists, but com.github.bric3.ha.Reproducer does.

@bric3
Copy link
Author

bric3 commented Feb 11, 2022

I have played a bit with kotlin, and discovered they have backticked identifiers. So fun `$$a while - space`(): Any is legal.

So for example the code below will produce these classes :

.rw-r--r--   885 brice.dutheil 11 Feb 00:59  Reproducer$$$a white - space$o$1.class
.rw-r--r--   820 brice.dutheil 11 Feb 00:59  Reproducer$o$1.class
.rw-r--r--  1.2k brice.dutheil 11 Feb 00:59  Reproducer.class
.rw-r--r--  2.0k brice.dutheil 11 Feb 00:59  ReproducerKt.class
import java.util.concurrent.Executors
import java.util.concurrent.TimeUnit

fun main(args: Array<String>) {
    Executors.newSingleThreadScheduledExecutor()
        .scheduleAtFixedRate(
            Runnable {
                print(Reproducer().`$$a white - space`())
                print(" ; ")
                println(Reproducer().o)

            },
            2,
            2,
            TimeUnit.SECONDS
        )
}


class Reproducer {
    val o = object {
        override fun toString(): String {
            return "foo-bar-qux"
        }
    }

    fun `$$a white - space`(): Any {
        val o = object {
            override fun toString(): String {
                return "foo   bar"
            }
        }
        return o
    }
}

So I have made a simple algorithm to find the Kotlin enclosing class:

@@ -101,6 +101,15 @@ public class AnonymousClassPatchPlugin {
         if (classPool.find(className) == null)
             return null;
 
+        while (classPool.find(mainClass) == null) {
+            // identifiers like this are possible in Kotlin
+            // - val `$$$something` = 1
+            // - val `a white space and a dash -`() { }
+            mainClass = mainClass.substring(0, mainClass.lastIndexOf('$'));
+            System.err.println("Infer mainclass, trying " + mainClass);
+        }
+
         AnonymousClassInfos info = getStateInfo(classLoader, classPool, mainClass);
 
         String compatibleName = info.getCompatibleTransition(javaClass);

With this change the NPE is gone, however now the replacement does not behave as expected.

eg, if I change

             Runnable {
                 print(Reproducer().`$$a white - space`())
-                print(" ; ")
+                print(" # ")
                 println(Reproducer().o)
             },

The program don't print the correct toString

Connected to the target VM, address: '127.0.0.1:58057', transport: 'socket'
HOTSWAP AGENT: 01:23:02.607 INFO (org.hotswap.agent.HotswapAgent) - Loading Hotswap agent {1.4.2-SNAPSHOT} - unlimited runtime class redefinition.
HOTSWAP AGENT: 01:23:02.892 INFO (org.hotswap.agent.config.PluginRegistry) - Discovered plugins: [JdkPlugin, ClassInitPlugin, AnonymousClassPatch, WatchResources, Hotswapper, Hibernate, Hibernate3JPA, Hibernate3, Spring, Jersey1, Jersey2, Jetty, Tomcat, ZK, Logback, Log4j2, MyFaces, Mojarra, Omnifaces, ELResolver, WildFlyELResolver, OsgiEquinox, Owb, Proxy, WebObjects, Weld, JBossModules, ResteasyRegistry, Deltaspike, GlassFish, Weblogic, Vaadin, Wicket, CxfJAXRS, FreeMarker, Undertow, MyBatis, IBatis, JacksonPlugin, Idea]
foo   bar  qux ; foo-bar-qux
foo   bar  qux ; foo-bar-qux
foo   bar  qux ; foo-bar-qux
foo   bar  qux ; foo-bar-qux
foo   bar  qux ; foo-bar-qux
foo   bar  qux ; foo-bar-qux
foo   bar  qux ; foo-bar-qux
HOTSWAP AGENT: 01:23:18.138 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
HOTSWAP AGENT: 01:23:18.146 INFO (org.hotswap.agent.plugin.jvm.AnonymousClassPatchPlugin) - Creating new infos for className com.github.bric3.ha.Reproducer
Infer mainclass, trying com.github.bric3.ha.Reproducer
HOTSWAP AGENT: 01:23:18.155 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
HOTSWAP AGENT: 01:23:18.157 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
HOTSWAP AGENT: 01:23:18.159 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
com.github.bric3.ha.Reproducer$$$a white - space$o$1@798d04f6 # com.github.bric3.ha.Reproducer$o$1@3dc8d3ae
com.github.bric3.ha.Reproducer$$$a white - space$o$1@3359eaa4 # com.github.bric3.ha.Reproducer$o$1@2c305661
com.github.bric3.ha.Reproducer$$$a white - space$o$1@3d8b4da5 # com.github.bric3.ha.Reproducer$o$1@46a5bc19
com.github.bric3.ha.Reproducer$$$a white - space$o$1@3a5572a3 # com.github.bric3.ha.Reproducer$o$1@1478b8d6
Disconnected from the target VM, address: '127.0.0.1:58057', transport: 'socket'

@bric3
Copy link
Author

bric3 commented Feb 11, 2022

I wondered if this last bit was something particular to the toString method, but even something different brings unexpected results. In this example I am using a JLabel.

import java.util.concurrent.Executors
import java.util.concurrent.TimeUnit
import javax.swing.JLabel

fun main(args: Array<String>) {
    Executors.newSingleThreadScheduledExecutor()
        .scheduleAtFixedRate(
            Runnable {
                print(Reproducer().`$$a white - space`().text)
                print(" - ")
                println(Reproducer().o.text)
            },
            2,
            2,
            TimeUnit.SECONDS
        )
}


class Reproducer {
    val o = object : JLabel() {
        override fun getText(): String {
            return "Hello from o"
        }
    }

    fun `$$a white - space`(): JLabel {
        val o = object : JLabel() {
            override fun getText(): String {
                return "Hello from white space"
            }
        }
        return o
    }
}

Again just changing this line

             Runnable {
                 print(Reproducer().`$$a white - space`())
-                print(" ; ")
+                print(" # ")
                 println(Reproducer().o)
             },

Produces this

OTSWAP AGENT: 01:31:14.603 INFO (org.hotswap.agent.HotswapAgent) - Loading Hotswap agent {1.4.2-SNAPSHOT} - unlimited runtime class redefinition.
HOTSWAP AGENT: 01:31:14.931 INFO (org.hotswap.agent.config.PluginRegistry) - Discovered plugins: [JdkPlugin, ClassInitPlugin, AnonymousClassPatch, WatchResources, Hotswapper, Hibernate, Hibernate3JPA, Hibernate3, Spring, Jersey1, Jersey2, Jetty, Tomcat, ZK, Logback, Log4j2, MyFaces, Mojarra, Omnifaces, ELResolver, WildFlyELResolver, OsgiEquinox, Owb, Proxy, WebObjects, Weld, JBossModules, ResteasyRegistry, Deltaspike, GlassFish, Weblogic, Vaadin, Wicket, CxfJAXRS, FreeMarker, Undertow, MyBatis, IBatis, JacksonPlugin, Idea]
Hello from white space # Hello from o
Hello from white space # Hello from o
HOTSWAP AGENT: 01:31:40.544 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
HOTSWAP AGENT: 01:31:40.554 INFO (org.hotswap.agent.plugin.jvm.AnonymousClassPatchPlugin) - Creating new infos for className com.github.bric3.ha.Reproducer
HOTSWAP AGENT: 01:31:40.567 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
HOTSWAP AGENT: 01:31:40.573 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
HOTSWAP AGENT: 01:31:40.576 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
 - 
 - 
 - 

It's like the method overrides of the anonymous class are gone.

@scosenza
Copy link

Using a recent version HotswapPlugin (compiled from head a few weeks ago), I need to disable the AnonymousClassPatch plugin for any projects using Kotlin. Is this the only option at this point, or do others have any fixes or workarounds? Thanks for any pointers!

@skybber
Copy link
Contributor

skybber commented Mar 16, 2024

As far as the issue causing the NullPointerException in the org.hotswap.agent.plugin.jvm.AnonymousClassInfos.lastModified method has been resolved as of today.

@scosenza
Copy link

Amazing news! Thanks so much for all the recent fixes!!!

@skybber
Copy link
Contributor

skybber commented Mar 16, 2024

I recommend to use also the last jbr17/21 versions since there is fixed following problem:

https://youtrack.jetbrains.com/issue/JBR-6647/XXAllowEnhancedClassRedefinition-does-not-work-with-Kotlin

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants