Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid spamming logs with errors about SR if SR not configured #4319

Closed
big-andy-coates opened this issue Jan 15, 2020 · 0 comments · Fixed by #4325
Closed

Avoid spamming logs with errors about SR if SR not configured #4319

big-andy-coates opened this issue Jan 15, 2020 · 0 comments · Fixed by #4325
Assignees

Comments

@big-andy-coates
Copy link
Contributor

It is not required to use KSQL with the Schema Registry. Yet if it is not configured the logs get spammed with lines such as:

[2020-01-15 12:19:08,701] WARN Could not clean up the schema registry for query: _confluent-ksql-default_transient_862477730803062421_1579090748245 (io.confluent.ksql.schema.registry.SchemaRegistryUtil:60)
java.net.ConnectException: Connection refused (Connection refused)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
	at sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
	at sun.net.www.http.HttpClient.New(HttpClient.java:339)
	at sun.net.www.http.HttpClient.New(HttpClient.java:357)
	at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1220)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)
	at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1040)
	at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1038)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1037)
	at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:984)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1564)
	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91)
	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1484)
	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1482)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1481)
	at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
	at io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:253)
	at io.confluent.kafka.schemaregistry.client.rest.RestService.httpRequest(RestService.java:325)
	at io.confluent.kafka.schemaregistry.client.rest.RestService.getAllSubjects(RestService.java:724)
	at io.confluent.kafka.schemaregistry.client.rest.RestService.getAllSubjects(RestService.java:719)
	at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.getAllSubjects(CachedSchemaRegistryClient.java:457)
	at io.confluent.ksql.schema.registry.SchemaRegistryUtil.getSubjectNames(SchemaRegistryUtil.java:58)
	at io.confluent.ksql.schema.registry.SchemaRegistryUtil.getInternalSubjectNames(SchemaRegistryUtil.java:75)
	at io.confluent.ksql.schema.registry.SchemaRegistryUtil.cleanUpInternalTopicAvroSchemas(SchemaRegistryUtil.java:45)
	at io.confluent.ksql.engine.KsqlEngine.unregisterQuery(KsqlEngine.java:253)
	at io.confluent.ksql.engine.EngineContext.unregisterQuery(EngineContext.java:228)
	at io.confluent.ksql.util.QueryMetadata.close(QueryMetadata.java:155)
	at io.confluent.ksql.util.TransientQueryMetadata.close(TransientQueryMetadata.java:108)
	at io.confluent.ksql.rest.server.resources.streaming.PushQueryPublisher$PushQuerySubscription.close(PushQueryPublisher.java:105)
	at io.confluent.ksql.rest.server.resources.streaming.PollingSubscription.lambda$request$1(PollingSubscription.java:86)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Whenever a query is closed.

KsqlConfig.DEFAULT_SCHEMA_REGISTRY_URL probably has something to do with it. We should't have a default! (Though when we remove it we should ensure KSQL is still hooked up to CP's SR by default).

Instead, is no SR url is provided we should assume its disabled. So we shouldn't try to clean up this stuff.

Probably the cleanest, least invasive, way of implementing this would be to provide an alternative impl of SchemaRegisteryClient. One that returns an empty list from getAllSubjects (so we don't try to clean anything up), and throws on any 'put' style operation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants