Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Random Startup error in first clustered node #2008

Closed
robgratz opened this issue Jan 29, 2014 · 1 comment
Closed

Random Startup error in first clustered node #2008

robgratz opened this issue Jan 29, 2014 · 1 comment
Assignees
@lvca
Milestone
1.7rc1

Comments

@robgratz
Copy link
Contributor

Steps:

  • start node non-clustered
  • shutdown
  • start node clustered

I have reproduced this in the test app

Bad level value for property: .level
Bad level value for property: java.util.logging.ConsoleHandler.level
Bad level value for property: java.util.logging.FileHandler.level
Bad level value for property: com.orientechnologies.orient.server.distributed.level
Can't set level for java.util.logging.ConsoleHandler
Can't set level for java.util.logging.FileHandler

2014-01-29 08:05:03:138 INFO URL=plocal:D:/Dev-SVN/v9.0/development/tools/orientdb-cluster-test/servers/0/databases/platform [OrientDBCluster]
2014-01-29 08:05:03:138 INFO Starting EXECUTE test against database plocal:D:/Dev-SVN/v9.0/development/tools/orientdb-cluster-test/servers/0/databases/platform count = 200000 [OrientDBCluster]
2014-01-29 08:05:03:311 INFO Loading configuration from input stream [OServerConfigurationLoaderXml]
2014-01-29 08:05:03:505 INFO OrientDB Server v1.7-SNAPSHOT is starting up... [OServer]
2014-01-29 08:05:03:517 INFO Databases directory: D:\Dev-SVN\v9.0\development\tools\orientdb-cluster-test\servers\0\databases [OServer]
2014-01-29 08:05:03:551 INFO Listening binary connections on 0.0.0.0:2424 (protocol v.19) [OServerNetworkListener]
2014-01-29 08:05:03:551 INFO Listening http connections on 0.0.0.0:2480 (protocol v.10) [OServerNetworkListener]
2014-01-29 08:05:03:585 INFO Starting distributed server 'europe0'... [OHazelcastPlugin]
2014-01-29 08:05:03:624 INFO Configuring Hazelcast from 'D:\Dev-SVN\v9.0\development\tools\orientdb-cluster-test\src\test\resources\hazelcast-0.xml'. [FileSystemXmlConfig]
2014-01-29 08:05:03:976 INFO Interfaces is disabled, trying to pick one address from TCP-IP config addresses: [127.0.0.1] [DefaultAddressPicker]
2014-01-29 08:05:03:987 INFO Picked Address[127.0.0.1]:2434, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=2434], bind any local is true [DefaultAddressPicker]
2014-01-29 08:05:04:164 INFO [127.0.0.1]:2434 [orientdb] Hazelcast Community Edition 3.1.5 (20140115) starting at Address[127.0.0.1]:2434 [system]
2014-01-29 08:05:04:164 INFO [127.0.0.1]:2434 [orientdb] Copyright (C) 2008-2013 Hazelcast.com [system]
2014-01-29 08:05:04:165 INFO [127.0.0.1]:2434 [orientdb] Creating TcpIpJoiner [Node]
2014-01-29 08:05:04:167 INFO [127.0.0.1]:2434 [orientdb] Address[127.0.0.1]:2434 is STARTING [LifecycleService]
2014-01-29 08:05:04:265 INFO [127.0.0.1]:2434 [orientdb] Connecting to possible member: Address[127.0.0.1]:2436 [TcpIpJoiner]
2014-01-29 08:05:04:267 INFO [127.0.0.1]:2434 [orientdb] Connecting to possible member: Address[127.0.0.1]:2435 [TcpIpJoiner]
2014-01-29 08:05:04:268 INFO [127.0.0.1]:2434 [orientdb] Connecting to possible member: Address[127.0.0.1]:2437 [TcpIpJoiner]
2014-01-29 08:05:04:268 INFO [127.0.0.1]:2434 [orientdb] Connecting to /127.0.0.1:2436, timeout: 0, bind-any: true [SocketConnector]
2014-01-29 08:05:04:268 INFO [127.0.0.1]:2434 [orientdb] Connecting to /127.0.0.1:2435, timeout: 0, bind-any: true [SocketConnector]
2014-01-29 08:05:04:268 INFO [127.0.0.1]:2434 [orientdb] Connecting to /127.0.0.1:2437, timeout: 0, bind-any: true [SocketConnector]
2014-01-29 08:05:05:270 INFO [127.0.0.1]:2434 [orientdb] Could not connect to: /127.0.0.1:2436. Reason: SocketException[Connection refused: connect to address /127.0.0.1:2436] [SocketConnector]
2014-01-29 08:05:05:270 INFO [127.0.0.1]:2434 [orientdb] Could not connect to: /127.0.0.1:2435. Reason: SocketException[Connection refused: connect to address /127.0.0.1:2435] [SocketConnector]
2014-01-29 08:05:05:271 INFO [127.0.0.1]:2434 [orientdb] Could not connect to: /127.0.0.1:2437. Reason: SocketException[Connection refused: connect to address /127.0.0.1:2437] [SocketConnector]
2014-01-29 08:05:06:268 INFO [127.0.0.1]:2434 [orientdb] 


Members [1] {
    Member [127.0.0.1]:2434 this
}
 [TcpIpJoiner]
2014-01-29 08:05:06:306 INFO [127.0.0.1]:2434 [orientdb] Hazelcast JMX agent enabled. [ManagementService]
2014-01-29 08:05:06:386 INFO [127.0.0.1]:2434 [orientdb] Address[127.0.0.1]:2434 is STARTED [LifecycleService]
2014-01-29 08:05:06:452 INFO [127.0.0.1]:2434 [orientdb] Initializing cluster partition table first arrangement... [PartitionService]
2014-01-29 08:05:06:573 WARN [europe0] opening database 'platform'... [OHazelcastPlugin]
2014-01-29 08:05:06:574 INFO [europe0] loaded database configuration from disk: src\test\resources\default-distributed-db-config.json [OHazelcastPlugin]
2014-01-29 08:05:06:587 INFO updated distributed configuration for database: platform:
----------
{
  "replication":true,
  "autoDeploy":true,
  "hotAlignment":false,
  "resyncEvery":15,"clusters":{
    "internal":{
  "replication":false
},
    "index":{
  "replication":false
},
    "ODistributedConflict":{
  "replication":false
},
    "*":{
  "replication":true,
  "readQuorum":1,
  "writeQuorum":2,
  "failureAvailableNodesLessQuorum":false,
  "readYourWrites":true,"partitioning":{
    "strategy":"round-robin",
    "default":0,
    "partitions":[["<NEW_NODE>"]]
    }
}
    }
}
---------- [OHazelcastPlugin]
2014-01-29 08:05:06:599 INFO [europe0] adding node 'europe0' in partition: platform.*.0 [OHazelcastDistributedDatabase]
2014-01-29 08:05:06:601 INFO updated distributed configuration for database: platform:
----------
{
  "replication":true,
  "autoDeploy":true,
  "hotAlignment":false,
  "resyncEvery":15,"clusters":{
    "internal":{
  "replication":false
},
    "index":{
  "replication":false
},
    "ODistributedConflict":{
  "replication":false
},
    "*":{
  "replication":true,
  "readQuorum":1,
  "writeQuorum":2,
  "failureAvailableNodesLessQuorum":false,
  "readYourWrites":true,"partitioning":{
    "strategy":"round-robin",
    "default":0,
    "partitions":[["<NEW_NODE>","europe0"]]
    }
}
    }
}
---------- [OHazelcastPlugin]
2014-01-29 08:05:06:610 INFO updated distributed configuration for database: platform:
----------
{
  "replication":true,
  "autoDeploy":true,
  "hotAlignment":false,
  "resyncEvery":15,"clusters":{
    "internal":{
  "replication":false
},
    "index":{
  "replication":false
},
    "ODistributedConflict":{
  "replication":false
},
    "*":{
  "replication":true,
  "readQuorum":1,
  "writeQuorum":2,
  "failureAvailableNodesLessQuorum":false,
  "readYourWrites":true,"partitioning":{
    "strategy":"round-robin",
    "default":0,
    "partitions":[["<NEW_NODE>"]]
    }
}
    },
  "version":1
}
---------- [OHazelcastPlugin]
2014-01-29 08:05:07:561 WARN [europe0] updated node status to 'ONLINE' [OHazelcastPlugin]
2014-01-29 08:05:07:563 INFO OrientDB Server v1.7-SNAPSHOT is active. [OServer]
2014-01-29 08:05:07:569 SEVE [europe0] No nodes configured for partition 'platform.ouser' request: id=1 task=record_read [OHazelcastDistributedDatabase]Cannot route READ_RECORD operation against #5:0 to the distributed node
No nodes configured for partition 'platform.ouser' request: id=1 task=record_read
-> com.orientechnologies.orient.server.hazelcast.OHazelcastDistributedDatabase.send(OHazelcastDistributedDatabase.java:133)
-> com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin.sendRequest(OHazelcastPlugin.java:322)
-> com.orientechnologies.orient.server.distributed.ODistributedStorage.readRecord(ODistributedStorage.java:186)
-> com.orientechnologies.orient.core.db.raw.ODatabaseRaw.read(ODatabaseRaw.java:244)
-> com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.executeReadRecord(ODatabaseRecordAbstract.java:715)
-> com.orientechnologies.orient.core.tx.OTransactionNoTx.loadRecord(OTransactionNoTx.java:63)
-> com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.load(ODatabaseRecordTx.java:202)
-> com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.load(ODatabaseRecordAbstract.java:1)
-> com.orientechnologies.orient.core.id.ORecordId.getRecord(ORecordId.java:296)
-> com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.fillSearchIndexResultSet(OCommandExecutorSQLSelect.java:828)
-> com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.searchForIndexes(OCommandExecutorSQLSelect.java:712)
-> com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.searchInClasses(OCommandExecutorSQLSelect.java:610)
-> com.orientechnologies.orient.core.sql.OCommandExecutorSQLResultsetAbstract.assignTarget(OCommandExecutorSQLResultsetAbstract.java:139)
-> com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.assignTarget(OCommandExecutorSQLSelect.java:354)
-> com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.executeSearch(OCommandExecutorSQLSelect.java:328)
-> com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.execute(OCommandExecutorSQLSelect.java:313)
-> com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.execute(OCommandExecutorSQLDelegate.java:57)
-> com.orientechnologies.orient.core.storage.OStorageEmbedded.executeCommand(OStorageEmbedded.java:96)
-> com.orientechnologies.orient.server.distributed.ODistributedStorage.command(ODistributedStorage.java:99)
-> com.orientechnologies.orient.core.sql.query.OSQLQuery.run(OSQLQuery.java:69)
-> com.orientechnologies.orient.core.sql.query.OSQLSynchQuery.run(OSQLSynchQuery.java:82)
-> com.orientechnologies.orient.core.query.OQueryAbstract.execute(OQueryAbstract.java:29)
-> com.orientechnologies.orient.core.metadata.security.OSecurityShared.getUser(OSecurityShared.java:181)
-> com.orientechnologies.orient.core.metadata.security.OSecurityShared.authenticate(OSecurityShared.java:148)
-> com.orientechnologies.orient.core.metadata.security.OSecurityProxy.authenticate(OSecurityProxy.java:83)
-> com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.open(ODatabaseRecordAbstract.java:139)
-> com.orientechnologies.orient.core.db.ODatabaseWrapperAbstract.open(ODatabaseWrapperAbstract.java:53)
-> com.orientechnologies.orient.core.db.document.ODatabaseDocumentTxPooled.<init>(ODatabaseDocumentTxPooled.java:44)
-> com.orientechnologies.orient.core.db.document.ODatabaseDocumentPool.createResource(ODatabaseDocumentPool.java:44)
-> com.orientechnologies.orient.core.db.document.ODatabaseDocumentPool.createResource(ODatabaseDocumentPool.java:1)
-> com.orientechnologies.orient.core.db.ODatabasePoolBase$1.createNewResource(ODatabasePoolBase.java:74)
-> com.orientechnologies.orient.core.db.ODatabasePoolBase$1.createNewResource(ODatabasePoolBase.java:1)
-> com.orientechnologies.common.concur.resource.OResourcePool.getResource(OResourcePool.java:68)
-> com.orientechnologies.orient.core.db.ODatabasePoolAbstract.acquire(ODatabasePoolAbstract.java:93)
-> com.orientechnologies.orient.core.db.ODatabasePoolAbstract.acquire(ODatabasePoolAbstract.java:78)
-> com.orientechnologies.orient.core.db.ODatabasePoolBase.acquire(ODatabasePoolBase.java:121)
-> test.OrientDBCluster.execute(OrientDBCluster.java:72)
-> test.OrientDBCluster.main(OrientDBCluster.java:172)Exception in thread "main" com.orientechnologies.orient.core.exception.ODatabaseException: Error on retrieving record #5:0 (cluster: ouser)
    at com.orientechnologies.orient.core.db.raw.ODatabaseRaw.read(ODatabaseRaw.java:250)
    at com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.executeReadRecord(ODatabaseRecordAbstract.java:715)
    at com.orientechnologies.orient.core.tx.OTransactionNoTx.loadRecord(OTransactionNoTx.java:63)
    at com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.load(ODatabaseRecordTx.java:202)
    at com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.load(ODatabaseRecordAbstract.java:1)
    at com.orientechnologies.orient.core.id.ORecordId.getRecord(ORecordId.java:296)
    at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.fillSearchIndexResultSet(OCommandExecutorSQLSelect.java:828)
    at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.searchForIndexes(OCommandExecutorSQLSelect.java:712)
    at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.searchInClasses(OCommandExecutorSQLSelect.java:610)
    at com.orientechnologies.orient.core.sql.OCommandExecutorSQLResultsetAbstract.assignTarget(OCommandExecutorSQLResultsetAbstract.java:139)
    at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.assignTarget(OCommandExecutorSQLSelect.java:354)
    at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.executeSearch(OCommandExecutorSQLSelect.java:328)
    at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.execute(OCommandExecutorSQLSelect.java:313)
    at com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.execute(OCommandExecutorSQLDelegate.java:57)
    at com.orientechnologies.orient.core.storage.OStorageEmbedded.executeCommand(OStorageEmbedded.java:96)
    at com.orientechnologies.orient.server.distributed.ODistributedStorage.command(ODistributedStorage.java:99)
    at com.orientechnologies.orient.core.sql.query.OSQLQuery.run(OSQLQuery.java:69)
    at com.orientechnologies.orient.core.sql.query.OSQLSynchQuery.run(OSQLSynchQuery.java:82)
    at com.orientechnologies.orient.core.query.OQueryAbstract.execute(OQueryAbstract.java:29)
    at com.orientechnologies.orient.core.metadata.security.OSecurityShared.getUser(OSecurityShared.java:181)
    at com.orientechnologies.orient.core.metadata.security.OSecurityShared.authenticate(OSecurityShared.java:148)
    at com.orientechnologies.orient.core.metadata.security.OSecurityProxy.authenticate(OSecurityProxy.java:83)
    at com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.open(ODatabaseRecordAbstract.java:139)
    at com.orientechnologies.orient.core.db.ODatabaseWrapperAbstract.open(ODatabaseWrapperAbstract.java:53)
    at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTxPooled.<init>(ODatabaseDocumentTxPooled.java:44)
    at com.orientechnologies.orient.core.db.document.ODatabaseDocumentPool.createResource(ODatabaseDocumentPool.java:44)
    at com.orientechnologies.orient.core.db.document.ODatabaseDocumentPool.createResource(ODatabaseDocumentPool.java:1)
    at com.orientechnologies.orient.core.db.ODatabasePoolBase$1.createNewResource(ODatabasePoolBase.java:74)
    at com.orientechnologies.orient.core.db.ODatabasePoolBase$1.createNewResource(ODatabasePoolBase.java:1)
    at com.orientechnologies.common.concur.resource.OResourcePool.getResource(OResourcePool.java:68)
    at com.orientechnologies.orient.core.db.ODatabasePoolAbstract.acquire(ODatabasePoolAbstract.java:93)
    at com.orientechnologies.orient.core.db.ODatabasePoolAbstract.acquire(ODatabasePoolAbstract.java:78)
    at com.orientechnologies.orient.core.db.ODatabasePoolBase.acquire(ODatabasePoolBase.java:121)
    at test.OrientDBCluster.execute(OrientDBCluster.java:72)
    at test.OrientDBCluster.main(OrientDBCluster.java:172)
Caused by: com.orientechnologies.orient.core.exception.OStorageException: Cannot route READ_RECORD operation against #5:0 to the distributed node
    at com.orientechnologies.orient.server.distributed.ODistributedStorage.handleDistributedException(ODistributedStorage.java:597)
    at com.orientechnologies.orient.server.distributed.ODistributedStorage.readRecord(ODistributedStorage.java:199)
    at com.orientechnologies.orient.core.db.raw.ODatabaseRaw.read(ODatabaseRaw.java:244)
    ... 34 more
Caused by: com.orientechnologies.orient.server.distributed.ODistributedException: No nodes configured for partition 'platform.ouser' request: id=1 task=record_read
    at com.orientechnologies.orient.server.hazelcast.OHazelcastDistributedDatabase.send(OHazelcastDistributedDatabase.java:133)
    at com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin.sendRequest(OHazelcastPlugin.java:322)
    at com.orientechnologies.orient.server.distributed.ODistributedStorage.readRecord(ODistributedStorage.java:186)
    ... 35 more

2014-01-29 08:05:21:600 SEVE [europe0] No nodes configured for partition 'platform.null' request: id=2 task=resynch [OHazelcastDistributedDatabase]
2014-01-29 08:05:36:600 SEVE [europe0] No nodes configured for partition 'platform.null' request: id=3 task=resynch [OHazelcastDistributedDatabase]
2014-01-29 08:05:51:601 SEVE [europe0] No nodes configured for partition 'platform.null' request: id=4 task=resynch [OHazelcastDistributedDatabase]
2014-01-29 08:06:06:601 SEVE [europe0] No nodes configured for partition 'platform.null' request: id=5 task=resynch [OHazelcastDistributedDatabase]
2014-01-29 08:06:21:602 SEVE [europe0] No nodes configured for partition 'platform.null' request: id=6 task=resynch [OHazelcastDistributedDatabase]
2014-01-29 08:06:36:602 SEVE [europe0] No nodes configured for partition 'platform.null' request: id=7 task=resynch [OHazelcastDistributedDatabase]
2014-01-29 08:06:51:603 SEVE [europe0] No nodes configured for partition 'platform.null' request: id=8 task=resynch [OHazelcastDistributedDatabase]
2014-01-29 08:07:06:603 SEVE [europe0] No nodes configured for partition 'platform.null' request: id=9 task=resynch [OHazelcastDistributedDatabase]
2014-01-29 08:07:21:604 SEVE [europe0] No nodes configured for partition 'platform.null' request: id=10 task=resynch [OHazelcastDistributedDatabase]
2014-01-29 08:07:36:604 SEVE [europe0] No nodes configured for partition 'platform.null' request: id=11 task=resynch [OHazelcastDistributedDatabase]
@lvca
Copy link
Member

lvca commented Jan 29, 2014

Perfect

@ghost ghost assigned lvca Jan 29, 2014
lvca added a commit that referenced this issue Jan 30, 2014
@lvca
Distributed: fixed issue #2008 on deploying database
2a590b7
@lvca lvca closed this as completed Jan 30, 2014
@lvca lvca modified the milestones: 1.7rc1, 1.7 Feb 5, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lvca lvca

Labels
None yet
Milestone
1.7rc1
Development

No branches or pull requests
2 participants
@lvca @robgratz