Skip to content

Distributed Configuration

Jump to bottom Edit New page
lvca edited this page Dec 22, 2012 · 3 revisions

Distributed Configuration

orientdb-cserver-config.xml

Database configuration

To enable the clustering between nodes, add and enable the OHazelcastPlugin. Below an example of configuration.

Example in orientdb-cserver-config.xml file:

    <orient-server>
      <handlers>
        <!-- CLUSTER PLUGIN, TO TURN ON SET THE 'ENABLED' PARAMETER TO 'true' -->
        <handler class="com.orientechnologies.orient.server.cluster.hazelcast.OHazelcastPlugin">
          <parameters>
            <parameter name="enabled" value="true" />
            <parameter name="alias" value="europe2012" />
            <parameter name="configuration" value="${ORIENTDB_HOME}/config/hazelcast.xml" />
            <parameter name="db.*"
                       value="
    {
      'servers' : {
        '*' : {
          'synchronization' : true,
          'clusters' : {
            '*' : { 
              'synchronization' : true,
              'operations' : {
                'create' : { 'direction' : 'in-out', 'mode' : 'synchronous' },
                'read'   : { 'direction' : 'in',     'mode' : 'synchronous' },
                'update' : { 'direction' : 'in-out', 'mode' : 'synchronous' },
                'delete' : { 'direction' : 'in-out', 'mode' : 'synchronous' }
              },
              'offlineMaxBuffer' : 1000000
             },
            'internal' : { 'synchronization' : false },
            'index' : { 'synchronization' : false }
          },
          'classes': {}
        }
      }
    }" />
          </parameters>
        </handler>
      </handlers>
    </orient-server>

Where:

  • <enabled> to enable or disable the plugin. Can be true or false
  • <alias> the node's alias. If the alias is not specified, then <ip-address>:<port> will be taken, i.e. 192.168.0.100:2424. Use this to assign a name that makes sense like "europe1" to the first european node.
  • <configuration> specifies the Hazelcast's file name

<db.>** is the configuration for the default database. To define ad-hoc configuration specify db.<db-name> where db-name is the database's name to configure in clustering.

Node configuration

The configuration can be set per single node. Use '**' to specify default configuration. Each server node can configure the behavior for clusters and classes. Syntax:

    <node-id> : {
      'defaultCluster' : <default-cluster-id>|<default-cluster-name>,
      'clusters' : { },
      'classes' : { }
    }

Where:

  • <node-id> is the node's id. If the alias is not specified, then <ip-address>:<port> will be taken, i.e. 192.168.0.100:2424
  • <default-cluster-id> is the cluster's id
  • <default-cluster-name> is the cluster's name as string

Record cluster configuration

The record cluster configuration specifies how to replicate a record cluster. Syntax:

    <node-id> : {
      'clusters' : {
        <cluster-name> : { 
          'synchronization' : true|false,
          'operations' : {
            <op-name> : { 'direction' : <direction>, 'mode' : <mode> },
          },
          'offlineMaxBuffer' : <offline-max-buffer>
        }
      }
    }

Where:

  • <cluster-name> is the cluster's name as string
  • <op-name can be:
  • create for creation
  • read for read by other nodes
  • update for update
  • delete for deletion
  • <direction> can be:
  • 'in': if the operation can be executed on incoming requests from other nodes
  • 'out': if the operation can be executed on outgoing requests against other nodes
  • 'in-out': for both above
  • <mode> is the execution mode and can be:
  • synchronous when the requester (the client) waits until the execution is completed. This assure the data is physically persisted in multiple places when the client receives the ACK. In case of error a conflict will be created.
  • asynchronous to don't wait for the replication because will be executed in background. In case of error a conflict will be created.
  • fire_and_forget to don't wait for the replication because will be executed in background forgetting the result. In case of error a conflict will NOT be created.
  • <offline-max-buffer> is the maximum size of log file before to mark the cluster as unaligned and therefore requiring a manual synchronization

The default cluster configuration is:

    '*' : {
      'operations' : {
        'create' : { 'direction' : 'in-out', 'mode' : 'synchronous' },
        'read'   : { 'direction' : 'in' },
        'update' : { 'direction' : 'in-out', 'mode' : 'synchronous' },
        'delete' : { 'direction' : 'in-out', 'mode' : 'synchronous' }
      },
      'synchronization' : true, 
      'offlineMaxBuffer' : 1000000
    }

Record class configuration

The record class configuration specifies how to replicate a class. Syntax:

    <node-id> : {
      'classes' : {
        <class-name> : {
          'defaultCluster' : <default-cluster> }
        }
      }
    }

Where:

  • <class-name> is the class's name as string
  • <default-cluster> is the default cluster for the class.

Real world example

    {
      'servers' : {
    
        '*' : {
          'clusters' : {
            '*' : { 
              'synchronization' : true,
              'operations' : {
                'create' : { 'direction' : 'in-out', 'mode' : 'synchronous' },
                'read'   : { 'direction' : 'in',     'mode' : 'synchronous' },
                'update' : { 'direction' : 'in-out', 'mode' : 'synchronous' },
                'delete' : { 'direction' : 'in-out', 'mode' : 'synchronous' }
              }
            },
            'mode' : 'synch', 'offlineMaxBuffer' : 1000000 },
            'internal' : { 'synchronization' : false },
            'index' : { 'synchronization' : false }
          },
          'classes': {}
        },
    
        'china' : {
          'defaultCluster' : 2,
          'clusters' : {
            'customer-china' : {
              'operations' : {
                'create' : { 'direction' : 'out', 'mode' : 'asynchronous' }
              }
            },
            'invoice-china' : {
              'operations' : {
                'create' : { 'direction' : 'out', 'mode' : 'asynchronous' }
              }
            },
          },
          'classes' : {
            'customer' : { 'defaultCluster' : 'customer-china' },
            'invoice' : { 'defaultCluster' : 'invoice-china' }
          },
        },
    
        'europe' : {
          'defaultCluster' : 'default-2',
          'clusters' : {
            'customer-europe' : {
              'operations' : {
                'create' : { 'direction' : 'out', 'mode' : 'asynchronous' }
              }
            },
            'invoice-europe' : {
              'operations' : {
                'create' : { 'direction' : 'out', 'mode' : 'asynchronous' }
              }
            },
          },
          'classes' : {
            'customer' : { 'defaultCluster' : 'customer-europe' },
            'invoice' : { 'defaultCluster' : 'invoice-europe' }
          }
        }
      }
    }

Distributed 2nd Level cache

To enable the 2nd level cache using the distributed intra-JVM cache, set the cache.level2.impl parameter to com.orientechnologies.orient.server.hazelcast.OHazelcastCache.

Note that this will slow down massive insertion but will improve query and lookup operations.

Example in orientdb-cserver-config.xml file:

    ...
    <properties>
      <!-- Uses the Hazelcast's distributed cache as 2nd level cache -->
      <entry name="cache.level2.impl" value="com.orientechnologies.orient.server.hazelcast.OHazelcastCache" />
    </properties>

Hazelcast configuration

A OrientDB cluster is composed by two or multiple servers that are the nodes of the cluster. All the server nodes that want to be part of the same cluster must to define the same group. By default "orientdb" is the group name. Look at the hazelcast.xml file below:

    <group>
      <name>orientdb</name>
      <password>orientdb</password>
    </group>

NOTE: Change the name and password of the group to avoid external nodes can join it!

Misc

Load balancing

The simpler and most powerful way to achieve load balancing seems to use some hidden properties (hidden to someone) of DNS. The trick is to create a TXT record telling the list of servers.

The format is:

v=opf<version> (s=<hostname[:<port>]> )*

Example of TXT record for domain test.nuvolabase.com:

v=opf1 s=s1.nuvolabase.com s=s2.nuvolabase.com:3434

In this way if you open a database against the URL remote:test.nuvolabase.com/demo the OrientDB client library will try to connect to the address s1.nuvolabase.com port 2424. If the connection fails, then the next address s2.nuvolabase.com port 3434 is tried.

Add a custom footer
Add a custom sidebar
Clone this wiki locally