messages_overview.md

Messages

Messages are the json diffs that are sent to the destinations. Each watch_type will be unique because each json watch type has it's own json formatting. Please look the documentation for each watch type for an example of that types message format. The following is an overview of the shared properties that each watch type shares. As well as the actual json diff, there is a little bit of meta data included as well.

Example of a key watch message

{
  "watch_type": "key",
  "id": "key.test",
  "diff": [
    "~",
    [
      "test",
      "Value"
    ],
    "old_value",
    "new_value"
  ]
}

watch_type

The top level of the data structure is a hash with 3 elements. watch_type, id and diff. watch_type should be obvious. It matches both the --type sent to the consul watch command, as well as the --watch-type. The example output above was generated by the following command;

consul watch --type keyprefix --prefix / /usr/bin/consul_watcher --config-file /etc/consul_watch/config.json --storage-name testing --watch-type key

Note that the --type keyprefix and --type key specified to the consul watch command share the same json output format. For these two, they both just use --watch-type key parameter for the consul_watcher ruby command.

id

This is a unique id that can be used to identify what change. For a key watch, this will result in an id of key_(path to key that change).

diff

The comparision between the previous json and the current json is handled by a ruby gem called hashdiff. The hashdiff github page has a good explaination of the diff output. Because arrays can be unordered in the consul watch output, this can lead to false positives in diff changes. To avoid this, data manipulation is done in the consul_watcher logic to remove arrays. The json diff format can be summed up as follows.

The first element in the diff array is the operation. '+' refers to a new element. '-' refers to an element that was removed. '~' refers to an element that changed.

The number of elements for diff arrays of '+' and '-' are both size 3. The size for a '~' is size for.

'+' operation The first element defines it as an addition operation, and will always be '+' The second element is an array that represents the nested path inside the json to the key that was added The third element is the value of the new key

'-' operation The first element defines it as a removal operation, and will always be '-' The second element is an array that represents the nested path inside the json to the key that was removed The third element is the value of the key that was removed

'' operation ( note the 4th array element ) The first element defines it as a modification operation, and will always be '' The second element is an array that represents the nested path inside the json to the key that was change The third element is the previous value of the key that was changed The fourth element is the new value of the key that was changed

Wrapping up

Summing up our example using this information:

• There was a modification to the key located at 'test' in the consul kv store
• This was represented in the consul watch json output as { "test": { "Value": "new value" } }
• The old value was a string of 'old_value'
• and the new value a string of 'new_value'

This will enable consumers of the messages to have all the information they need; the type of operation, the location of what changed, and old/new value(s)