diff --git a/doc/Analytics.xml b/doc/Analytics.xml index 5ee0ee89b..618a443b5 100755 --- a/doc/Analytics.xml +++ b/doc/Analytics.xml @@ -254,6 +254,8 @@ <> ISO 3166-1:2013 Codes for the representation of names of countries and their subdivisions -- Part 1: Country codes <> + JSON-LD 1.1 A JSON-based Serialization for Linked Data + <> Terms and Definitions @@ -500,25 +502,12 @@ Radiometry types, modules and rules are defined in the RADIOMETRY Schema file http://www.onvif.org/ver20/analytics/radiometry.xsd. - Service - This section covers the following main areas of this architecture: - - - Analytics Module interface - - - Scene description - - - Rules interface - - - The analytics service allows fine-grained configuration of individual rules and individual analytics modules (see and ). introduces the XML-based scene description, which can be streamed as metadata to clients via RTP as defined in the ONVIF Streaming Specification. -
- Scene Description Interface + Scene Description
Overview - This specification defines the XML schema that shall be used to encode Scene Descriptions by a device. The scope of the Scene Description covers basic Scene Elements which can be displayed in a video overlay to the end-user as well as a framework for vendor-specific extensions. Annex A shows additional Scene Elements that may be used for processing vendor-specific rules. + This chapter defines the XML-based scene description, which can be streamed as metadata to clients via RTP as defined in the ONVIF Streaming Specification. + The scope of the Scene Description covers basic Scene Elements which can be displayed in a video overlay to the end-user as well as a framework for vendor-specific extensions. + Section defines an alternate JSON based representation.
Frame Related Content @@ -2204,7 +2193,492 @@
+
+ ONVIF Metadata in JSON over MQTT + Increasing demand in analytics use cases presented a need for + standardizing format and transport of ONVIF metadata to be consumed by + non-traditional clients (e.g. cloud vendors) to bridge gap between + traditional surveillance systems and IoT ecosystems. For easy + consumption in such systems ONVIF metadata can be published in JSON + format over the MQTT protocol. + + MQTT is a pub/sub messaging protocol centered around topics. + Consumers can subscribe to topics published by producers to get + continuous updates on the data content. ONVIF metadata, like video + analytics scene descriptions, can be published to an MQTT broker and + subscribed to by a consumer. + +
+ Metadata publish sequence + This is sequence of operations to set up the metadata flow: + + An ONVIF client calls AddEventBroker in the device's Event + service to set up: + + Address and credentials to the MQTT broker to use + + + + A unique TopicPrefix for the device + + + + A MetadataFilter listing topics to which the device + shall publish + + + + + + The producer (ONVIF device) connects to the MQTT broker and starts publishing + data on the selected topics. + + + + The consumer connects to the MQTT broker, specifying which topics + that it wants to subscribe to. A consumer can be any MQTT capable + client, for example, an event engine like Node-RED, IoT cloud + platforms, VSaaS platforms, etc. Multiple comsumers using + different topics are possible. + + + + The consumer receives updates on subscribed topics and can + implement actions like statistics aggregation and complex + analytics rules with data from multiple sources. + + + + The following sections describe how topics and payload shall be + structured by an ONVIF metadata producer when published to an MQTT + broker. +
+
+ MQTT Topic Structure + + This section describes how an MQTT topic is structured based on + metadata type and source expressed as ABNF rules according to RFC + 5234. + + Topic = TopicPrefix "/" PayloadPrefix "/" MetadataType "/" MetadataProducer + Note that special characters like, ‘/’, ‘#’ and ‘+’ shall be omitted from the topic. + with + + TopicPrefix - uniquely identifies the producer and is + configurable through the AddEventBroker command in the Event + service. It shall not be empty. + + + + PayloadPrefix - signals in what format the data is + published. See below for + possible values. + + + + MetadataType - signals what type of metadata it is. See + below for possible + values. + + + + MetadataProducer - specifies additional source information for the metadata. The + format depends on the metadata type. See below + for definition. + + + + + PayloadPrefix types + + + + + Prefix + + Description + + + + + + "onvif-mj" + + ONVIF metadata with JSON payload + + + +
+ + + MetadataType categories with corresponding MetadataProducer + + + + + MetadataType + + MetadataProducer (template in ABNF) + + Description + + + + + + "VideoAnalytics" + + ProfileToken "/" + AnalyticsModuleName + + ONVIF metadata from a VideoAnalytics module + active via VideoAnalyticsConfiguration included in Profile + configuration + + + + "AudioAnalytics" + + ProfileToken "/" + AnalyticsModuleName + + ONVIF metadata from an AudioAnalytics module + active via AudioAnalyticsConfiguration included in Profile + configuration + + + + "PTZ" + + ProfileToken "/" PTZConfigurationToken + + ONVIF metadata from a PTZ node included in + Profile configuration + + + +
+ + with + + ProfileToken = The media profile token + + + + AnalyticsModuleName = The name of the analytics + module + + + + PTZConfigurationToken = The PTZ configuration token + + +
+ +
+ JSON Payload Structure + + Instead of creating a full fledged schema for XML to JSON + conversion, provided below are a set of generic rules that devices + shall need to implement to express ONVIF metadata in JSON + format. + + + ONVIF Metadata XML to JSON conversion + + + + + ONVIF XML Element + + ONVIF JSON + Representation + + Description + + + + + + <xmltag/> + + “xmltag”: null + + Null tag + + + + <xmltag>text</xmltag> + + “xmltag”: "text" + + Simple tag with value + + + + < xmltag name="value" /> + + “xmltag”:{"@name": "value"} + + Tag with attribute + + + + <xmltag name="value">text</ + xmltag> + + “xmltag”: { "@name": "value", "#text": "text" + } + + Tag with attribute and value + + + + <xmltag> <tag1>text</tag1> + <tag2>text</tag2> </xmltag + + “xmltag”: { "tag1": "text", "tag2": "text" + } + + Tag with multiple child tags + + + + <xmltag> <tag1>text</tag1> + <tag1>text</tag1> </ xmltag> + + “xmltag”: { "tag1": ["text", "text"] + } + + Tag with multiple child tags of same type + (similar to minOccurs > 1) + + + +
+ + All extension elements and attributes shall be included within + the same parent JSON object. + + Namespace prefixes for ONVIF defined namespaces shall be + dropped, i.e. elements and attributes that belong to the following + namespaces: + + + + http://www.onvif.org/ver10/schema + + + + http://www.onvif.org/ver20/analytics/humanface + + + + http://www.onvif.org/ver20/analytics/humanbody + + + + http://www.onvif.org/ver20/analytics/radiometry + + + + XML elements and attributes that belong to a different namespace + shall have their names prepended with their corresponding namespace + prefix joined by a ':' as shown in the example while defining + the namespace in the "context" object as per JSON-LD specification.. See specifically the acme:ColorName + exampe. +
+ +
+ Example + + Sample metadata frame in XML format: + + <?xml version="1.0" encoding="UTF-8"?> +<tt:MetadataStream xmlns:tt="http://www.onvif.org/ver10/schema" +xmlns:fc="http://www.onvif.org/ver20/analytics/humanface" +xmlns:bd="http://www.onvif.org/ver20/analytics/humanbody" +xmlns:acme="http://www.acme.com/schema"> + <tt:VideoAnalytics> + <tt:Frame UtcTime="2021-10-05T15:13:27.321" Source="MyClassifier"> + <tt:Transformation> + <tt:Translate x="-1.0" y="-1.0"/> + <tt:Scale x="0.003125" y="0.00416667"/> + </tt:Transformation> + <tt:Object ObjectId="15"> + <tt:Appearance> + <tt:Shape> + <tt:BoundingBox left="20.0" top="80.0" right="100.0" bottom="30.0"/> + <tt:CenterOfGravity x="60.0" y="50.0"/> + <tt:Polygon> + <tt:Point x="20.0" y="30.0"/> + <tt:Point x="100.0" y="30.0"/> + <tt:Point x="100.0" y="80.0"/> + <tt:Point x="20.0" y="80.0"/> + </tt:Polygon> + </tt:Shape> + <tt:Color> + <tt:ColorCluster> + <tt:Color X="213" Y="135" Z="126" /> + <tt:Weight>0.9</tt:Weight> + <acme:ColorName>White</acme:ColorName> + </tt:ColorCluster> + <tt:ColorCluster> + <tt:Color X="102" Y="176" Z="119" /> + <tt:Weight>0.5</tt:Weight> + <acme:ColorName>Blue</acme:ColorName> + </tt:ColorCluster> + </tt:Color> + <tt:Class> + <tt:Type Likelihood="0.8">Human</tt:Type> + </tt:Class> + <tt:HumanFace> + <fc:Gender>Male</fc:Gender> + <fc:Age> + <tt:Min>20</tt:Min> + <tt:Max>30</tt:Max> + </fc:Age> + <fc:Accessory> + <fc:Opticals> + <fc:Wear>True</fc:Wear> + </fc:Opticals> + <fc:Hat> + <fc:Wear>True</fc:Wear> + </fc:Hat> + </fc:Accessory> + </tt:HumanFace> + <tt:HumanBody> + <bd:Clothing> + <bd:Tops> + <bd:Category>LongSleeve</bd:Category> + </bd:Tops> + <bd:Bottoms> + <bd:Category>Shorts</bd:Category> + </bd:Bottoms> + </bd:Clothing> + <bd:Belonging> + <bd:Bag> + <bd:Category>Backpack</bd:Category> + </bd:Bag> + </bd:Belonging> + </tt:HumanBody> + </tt:Appearance> + </tt:Object> + </tt:Frame> + </tt:VideoAnalytics> +</tt:MetadataStream> + + This data will be published under this topic: + + MyDevice/onvif-mj/VideoAnalytics/1/MyClassifier + + Where "MyDevice" is the given TopicPrefix and "1" is the + profile token. Shown below is how the sample metadata + frame is formatted in JSON: + + { + "Frame": [{ + "@UtcTime": "2021-10-05T15:13:27.321", + "@Source": "MyClassifier", + "@context": { + "acme": "http://www.acme.com/schema" + }, + "Transformation": { + "Translate": { + "@x": "-1.0", "@y": "-1.0" + }, + "Scale": { + "@x": "0.003125", "@y": "0.00416667" + } + }, + "Object": [{ + "@ObjectId": "15", + "Appearance": { + "Shape": { + "BoundingBox": { + "@left": "20.0", "@top": "20.0", "@right": "100.0", "@bottom": "30.0" + }, + "CenterOfGravity": { + "@x": "60.0", "@y": "50.0" + }, + "Polygon": [{ + "@x": "20.0", "@y": "30.0" + }, { + "@x": "100.0", "@y": "30.0" + }, { + "@x": "100.0", "@y": "80.0" + }, { + "@x": "20.0", "@y": "80.0" + }] + }, + "Color": { + "ColorCluster": [{ + "Color": { + "@X": "213", "@Y": "135", "@Z": "126" + }, + "Weight": 0.9, + "acme:ColorName": "White" + }, { + "Color": { + "@X": "102", "@Y": "176", "@Z": "119" + }, + "Weight": 0.5, + "acme:ColorName": "Blue" + }] + }, + "Class": { + "@Likelihood": "0.8", + "#text": "Human" + }, + "HumanFace": { + "Gender:": "Male", + "Age": { + "@Min": "20", "@Max": "30" + }, + "Accessory": { + "Opticals": { + "Wear": "true" + }, + "Hat": { + "Wear": "true" + } + } + }, + "HumanBody": { + "Clothing": { + "Tops": { + "Category": "LongSleeve" + }, + "Bottoms": { + "Category": "Shorts" + } + }, + "Belonging": { + "Bag": { + "Category": "Backpack" + } + } + } + } + }] + }] +} +
+ + + Service + This section covers the following main areas of this architecture: + + + Analytics Module interface + + + Rules interface + + + The analytics service allows fine-grained configuration of individual rules and individual analytics modules (see and ).
Configuration description language This specification introduces a description language which is used to configure rules and analytics modules. Beside configuration parameters the config description additionally contains an event description according to the OASIS topic notification. @@ -3091,7 +3565,7 @@
-
+
GetServiceCapabilities The capabilities reflect optional functions and functionality of a service. The information is static and does not change during device operation. The following capabilites are available: diff --git a/doc/Core.xml b/doc/Core.xml index f08f1248b..c87decf59 100644 --- a/doc/Core.xml +++ b/doc/Core.xml @@ -7231,6 +7231,10 @@ http://www.onvif.org/ver10/tev/topicExpression/ConcreteSet MaxEventBrokers Maxiumum number of event broker configurations that can be added to the device. + + + MetadataOverMQTT + Indicates that metadata streaming over MQTT is supported. @@ -7606,8 +7610,16 @@ http://www.onvif.org/ver10/tev/topicExpression/ConcreteSet PublishFilter - Concrete TopicExpression to select specific topics to publish, see section . Example TopicExpression: "tns1:VideoAnalytics//.|tns1:RuleEngine//." + Concrete Topic Expression to select specific event topics to publish, see section . Example TopicExpression: "tns1:VideoAnalytics//.|tns1:RuleEngine//." + If PublishFilter is empty, then device shall send all events but not metadata, for e.g. "MyDevice/onvif-ej//." . + + + MetadataFilter + + Concrete Topic Expression to select specific metadata topics to publish, see section . Example TopicExpression: "tns1:VideoAnalytics//. |tns1:AudioAnalytics//. |tns1:PTZ//." + If MetadataFilter is empty, then device shall send metadata from all active providers, for e.g. "MyDevice/onvif-mj//." . + QoS diff --git a/wsdl/ver10/events/wsdl/event.wsdl b/wsdl/ver10/events/wsdl/event.wsdl index a53c83b9f..706b4e471 100644 --- a/wsdl/ver10/events/wsdl/event.wsdl +++ b/wsdl/ver10/events/wsdl/event.wsdl @@ -80,6 +80,11 @@ IN NO EVENT WILL THE CORPORATION OR ITS MEMBERS OR THEIR AFFILIATES BE LIABLE FO Maxiumum number of event broker configurations that can be added to the device. + + + Indicates that metadata streaming over MQTT is supported + + @@ -365,9 +370,14 @@ IN NO EVENT WILL THE CORPORATION OR ITS MEMBERS OR THEIR AFFILIATES BE LIABLE FO - Concrete Topic Expression to select specific topics to publish. + Concrete Topic Expression to select specific event topics to publish. + + + Concrete Topic Expression to select specific metadata topics to publish. + + Quality of service level to use when publishing. This defines the guarantee of delivery for a specific message: 0 = At most once, 1 = At least once, 2 = Exactly once.