Defined-Trust Transport (DeftT) Protocol for Limited Domains

1.1. Environment and use

Due to physical deployment constraints and the high cost of wiring, OT networks overwhelmingly prefer radio as a communication medium. Use of wires is impossible in many installations (untethered Things, adding smart devices to home and infrastructure networks, vehicular uses, etc.). Wiring costs far exceed the cost of current System-on-Chip Wi-Fi IoT devices and the cost differential is increasing [WSEN][COST]. For example, the popular ESP32 is a 32bit/320KB SRAM RISC with 60 analog and digital I/O channels plus complete 802.11b/g/n and bluetooth radios on a 5mm die that consumes 70uW in normal operation. It costs $0.13 in small quantities while the estimated cost of pulling cable to retrofit nuclear power plants is $2000/ft [NPPI].¶

OT communications are frequently local with a many-to-many communication pattern using application-specific identifiers ("topics") for rendezvous. This fits the generic Publish/Subscribe communications model and, as table 1 in [PRAG] shows, nine of the eleven most widely used IoT protocols use a topic-based pub/sub transport. For example MQTT, an open standard developed in 1999 to monitor oil pipelines over satellite [MQTT][MHST], is now probably the most widely used IoT protocol (https://mqtt.org/use-cases/). In addition to providing local pub/sub in buildings, factories and homes, Microsoft Azure, Amazon AWS, Google Cloud, and Cloudflare all offer hosted MQTT brokers for collecting and connecting sensor and control data.¶

Pub/sub protocols are not connection or session oriented but instead organized around publishing and subscribing to topics. To communicate, publishers and subscribers need to use the same topic but need no knowledge of one another. IoT pub/sub is typically implemented as an application layer protocol over an Internet transport like TCP or TLS. These endpoint-based transport protocols do require in-advance configuration of peer addresses and credentials at each endpoint.¶

1.2. Transporting information

The smart lighting example of Figure 1 shows a topic-based publish/subscribe application layer protocol and a wireless broadcast domain. Each switch is set up to do triple-duty: one click of its on/off paddle controls some particular light(s), two clicks control all the lights in the room, and three clicks control all available lights (five kitchen plus the four den ceiling). Thus a switch button push may require a message to as many as nine light devices. On a broadcast physical network each message published by the switch is heard by all nine devices. IPv6 multicast provides a network layer that can take advantage of this but current IP transport protocols cannot. Instead, each switch needs to establish nine transport associations in order to send the published message for all lights to turn on. Communicating devices must be configured with each other's IP address and enrolled identity so, for n devices, both the configuration burden and traffic scale as O(n²). For example, when an "all" event is triggered, every light's radio will receive nine messages but discard the eight determined to be "not mine." If a device sleeps, is out-of-range, or has partial connectivity, additional application-level mechanisms have to be implemented to accommodate it.¶

MQTT and other broker-based pub/sub approaches mitigate this by adding a broker where all transport connections terminate (Figure 2). Each entity makes a single TCP transport connection with the broker and tells the broker the topics to which it subscribes. Thus the kitchen switch uses its single transport session to publish commands to topic kitchen/counter, topic kitchen or all. The kitchen counter light uses its broker session to subscribe to those same three topics. The kitchen ceiling lights subscribe to topics kitchen ceiling, kitchen and all while den ceiling lights subscribe to topics den ceiling, den and all. The broker reduces the configuration burden from O(n²) to O(n): 18 transport sessions to 11 for this simple example but for realistic deployments the reduction is often greater. There are other advantages: besides their own IP addresses and identities, devices only need to be configured with those of the broker. Further, the broker can store messages for temporarily unavailable devices and use the transport session to confirm the reception of messages. This approach is popular because the pub/sub application layer protocol provides an easy-to-use API and the broker reduces configuration burden while maintaining secure, reliable delivery and providing short-term in-network storage of messages. Still the broker implementation doubles the per-device configuration burden by adding an entity that exists only to implement transport and traffic still scales as O(n²). E.g., any switch publishing to all lights results in ten (unicast) message transfers over the wifi network.¶

A transport protocol not organized around bilateral associations ("connections") would better suit this problem. In the distributed systems literature, communication associated with coordinating shared objectives has long been modeled by the mathematics of distributed set reconciliation. In this approach, the domain of discourse is some named set, e.g., myhouse.iot, the event created by a button press on a switch is added as a new object to the instance of myhouse.iot at its point of origin, then the reconciliation process ensures that every entity holding myhouse.iot has the same set of objects by propagating this new object. Recent progress in distributed set reconciliation via Invertible Bloom Lookup Tables (IBLTs) [DIFF][IBLT][MPSR] enables solution of this problem by a simple, efficient multicast transport. DeftT implements IBLT set reconciliation and takes advantage of IPv6's ability to use link local multicast without requiring manual configuration or a routing agent, yielding Figure 1 where each device has a single, auto-configured transport that makes use of the broadcast radio medium without need for a broker or multiple transport associations. Each button push is broadcast exactly once to be added to the distributed set.¶

1.3. Securing information

In addition to reliability, current transport protocols provide some security via encrypting the sessions between end points. In the Internet, trust in the credentials of an endpoint (e.g., a website) is usually attested by a third party certificate authority (CA) and is bound to a DNS name. Each secure transport association requires the exchange of these credentials. Instead of third party certificates, OT tends to prefer locally created certificates, configuring them into on-device trusted enclaves [TPM][HSE][ATZ] as part of the device enrollment process. Thus, a transport for OT should work with a local root of trust. In Figure 2 each connection is a separate security association where each device needs to validate the broker's credential and the broker has to validate each device's (the validity check usually involves cryptographic verification of a signature chain terminating on a common trust root). This approach ensures that transport associations are between two enrolled devices (protecting against outsider and some MITM attacks) and allows for secure exchange of a nonce symmetric key that can be used to ensure transport privacy. However, once transport has been established there are no constraints whatsoever on what devices can say. Thus this type of security does not protect against the insider attacks that currently plague OT, e.g., [CHPT] description of a lightbulb taking over a network. Hardening an OT system against these attacks requires trust management operating at the individual publication level, not the connection or session level. For example, the basic function of a light switch requires that it be allowed to tell a light to turn on or off but it almost certainly shouldn't be allowed to tell the light to overwrite its firmware (fwupd), even though "on/off" and "fwupd" are both standard capabilities of most smart light APIs. Once the session is established, its security handles the "fwupd" publications the same way as the "on/off" publications. Per-publication trust decisions can enable the fwupd from the light switch to be rejected.¶

The requirement for per-publication trust decisions combined with OT's preponderance of many-to-many communications over broadcast infrastructure suggests that per-publication signing is preferable to session-based signing. Securing each publication rather than the path it arrives on deals with a wider spectrum of threats while avoiding the quadratic session state and traffic burden. This results in a trust management system such as [DLOG] where each publisher is responsible for supplying all of the "who/what/where/when" information needed for each subscriber to prove the publication complies with system policies. If, as described in [DLOG], the system's trust requirements are expressed using a declarative framework, they can be validated for consistency and completeness then converted to a compact runtime form which can be authorized and secured via signing with the system trust anchor, then distributed, validated and updated using the same mechanisms used for identity certificates. This trust schema can be used both to construct and validate publications, guaranteeing that all parts of the system always conform to and enforce the same rules, even as those rules evolve to meet new threats.¶

OT communication maps well to pub/sub because it consists of independent messages that, due to interoperability requirements, conform to rigid standards on syntax and semantics [IEC61850][ISO9506MMS][ONE][MATR][OSCAL][NMUD][ST][ZCL]. Using conventional session-based transports combines these independent publications under a single session key. Moving to a publication-based transport makes it possible to embed the trust management mechanism described above directly in the publish and subscribe data paths as shown below:¶

This style of trust management extends LangSec's [LANG] "be definite in what you accept" principle by using the authenticated common ruleset for belt-and-suspenders enforcement by both ends of the transport: If the trust schema shows the publication builder it doesn't have the credentials needed to produce a valid publication, an error is thrown and nothing is published. Independently, if the publication validator doesn't have a locally validated, complete signing chain for the credential that signed some pub, the schema shows the signing chain isn't appropriate to the pub, or the pub's signature doesn't validate, the pub is ignored. Since an app's subscriptions determine the pubs it will see, only certs of chains that sign pubs matching the subscriptions need to be validated or retained. Thus a device's communication state burden and computation costs are a function of how many different things are allowed to talk to it but not how many things it talks to or the total number of devices in the system. In particular, event driven, publish-only devices like sensors spend no time or space on validation. And, unlike most 'secure' systems, adding additional trust schema constraints to reduce attack surface results in devices doing less work.¶

The particular rules for any deployment are application-specific (e.g., Is it home IoT or a nuclear power plant?) and site-specific (specific form of credential and idiosyncrasies in rules) which DeftT accommodates by being invoked with a ruleset particular for a deployment. We anticipate that the efforts to create common data models for specific sectors will lead to easier and more forms-based configuration of DeftT deployments.¶

1.4. Current status

An open-source Defined-trust Communications Toolkit [DCT] with a reference implementation of DeftT is maintained by the corresponding author's company, Pollere. Pollere is also working on home IoT uses. Massive build out of the renewable energy sector is driving connectivity needs for both monitoring and control. Author King's company, Operant, is currently developing extensions of DeftT in a mix of open-source and proprietary software tailored for commercial deployment in support of distributed energy resources (DER). Current small scale use cases have performed well and expanded usage is planned. Pollere and Operant welcome collaborators with problems suitable for DeftT.¶

[DCT] has examples of using DeftT to implement secure brokerless message-based pub/sub using UDP/IPv6 multicast and unicast UDP/TCP and include extending a trust domain via a unicast connection or between two broadcast domains. As the needs of our use cases expand, the Defined-trust Communications (DC) architecture will evolve. Working implementations and performance improvements are occasionally added to the repository. This reflects the development philosophy of DC to start from solving useful problems with a well-defined scope and extend from there. DeftT's reference implementation code is open-source, as befits any communications protocol, but even more critical for one attempting to offer security. DCT itself makes use of the open-source cryptographic library libsodium [SOD] and the project is open to feedback on potential security issues.¶

3.1. DeftT role in application communications

DeftT's security enforces "who can say what to which" as well as providing required integrity, authenticity and confidentiality. Applications use DeftT to add to and access from a distributed collection of Publications. Transparently to applications, a DeftT instance both constructs and validates all Publications against a set of formal, validated rules. Each new DeftT is configured with these rules in certificate form along with a signing chain that includes its private signing identity and has the same root of trust as the certificate of trust rules. A DeftT is identified by its signing identity and the role and capabilities in that identity's signing chain. Each DeftT must add its credentials (signing chain) to a certificate collection prior to adding publications to the collection. DeftT validates credentials as a chain of trust against the shared trust rules and does not accept Publications without a fully validated signer identity.¶

Communicating DeftTs must be in the same trust domain (TD), i.e., identities derive from the same root of trust and follow an identical trust schema. Using this shared set of rules, DeftT provides fully distributed policy enforcement for the TD without relying on a secured-perimeter physical network and/or extensive per-device configuration. Trust schemas are distributed as a trust-root-signed certificate and that certificate's thumbprint uniquely identifies a TD in DeftT PDUs. DeftT can share an IP network with non-DeftT traffic as well as DeftT traffic of a different TD. Privacy via AEAD PDU content encryption is automatically handled within DeftT.¶

Figure 4 shows the data that flows in and out of a DeftT instance. An application-specific format is used for information exchanged with an application. DeftT uses its trust schema to turn this information into Publications that it adds to its set (sync collection). DeftT implements two types of transport PDU, both broadcast on its subnet, that it uses to manage the set. One represents the collection state (cState) of one or more DeftTs by an IBLT that contains all the Publications the DeftT currently has in its collection. The cState serves as a query for additional data that isn't reflected in its local state. The other PDU carries the collection additions (cAdds) that are sent in response to a cState. Reception of a cAdd with new data responds to a particular cState so a receiving DeftT removes that cState as a pending query (it will be replaced with a new cState with the addition of the new items). DeftT requires and implements signing and validation of publications as well as the cAdd PDUs that transport them. As these dynamics are exactly that of Named-Data Networking's Interest (cState) and Data (cAdd) packets, the DeftT reference implementation uses a restricted subset of that format for its PDUs along with its own broadcast-optimized Face protocol. (DeftT does not utilize a NDN forwarder or implement the full NDN protocol.)¶

Any DeftT that is missing a Publication (due to being out-of-range or asleep, etc.) can receive it from any other DeftT. The reconciliation protocol will continue to send cAdds as long as cStates are received that don't contain some of their publications. This results in reliability that is subscriber-oriented, not publisher-oriented and is efficient for broadcast media, particularly with protocol features designed to prevent multiple redundant broadcasts. DeftT's reference implementation includes an efficient multicast-optimized approach.¶

3.2. Information movement is based on trust rules

The Internet's transport and routing protocols emphasize universal reachability and packet forwarding is based on destination. A significant number of uses neither need nor desire to transit the Internet. For a wide class of OT applications, depending on the good sending practices of others while accepting packets has left critical applications open to misconfiguration and attacks. DC only moves its Publications if they follow trust rules, an approach that differs from Internet forwarding but offers new opportunities.¶

DeftTs on the same subnet may not be in the same trust domain (TD) and DeftTs in the same TD may not be on the same subnet. In the former case, cState and cAdd PDUs of different TDs are differentiated by the trust domain id (thumbprint of the certificate holding the domain's trust rules) which can be used to determine whether or not to process a PDU. A particular sync zone is managed on a single subnet: cState and cAdds are not forwarded off that subnet. Trust-based relays connect separate sync zones while preserving a trust domain for the publications. A relay is an application running on a device with a network interface on each subnet that has two or more DeftT instances. Each DeftT participates in a different sync zone and has a signing identity valid for its sync zone. Each subnet's DeftT uses the same trust rules to validate Publications and those that arrive via one DeftT are added to the collections of all other DeftTs of the relay (after validation). Only Publications are relayed between subnets and the Publication must match a trust rule in the DeftT's trust engine. (Note that cAdd encryption is unique per subnet/sync collection.)¶

3.3. Relays extend a trust domain

Relay DeftTs use a special API module (a "shim", see Appendix A.6) that performs "pass-through" of valid Publications rather than creating Publications. The relay of Figure 5-left has three separate wireless subnets. If all three DeftTs use the same trust schema, a new validated cert added to the cert store of one DeftT is passed to the other two, which will validate it before adding to its own cert store (superfluous in this case, but not a lot of overhead for additional security). When a valid Publication is received by one DeftT, it is passed to the other two DeftTs to validate against their trust schema copies and published if it passes.¶

Relays can also connect subsets of a TD, either proper subsets or overlapping subsets. A relay may have different identities and trust schemas for each of its DeftT but must have the same trust anchor. Publications that are undefined for a particular DeftT will not pass its validation and will be silently discarded. This means the relay application of Figure 5-left can remain the same but Publications will only be published to a different subnet if its DeftT has that format in its ruleset. If the added efficiency of filtering Publications prior to validation checks is desired, a relay can be altered to limit subscriptions (a one-line change) on some DeftT(s) or may add code to filter Publications before passing to other DeftT(s). Figure 5-right shows extending a trust domain geographically by using a unicast connection (e.g., over a cell line or tunnel over the Internet) between two relays which also interface to local broadcast networks. Everything on each local network shows up on the other. A TD subset could be used here to limit the types of Publications sent on the remote link, e.g., logs or alerts. Using this approach in Figure 5-right, local communications for subnet 1 can be kept local while subnet 2 might send commands and/or collect log files from subnet 1.¶

More generally, relays can form a mesh of broadcast networks with no additional configuration (i.e., relays on a broadcast network do not need to be configured with others' identities). The mesh is efficient: publications are only added to an individual DeftT's collection once regardless of how it is received. More on the applicability of DeftT meshes is in Section 4.¶

3.4. DeftT and congestion control

Each DeftT instance manages its collection on a single broadcast subnet (since unicast is a proper subset of multicast, a point-to-point connection is viewed as a trivial broadcast subnet) thus only has to deal with that subnet's congestion. (As described in the previous section, a device connected to two or more subnets may create DeftT instances having the same collection name on each subnet with an app-level publication relay between them but DeftT never forwards PDUs between subnets. It is, of course, possible to run DeftT over an extended broadcast network like a PIM multicast group (or NDN NFD forwarding mesh?) but the result will generally require more configuration and be less reliable, efficient and secure than DeftT's self-configuring peer-to-peer relay mesh described in the previous section.¶

DeftT will send at most one copy of any publication over any subnet, independent of the number of publishers and subscribers on the subnet. Thus the total DeftT traffic on a subnet is strictly upper bounded by the app-level publication rate. As described in Section 3.1, instances publish a cState specifying the set elements they currently hold. If an instance receives a cState specifying the same elements it holds, it doesn't send its cState. Thus the upper bound on cState publication rate is the number of peers on the subnet divided by the cState lifetime (typically seconds to minutes) but is typically one per cState lifetime due to the duplicate suppression. Each peer can send at most one cAdd in response to a cState. This creates a strict request/response flow balance which upper bounds the cAdd traffic rate to number of peers - 1 times the cState publication rate. The flow balance ensures an instance can't send a new cState until it's previous one is either obsoleted by a cAdd or times out. Similarly a cAdd can only be sent in response to the cState which it obsoletes. Thus the number of outstanding PDUs per instance is at most one and DeftT cannot cause subnet congestion collapse.¶

If a relay is used to extend a TD over a path whose bandwidth delay product is many times larger than typical subnet MTUs (1.5-9KB), the one-outstanding-PDU per peer constraint can result in poor performance (1500 bytes per 100ms transcontinental RTT is only 120Kbps). DeftT can run over any lower layer transport and stream-oriented transports like TCP or QUIC allow for a 'virtual MTU' that can be set large enough for DeftT to relay at or above the average publication rate (the default is 64KB which can relay up to 5Mbps of publications into a 100ms RTT). In this case there can be many lower layer packets in flight for each DeftT PDU but their congestion control is handled by TCP or QUIC.¶

3.5. Defined-trust management

OT applications are distinguished (from general digital communications) by well-defined roles, behaviors and relationships that constrain the information to be communicated. Structured abstract profiles characterize the capabilities and attributes of Things and can be machine-readable. Energy applications in particular have defined strict role-based access controls [IEC] though proposed enforcement approaches require interaction of a number of mechanisms across the communications stack [NERC]. These structured profiles and rules strictly define permitted behaviors including what types of messages can be issued or acted on; undefined behaviors should not be permitted. DC can incorporate these rules along with local configuration directly into the trust schemas used in DeftT's integrated trust management engine. This not only provides a fine-grained security but a highly usable security, an approach that can make an application writer's job easier since applications do not need to contain local configuration and security considerations.¶

DeftT's trust engine modules use trust schemas which are compiled into a binary format to become the content of a certificate signed by the trust domain's root of trust. The trust schema contains the defined rules that describe the format of PDUs and the specific roles and credentials that must be in the signing chain of entities that create them. Defined-trust Communications includes the use of trust schemas, a language for expressing the trust schemas, and the use of a compiler and other tools to create the credentials a DeftT needs at run-time to function in a particular TD.¶

#pub: /_domain/trgt/topic/loc/arg/_ts & { _ts: timestamp() } <= roleCert
roleCert:       _domain/_role/_roleId/_keyinfo <= netCert
netCert:        _domain/_keyinfo
#pubPrefix:     _domain
#pubValidator:  "EdDSA"
#cAddValidator: "EdDSA"
_domain:        "example"
_keyinfo:       "KEY"/_/"dct"/_

Publication #pub:
  parameters: trgt topic loc arg
  tags: /_domain/trgt/topic/loc/arg/_ts
Publication #pubPrefix:
  parameters:
  tags: /_domain
Publication #pubValidator:
  parameters:
  tags: /"EdDSA"
Publication #cAddValidator:
  parameters:
  tags: /"EdDSA"
Certificate templates:
  cert roleCert: /"example"/_role/_roleId/"KEY"/_/"dct"/_
  cert netCert: /"example"/"KEY"/_/"dct"/_
binary schema  is 301 bytes

adminCert:  roleCert & { _role: "admin" } <= netCert
sensorCert: roleCert & { _role: "sensor" } <= kmCap
capCert:    _network/"CAP"/_capId/_capArg/_keyinfo <= netCert
kmCap:      capCert & { _capId: "KM" }
#reportPub: #pub & {topic:"status"} <= sensorCert
#commandPub: #pub & {topic:"command"} <= adminCert
#cAddValidator: "EdDSA"

3.6. DC certificates and identity bundles

To participate in a trust domain, an entity needs the domain's trust anchor, signed trust schema cert and an identity signing cert (public key cert that includes private key) complete with its signing chain (public certs only) terminating at the zone's trust anchor. The public signing certs of other entities are obtained and validated (using the common trust schema and anchor) at run-time and as entities join. Well-formed certificates and identity bundle deployment are critical elements of DC. This section describes certificate requirements and the construction and installation of an identity bundle. DCT includes utilities to create certs and bundles.¶

4.1. Secure Industrial IoT

IIoT sensors offer significant advantages in industrial process control including: improved accuracy, process optimization, predictive maintenance and analysis, higher efficiency, low-cost remote accessibility and monitoring, reduced downtime, power savings, and reduced costs [IIOT]. The large physical scale of many industrial processes necessitates that expensive cabling costs be avoided through wireless transport and battery power. This is particularly an issue in nuclear power plant applications where radioactive shielding walls are very thick concrete and security regulations make any plant modifications to add cabling subject to expensive and time-consuming reviews and permitting. Wireless sensor deployments in an industrial environment can suffer from signal outages due to shielding walls and interference caused by rotating machinery and electrical generators. Resiliency through the use of multiple gateway devices, that can receive sensor information and transmit to monitor/controllers and servers, can be used to create a robust architecture. Gateways that use DeftT can form a robust wireless mesh that is resilient against transmission outages, facilitating reliability. DeftT forms meshes with no additional configuration as Publications missing from one DeftT's set can be supplied by another within range. Several gateways are typically within a single sensor's wireless range, reducing the number of lost sensor packets. Other meshed gateways can relay the sensor's publications either wirelessly or via a wired ethernet backhaul.¶

IIoT sensors require tight security. Critical Digital Assets (CDA) are a class of industrial assets such as power plants or chemical factories which must be carefully controlled to avoid loss-of-life accidents. Even when IIoT sensors are not used for direct control of CDA, spoofed sensor readings can lead to destructive behavior. There are real-life examples (such as uranium centrifuges) of nation-state actors changing sensor readings through cyberattacks leading to equipment damage. These risks result in a requirement for stringent security reviews and regulation of CDA sensor networks. Despite the advantages of deploying CDA sensors, adequate security is prerequisite to deploying the CDA sensors. Information conveyed via DeftT has an ensured provenance and may be encrypted in an efficient implementation making it ideal for this use.¶

IIoT sensors may be mobile (including drone-based) and different gateways may receive the wireless broadcast of a particular sensor over time. A DeftT mesh captures Publications anywhere within its combined wireless network coverage area and ensures it efficiently reaches all other subscribing DeftTs as long as they are in range of at least one that has received the information. An out-of-service or out-of-range DeftT can receive all active subscribed publications once it is in range and/or able to communicate. This type of use of DeftT is illustrated in Figure 11 where gateway devices are deployed with DeftT relay applications that have a Bluetooth (BT) interface and a WiFi interface. A number of BT devices are deployed as sensors, switches, cameras, lock openers, etc. The WiFi network includes tablet devices and a monitor/controller computer. Gateways are placed so that there is always at least one gateway in range of a BT device and at least one gateway (or the controller) in its WiFi range. WiFi tablets can move around within range of one or more gateways. If all the DeftTs have an identical trust schema, devices on the WiFi network have access to all of the BT devices though applications on any particular device may subscribe to any subset of the information available. The WiFi DeftTs can be given a trust schema that requires encrypting its cAdds so that longer-range data is kept private. These configuration choices are made by changes in the trust schemas alone, the application code is exactly the same. No configuration is needed to make devices recognize one another and the dynamics of syncps will keep communications efficient, ensuring that all DeftTs in the trust domain know what information is available. The Face implementation ensures that requested Publications are only sent once (within a particular range). These features mean that DeftT forms efficient broadcast meshes with no additional configuration, an important advantage.¶

In addition to specifying encryption and signing types, trust rules control which users can access specific sensors. For example, an outside predictive maintenance analysis vendor can be allowed access to the vibration sensor data from critical motors which is relayed through the internet, while only plant Security can see images from the site cameras due to privacy concerns.¶

4.2. Secure access to Distributed Energy Resources (DER)

The electrical power grid is evolving to encompass many smaller generators with complex interconnections. Renewable energy systems such as smaller-scale wind and solar generator sites must be economically accessed by multiple users such as building owners, renewable asset aggregators, utilities, and maintenance personnel with varying levels of access rights. North American Electric Reliability Corporation Critical Infrastructure Protection (NERC CIP) regulations specify requirements for communications security and reliability to guard against grid outages [DER]. Legacy NERC CIP compliant utility communications approaches, using dedicated physically secured links to a few large generators, are no longer practical. DeftT offers multiple advantages over point-point TLS connections for this use case:¶

• Security. Encryption, authentication, and authorization of all information objects. Secure broker-less pub/sub avoids single-point broker vulnerabilities. Large generation assets of hundreds of megawatts to more than 1 gigawatt, particularly nuclear power plants must be controlled securely or risk large-scale loss of life accidents. Hence, they are attractive targets for sophisticated nation-state cyber attackers seeking damage with national security implications. Even small-scale DER generators are susceptible to a coordinated attack which could still bring down the electric grid.¶
• Scalability. Provisioning, maintaining, and distributing multiple keys with descriptive, institutionalized, hierarchical names. DeftT allows keys to be published and securely updated on-line. Where historically a few hundred large-scale generators could supply all of the energy needs for a wide geographic area, now small-scale DER such as residential solar photovoltaic (PV) systems are located at hundreds of thousands of geographically dispersed sites. Many new systems are added daily and must be accommodated economically to spur wider adoption.¶
• Resiliency. A mesh network of multiple client users, redundant servers, and end devices adds reliability without sacrificing security. Generation assets must be kept on-line continuously or failures risk causing a grid-wide blackout. Climate change is driving frequent natural disasters including wildfires, hurricanes, and temperature extremes which can impact the communications infrastructure. If the network is not resilient communications breakdowns can disable generators on the grid leading to blackouts.¶
• Efficiency. Data can be published once from edge gateways over expensive cellular links and be accessed through servers by multiple authorized users, without sacrificing security. For small residential DER systems, economical but reliable connectivity is required to spur adoption of PV compared to purchasing from the grid. However, for analytics, maintenance and grid control purposes, regular updates from the site by multiple users are required. Pub/sub via DefT allows both goals to be met efficiently.¶
• Flexible Trust rules: Varying levels of permissions are possible on a user-by-user and site-by-site basis to tightly control user security and privacy at the information object level. In an energy ecosystem with many DER, access requirements are quite complex. For example, a PV and battery storage system can be monitored on a regular basis by a homeowner. Separate equipment vendors for batteries and solar generation assets, including inverters, need to perform firmware updates or to monitor that the equipment is operating correctly for maintenance and warranty purposes. DER aggregators may contract with a utility to supply and control multiple DER systems, while the utility may want to access production data and perform some controls themselves such as during a fire event where the system must be shut down. Different permissions are required for each user. For example, hourly usage data which gives detailed insight into customer behaviors can be seen by the homeowner, but for privacy reasons might only be shared with the aggregator if permission is given. These roles and permissions can be expressed in the trust rules and then secured by DeftT's use of compiled trust schemas.¶

The specificity of the requirements of NERC CIP can be used to create trust schemas that contain site-specifics, allowing applications to be streamlined and generic for their functionality, rather than containing security and site-specifics.¶

A.1. Syncps set reconciliation

Syncps has roots in the sync protocols developed for Information-Centric Networking, e.g., [CBIS], over the past decade. A DC sync protocol MUST provide mechanisms that are end-point agnostic, be broadcast-friendly, and have an efficient implementation. Syncps has been in development to meet these needs for some time, starting with the first version (covered in [DNMP]). It currently appears that syncps is the only sync protocol DC needs so discussion of the sync protocol module is explicitly syncps. Future uses and research developments may lead to new sync protocols suitable for DC but they must fulfill the same role and provide the same interfaces as syncps.¶

The sync module performs set reconciliation over the Publications of a Collection, providing the enabling protocol for any-to-any communications. A single syncps instance manages a single collection. Each syncps announces the Publications it currently has in its Collection by sending a cState containing an IBLT [IBLT]. IBLTs solve the multi-party set-difference problem efficiently without the use of prior context and with communication proportional to the size of the difference between the sets being compared. Syncps manages both active and inactive Publications to know when and if to communicate them to peers (though Face) and subscribers (via upcalls), but it knows nothing about the format or semantics of Publications. Upcalls from syncps to other modules provide validation and expiration information for Publications as well as validation and signing of cAdds. If a cAdd fails to validate, it is silently discarded. Syncps can confirm that Publications have reached their collection if an optional callback handler is provided. This feature piggybacks on normal syncps dynamics, reporting when that Publication appears in the state summary of some other entity. As such, it should not be used as a measure of the transmission time of a Publication.¶

Syncps keeps its DeftT instance synchronized with the sync zone at the attached Face. It adds new DeftT-local Publications to the collection and transfers validated, subscribed Publications to a shim (or distributor). Syncps creates two PDUs to manage collections: a cState, which summarizes the current state of its Collection and a cAdd, used to respond to received cStates with any applicable Publications (i.e., cAdd carry Publications). Syncps subscribes to everything in a collection. The Publications that initiate upcalls can be limited by the shim in setting up subscriptions with syncps.¶

A.2. SchemaLib trust management

SchemaLib implements the trust management engine of DeftT. It validates certificates, uses the trust schema, and instantiates the distributors required by the trust schema.¶

When a DeftT is instantiated, it is handed its identity bundle which contains the trust anchor for the trust domain, the particular (compiled) trust schema for the domain and the signing identity chain for the DeftT. This component performs validation of these certificates and they are stored locally. Templates for all the legal (according the the trust schema and the signing identity) Publications are created and saved to use whenever a new Publication is constructed, ensuring its structural validity.¶

The DeftT's certificate distributor publishes all the public certs in its chain in order to connect to the collections used by this DeftT. A DeftT is not connected until a delivery indication has been received for its identity certificate chain. If information privacy is required in the trust schema, the distributors for the required keys are instantiated.¶

Signing identities published by others are received, validated, stored and used to create templates for structural validation of Publications that originate with those signing identities.¶

A.3. Signature managers

Signature Managers (sigmgrs) implement the signing and validation of Data (which may include encryption/decryption) selected by the trust schema for Publications and their cAdd as well as by the particular signing and validation required by distributors. Use of an encryption sigmgr requires a group key distributor (see A.5). Integrity signing and null signing is not available in trust schemas (i.e., not used to specify Publications or their cAdd) and are only used in certificate and key distribution.¶

Six sigmgrs are currently implemented in DCT, supplying the following signing and validation algorithms:¶

EdDSA using the DCT identity cert associated with the particular DeftT¶

RFC7693 and SHA256: integrity (not available to trust schemas)¶

AEAD encryption/decryption for an entire sync zone where the key is created, distributed and updated by the group key distributor. The key distributor encrypts the group key individually for each valid signing identity that has been published in the cert Collection (validated and stored locally). Members are added to the group as their validated signing identities become known; no members are added apriori.¶

PPAEAD is a version of AEAD encryption/decryption where the encryption key is unique to a particular publisher and the group of authorized subscribers. Authorized subscribers must have the required capability in their signing chain and the subscriber group key pair is distributed by a subscriber group key distributor which creates (and updates) a key pair for the subscriber group, putting the public key in the clear and encrypting the secret key for each subscriber group member. Data can only be decrypted by authorized subscribers (subscriber group members).¶

PPsigned adds EdDSA signing and validation to PPAEAD. Its use is indicated if there is a need to protect against authorized members of the subscriber group forging packets from Collection publishers. The encrypted packet is also signed by the publisher. Uses the same subscriber group key distributor as PPAEAD.¶

A.4. Distributors

Distributors manage certificate and key Collections transparently to applications through their own syncps. A certificate distributor MUST be provided in order to manage the signing certificate collection for any application. Its Publications are signing chain certificates and when a DeftT is first instantiated it is used to publish the certs of its own signing chain. The cert distributor must confirm their delivery to the cert Collection (i.e., shows up in an IBLT summary originated by a different DeftT) before the DeftT upcalls to the application that it can start communications. The cert distributor passes subscribed certs to schemaLib where validated certificates are stored; invalid certificates are silently discarded.¶

The use of other distributors is dependent on the signature manager selected. AEAD, PPAEAD, and PPsigned require group key distributors to manage a key Collection. DCT contains a group key distributor for AEAD as well as a group key distributor for publisher privacy. These use the capabilities contained in signing identities to determine eligibility to be a group's key maker and eligibility to subscribe (decrypt) Publications in a publisher privacy collection. Key maker election, key creation (and re-creation), and key distribution are all handled by the distributors and transparent to the application.¶

A.5. Faces

Faces translate between the cAdd and cStates of the sync protocol and the system packet transport used for a particular DeftT instance. Any packet transport can be used as long as it provides:¶

• send packet and register callback for received packet¶
• connect registration callback invoked when packets can be sent or received¶
• information callback - MTU is returned¶

DCT currently implements a broadcast Face that is used with UDP multicast over IPv6 and a unicast face over UDP or TCP. The DCT Face derives that of NDN, where cState are Interests and cAdds are Data, but its structure has been optimized for use on broadcast media and to interact directly with a sync module, not a forwarder node. The Face keeps:¶

• Pending Interest Table (PIT): similar to its use in CCN and NDN, this is a table of unsatisfied Interests. Interests are removed when a corresponding Data arrives ("satisfies the Interest") or the Interest times out (when its lifetime has been exceeded).¶
• Registered Interest Table (RIT) : this is used to hold the Interest type that this DeftT can respond to¶
• Duplicate Interest Table (DIT): is used to keep Interests from looping (without the need for a spanning tree) and tracks the Interest plus its nonce. (A cAdd is never sent to the Interface on which it arrived.)¶

A.6. Shims

A shim is application-class-specific, converting between application-meaningful messages and DeftT's Publications. A shim is passed the salient information about a particular communication and its application level data unit and uses these to create trust schema compliant Publications, using knowledge about this application class and calls to schemaLib modules. A shim parses received Publications for call back(s) to the application. A shim can be customized to a particular application (the approach taken in the now-obsolete [DNMP]) or can provide more general communication models, such as pub/sub, streaming, request/response.¶

This latter approach has been used to create a DCT library shim, a message-based publish/subscribe (MBPS) API whose semantics resemble MQTT but the protocol is brokerless and collection-secure, unlike MQTT. MBPS handles breaking application-level messages into trust schema specified Publications and provides an option for delivery confirmation to be passed to the application. MBPS has been used for multiple applications: two examples are in DCT and Operant has used for other applications. MBPS provides a simple API that hides network layer and security details and offers two levels of message QoS: a default unacknowledged delivery and a confirmation that the publication has reached at least one other member of the collection. Applications can use the following MBPS methods:¶

connect(successCB, <opt>failureCB): Performs set up (if any) necessary to allow communications to start. (e.g., signing key distribution is carried out). Invokes appropriate callback, success or failure.publish(msg, args, <opt>confCB): Publishes the given message content and returns a unique message ID. If a confirmation callback is included, mbps invokes confCB with an indicator of success or failure of the message.¶

subscribe(handler): subscribes to all the topics in the pubs Collection. A received message's underlying publication(s) is validated before the handler is invoked. subscribe(topic, handler): distinguishes application-level subscriptions further by topic (component(s) of name that mbps will append to the pubPrefix of the trust schema) and passes a handler to use for a particular topic.¶

run(): once application set up is finished, this turns over control to the transport.¶

The API simplicity is shown in this application snippet:¶

void msgPubr(mbps &cm, const auto& toSnd) {
    //... lines of code to prepare arguments
    cm.publish(toSnd, a); //load arguments in a
}
static void msgRecv(mbps &cm, const auto& msg, const msgArgs& a) {
    //... do something with msg
}
int main(int argc, char∗ argv[]) {
    mbps cm(argv[optind]); // make shim using identity bundle
    cm.subscribe(msgRecv);
    cm.connect([&cm]() {
        ... // prepare toSnd
        msgPubr(cm, toSnd);
    });
    cm.run();
}

B.1. Publications

Publications are used throughout DeftT's modules. A Name TLV is used to encode the name defined in the trust schema. A Publication is valid if it starts with the correct TLV, its Name validates against the trust schema and it contains the five required Level 1 TLVs in the right order (top-down in Table 1) and nothing else. MetaInfo contains the ContentType (in DeftT either type Key or Blob). The Content carries the named information and MAY be empty. SignatureInfo indicates the SignatureType used to select the appropriate signature manager (Appendix A.3). The SignatureType for a collection's Publications is specified in the trust schema and each Publication MUST match it. (A list of current types can be found in [DCT] file include/dct/sigmgrs/sigmgr.hpp.) The KeyLocator associated with the SignatureType follows then the ValidityPeriod (if the Publication is a Certificate). Finally, SignatureValue is determined by the SignatureType and its format is verified by the signature manager. .¶

B.2. Certificates

Certificates (certs) are Publications with the ContentType set to Key and both a KeyLocator and a ValidityPeriod. DCT certs are compatible with the NDN Certificate standard V2 but adhere to a stricter set of conventions to make them resistant to substitution, work factor and DoS attacks. The only KeyLocator type allowed in a DCT cert is a KeyDigest type that MUST contain the 32 byte SHA256 digest of the entire signing cert (including SignatureValue). A self-signed cert (such as a trust anchor) MUST set this digest to all zero. This digest, a cert thumbprint [IOTK], is the only locator allowed in any signed DC object (e.g., Publications, cAdd, schemas, certs) and MUST be present in every signed object. A signed object using any other type of locator will be considered unverifiable and silently ignored. Certificate Names use a suffix:¶

  KEY/<keyID>/dct/<version>

where the cert's thumbprint is the keyID and its creation time is the version.¶

The original publisher of any signed object MUST ensure that that all certs, schemas, etc., needed to validate the object have been published before the object is published. If an entity receives a signed object but is missing any of its signing dependencies, the object should be considered unverifiable and silently ignored. Such objects MUST NOT be propagated to other entities.¶

B.3. cState

cState and cAdds are are the PDUs exchanged with the system-level transport in use (e.g., UDP) but are only used by the Sync (sec A.1) and Face (sec A.5) modules. Sync creates cState and cAdd PDUs while the Face manages the protocol interaction within the trust domain. A cState PDU (see Table 2) is used to report the state of a Collection at its originator. A cState serves to inform all subscribing entities of a trust domain about Publications currently in the Collection, both so an entity can obtain Publications it is missing and so an entity can add Publications it has that are not reflected in the received cState.¶

A cState is valid if it starts with the correct TLV and it contains the three required Level 1 TLVs in the right order (top-down in Table 2) and nothing else. Its Name MUST start with the trust domain id of the DeftT, then a descriptive Collection name (of at least one component) and finally a representation of the the state of the Collection at the originator. There is no signature for a cState PDU. (The cState format is a restricted subset of an NDNv3 Interest.)¶

B.4. cAdd

A cAdd PDU is used to add Publications to a Collection and carries Publications as Content. A cAdd PDU is created after a cState is received and only if the recipient has Publications that are not reflected in the recipient's local state. A cAdd is valid if it starts with the correct TLV, contains the five required Level 1 TLVs in the right order (top-down in Table 3) and nothing else. A cAdd name MUST be identical to the cState to which it responds.¶