Skip to content

JSON and YANG ​

Every operation that reads or writes datastore content takes RFC 7951 JSON as well as XML: YANG's JSON encoding, the one RESTCONF uses. You read configuration as plain objects, change them in code and send them back:

ts
const IFS = 'ietf-interfaces:interfaces';

const config = await client.ops.getConfig({
  filter: { [IFS]: {} },
  format: 'json',
});

config[IFS].interface[0].description = 'uplink';

await client.ops.editConfig({ target: 'running', config });

That example only works because the library uses the server's YANG schema to convert between JSON and NETCONF's XML. This page explains why the schema is needed, what goes wrong without one, and how the conversion works in each direction.

Why the schema matters ​

XML and JSON carry different information ​

NETCONF sends data as XML, and that XML holds only text. Each element has a name and a namespace, and its value is a string. Nothing in <mtu>1500</mtu> says whether 1500 is a number. Nothing in a single <interface> element says whether more can follow. The rules for both are in the YANG modules the server implements, and the server checks everything it receives against them. An XML client can ignore the schema because it passes text through unchanged and lets the server interpret it.

JSON is where you want the data in code, because it has the things XML lacks: arrays, numbers, booleans and null. Converting XML to JSON means deciding each of those, and converting JSON back means restoring what JSON leaves out. That information isn't in the data. It is in the schema.

Reading: what you're missing without a schema ​

Here is one interface, as a server sends it:

xml
<interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces"
            xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">
  <interface>
    <name>eth0</name>
    <type>ianaift:ethernetCsmacd</type>
    <enabled>false</enabled>
    <ipv4 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip">
      <mtu>1500</mtu>
    </ipv4>
  </interface>
</interfaces>

Here is the same data converted with the schema, and converted without one:

ts
// Typed by the schema
{
  'ietf-interfaces:interfaces': {
    interface: [
      {
        name: 'eth0',
        type: 'iana-if-type:ethernetCsmacd',
        enabled: false,
        'ietf-ip:ipv4': { mtu: 1500 }
      }
    ]
  }
}

// Without a schema
{
  'ietf-interfaces:interfaces': {
    interface: {
      name: 'eth0',
      type: 'ianaift:ethernetCsmacd',
      enabled: 'false',
      'ietf-ip:ipv4': { mtu: '1500' }
    }
  }
}

The second version looks reasonable, but each difference is a bug waiting to happen:

  • Which nodes are lists. XML repeats an element once per list entry, so a list with one entry looks exactly like a container. Without a schema, interface is an object while the device has one interface and an array once it has two. Code such as interface.map(…) works in the lab and fails on a device with a different number of entries. The same applies to leaf-lists.
  • Value types. Every value stays a string. enabled: 'false' is truthy, so if (entry.enabled) takes the wrong branch. '10000' > '9000' is false, because strings are compared character by character. An empty leaf such as <shutdown/> becomes '', which is falsy, even though the leaf being present means "set".
  • Qualified names in values. ianaift:ethernetCsmacd uses a prefix the server declared in this particular reply. Another server, or the next software release, may write ift: or use a default namespace, so you can't compare the value with a constant. The schema shows that type is an identityref, so it can be rewritten to the stable iana-if-type:ethernetCsmacd. Without the schema, you can't tell an identityref from a string that happens to contain a colon.

Why not guess the types from the text? Because a guess is sometimes wrong, and wrong silently:

  • An interface named 0001, a string leaf that holds true, and a description of 1500 would all change type.
  • 64-bit counters such as in-octets would lose precision above 2⁵³. RFC 7951 encodes int64 and uint64 as strings for that reason.

Only the schema can say which values are which.

Writing: what you're missing without a schema ​

Here is an edit to the same interface, written in JSON with the description first:

ts
{
  'ietf-interfaces:interfaces': {
    interface: [
      {
        description: 'uplink',
        name: 'eth0',
        type: 'iana-if-type:ethernetCsmacd',
        enabled: false,
        'ietf-ip:ipv4': { mtu: 1500 }
      }
    ]
  }
}

Here is the XML it becomes without a schema, and with one:

xml
<!-- Without a schema -->
<interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces">
  <interface>
    <description>uplink</description>
    <name>eth0</name>
    <type>iana-if-type:ethernetCsmacd</type>
    <enabled>false</enabled>
    <ipv4 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip">
      <mtu>1500</mtu>
    </ipv4>
  </interface>
</interfaces>

<!-- With the schema -->
<interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces">
  <interface>
    <name>eth0</name>
    <description>uplink</description>
    <type
      xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type"
    >ianaift:ethernetCsmacd</type>
    <enabled>false</enabled>
    <ipv4 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip">
      <mtu>1500</mtu>
    </ipv4>
  </interface>
</interfaces>

The names and namespaces are the same in both, but a JSON object doesn't hold everything the XML has to carry:

  • Element order. A list key is the leaf, or leaves, that identify each entry of a list, like a primary key in a database: ietf-interfaces declares list interface { key "name"; … }, so each interface is identified by its name. RFC 7950 §7.8.5 requires an entry's keys to come first, in the order of its key statement. A JavaScript object keeps members in the order you wrote them, so without a schema the key name is sent after description. A server can reject that. If you write the keys first, in key-statement order, the schema-less XML is correct.
  • Namespace prefixes in values. type: 'iana-if-type:ethernetCsmacd' must be sent as ianaift:ethernetCsmacd, with xmlns:ianaift declared on the element. Sent as plain text, the value uses a prefix nobody declared, and the server rejects it as an invalid value. A description of 'note: check' must be left exactly as it is. Only the schema tells the two apart. instance-identifier values have the same problem.
  • Canonical values. A subtree filter's content-match value is compared with the text the device holds, which is the value's canonical form. One number can be written several ways: '01500' and '+1500' are both 1500, and the canonical form is 1500. The schema shows that mtu is a uint16, so the value is sent as 1500 and matches. Without the schema, '01500' could be a string, so it is sent as written. A server that compares the text then finds no match and returns nothing, with no error. Some servers parse the value with their own schema first and match anyway, but a filter typed by the schema matches on every server.
  • Mistakes found before sending. With the schema, a misspelt node, a value out of range, a missing list key or an edit to a config-false node is rejected locally. The error lists every problem with its path, and nothing reaches the server. Without the schema, you rely on the device's <rpc-error>, which usually reports one problem at a time in the device's own words.

What doesn't need a schema ​

The names of nodes don't. RFC 7951 names the module wherever it changes ('ietf-ip:ipv4'), so a map from module names to namespaces is enough to convert names. That map comes from the server's YANG library or hello. This is why the library can fall back to converting without a schema rather than failing outright.

InformationIn the XMLIn the JSONRead without a schemaWritten without a schema
Node names and namespacesnamespace URIsmodule names✅ converted, using the YANG library or hello; the namespace URI stands in for an unknown module✅ converted the same way; an unknown module is an error
List or container✗ (only by repetition)array or object✗ an array only when an element repeats✅ an array becomes repeated elements
Value types✗ (all text)number, boolean, [null], string✗ every value is a string, and an empty leaf or container is ""✅ sent as text
Identityref and instance-identifier valuesprefixes declared in each replymodule names✗ left with the reply's prefixes, whose declarations are lost✗ sent with a prefix nobody declared
List keys firstrequired✗ (member order is free)— (the server sends them first)✗ sent in the order you wrote them
Canonical valuescompared as text in a filterany form—✗ a filter value is sent as written, and may match nothing
Valid datachecked by the servernot checked—✗ the server finds the mistakes

Do you have to think about schemas? ​

Usually not. The library finds the schema itself: it reads the server's module list, fetches each module's text with <get-schema> or uses the bundled IETF modules, compiles only the modules your data uses, and keeps them for the session. You need to step in when:

  • the device can't supply its modules (no <get-schema>, as on a classic-CLI Nokia 7210): put the .yang files in a directory or a device profile;
  • you want the conversion to be strict: by default, data the library can't type is still converted schema-less, with a warning. Set requireSchema to make that an error instead;
  • you run many sessions: share a YangCache so each module set is compiled once.

If you'd rather not use a schema at all, use XML. It goes to the server unchanged, and the server does the checking.

Where JSON is accepted ​

OperationJSON
get(), getConfig(), nmda getData()filter, and the reply with format: 'json'
compare()filter
editConfig(), copyConfig({ source: { config } }), validate({ config }), nmda editData()config
withCandidate()'s txn.editConfig() and txn.getConfig()config, filter
configure()'s txn.merge(json), txn.replace(json)whole subtrees
read(), readValue()the result, by default
client.ops.run()the same arguments, for the operations above

format and json options are never sent to the server.

Reading JSON ​

With format: 'json', a read resolves with the data as RFC 7951 JSON, typed by the schema of the datastore it came from. The TypeScript return type follows the option: YangJsonObject with format: 'json', and XmlElement without it. The path API's read() and readValue() are the other way round: JSON is their default, and format: 'xml' gives the elements.

ts
const data = await client.ops.getConfig({ filter, format: 'json' });

The output follows RFC 7951:

YANGJSON
member namesmodule:name at the top level and wherever the module changes (an augmented node); a bare name otherwise
list, leaf-listalways an array, even with one entry
int8–int32, uint8–uint32number
int64, uint64, decimal64string ("18446744073709551615", "1.5")
booleantrue / false
empty[null]
identityref"module:identity"
string and everything elsethe text exactly, never trimmed

XML attributes that are YANG metadata become RFC 7952 annotations, always module-qualified: an "@" member inside a container or list entry, an "@name" member beside a leaf, and a parallel array for a leaf-list. NMDA origins (withOrigin: true) come out as "ietf-origin:origin": "ietf-origin:intended", and with-defaults report-all-tagged as "ietf-netconf-with-defaults:default": true.

JSON filters ​

A plain object as filter is a subtree filter written as RFC 7951 JSON:

ts
// The eth0 entry of the interface list
const filter = {
  'ietf-interfaces:interfaces': {
    interface: {
      name: 'eth0'
    }
  }
};

const state = await client.ops.get({ filter });
JSONFilter nodeExample
an objectcontainment node: only what's inside itinterface: { name: 'eth0' }
{} (or null) on a container or listselects the whole nodeinterface: {}: every interface, in full
null on a leaf or leaf-listselection node: return the leafmtu: null: return each entry's mtu
a value on a leafcontent-match node, sent in the schema's canonical form ('01500' for a uint16 is sent as 1500, '1.50' for a decimal64 as 1.5)name: 'eth0': only the eth0 entry
a list as one object, or an array of objectsone entry pattern eachinterface: [{ name: 'eth0' }, { name: 'eth1' }]: eth0 and eth1

A <get> filter is typed by the operational schema (state data allowed), a get-config filter by its source, and a get-data or compare filter by its datastore. The filter's format doesn't change the reply's format: add format: 'json' to get JSON back.

Writing JSON ​

A plain object as config is RFC 7951 JSON, so data read as JSON can be changed and sent straight back. The JSON is typed by the schema of the datastore being edited: values are sent in canonical form, list keys come first, identityref and instance-identifier values get their namespace prefixes declared, and config-false nodes are refused.

RFC 7952 metadata sets the edit attributes, as libyang reads them:

JSONSent as
"@": { "ietf-netconf:operation": "delete" } in a container or list entrync:operation="delete" (also merge, replace, create, remove)
"@mtu": { "ietf-netconf:operation": "remove" } beside a leaf, or a parallel array beside a leaf-listnc:operation on that leaf or entry
"mtu": null with a delete or remove operationan empty <mtu/>
"@": { "yang:insert": "after", "yang:key": "[name='alice']" } in an ordered-by-user list entryyang:insert, yang:key (RFC 7950 §7.8.6)
"@dns": [{ "yang:insert": "first" }] beside an ordered-by-user leaf-listyang:insert, with yang:value for before/after
ts
// Delete bob from the ops group,
// and move the ops-rules list entry after admin-rules
const config = {
  'ietf-netconf-acm:nacm': {
    groups: {
      group: [
        {
          name: 'ops',
          'user-name': ['bob'],
          '@user-name': [{ 'ietf-netconf:operation': 'delete' }]
        }
      ]
    },
    'rule-list': [
      {
        name: 'ops-rules',
        '@': {
          'yang:insert': 'after',
          'yang:key': "[name='admin-rules']"
        }
      }
    ]
  }
};

await client.ops.editConfig({ target: 'running', config });

Mistakes ​

JSON that doesn't match the schema is never sent. The call rejects with a YangDataError, whose issues list every problem with its data path: an unknown node, a value of the wrong type or out of range, a missing list key, or a config-false node in an edit. Cross-node rules (must, when, leafref and similar) are left to the device. See Errors.

json: { validate: false } sends a member that doesn't match as schema-less instead of rejecting it. json.datastore uses a different datastore's schema.

Where the schema comes from ​

The first JSON conversion compiles a schema from only the modules the data uses, plus the modules that augment or deviate them. The session keeps it for later conversions. The module list comes from the server's YANG library, which is read once per session (the path API reads it the same way), or else from its hello. The module text comes from these sources, in this order:

  1. connect({ yang: { modules } }): one or more directories of .yang files (named module.yang or module@revision.yang), or a @node-net/yang ModuleSource;
  2. the device profile's yang.modules;
  3. the IETF modules bundled with @node-net/yang (YANG library, NMDA, origin, with-defaults, NETCONF and the common types);
  4. <get-schema> (RFC 6022), when the server supports it.

A device without <get-schema>, such as a classic-CLI Nokia 7210, needs its modules supplied. Until they are, its JSON reads are schema-less. The 7210's alu-conf-r13 modules aren't bundled, because they aren't published openly: take them from the device's software release, then give their directory as yang.modules in connect(), or in a profile that extends nokia-7210-sas. Clients given the same YangCache share fetched module text and compiled schemas, so a service with many sessions compiles each module set once:

ts
import { connect, YangCache } from '@node-net/netconf';

const cache = new YangCache();
const client = await connect({
  transport,
  yang: { modules: './yang/nokia', cache },
});

Without a schema ​

Each top-level member is converted separately. A member can't be typed when no module the server lists has its namespace, when its module's text can't be found, or when it doesn't match the schema. By default it is then converted schema-less and reported in the session's warning event and the logger. The warning says how to supply the module when one is missing. For a read, it also suggests format: 'xml', which returns the data as the server sent it. The other members are still fully typed.

Read as JSON, a schema-less member has:

  • names qualified as above, with the namespace URI in place of an unknown module;
  • an array only where an element repeats, so a one-entry list is not an array;
  • every value as its text, verbatim, and an element with no content as "", whether it's an empty leaf (typed: [null]), an empty container (typed: {}) or an empty string;
  • identityref and instance-identifier values with the reply's prefixes, whose xmlns declarations aren't carried into the JSON.

Sent as JSON, its names map to namespaces from the YANG library, hello, profile or connect({ namespaces }). A module whose namespace none of those knows is an error, and so is a top-level member with no module name. Arrays become repeated elements, null an empty element, and values are sent as their text. That loses less than a read, but:

  • prefixes in identityref and instance-identifier values are sent undeclared;
  • members are sent in the order written, so list keys must be written first to be sent first;
  • a filter's content-match values are sent as written, not in canonical form;
  • nothing is checked before the server sees it.

Why the schema matters shows what each of those costs. To refuse schema-less conversion, set requireSchema. The call then rejects with a SchemaUnavailableError (no schema could be built) or a YangDataError (the data doesn't match the schema). On a read, the error's message suggests format: 'xml':

ts
await client.ops.get({ format: 'json', json: { requireSchema: true } });
await client.ops.editConfig({
  target: 'running',
  config,
  json: { requireSchema: true },
});

requireSchema can also be set for a whole client (connect({ yang: { requireSchema: true } })) or in a device profile. A call's own setting overrides the client's, and the client's overrides the profile's.

Converting directly ​

client.content.toJson() and client.content.toXml() make these conversions on their own, for data that didn't come from (or isn't going to) an operation above, such as the result of client.session.rpc() or a notification's content. See Converting between XML and JSON. Extensions get toXml() as ctx.toXml.

Released under the MIT Licence.