#UniConfig NETCONF
#Overview
NETCONF is an Internet Engineering Task Force (IETF) protocol used for configuration and monitoring of devices in a network. It can be used to “create, recover, update, and delete configurations of network devices”. The base NETCONF protocol is described in RFC-6241.
NETCONF operations are overlaid on the Remote Procedure Call (RPC) layer and may be described in either XML or JSON.
#NETCONF southbound plugin
#Introduction to southbound plugin and netconf-connectors
The NETCONF southbound plugin is capable of connecting to remote NETCONF devices and exposing their configuration/operational datastores, RPCs and notifications as MD-SAL mount points. These mount points allow applications and remote users (over RESTCONF) to interact with the mounted devices.
In terms of RFCs, the southbound plugin supports:
- Network Configuration Protocol (NETCONF) - RFC-6241
- NETCONF Event Notifications - RFC-5277
- YANG Module for NETCONF Monitoring - RFC-6022
- YANG Module Library - draft-ietf-netconf-yang-library-06
NETCONF is fully model-driven (utilizing the YANG modelling language) so in addition to the above RFCs, it supports any data/RPC/notifications described by a YANG model that is implemented by the device.
By mounting of NETCONF device a new netconf-connector is created. This connector is responsible for:
- keeping state of NETCONF session between NETCONF client that resides on FRINX UniConfig distribution and NETCONF server (remote network device)
- sending / receiving of NETCONF RPCs that are used for reading / configuration of network device
- interpreting of NETCONF RPCs by mapping of their content using loaded device-specific YANG schemas
There are 2 ways for configuring a new netconf-connector: NETCONF or RESTCONF. This guide focuses on using RESTCONF.
#Spawning of netconf-connectors while the controller is running
To configure a new netconf-connector (NETCONF mount-point) you need to create a node in configuration data-store under 'topology-netconf'. Adding of new node under NETCONF topology automatically triggers data-change-event that at the end triggers mounting process of the NETCONF device. The following example shows how to mount device with node name 'example' (make sure that the same node name is specified in URI and request body under 'node-id' leaf).
This spawns a new netconf-connector with name 'example' which tries to connect to the NETCONF device at '192.168.1.100' and port '22'. Both username and password are set to 'test' and SSH is used as channel for transporting of NETCONF RPCs (if 'tcp-only' leaf is set to 'true', NETCONF application protocol is running directly on top of the TCP protocol).
Right after the new netconf-connector is created, NETCONF layer writes some useful metadata into the operational data-store of MD-SAL under the network-topology subtree. This metadata can be found at:
Information about connection status, device capabilities, etc. can be found there.
You can check the configuration of device by accessing of 'yang-ext:mount' container that is created under every mounted NETCONF node. The new netconf-connector will now be present there. Just invoke:
The response will contain the whole configuration of NETCONF device. You can fetch smaller slice of configuration using more specific URLs under 'yang-ext:mount' too.
#Authentication with private/public key
This type of authentication is used when you want to connect to the NETCONF device via private/public key, it is necessary to save public key into device, then put private key into UniConfig and when trying to configure NETCONF mount-point to connect via ssh key and not password.
To accomplish that, follow these steps :
1. Generate private/public key-pair on your local machine
2. Change .pub format into .bin format
3. Copy public key into device directory. Password of the device will be required.
4. (Optional) Check if the public key is on device
5. Import public key to device
6. Log in with private key to device NETCONF subsystem. Passphrase for key will be required.
7. Start UniConfig and insert keystore with private key into it.
RPC request:
8. Create mount-point with key-id
RPC request:
Delete public key
Login to device, remove rsa public key and after that, it is also possible to delete key from device directory.
#PKI Data persistence in NETCONF
- PKI data is used for authentication of NETCONF sessions with the provided RSA private key. The corresponding public key must be stored on the device side.
- Keys are identified using a unique 'key-id'. This key identifier can be specified in the NETCONF installation request.
- Keys can be managed using the 'remove-keystore-entry' and 'add-keystore-entry' operations. These RPC calls are part of the UniConfig transaction. Changes are not applied until they are committed by the user or the immediate commit model is used to invoke the operation.
- Keys are stored in the UniConfig database. In a clustered environment, all nodes share the same set of keys.
#Registration of the new key
The following request demonstrates how to register a new RSA private key with a key-id of 'key1'. The private key must be specified in the PKCS#8 format. The passphrase is optional and must be specified only if the private key is encrypted.
Multiple keys can be registered at once if the user provides a list of the 'key-credential' in the input.
#Removing of the existing key
The following example shows how to remove the existing key 'key1' from UniConfig. It is possible to remove multiple keys at once.
#Reading list of the existing keys
The following example shows how to read list of the existing keys from UniConfig.
Note: Both 'passphrase' and 'private-key' are additionally encrypted by the UniConfig encryption system to protect confidential data.
#Keepalive settings
If the NETCONF session haven't been created yet, the session is tried to be established only within maximum connection timeout. If this timeout expires before NETCONF session is established, underlay NETCONF channel is closed (reconnection strategy will not be started). After the NETCONF session has been successfully created, there are two techniques how the connection state is kept alive:
- TCP acknowledgements - NETCONF is running on top of the TCP protocol that can handle dropped packets by decreasing of window size and resending of lost TCP segments. Working TCP connection doesn't imply working state of the application layer (NETCONF session) - keepalive messages are required too.
- Explicit NETCONF keepalive messages - Keepalive messages test whether NETCONF server is alive - server responds to keepalive messages within NETCONF RPC timeout.
If TCP connection is dropped or NETCONF server doesn't respond within keepalive timeout, NETCONF launches reconnection strategy. To summarize it all, there are 3 configurable parameters that can be set in mount-request:
- Initial connection timeout [seconds] - Specifies timeout in milliseconds after which initial connection to the NETCONF server must be established. By default, the value is set 20 s.
- Keepalive delay [seconds] - Delay between sending of keepalive RPC messages to the NETCONF server. Keepalive messages test state of the NETCONF session (application layer) - whether remote side is able to respond to RPC messages. Default keepalive delay is 120 seconds.
- Request transaction timeout [seconds] - Timeout for blocking RPC operations within transactions. Southbound plugin stops to wait for RPC reply after this timeout expires. By default, it is set to 60 s.
Example with set keepalive parameters at creation of NETCONF mount-point (connection timeout, keepalive delay and request timeout):
#Reconnection strategy
Reconnection strategies are used for recovering of the lost connection to the NETCONF server. The behaviour of the reconnection can be described by 3 configurable mount-request parameters:
- Maximum number of connection attempts [count] - Maximum number of initial connection retries. This is used when no connection is yet established for given node inside uniconfig transaction; when it is reached, the NETCONF won't try to connect to device anymore. By default, this value is set to 1, non-positive value or null is interpreted as infinity.
- Maximum number of reconnection attempts [count] - Maximum number of reconnection retries. This is used when connection was already successfully established and ongoing (not yet closed), but it dropped (from device side); when it is reached, the NETCONF won't try to reconnect to device anymore. By default, this value is set to 0, non-positive value or null is interpreted as infinity.
- Initial timeout between attempts [seconds] - The first timeout between reconnection attempts in milliseconds. The default timeout value is set to 2000 ms.
- Reconnection attempts multiplier [factor] - After each reconnection attempt, the delay between reconnection attempts is multiplied by this factor. By default, it is set to 1.5. This means that the next delay between attempts will be 3 s, then it will be 4,5 s, etc.
Example with set reconnection parameters at creation of NETCONF mount-point - maximum connection attempts, initial delay between attempts and sleep factor:
#Local NETCONF cache repositories
The netconf-connector in OpenDaylight relies on 'ietf-netconf-monitoring' support when connecting to remote NETCONF device. The 'ietf-netconf-monitoring' feature allows netconf-connector to list and download all YANG schemas that are used by the device. These YANG schemas are afterwards used by NETCONF southbound plugin for interpretation of RPCs. The following rules apply for maintaining of local NETCONF cache repositories:
- By default, for each device type, the separate local repository is prepared.
- All NETCONF repositories are backed up by separate sub-directory under 'cache' directory of UniConfig Distribution.
- NETCONF device types are distinguished by unique set of YANG source identifiers - module names and revision numbers. For example, if 2 NETCONF devices differ only in revision of one YANG schema, these NETCONF devices are recognized to have different device types.
- Format of the name of generated NETCONF cache directory at runtime is 'schema_id', where 'id' represents unique integer computed from hash of all source identifiers. This generation of cache directory name is launched only at mounting of new NETCONF device and only if another directory with the same set of source identifiers haven't been registered yet.
- You can still manually provide NETCONF cache directories with another format before starting of UniConfig Distribution or at runtime - such directories don't have to follow 'schema_id' format.
The NETCONF repository can be registered in 3 ways:
- Implicitly by mounting of NETCONF device that has NETCONF monitoring capability and another devices with the same type hasn't already been mounted.
- At booting of FRINX UniConfig distribution, all existing sub-directories of 'cache' root directory are registered as separate NETCONF repositories.
- At runtime, by invocation of 'schema-resources:register-repository' RPC.
Already registered schema repositories can be listed using following request:
It should return list of ODL nodes in cluster with list of all loaded repositories. Each repository have associated list of source identifiers. See the following example of GET request output:
#Local Netconf default cache repository
Before booting of FRINX UniConfig, the user can put the 'default' repository in the ‘cache’ directory. This directory should contain the most frequently missing sources. As mentioned above, if the device supports ‘ietf-netconf-monitoring’ and there is no directory in the 'cache' with all sources that the device requires, then NETCONF will generate directory with name ‘schema_id’, where ‘id’ represents unique integer. The generated repository may not contain all required schemas because device may not provide them. In such case, the missing sources will be searched in the 'default' repository and if sources will be located there, generated repository will be supplemented by the missing sources. In general, there are 2 situations that can occur:
- Missing imports
The device requires and provides a resource which for its work requires additional resources that are not covered by provided resources.
- Source that is not covered by provided sources
The device requires but does not provide a specific source.
note Using the 'default' directory in the 'cache' directory is optional.
#Connecting to a device not supporting NETCONF monitoring
NETCONF connector can only communicate with a device if it knows the set of used schemas (or at least a subset). However, some devices use YANG models internally but do not support NETCONF monitoring. Netconf-connector can also communicate with these devices, but you must load required YANG models manually. In general, there are 2 situations you might encounter:
- NETCONF device does not support 'ietf-netconf-monitoring' but it does list all its YANG models as capabilities in HELLO message
This could be a device that internally uses, for example, 'ietf-inet-types' YANG model with revision '2010-09-24'. In the HELLO message, that is sent from this device, there is this capability reported as the following string (other YANG schemas can be reported as capabilities in the similar format):
The format of the capability string is following:
- [NAMESPACE] - Namespace that is specified in the YANG schema.
- [MODULE_NAME] - Name of the YANG module.
- [REVISION] - The newest revision that is specified in the YANG schema (it should be specified as the first one in the file). note Revision number is not mandatory (YANG model doesn't have to contain revision number) - then, the capability is specified without the '&' and revision too. For such devices you have to side load all device YANG models into separate sub-directory under 'cache' directory (you can choose random name for this directory, but directory must contain only YANG files of one device type).
- NETCONF device does not support 'ietf-netconf-monitoring' and it does NOT list its YANG models as capabilities in HELLO message
Compared to device that lists its YANG models in HELLO message, in this case there would be no specified capabilities in the HELLO message. This type of device basically provides no information about the YANG schemas it uses so its up to the user of OpenDaylight to properly configure netconf-connector for this device. Netconf-connector has an optional configuration attribute called 'yang-module-capabilities' and this attribute can contain a list of 'yang-module-based' capabilities. By setting this configuration attribute, it is possible to override the 'yang-module-based' capabilities reported in HELLO message of the device. To do this, we need to mount NETCONF device or modify the configuration of existing netconf-connector by adding the configuration snippet with explicitly specified capabilities (it needs to be added next to the address, port, username etc. configuration elements). The following example shows explicit specification of 6 capabilities:
Remember to also put the YANG schemas into the cache folder like in the case 1.
#Registration or refreshing of NETCONF cache repository using RPC
This RPC can be used for registration of new NETCONF cache repository or updating of NETCONF cache repository. This is useful when user wants to add new NETCONF cache repository at runtime of FRINX UniConfig distribution for device that doesn't support 'ietf-netconf-monitoring' feature. It can also be used for refreshing of repository contents (YANG schemas) at runtime.
The following example shows how to register a NETCONF repository with name 'example-repository'. The name of the provided repository must equal to name of the directory which contains YANG schemas.
If the repository registration or refreshing process ends successfully, the output contains just set 'status' leaf with 'success' value:
On the other side, if the directory with input 'repository-name' does not exist, directory doesn't contain any YANG files, or schema context cannot be built using provided YANG sources the response body will contain 'failed' 'status' and set 'error-message'. For example, non-existing directory name produces following response:
Constraints:
- Only the single repository can be registered using one RPC request.
- Removal of registered repositories is not supported for now.
#Reconfiguring netconf-connector while the controller is running
It is possible to change the configuration of an already mounted NETCONF device while the whole controller is running. This example will continue where the last left off and will change the configuration for the existing netconf-connector after it was spawned. Using one RESTCONF request, we will change both username and password for the netconf-connector.
To update an existing netconf-connector you need to send following request to RESTCONF:
Since a PUT is a replace operation, the whole configuration must be specified along with the new values for username and password. This should result in a '2xx' response and the instance of netconf-connector called 'example' will be reconfigured to use username 'bob' and password 'passwd'. New configuration can be verified by executing:
With new configuration, the old connection will be closed and a new one established.
#Destroying of netconf-connector
Using RESTCONF one can also destroy an instance of a netconf-connector - NETCONF connection will be dropped and all resources associated with NETCONF mount-point on NETCONF layer will be cleaned (both CONFIGURATION and OPERATIONAL data-store information). To do this, simply issue a request to following URL:
The last element of the URL is the name of the mount-point.
#NETCONF TESTTOOL
#Testtool overview
NETCONF testtool is the Java application that:
- Can be used for simulation of 1 or more NETCONF devices (it is suitable for scale testing).
- Uses core implementation of NETCONF NORTHBOUND server.
- Provides broad configuration options of simulated devices.
- Supports YANG notifications.
NETCONF testtool is available at netconf repository of ODL (<https://git.opendaylight.org/gerrit/admin/repos/netconf under 'netconf/tools/netconf-testtool' module. After building of this module using maven (just invoke command 'mvn clean install' in this directory), the java executable can be found in appeared 'target' directory with name 'netconf-testtool-[version]-executable.jar' (version placeholder depends on used release).
Up-to-date NETCONF testtool is also available at frinx artifactory https://artifactory.frinx.io/.
Up-to-date NETCONF testtool is also available at DockerHub as Docker Image.
#Starting of the NETCONF testtool
After NETCONF testtool has been built, it can be used using the following command:
Please see used fields and placeholders explained below ...
Multi line example with values replacing placeholders e.g.:
Usually no need to debug (it can be omitted) and long jar executable can be renamed:
One liner example:
The following snippet shows output from successfully simulated NETCONF device (notice the last line that shows hint, on which TCP ports simulated devices have been started):
You can check occupied ports and netconf-testool process like this:
Description of some of the used fields and placeholders:
- SCHEMAS-DIR - Path to the directory that contains YANG schemas used for simulation of all NETCONF devices.
- DEVICE-COUNT - Number of NETCONF devices that should be simulated at once.
- ENABLE-DEBUGGING - It should be set to 'true', if you want to see detailed debugging messages from simulation of NETCONF device (for example, received and sent RPC messages); otherwise it should be set to 'false' (INFO logging level is used).
- STARTING-PORT - The first TCP port on which the first simulated device will listen on - other simulated devices will reserve next TCP ports in order by incrementing of this value.
- SSH - It should be set to 'true' if NETCONF session should be created on top of SSH session. If it is set to 'false', TCP is used as carrier protocol.
- MD-SAL - Whether to use md-sal datastore ('true') instead of default simulated datastore ('false').
All configurable parameters can be fetched using help modifier:
e.g.
Notes:
- while you use --md-sal true devices are started with empty datastore. You can put initial config via netconf session or via uniconfig operations.
- --md-sal true --md-sal-persistent true will preserves datastore content across netconf sessions
- --initial-config-xml-file data.xml overrides --md-sal true and send hardcoded response to get-config.
- --notification-file notif.xml notification support
#Starting of the NETCONF testtool from Docker Image
- download docker image using download command
- run it using docker run command
- you can use options of Netconf test tool starting script
note
To set value of xmx is required to separate -x or --xmx option and value of xmx with space.
When using xmx setting option together with override option, it is required to use xmx setting option first before override.
All options and their values should be separated with space.
Setting to enable ssh or md-sal requires only option ssh or md-sal to be present to enable these functions and no other value is required.
When override is used, then it is not possible to set any other option from simple interface except Java specific options, like xmx and partially also debug.
After override option there should be Netconf test tool complex interface options.
If you want to use external files like schemas or init config file, you should first mount them using -v option of docker.
You should use --publish option to publish ports if you want to use them.
You can use --rm option after docker run to automatically remove docker container after it is stopped.
- you can also use help option to show how to use it using -h or --help
- you can also override simple interface of starting script and use all options of test tool directly
note
If you override starting script options, you should set Java heap size using -x or --xmx option before --override option.
You could not use any other starting script options except xmx and debug option.
If you override starting script options, --debug option activate only Java and shell debug mode, but not debug mode of test tool itself.
If you want to start debug mode of netconf test tool, you should use debug options of Netconf test tool directly.
- you can also override docker entrypoint and run netconf test tool by custom way using all options including java options and run like bellow
and in docker container shell:
- in custom way of running test tool there are these files, placed in /opt folder
#Prepare init config via Uniconfig for simulated netconf-testtool device
- start Uniconfig - in cache folder there can be present the folder with device yang models to preload them faster
- start netconf-testool with schema-dir folder of device yang models
- install device
- send init config to the netconf-testtool device via uniconfig
#Example of notification file
#Increasing the maximum number of opened files
Since NETCONF testtool can be used for simulation of large number of NETCONF devices, it requires opening a lot of TCP sockets that listen on different TCP ports. In Linux systems TCP socket is also represented as file - from this reason such simulations can easily exhaust configured limit of maximum number of opened files. Then, if the buffering file for connection cannot be created on time it can cause continuous reconnection attempts.
Usually, the default soft limit for maximum number of opened files is set to 1024 (reaching this limit should produce warnings in logging messages) and hard limit to 4096 (it cannot be exceeded). For setting of custom soft and hard limits you must modify the following lines in "/etc/security/limits.conf" file:
Replace '[user-name]' by login-name of the user under whom you start NETCONF test-tool.
You can check the current limits using following commands:
Soft limit '4096' and hard limit '10240' should be enough, but it also depends on occupation by other applications and operating system too).
note Configured value should not reach the one that applies for all users - "cat /proc/sys/fs/file-max".
see also /etc/sysctl.conf:
#How does the FRINX UniConfig distribution use NETCONF?
FRINX UniConfig uses a NETCONF southbound connectors to communicate with downstream NETCONF-enabled devices. There are three options:
- uniconfig-native - Using of raw device models for interaction with devices and still using Uniconfig transactions.
- translation units - Translation units that map OpenConfig models to device models and vice-versa can be used for configuration of NETCONF devices using OpenConfig models.
- direct using of Netconf mount-points - Modification of data under NETCONF mount-point but without option to use Uniconfig RPCs (data is exposed under 'yang-ext:mount' container).
#UniConfig-native NETCONF
UniConfig Native allows to communicate with network devices using their native YANG data models (e.g.: Cisco YANG models, JunOS YANG models, Calix YANG models, CableLabs YANG models, SROS YANG models, ...) to manage configurations. With UniConfig Native is possible to mount devices through NETCONF, sync configurations in their native format and manage those devices without the need to develop translation units. Here are some examples of NETCONF Native installation.