OVN Interconnection¶

This document provides a guide for interconnecting multiple OVN deployements with OVN managed tunneling. More details about the OVN Interconnectiong design can be found in ovn-architecture(7) manpage.

This document assumes two or more OVN deployments are setup and runs normally, possibly at different data-centers, and the gateway chassises of each OVN are with IP addresses that are reachable between each other.

Setup Interconnection Databases¶

To interconnect different OVNs, you need to create global OVSDB databases that store interconnection data. The databases can be setup on any nodes that are accessible from all the central nodes of each OVN deployment. It is recommended that the global databases are setup with HA, with nodes in different avaialbility zones, to avoid single point of failure.

Install OVN packages on each global database node.
Start OVN IC-NB and IC-SB databases.

On each global database node
```
$ ovn-ctl [options] start_ic_ovsdb
```
Options depends on the HA mode you use. To start standalone mode with TCP connections, use
```
$ ovn-ctl --db-ic-nb-create-insecure-remote=yes \
          --db-ic-sb-create-insecure-remote=yes start_ic_ovsdb
```
This command starts IC database servers that accept both unix socket and TCP connections. For other modes, see more details with
```
$ ovn-ctl --help
```

Register OVN to Interconnection Databases¶

For each OVN deployment, set an availability zone name

$ ovn-nbctl set NB_Global . name=<availability zone name>

The name should be unique across all OVN deployments, e.g. ovn-east, ovn-west, etc.

For each OVN deployment, start the ovn-ic daemon on central nodes

$ ovn-ctl --ovn-ic-nb-db=<IC-NB> --ovn-ic-sb-db=<IC-SB> \
          --ovn-northd-nb-db=<NB> --ovn-northd-sb-db=<SB> [more options] start_ic

An example of <IC-NB> is tcp:<global db hostname>:6645, or for clustered DB: tcp:<node1>:6645,tcp:<node2>:6645,tcp:<node3>:6645. <IC-SB> is similar, but usually with a different port number, typically, 6646.

For <NB> and <SB>, use same connection methods as for starting northd.

Verify each OVN registration from global IC-SB database, using ovn-ic-sbctl, either on a global DB node or other nodes but with property DB connection method specified in options

$ ovn-ic-sbctl show

Configure Gateways¶

For each OVN deployment, specify some chassises as interconnection gateways. The number of gateways you need depends on the scale and bandwidth you need for the traffic between the OVN deployments.

For a node to work as an interconnection gateway, it must firstly be installed and configured as a regular OVN chassis, with OVS and ovn-controller running. To make a chassis as an interconnection gateway, simply run the command on the chassis

$ ovs-vsctl set open_vswitch . external_ids:ovn-is-interconn=true

After configuring gateways, verify from the global IC-SB database

$ ovn-ic-sbctl show

Create Transit Logical Switches¶

Transit Logical Switches, or Transit Switches, are virtual switches for connecting logical routers in different OVN setups.

$ ovn-ic-nbctl ts-add <name>

After creating a transit switch, it can be seen from each OVN deployment’s Northbound database, which can be seen using

$ ovn-nbctl find logical_switch other_config:interconn-ts=<name>

You will also see it with simply ovn-nbctl ls-list.

If there are multiple tenants that require traffic being isolated from each other, then multiple transit switches can be created accordingly.

Connect Logical Routers to Transit Switches¶

Connect logical routers from each OVN deployment to the desired transit switches just as if they are regular logical switches, which includes below steps (from each OVN, for each logical router you want to connect).

Assume a transit switch named ts1 is already created in IC-NB and a logical router lr1 created in current OVN deployment.

Create a logical router port.

$ ovn-nbctl lrp-add lr1 lrp-lr1-ts1 aa:aa:aa:aa:aa:01 169.254.100.1/24

(The mac and IP are examples.)

Create a logical switch port on the transit switch and peer with the logical router port.

$ ovn-nbctl lsp-add ts1 lsp-ts1-lr1 -- \
        lsp-set-addresses lsp-ts1-lr1 router -- \
        lsp-set-type lsp-ts1-lr1 router -- \
        lsp-set-options lsp-ts1-lr1 router-port=lrp-lr1-ts1

Assign gateway(s) for the logical router port.
```
$ ovn-nbctl lrp-set-gateway-chassis lrp-lr1-ts1 <gateway name> [priority]
```
Optionally, you can assign more gateways and specify priorities, to achieve HA, just as usual for a distributed gateway port.

Similarly in another OVN deployment, you can connect a logical router (e.g. lr2) to the same transit switch the same way, with a different IP (e.g. 169.254.100.2) on the same subnet.

The ports connected to transit switches will be automatically populated to IC-SB database, which can be verified by

$ ovn-ic-sbctl show

Create Static Routes¶

Now that you have all the physical and logical topologies ready, simply create static routes between the OVN deployments so that packets can be forwarded by the logical routers through transit switches to the remote OVN.

For example, in ovn-east, there are workloads using 10.0.1.0/24 under lr1, and in ovn-west, there are workloads using 10.0.2.0/24 under lr2.

In ovn-east, add below route

$ ovn-nbctl lr-route-add lr1 10.0.2.0/24 169.254.100.2

In ovn-west, add below route

$ ovn-nbctl lr-route-add lr2 10.0.1.0/24 169.254.100.1

Now the traffic should be able to go through between the workloads through tunnels crossing gateway nodes of ovn-east and ovn-west.

Route Advertisement¶

Alternatively, you can avoid the above manual static route configuration by enabling route advertisement and learning on each OVN deployment

$ ovn-nbctl set NB_Global . options:ic-route-adv=true \
                            options:ic-route-learn=true

With this setting, the above routes will be automatically learned and configured in Northbound DB in each deployment. For example, in ovn-east, you will see the route

$ ovn-nbctl lr-route-list lr1
IPv4 Routes
             10.0.2.0/24             169.254.100.2 dst-ip (learned)

In ovn-west you will see

$ ovn-nbctl lr-route-list lr2
IPv4 Routes
             10.0.1.0/24             169.254.100.1 dst-ip (learned)

Static routes configured in the routers can be advertised and learned as well. For more details of router advertisement and its configure options, please see <code>ovn-nb</code>(5).