VIF Plug Providers

Traditionally it has been the CMSes responsibility to create VIFs as part of instance life cycle, and subsequently manage plug/unplug operations on the integration bridge following the conventions described in the Open vSwitch Integration Guide for mapping of VIFs to OVN logical port.

With the advent of NICs connected to multiple distinct CPUs we can have a topology where the instance runs on one host and Open vSwitch and OVN runs on a different host, the smartnic control plane CPU. The host facing interfaces will be visible to Open vSwitch and OVN as representor ports.

The actions necessary for plugging and unplugging the representor port in Open vSwitch running on the smartnic control plane CPU would be the same for every CMS.

Instead of every CMS having to develop their own version of an agent to do the plugging, we provide a pluggable infrastructure in OVN that allows the ovn-controller to perform the plugging on CMS direction.

Hardware or platform specific details for initialization and lookup of representor ports is provided by an plugging provider library hosted inside or outside the core OVN repository, and linked at OVN build time.

Life Cycle of an OVN plugged VIF

CMS creates a record in the OVN Northbound Logical_Switch_Port table with the options column containing the vif-plug-type key with a value corresponding to the const char *type provided by the VIF plug provider implementation as well as a requested-chassis key with a value pointing at the name or hostname of the chassis it wants the VIF plugged on. Additional VIF plug provider specific key/value pairs must be provided for successful lookup.
ovn-northd looks up the name or hostname provided in the requested-chassis option and fills the OVN Southbound Port_Binding requested_chassis column, it also copies relevant options over to the Port_Binding record.
ovn-controller monitors Southbound Port_Binding entries with a requested_chassis column pointing at its chassis UUID. When it encounters an entry with option vif-plug-type and it has registered a VIF plug provider matching that type, it will act on it even if no local binding exists yet.
It will fill the struct vif_plug_port_ctx_in as defined in lib/vif-plug.h with op_type set to ‘PLUG_OP_CREATE’ and make a call to the VIF plug providers vif_plug_port_prepare callback function. VIF plug provider performs lookup and fills the struct vif_plug_port_ctx_out as defined in lib/vif-plug.h.
ovn-controller creates a port and interface record in the local OVSDB using the details provided by the VIF plug provider and also adds external-ids:iface-id with value matching the logical port name and external-ids:ovn-plugged with value matching the logical port vif-plug-type. When the port creation is done a call will first be made to the VIF plug providers vif_plug_port_finish function and then to the vif_plug_port_ctx_destroy function to free any memory allocated by the VIF plug implementation.
The Open vSwitch vswitchd will assign a ofport to the newly created interface and on the next ovn-controller main loop iteration flows will be installed.
On each main loop iteration the ovn-controller will in addition to normal flow processing make a call to the VIF plug provider again similar to the first creation in case anything needs updating for the interface record.
The port will be unplugged when an event occurs which would make the ovn-controller release a logical port, for example the Logical_Switch_Port and Port_Binding entry disappearing from the database or its requested_chassis column being pointed to a different chassis.

The VIF plug provider interface

The interface between internals of OVN and a VIF plug provider is a set of callbacks as defined by the struct vif_plug_class in lib/vif-plug-provider.h.

It is important to note that these callbacks will be called in the critical path of the ovn-controller processing loop, so care must be taken to make the implementation as efficient as possible, and under no circumstance can any of the callback functions make calls that block.

On ovn-controller startup, VIF plug providers made available at build time will be registered by the identifier provided in the const char *type pointer, at this time the init function pointer will be called if it is non-NULL.

> Note: apart from the const char *type pointer, no attempt will be made: to access VIF plug provider data or functions before the call to the init has been made.

On ovn-controller exit, the VIF plug providers registered in the above mentioned procedure will have their destroy function pointer called if it is non-NULL.

If the VIF plug provider has internal lookup tables that need to be maintained they can define a run function which will be called as part of the ovn-controller main loop. If there are any changes encountered the function should return ‘true’ to signal that further processing is necessary, ‘false’ otherwise.

On update of Interface records the ovn-controller will pass on a sset to the ovsport_update_iface function containing options the plug implementation finds pertinent to maintain for successful operation. This sset is retrieved by making a call to the plug implementation vif_plug_get_maintained_iface_options function pointer if it is non-NULL. This allows presence of other users of the OVSDB maintaining a different set of options on the same set of Interface records without wiping out their changes.

Before creating or updating an existing interface record the VIF plug provider vif_plug_port_prepare function pointer will be called with valid pointers to struct vif_plug_port_ctx_in and struct vif_plug_port_ctx_out data structures. If the VIF plug provider implementation is able to perform lookup it should fill the struct vif_plug_port_ctx_out data structure and return ‘true’. The ovn-controller will then create or update the port/interface records and then call vif_plug_port_finish when the transactions commits and vif_plug_port_ctx_destroy to free any allocated memory. If the VIF plug provider implementation is unable to perform lookup or prepare the desired resource at this time, it should return ‘false’ which will tell the ovn-controller to not plug the port, in this case it will not call vif_plug_port_finish nor vif_plug_port_ctx_destroy.

> Note: The VIF plug provider implementation should exhaust all: non-blocking options to succeed with lookup from within the vif_plug_port_prepare handler, including refreshing lookup tables if necessary.

Before removing port and interface records previously plugged by the ovn-controller as identified by presence of the Interface external-ids:ovn-plugged key, the ovn-controller will look up the vif-plug-type from external-ids:ovn-plugged, fill struct vif_plug_port_ctx_in with op_type set to ‘PLUG_OP_REMOVE’ and make a call to vif_plug_port_prepare. After the port and interface has been removed a call will be made to vif_plug_port_finish. Both calls will be made with the pointer to vif_plug_port_ctx_out set to ‘NULL’, and no call will be made to vif_plug_port_ctx_destroy.

Building with in-tree VIF plug providers

VIF plug providers hosted in the OVN repository live under lib/vif-plug-providers:

To enable them, provide the –enable-vif-plug-providers command line option to the configure script when building OVN.

Building with an externally provided VIF plug provider

There is also infrastructure in place to support linking OVN with an externally built VIF plug provider.

This external VIF plug provider must define a NULL-terminated array of pointers to struct vif_plug_class data structures named vif_plug_provider_classes. Example:

const struct vif_plug_class *vif_plug_provider_classes[] = {
    &vif_plug_foo,
    NULL,
};

The name of the repository for the external VIF plug provider should be the same as the name of the library it produces, and the built library artifact should be placed in lib/.libs. Example:

ovn-vif-foo/
ovn-vif-foo/lib/.libs/libovn-vif-foo.la

To enable such a VIF plug provider provide the –with-vif-plug-provider=/path/to/ovn-vif-foo command line option to the configure script when building OVN.