VPP-651: Ensure sw_if_index to node mapping for L2 output path is only done via l2output_main.next_nodes

Before this commit, several output features that happen to be the
last in the list of features to be executed, send the packets directly
to <interfaceName>-output. To do this, they use l2_output_dispatch,
which builds a list of sw_if_index to next index mappings.

When interfaces are deleted and the new interfaces are created,
these mappings become stale, and cause the packets being sent to wrong
interface output nodes.

This patch (thanks John Lo for the brilliant idea!) adds a feature node "output",
whose sole purpose is dispatching the packets to the correct interface output
nodes. To do that, it uses the l2output_main.next_nodes, which is already
taken care of for the case of the sw_if_index reuse, so this makes the dependent
features all work correctly.

Since this changes the packet path, for the features that were always the last ones
it has triggered a side problem of the output feat_next_node_index not being properly
initalized. These two users are l2-output-classify node and the output nodes belonging
to the acl-plugin.

For the first one the less invasive fix is just to initialize that field.
For the acl-plugin nodes, rewrite the affected part of the code to use
feat_bitmap_get_next_node_index since this is essentially what the conditional
in l2_output_dispatch does, and fix the compiler warnings generated.

This fix was first made in stable/1701 under commit e7dcee4027854b0ad076101471afdfff67eb9011.

Change-Id: I32e876ab1e1d498cf0854c19c6318dcf59a93805
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>

3 files changed, 17 insertions, 3 deletions

diff --git a/src/vnet/l2/l2_input.c b/src/vnet/l2/l2_input.c
index fbd75f222e0..3aa5331bf7a 100644
--- a/src/vnet/l2/l2_input.c
+++ b/src/vnet/l2/l2_input.c
@@ -703,10 +703,11 @@ set_int_l2_mode (vlib_main_t * vm, vnet_main_t * vnet_main,	/*           */
 	  shg = 0;		/* not used in xconnect */
 	}
 
-      /* set up split-horizon group */
+      /* set up split-horizon group and set output feature bit */
       config->shg = shg;
       out_config = l2output_intf_config (sw_if_index);
       out_config->shg = shg;
+      out_config->feature_bitmap |= L2OUTPUT_FEAT_OUTPUT;
 
       /*
        * Test: remove this when non-IP features can be configured.
diff --git a/src/vnet/l2/l2_output.h b/src/vnet/l2/l2_output.h
index c683b1ade73..9597205caed 100644
--- a/src/vnet/l2/l2_output.h
+++ b/src/vnet/l2/l2_output.h
@@ -94,6 +94,7 @@ l2output_main_t l2output_main;
 
 /* Mappings from feature ID to graph node name */
 #define foreach_l2output_feat \
+ _(OUTPUT,            "interface-output")           \
  _(SPAN,              "feature-bitmap-drop")        \
  _(CFM,               "feature-bitmap-drop")        \
  _(QOS,               "feature-bitmap-drop")        \
@@ -217,9 +218,21 @@ l2_output_dispatch (vlib_main_t * vlib_main,
 		    vlib_buffer_t * b0,
 		    u32 sw_if_index, u32 feature_bitmap, u32 * next0)
 {
-  if (feature_bitmap)
+  /*
+   * The output feature bitmap always have at least the output feature bit set
+   * for a normal L2 interface (or all 0's if the interface is changed from L2
+   * to L3 mode). So if next_nodes specified is that from the l2-output node and
+   * the bitmap is all clear except output feature bit, we know there is no more
+   * feature and will fall through to output packet. If next_nodes is from a L2
+   * output feature node (and not l2-output), we always want to get the node for
+   * the next L2 output feature, including the last feature being interface-
+   * output node to output packet.
+   */
+  if ((next_nodes != &l2output_main.next_nodes)
+      || ((feature_bitmap & ~L2OUTPUT_FEAT_OUTPUT) != 0))
     {
       /* There are some features to execute */
+      ASSERT (feature_bitmap != 0);
 
       /* Save bitmap for the next feature graph nodes */
       vnet_buffer (b0)->l2.feature_bitmap = feature_bitmap;
diff --git a/src/vnet/l2/l2_output_classify.c b/src/vnet/l2/l2_output_classify.c
index 27d5eb39514..832be1a11c0 100644
--- a/src/vnet/l2/l2_output_classify.c
+++ b/src/vnet/l2/l2_output_classify.c
@@ -493,7 +493,7 @@ l2_output_classify_init (vlib_main_t * vm)
 			       l2_output_classify_node.index,
 			       L2OUTPUT_N_FEAT,
 			       l2output_get_feat_names (),
-			       cm->feat_next_node_index);
+			       cm->next_nodes.feat_next_node_index);
   rt->l2cm = cm;
   rt->vcm = cm->vnet_classify_main;