vlib: queue_hi_thresh fix to avoid deadlock

Adapt queue_hi_thresh value using num_threads to avoid risk of deadlock
between threads which could happen for example when different NAT
threads try to handoff work to each other at the same time when their
frame queues are congested. This change ensures that each thread can
reserve a queue entry without causing problems even in the most extreme
case when all threads attempt to add to the same queue simultaneously
when the queue is nearly full.

Type: fix

Signed-off-by: Elias Rudberg <elias.rudberg@bahnhof.net>
Change-Id: I9e02f753bd00833d8dd500d181b0d4f9a454d703

1 files changed, 4 insertions, 2 deletions

diff --git a/src/vlib/threads.c b/src/vlib/threads.c
index a6915c5949b..82263797d93 100644
--- a/src/vlib/threads.c
+++ b/src/vlib/threads.c
@@ -1814,17 +1814,19 @@ vlib_frame_queue_main_init (u32 node_index, u32 frame_queue_nelts)
   vlib_frame_queue_main_t *fqm;
   vlib_frame_queue_t *fq;
   int i;
+  u32 num_threads;
 
   if (frame_queue_nelts == 0)
     frame_queue_nelts = FRAME_QUEUE_MAX_NELTS;
 
-  ASSERT (frame_queue_nelts >= 8);
+  num_threads = 1 /* main thread */  + tm->n_threads;
+  ASSERT (frame_queue_nelts >= 8 + num_threads);
 
   vec_add2 (tm->frame_queue_mains, fqm, 1);
 
   fqm->node_index = node_index;
   fqm->frame_queue_nelts = frame_queue_nelts;
-  fqm->queue_hi_thresh = frame_queue_nelts - 2;
+  fqm->queue_hi_thresh = frame_queue_nelts - num_threads;
 
   vec_validate (fqm->vlib_frame_queues, tm->n_vlib_mains - 1);
   vec_validate (fqm->per_thread_data, tm->n_vlib_mains - 1);