aboutsummaryrefslogtreecommitdiffstats
path: root/docs/lab/testbed_specifications.md
AgeCommit message (Expand)AuthorFilesLines
2022-01-24feat(core): Add ICXPeter Mikus1-1/+1
2021-12-14docs(lab): Intel Icelakepmikus1-111/+548
2021-12-02lab WIP: add new 3n-alt Arm testbedJuraj Linkeš1-18/+107
2021-08-26Docs: Cleanup loopback linkspmikus1-14/+36
2021-08-25lab: update Arm lab docsJuraj Linkeš1-187/+184
2021-08-04Infra: Decommission 3n-hsw - pxe/ansiblepmikus1-10/+9
2021-06-10Infra: vpp_device -> Ubuntu 20.04pmikus1-2/+10
2021-06-10Infra: Decommission 3n-hsw - docspmikus1-296/+31
2020-12-13Ansible: Add 10.32.8.17 into nomad poolpmikus1-16/+16
2020-12-01Intel E810 hardware onboardingpmikus1-14/+35
2020-11-24ansible: update 3n-tsh hugepagesJuraj Linkeš1-8/+12
2020-11-24lab: ThunderX2 updatesJuraj Linkeš1-27/+100
2020-10-07docs: update arm hardware testbed specificationsJuraj Linkeš1-116/+53
2020-09-30Update 1n-tx2 testbed docs for new thx2 serversJieqiang Wang1-42/+103
2020-09-18Added amd 2n-zen2 testbed specificationMaciek Konstantynowicz1-27/+118
2020-07-01Infra: Update Ansible ARM NomadJuraj Linkeš1-4/+12
2020-06-09Infra: Ansible Nomadpmikus1-2/+14
2020-05-25Infra: Ansible Nomad IIIpmikus1-11/+13
2020-05-22Infra: Ansible Nomad IIpmikus1-16/+25
2020-05-21Infra: Ansible Nomadpmikus1-2/+2
2019-08-16labs: add new Cascade Lake IPsMohammed Naser1-28/+28
2019-07-25docs: fixed TOC and list indexes in testbed specificationMaciek Konstantynowicz1-63/+63
2019-07-25docs: updated testbed specification with clx buildMaciek Konstantynowicz1-142/+408
2019-06-19Add 2-node and 3-node denverton to specificationsYulong Pei1-7/+82
2019-04-05Fixes for Taishan testbedjuraj.linkes1-6/+6
2019-03-06docs/lab: re-fixed TOC hyperlinks and editing nitsMaciek Konstantynowicz1-23/+27
2019-03-06doc/lab: fixed TOC hyperlinksMaciek Konstantynowicz1-54/+53
2019-03-06docs/lab: merged testbed specifications, separated out detailed HW and BIOS c...Maciek Konstantynowicz1-0/+1469
" can be solved offline. More often than not, one discovers that a control-plane client misprograms the data plane after a long time or under complex circumstances. Without direct evidence, "it's a data-plane problem!" See @ref src/vlibmemory/memory_vlib.c @ref vl_msg_api_process_file(), and @ref src/vlibapi/api_shared.c. See also the debug CLI command "api trace" ## Client connection details Establishing a binary API connection to VPP from a C-language client is easy: ```{.c} int connect_to_vpe (char *client_name, int client_message_queue_length) { vat_main_t *vam = &vat_main; api_main_t *am = &api_main; if (vl_client_connect_to_vlib ("/vpe-api", client_name, client_message_queue_length) < 0) return -1; /* Memorize vpp's binary API message input queue address */ vam->vl_input_queue = am->shmem_hdr->vl_input_queue; /* And our client index */ vam->my_client_index = am->my_client_index; return 0; } ``` 32 is a typical value for client_message_queue_length. VPP cannot block when it needs to send an API message to a binary API client, and the VPP-side binary API message handlers are very fast. When sending asynchronous messages, make sure to scrape the binary API rx ring with some enthusiasm. ### binary API message RX pthread Calling @ref vl_client_connect_to_vlib spins up a binary API message RX pthread: ```{.c} static void * rx_thread_fn (void *arg) { unix_shared_memory_queue_t *q; memory_client_main_t *mm = &memory_client_main; api_main_t *am = &api_main; q = am->vl_input_queue; /* So we can make the rx thread terminate cleanly */ if (setjmp (mm->rx_thread_jmpbuf) == 0) { mm->rx_thread_jmpbuf_valid = 1; while (1) { vl_msg_api_queue_handler (q); } } pthread_exit (0); } ``` To handle the binary API message queue yourself, use @ref vl_client_connect_to_vlib_no_rx_pthread. In turn, vl_msg_api_queue_handler(...) uses mutex/condvar signalling to wake up, process VPP -> client traffic, then sleep. VPP supplies a condvar broadcast when the VPP -> client API message queue transitions from empty to nonempty. VPP checks its own binary API input queue at a very high rate. VPP invokes message handlers in "process" context [aka cooperative multitasking thread context] at a variable rate, depending on data-plane packet processing requirements. ## Client disconnection details To disconnect from VPP, call @ref vl_client_disconnect_from_vlib. Please arrange to call this function if the client application terminates abnormally. VPP makes every effort to hold a decent funeral for dead clients, but VPP can't guarantee to free leaked memory in the shared binary API segment. ## Sending binary API messages to VPP The point of the exercise is to send binary API messages to VPP, and to receive replies from VPP. Many VPP binary APIs comprise a client request message, and a simple status reply. For example, to set the admin status of an interface, one codes: ```{.c} vl_api_sw_interface_set_flags_t *mp; mp = vl_msg_api_alloc (sizeof (*mp)); memset (mp, 0, sizeof (*mp)); mp->_vl_msg_id = clib_host_to_net_u16 (VL_API_SW_INTERFACE_SET_FLAGS); mp->client_index = api_main.my_client_index; mp->sw_if_index = clib_host_to_net_u32 (<interface-sw-if-index>); vl_msg_api_send (api_main.shmem_hdr->vl_input_queue, (u8 *)mp); ``` Key points: * Use @ref vl_msg_api_alloc to allocate message buffers * Allocated message buffers are not initialized, and must be presumed to contain trash. * Don't forget to set the _vl_msg_id field! * As of this writing, binary API message IDs and data are sent in network byte order * The client-library global data structure @ref api_main keeps track of sufficient pointers and handles used to communicate with VPP ## Receiving binary API messages from VPP Unless you've made other arrangements (see @ref vl_client_connect_to_vlib_no_rx_pthread), *messages are received on a separate rx pthread*. Synchronization with the client application main thread is the responsibility of the application! Set up message handlers about as follows: ```{.c} #define vl_typedefs /* define message structures */ #include <vpp/api/vpe_all_api_h.h> #undef vl_typedefs /* declare message handlers for each api */ #define vl_endianfun /* define message structures */ #include <vpp/api/vpe_all_api_h.h> #undef vl_endianfun /* instantiate all the print functions we know about */ #define vl_print(handle, ...) #define vl_printfun #include <vpp/api/vpe_all_api_h.h> #undef vl_printfun /* Define a list of all message that the client handles */ #define foreach_vpe_api_reply_msg \ _(SW_INTERFACE_SET_FLAGS_REPLY, sw_interface_set_flags_reply) static clib_error_t * my_api_hookup (vlib_main_t * vm) { api_main_t *am = &api_main; #define _(N,n) \ vl_msg_api_set_handlers(VL_API_##N, #n, \ vl_api_##n##_t_handler, \ vl_noop_handler, \ vl_api_##n##_t_endian, \ vl_api_##n##_t_print, \ sizeof(vl_api_##n##_t), 1); foreach_vpe_api_msg; #undef _ return 0; } ``` The key API used to establish message handlers is @ref vl_msg_api_set_handlers , which sets values in multiple parallel vectors in the @ref api_main_t structure. As of this writing: not all vector element values can be set through the API. You'll see sporadic API message registrations followed by minor adjustments of this form: ```{.c} /* * Thread-safe API messages */ am->is_mp_safe[VL_API_IP_ADD_DEL_ROUTE] = 1; am->is_mp_safe[VL_API_GET_NODE_GRAPH] = 1; ```