summaryrefslogtreecommitdiffstats
path: root/src/vnet/session/segment_manager.c
AgeCommit message (Collapse)AuthorFilesLines
2019-08-01vppinfra: refactor clib_rwlock_t to use single condition variablejaszha031-1/+0
Previous implementation of clib_rwlock_t used two spinlocks: one writer lock, and one to guard the counter for the number of readers. This implementation uses a single condition variable rw_cnt which has the following properties: if a writer has the rwlock, rw_cnt = -1 if the rwlock is free, rw_cnt = 0 otherwise, rw_cnt > 0 and rw_cnt = number of readers rw_cnt will never be less than -1 Benchmarking: The results below are the cycle counts from test_rwlock.c, configured so that for 10000 iterations, 6 reader and 6 writer threads on separate cores are spawned such that each writer thread increments a global counter 10000 times in each iteration. For Taishan, 4 reader and 4 writer threads are spawned in each test. x86 Xeon old rwlock: 12.473e8, 11.655e8, 13.201e8, 11.347e8, 13.182e8 x86 Xeon new rwlock: 5.881e8, 5.796e8, 6.536e8, 5.540e8, 5.890e8 Aarch64 ThX2* old rwlock: 9.263e7, 8.933e7, 9.074e7, 8.979e7, 9.378e7 Aarch64 ThX2* new rwlock: 7.221e7, 8.107e7, 7.515e7, 7.672e7, 7.386e7 A72 old rwlock: 3.268e6, 3.200e6, 3.086e6, 3.176e6, 3.170e6 A72 new rwlock: 1.261e6, 1.288e6, 1.251e6, 1.229e6, 1.234e6 *ThunderX2 used additional gcc options "-march=armv8.1-a+crc+crypto+lse" Type: refactor Change-Id: I7c347d3037b36205ab532cbcb52a374c846eb275 Signed-off-by: Jason Zhang <jason.zhang2@arm.com> Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com> Reviewed-by: Lijian Zhang <Lijian.Zhang@arm.com>
2019-06-09svm: improve fifo segment showFlorin Coras1-18/+5
Type: refactor Change-Id: I6933205cfb04bc31cabe6e3b1a8044cace93f84c Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-05-14svm: improve fifo segment prealloc supportFlorin Coras1-3/+8
- track fifo segment free and chunk freelist memory - improve fifo alloc. If there are enough chunks to satisfy a fifo allocation request but not enough free memory, allocate a multi-chunk fifo - add apis to preallocate chunks and fifo headers - more tests Change-Id: If18dba7ab856272c9f565d36ac36365139793e0b Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-05-09session: segment manager fixesFlorin Coras1-1/+3
- fix segment size rounding for 4GB segments - fix initialization of first segment size - cleanup fifo segment info retrieval Change-Id: I5ebf20f71ea797087653e7e76fa2e37b2686ec40 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-05-03session/svm: apis for fifo shrinkingFlorin Coras1-0/+31
Change-Id: Ie519683bb90aae6fb95f2a09e251cded1890ed41 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-04-26svm: fifo segment support for chunk allocationFlorin Coras1-1/+14
Change-Id: Ie96706b4d8bcb32d2d5f065bc765f95f4e9369e7 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-04-25session: cleanup segment manager and fifo segmentFlorin Coras1-226/+321
Change-Id: I984f347fb465c0c405cef668d8690457e81788e2 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-04-24session: remove unused fifo allo return valueFlorin Coras1-6/+5
Change-Id: I50a6bcc127e4b44becc4b694bdd3018ac9bfab5c Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-04-17session: fix segment manager init conditionFlorin Coras1-1/+1
Change-Id: I0ef3115bd29a11538090c582a4eacdbb7cd86d7a Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-04-12session: drop lock on segment allocation errorFlorin Coras1-12/+9
Change-Id: Ib346570daa3e40f4f53100a05e9355ce60d533a4 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-03-06session: use vpp to switch io events for ct sessionsFlorin Coras1-6/+4
Instead of allocating pairs of message queues per cut-thru session and having the applications map them, this uses vpp as an io event message switch. Change-Id: I51db1c7564df479a7d1a3288342394251fd188bb Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-03-04session: remove svm_segment_index from session structFlorin Coras1-2/+3
Change-Id: I42ef9ee38f93600a0d6f2699b1b2a0a201fcec9c Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-03-02session: notify app of segment removalFlorin Coras1-1/+12
Change-Id: I3774f7ffd14be73e9bbbf4cdd6d3b22f24fee730 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-03-01session: refactor local connectsFlorin Coras1-13/+12
- Switches local connects to cut-thru transport - Removes local sessions as a separate session type Change-Id: I997c6355d8c8e4f2110678f785b0f5d96bba47f7 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-02-23session: separate local session logicFlorin Coras1-0/+1
Move local session to separate header and source files. First step to refactoring local sessions. Change-Id: I280fdfef20ba8a0977d15c1c8ce030ea2fb72dde Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-02-05session: cleanup part 2Florin Coras1-1/+1
Move app worker logic to app_worker.c Change-Id: Ic5e5735b2884f006c064d023f491aa6888114810 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-02-04session: cleanup part 1Florin Coras1-2/+2
Rename core data structures. This will break compatibility for out of tree builtin apps. - stream_session_t to session_t - server_rx/tx_fifo to rx/tx_fifo - stream_session.h to session_types.h - update copyright Change-Id: I414097c6e28bcbea866fbf13b8773c7db3f49325 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-01-17tls: preallocate app sessions on connect/acceptFlorin Coras1-0/+3
Avoid allocating session and possibly reallocating thread session pool on builtin session rx. Change-Id: I70e7c604678b44ce8d22603489e247a2c5faa439 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-01-02Fixes for buliding for 32bit targets:David Johnson1-3/+3
* u32/u64/uword mismatches * pointer-to-int fixes * printf formatting issues * issues with incorrect "ULL" and related suffixes * structure alignment and padding issues Change-Id: I70b989007758755fe8211c074f651150680f60b4 Signed-off-by: David Johnson <davijoh3@cisco.com>
2018-12-28session: free session after transport and app confirmFlorin Coras1-1/+1
In addition to that, a bit of refactoring. Change-Id: Iea1eabc2167bcdef185ec53bc09bae087c5398e6 Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-11-30session: segment handle in accept/connect notificationsFlorin Coras1-3/+39
Change-Id: I03884b6cde9d4c38ae13d1994fd8d37d44016ef0 Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-10-23c11 safe string handling supportDave Barach1-4/+4
Change-Id: Ied34720ca5a6e6e717eea4e86003e854031b6eab Signed-off-by: Dave Barach <dave@barachs.net>
2018-10-05session: fix local session tagging and cleanupFlorin Coras1-2/+2
Change-Id: I0d42a0c71fea7dd669fb1fe5ded7e6e944245c7d Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-09-19session: improve close procedureFlorin Coras1-1/+3
Change-Id: I6201a044e70ab6a58db8212960c57edc77c41f96 Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-09-14vcl: keep track of unexpected eventsFlorin Coras1-3/+2
If sessions are marked as blocking, events for other sessions received while waiting for the blocking sessions, are added to a pending list and processed later. Change-Id: Ia6c71006b1c2bcb78af708390da0cd436af397cc Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-09-06session: support multiple worker bindsFlorin Coras1-1/+2
Allows app workers to listen on the same session endpoint. Incoming connects are spread across the workers in a round-robin fashion Change-Id: Ib5f5817230d9abc6127a85cdbdcad70d980c0f7f Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-08-24session: add support for multiple app workersFlorin Coras1-8/+8
Refactor session layer to support multiple workers per application. Change-Id: Ie67354688d396449d14bbbb8c56050206e307cd8 Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-08-10vcl: support for eventfd mq signalingFlorin Coras1-9/+13
- support eventfd based mq signaling. Based on configuration, vcl epoll/select can use either condvars or epoll on mq eventfds. - add vcl support for memfd segments - vpp explicitly registers cut-through segments with apps/vcl - if using eventfd, make ldp allow one call to libc_epoll_create. Needed for the message queue epfd - update svm_queue_t to allow blocking calls with eventfd signaling. Change-Id: I064151ac370bbe29bb16c968bf4e3659c8286bea Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-07-17session: send ctrl msg over mqFlorin Coras1-2/+2
Change-Id: I242056bc46ddb671064665916b2687860292dcb2 Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-07-17session: use msg queue for eventsFlorin Coras1-8/+35
Change-Id: I3c58367eec2243fe19b75be78a175c5261863e9e Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-07-17session: make sure segment sizes are page alignedFlorin Coras1-1/+3
Change-Id: Ibbba75d069ca1bbf9e5a1b8bd2f405d32021c656 Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-03-25session: fix local session disconnectsFlorin Coras1-0/+8
Select the right segment manager for local sessions established via global table. Change-Id: I88ad4bf70d0cae160a0c744950098a954dfbc911 Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-02-15session: fix fifo allocation for empty connect seg-managersFlorin Coras1-1/+0
Change-Id: Ibc5b1149f3fbbe3dfe1f069ab69bfed7de660582 Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-02-14session: support local sessions and deprecate redirectsFlorin Coras1-24/+29
Memfd backed shared memory segments can only be negotiated over sockets. For such scenarios, the existing redirect mechanism that establishes cut-through sessions does not work anymore as the two peer application do not share such a socket. This patch adds support for local sessions, as opposed to sessions backed by a transport connection, in a way that is almost transparent to the two applications by reusing the existing binary api messages. Moreover, all segment allocations are now entirely done through the segment manager valloc, so segment overlaps due to independent allocations previously required for redirects are completely avoided. The one notable characteristic of local sessions (cut-through from app perspective) notification messages is that they carry pointers to two event queues, one for each app peer, instead of one. For transport-backed sessions one of the queues can be inferred but for local session they cannot. Change-Id: Ia443fb63e2d9d8e43490275062a708f039038175 Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-02-05session: segment manager refactorFlorin Coras1-333/+413
- use valloc as a 'central' segment baseva manager - use per segment manager segment pools and use rwlocks to guard them - add session test that exercises segment creation - embed segment manager properties into application since they're shared - fix rw locks Change-Id: I761164c147275d9e8a926f1eda395e090d231f9a Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-01-27session: disconnect and echo client improvementsFlorin Coras1-8/+3
Change-Id: If421bad17b6cfe8e321257c93bb38931e37d7b64 Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-01-25session: add support for memfd segmentsFlorin Coras1-108/+87
- update segment manager and session api to work with both flavors of ssvm segments - added generic ssvm slave/master init and del functions - cleanup/refactor tcp_echo - fixed uses of svm fifo pool as vector Change-Id: Ieee8b163faa407da6e77e657a2322de213a9d2a0 Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-01-09api: refactor vlibmemoryFlorin Coras1-9/+7
- separate client/server code for both memory and socket apis - separate memory api code from generic vlib api code - move unix_shared_memory_fifo to svm and rename to svm_fifo_t - overall declutter Change-Id: I90cdd98ff74d0787d58825b914b0f1eafcfa4dc2 Signed-off-by: Florin Coras <fcoras@cisco.com>
2017-12-14session: cleanup attach flagsFlorin Coras1-14/+7
Change-Id: I39d21d15677f57e10b69b8842f2cbca277abddf0 Signed-off-by: Florin Coras <fcoras@cisco.com>
2017-11-10session: use pool for segment manager propertiesFlorin Coras1-22/+59
Change-Id: I280fea2610dcfc0b2da84973b9f567daec42f1f6 Signed-off-by: Florin Coras <fcoras@cisco.com>
2017-11-07session: fix v6 double bindsFlorin Coras1-0/+7
Change-Id: Ie747b490901254e962cf61814491851b891129ee Signed-off-by: Florin Coras <fcoras@cisco.com>
2017-10-16udp: refactor udp codeFlorin Coras1-1/+1
Change-Id: I44d5c9df7c49b8d4d5677c6d319033b2da3e6b80 Signed-off-by: Florin Coras <fcoras@cisco.com>
2017-10-10session: add support for application namespacingFlorin Coras1-1/+1
Applications are now provided the option to select the namespace they are to be attached to and the scope of their attachement. Application namespaces are meant to: 1) constrain the scope of communication through the network by association with source interfaces and/or fib tables that provide the source ips to be used and limit the scope of routing 2) provide a namespace local scope to session layer communication, as opposed to the global scope provided by 1). That is, sessions can be established without assistance from transport and network layers. Albeit, zero/local-host ip addresses must still be provided in session establishment messages due to existing application idiosyncrasies. This mode of communication uses shared-memory fifos (cut-through sessions) exclusively. If applications request no namespace, they are assigned to the default one, which at its turn uses the default fib. Applications can request access to both local and global scopes for a namespace. If no scope is specified, session layer defaults to the global one. When a sw_if_index is provided for a namespace, zero-ip (INADDR_ANY) binds are converted to binds to the requested interface. Change-Id: Ia0f660bbf7eec7f89673f75b4821fc7c3d58e3d1 Signed-off-by: Florin Coras <fcoras@cisco.com>
2017-09-19session/tcp: improve preallocated segment handlingFlorin Coras1-33/+40
- add preallocated segment flag - don't remove pre-allocated segments except if application detaches - when preallocating fifos in multiple segments, completely fill a segment before moving to the next - detach server application from segment-managers when deleting app - batch syn/syn-ack/fin (re)transmissions - loosen up close-wait and time-wait times Change-Id: I412f53ce601cc83b3acc26aeffd7fa2d52d73b03 Signed-off-by: Florin Coras <fcoras@cisco.com>
2017-09-06Improve "show segment-manager segments"Dave Barach1-6/+48
Clean up private-segment fifo preallocation Change-Id: I53c630ed81d48f2832a204992d73635536926983 Signed-off-by: Dave Barach <dave@barachs.net>
2017-08-29session: segment manager improvementsFlorin Coras1-115/+250
- cleanup connects segment manager even if first - fix segment manager allocation for listen sessions - improve handling of process private segments (mheaps/main heap) - added segment manager cli Change-Id: Ic2ca97c3622ab2286d5fb5772aeb57680e64f769 Signed-off-by: Florin Coras <fcoras@cisco.com> Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2017-08-25tcp: retransmit and multi-buffer segment fixes and improvementsFlorin Coras1-0/+1
- set session state as closed on session manager delete - enable retransmit as opposed to persist timer after persist timer completes - properly discard buffer chain bytes when new data overlaps ooo segments - don't use rxt bytes in snd space estimate used on tx path Change-Id: Id9cab686e532e5fe70c775d5440260e8eb890a9f Signed-off-by: Florin Coras <fcoras@cisco.com>
2017-07-11Horizontal (nSessions) scaling draftDave Barach1-10/+16
- Data structure preallocation. - Input state machine fixes for mid-stream 3-way handshake retries. - Batch connections in the builtin_client - Multiple private fifo segment support - Fix elog simultaneous event type registration - Fix sacks when segment hole is added after highest sacked - Add "accepting" session state for sessions pending accept - Add ssvm non-recursive locking - Estimate RTT for syn-ack - Don't init fifo pointers. We're using relative offsets for ooo segments - CLI to dump individual session Change-Id: Ie0598563fd246537bafba4feed7985478ea1d415 Signed-off-by: Dave Barach <dbarach@cisco.com> Signed-off-by: Florin Coras <fcoras@cisco.com>
2017-07-06Unlink /dev/shm files on application detach.Dave Wallace1-3/+22
Change-Id: Ieb5522cd5cb27bcbce808d4cd6df5248716da43c Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2017-06-19Overall tcp performance improvements (VPP-846)Florin Coras1-0/+5
- limit minimum rto per connection - cleanup sack scoreboard - switched svm fifo out-of-order data handling from absolute offsets to relative offsets. - improve cwnd handling when using sacks - add cc event debug stats - improved uri tcp test client/server: bugfixes and added half-duplex mode - expanded builtin client/server - updated uri socket client/server code to work in half-duplex - ensure session node unsets fifo event for empty fifo - fix session detach Change-Id: Ia446972340e32a65e0694ee2844355167d0c170d Signed-off-by: Florin Coras <fcoras@cisco.com>