summaryrefslogtreecommitdiffstats
path: root/src/svm
AgeCommit message (Collapse)AuthorFilesLines
2021-06-24svm: fix asan check failed @svm_map_region on armTianyu Li1-4/+3
==283032==AddressSanitizer CHECK failed: compiler-rt/lib/asan/asan_mapping.h:366 "((AddrIsInMem(p))) != (0)" (0x0, 0x0) #0 0x49c128 in __asan::AsanCheckFailed #1 0x4ae8dc in __sanitizer::CheckFailed #2 0x495dec in __asan::ShadowSegmentEndpoint::ShadowSegmentEndpoint #3 0x495e48 in __asan_unpoison_memory_region #4 0xfffff4e851f8 in svm_map_region /home/vpp/src/svm/svm.c:611:7 #5 0xfffff4e86d9c in svm_region_init_internal /home/vpp/src/svm/svm.c:797:8 #6 0xfffff4e87ce4 in svm_region_init_args /home/vpp/src/svm/svm.c:880:3 #7 0xfffff7f30d30 in vlibmemory_init /home/vpp/src/vlibmemory/memory_api.c:974:3 #8 0xfffff4fd5368 in vlib_main /home/vpp/src/vlib/main.c:1986:16 svm_global_region_base_va 0x200000000000 is not in the aarch64 mapping range, leading check failure and vpp cannot start. aarch64 asan mapping || `[0x201000000000, 0xffffffffffff]` || HighMem || || `[0x041200000000, 0x200fffffffff]` || HighShadow || || `[0x001200000000, 0x0411ffffffff]` || ShadowGap || || `[0x001000000000, 0x0011ffffffff]` || LowShadow || || `[0x000000000000, 0x000fffffffff]` || LowMem || x86 asan mapping || `[0x10007fff8000, 0x7fffffffffff]` || HighMem || || `[0x02008fff7000, 0x10007fff7fff]` || HighShadow || || `[0x00008fff7000, 0x02008fff6fff]` || ShadowGap || || `[0x00007fff8000, 0x00008fff6fff]` || LowShadow || || `[0x000000000000, 0x00007fff7fff]` || LowMem || Type: fix Signed-off-by: Tianyu Li <tianyu.li@arm.com> Change-Id: I55ddbdcd361d66d4cfaf6459b2fa20fd8b64af37
2021-06-17session: optimize ct fifo segment allocationsFlorin Coras1-0/+1
Allocate per app pair segments with space for more than one fifo. Type: feature Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Ib96fe12b899cb14ff20c0be607814011e2c3fc6a
2021-06-16vcl: fix fifo sharingFlorin Coras2-0/+9
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Iec1fe8315a057214901250d5fb06d1c1e33dda46
2021-06-03svm: release mem order for fifo chunk list CASFlorin Coras1-6/+9
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Signed-off-by: Dave Wallace <dwallacelf@gmail.com> Change-Id: Ifad679f46abd6e9c18a3eaf7e55800a09f3791ab
2021-05-28svm: fix inadequate atomic load for chunk mgrDave Wallace1-3/+3
- Fixes intermittent LDP cut thru iperf3 test failure on AARCH64 Type: fix Change-Id: Id21a078d642e03d974bacacd1f4d0faa42fb6652 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2021-05-20tls svm: prealloc tcp fifo chunks before ssl writeFlorin Coras1-0/+3
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I7c47b55ec6f0c83f2d13e0e737d0559a32f7c837
2021-05-10svm: try chunk alloc only if enough space availableFlorin Coras1-1/+2
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I3c52dd783601d50e981f75bd05a7b95c03bb5c22
2021-04-20svm: allow external chunk alloc on segmentFlorin Coras2-0/+32
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I89f9caa8dd44dc640615a58fe7708f388fdd84e9
2021-04-02svm: lock-free fifo chunk list push and popFlorin Coras2-43/+57
This avoids chunk allocation/collection deadlocks if either of the sides crashes. Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I98619e6e035fa8688889ca34db2143c8898732df
2021-03-30svm: cleanup mq private data on fifo segment cleanupFlorin Coras3-2/+16
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I65373f2cd5a33381da1e51ed7ab7a8b1358cef29
2021-03-29svm session vcl: per app rx message queuesFlorin Coras1-1/+1
Add option to use per app private segments for app to vpp message queues, as opposed to exposing internal message queues segment. When so configured, internal message queues are still polled by the session queue node but external app message queues are handled by a new input node (appsl-rx-mqs-input) that runs in interrupt state. Signaling of the node, when mqs receive new messages, is done through eventfds epolled by worker epoll input nodes. Type: feature Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Iffe8ce5a9944a56a14e6d0f492a850cb9e392d16
2021-03-25svm: add producer wait functionFlorin Coras2-13/+46
Type: refactor Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I9488ad7e045c908b60b5821d9c48583f6d513c2f
2021-03-25svm: fix producer deadlock in svm_msg_q_wait()nandfan1-5/+9
1. When producer invokes svm_msg_q_wait() in svm_msg_q_lock_and_alloc_msg_w_ring(), queue mutex is held by itself. 2. Sometimes, svm msg queue is not full and ring is full, svm_msg_q_wait() do nothing with mutex held, consumer will blocking at svm_msg_q_send_signal(). Type: fix Signed-off-by: nandfan <fanyufei521@outlook.com> Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Ib90b87ab76534cd42e9a4c3e11703e80d93ca678
2021-02-28session svm: segment manager and fifo segment leaksFlorin Coras1-0/+2
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I4e00dd7f8ce1e56092dde9a073decae62d5475de
2021-02-26svm: fix shared hdr migrationFlorin Coras2-7/+9
Avoid changing the header on attach as it may be in use. Instead, as for chunks, allocate header to be collected on detach. Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Ib316ecb5d61ae161032869b6f6a1863f1105a1d9
2021-02-22vcl: cleanup fifos detached from segmentsFlorin Coras2-3/+26
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I412024731c1f561680736ad7bfabb99b595e3dff
2021-02-22svm: free shared fifo on detachFlorin Coras1-2/+4
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I639560ee3dd0a1d605ec2866dce5cdd13fda8201
2021-02-20svm: return chunks to slice on fifo detachFlorin Coras2-18/+14
Ensure chunk alloc distribution is maintained on fifo detach. Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I5aa5524e06a703dc50e90da6d177663d2d997aa4
2021-02-19session vppinfra: asan fixesFlorin Coras1-0/+1
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Ie709d76438542783cbc8c6174b5e712ef18a6276
2021-02-18svm: fix active fifo ll on attachFlorin Coras1-0/+4
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Idf44f8d54c97fc43da5d5760e5ce477af07e5fbf
2021-02-15svm: fix mq coverity warningFlorin Coras1-2/+7
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I1914366c17fa05305d57d842793fad372319256a
2021-02-15misc: coverity fixesDave Barach1-0/+1
Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I6a3348c7edd1cce6b407d336443103f77392bc5d
2021-02-05session svm: non blocking mqFlorin Coras2-74/+181
Avoid synchronizing producers and the consumer. Instead, only use mutex or spinlock (if eventfds are configured) to synchronize producers. Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Ie2aafbdc2e07fced5d5e46ee2df6b30a186faa2f
2021-01-28svm vcl: add helper fn that discovers mqs in segmentFlorin Coras3-0/+43
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I1b083ee793a7cf91b1001bfe88353fa5e6515c42
2021-01-25svm: add custom q implementation for mqFlorin Coras3-95/+288
Add separate queue implementation for the message queue as it's custom tailored for fifo segments as opposed to binary api. Also move eventfds to the private data structures. Type: refactor Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I6df0c824ecd94c7904516373f92a9fffc6b04736
2021-01-24svm: use standard function to reset stale mutexBenoît Ganne1-2/+7
Avoid accessing the private data structure of mutexes which is implementation-dependent, eg. musl is different from glibc. Type: improvement Change-Id: I20ec0c1c9faef0749d89a1969cd2430c80ac04b3 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-01-20svm: add standard include for __WORDSIZENathan Moos1-0/+1
Type: fix In glibc, the __WORDSIZE macro is defined in a header included indirectly from svm_common.h. In musl, the __WORDSIZE macro is not indirectly included, so this patch directly includes <sys/user.h> to ensure the __WORDSIZE macro is available. Change-Id: If1b1fc97161b7a0b2e82351d4776c81c36a323cb Signed-off-by: Nathan Moos <nmoos@cisco.com>
2021-01-09session svm: fix fifo migrationFlorin Coras3-27/+41
Allocate and attach a new pair of private fifos in the right private slice when a session is cloned. This ensures that private fifos are not shared between workers. Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Ib700d18104d2ca79aa8a07434cdcdcab0bef13a5
2021-01-06svm: force fixed memfd mapping if va providedFlorin Coras1-3/+3
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I6a2cc596cf0b7f29d3ae1c05b1bb6b748b558175
2020-12-30svm: fifo segment sptr for fifo hdr free listFlorin Coras4-49/+88
With this there are no more pointers in data structures allocated on fifo segments. Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Ibe584b7b6809fa360a105974655a91674db69ab6
2020-12-29session: remove fifo segment va allocatorFlorin Coras1-1/+1
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I7b2fd896dfa6df46916f46327975b95561809f00
2020-12-29svm vcl: allow random offsets for fifo segments in appsFlorin Coras2-10/+34
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I1658a9c19d8eae4c9a42c0a111d4ad343b8eb8a4
2020-12-29svm: allow mq attachments at random offsetsFlorin Coras6-70/+198
Type: feature Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Ic373cd2c11272da539eb4b0db27227f36f2f9688
2020-12-28svm: move active fifo list to private sliceFlorin Coras2-16/+30
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I7672d0715348d30c0aa718e21ed86c96402c6d69
2020-12-26svm: change fifo pointers to offsetsFlorin Coras4-156/+230
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I113a630d6db4741f54e508f7e0ed77be139df0e4
2020-12-24svm: split fifo into private and shared structsFlorin Coras5-214/+318
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Id8e77e8b2623be719fd43a95e181eaa5b7df2b6e
2020-12-23svm: separate private from shared fifo segment ptrsFlorin Coras3-51/+49
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Idcc8d862b98dba4a67f829c1778377f35ad47b00
2020-12-23svm: remove fifo segment heapFlorin Coras5-150/+181
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I518e096fe13847759806ff62009e73fd8f7451b7
2020-12-21svm: honor max requested bytes in fifo segmentsFlorin Coras1-1/+1
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I00a6be53c4519937ac900560c669587921990821
2020-12-20tls: use fifo segments instead of chunksFlorin Coras2-0/+51
Type: refactor Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Id67cf8a3e1c5c9b4160689fde5de9ce7ed8a2ee3
2020-12-14misc: move to new pool_foreach macrosDamjan Marion2-6/+6
Type: refactor Change-Id: Ie67dc579e88132ddb1ee4a34cb69f96920101772 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-12-08svm: fix ssvm_va for private segmentsFlorin Coras1-1/+1
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I8288a1ed97b375626d69ac7efa0256211834d89c
2020-12-03svm: fix mutex consistency on trylockFlorin Coras2-4/+16
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I1370943935b460a1a1a22847fed2c97b1272a5e1
2020-11-30svm: make svm queue mutex robustFlorin Coras2-2/+9
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I256774c25b3c4105139a4728724a1a0c775e740f
2020-11-26svm: use internal function to prealloc fifo hdrsFlorin Coras1-30/+3
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Ic13828e235d01ec5b73658e4bdd19e510932713d
2020-11-26svm: only try to pre-allocate requested number of fifo pairsBenoît Ganne1-1/+8
Type: fix Change-Id: I6f7679fd48d0ed98677c58f8c08d7e969e8e8220 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-11-24svm: support for multi-segment enqueuesFlorin Coras2-0/+92
Type: feature Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I06c7022a6afbb146b23cbd3a430497ec9e8be73d
2020-11-20svm: fix coverity warningFlorin Coras1-6/+7
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Icead1cbff69cf652fa566a79645c89864b6f3a0f
2020-11-19svm: move chunk locks to linked listFlorin Coras1-37/+48
Type: improvement We only need to protect the linked lists. Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Ie1542073f3993acfc66d99096b08bf9ecd10a49b
2020-11-19svm: init chunk rb tree indices in fifoFlorin Coras2-4/+2
Type: improvement Let fifo segment mainly deal with fifo and chunk allocations not initialization. Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I1f1eb5a1423ba52cb950ae771641fd7eeff4e76c