summaryrefslogtreecommitdiffstats
path: root/src/svm/svm_fifo.h
AgeCommit message (Collapse)AuthorFilesLines
2022-01-19svm: update number of segments in svm_fifo_segmentsFlorin Coras1-2/+2
In addition to returning the number of bytes also update the number of segments to reflect the number used. Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Ia87dc2aa62cea38b18dfa83df94dc2abe29d5121
2021-12-30svm: merge want deq and leq threshold notificationsFlorin Coras1-12/+3
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I2775af35386a4e52ba82991c68bce6c56f13410f
2021-12-29svm: Add support for thresh dequeue notificationMichal Kalderon1-2/+23
Add the ability to set a threshold on the fifo that will be used to notify caller when de-queue from the fifo reached a value that is less than or equal to the threshold. Type: feature Change-Id: I70ba1a05f783ce5247409e9beebe4e336b7c1eb5 Signed-off-by: Michal Kalderon <mkalderon@marvell.com> Signed-off-by: Yuval Caduri <cyuval@marvell.com>
2021-09-23svm: remove unused fifo functionsFlorin Coras1-57/+0
Type: refactor Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I4525bc669d1c42c41dbc398cf239a093b6853298
2020-12-30svm: fifo segment sptr for fifo hdr free listFlorin Coras1-3/+3
With this there are no more pointers in data structures allocated on fifo segments. Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Ibe584b7b6809fa360a105974655a91674db69ab6
2020-12-26svm: change fifo pointers to offsetsFlorin Coras1-6/+51
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I113a630d6db4741f54e508f7e0ed77be139df0e4
2020-12-24svm: split fifo into private and shared structsFlorin Coras1-30/+35
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Id8e77e8b2623be719fd43a95e181eaa5b7df2b6e
2020-12-23svm: separate private from shared fifo segment ptrsFlorin Coras1-1/+1
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Idcc8d862b98dba4a67f829c1778377f35ad47b00
2020-12-20tls: use fifo segments instead of chunksFlorin Coras1-0/+15
Type: refactor Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Id67cf8a3e1c5c9b4160689fde5de9ce7ed8a2ee3
2020-11-24svm: support for multi-segment enqueuesFlorin Coras1-0/+11
Type: feature Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I06c7022a6afbb146b23cbd3a430497ec9e8be73d
2020-10-12vcl svm: segments improvementsFlorin Coras1-2/+3
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I717c64666972bb4e440cb3d1180a5cb26ee25577
2020-09-29vcl svm: provide apps access to fifo chunksFlorin Coras1-2/+16
Type: feature Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I2191b8594b1e87ecc00f237316457db249f73603
2020-09-08svm session: document unsupported fifo deq combinationsFlorin Coras1-2/+4
Type: fix - Document that ooo dequeues with ooo lookups cannot be done in combination with in order dequeues. - Added assert to capture this scenario and de-initialized rbtrees for cut-through tx fifo Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Ic40d020b3f0391fcf022ea3c906b86121744144f
2020-04-20svm: check if fifo free list index is valid on allocFlorin Coras1-0/+2
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Signed-off-by: Benoît Ganne <bganne@cisco.com> Change-Id: Ib85c2f01dc7ec9858f2f88b89e209f989d78c5d9
2020-03-09svm: track fifo segment virtual memory consumptionFlorin Coras1-0/+1
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I89d33d8e7e7ab049750439b22ba50372fd2261ca
2020-02-26svm: remove chunk add functionFlorin Coras1-6/+45
Type: refactor Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I606eabbe0f3f387529e77d2483f5d63db7bde88a
2020-02-25svm: refactor fifoFlorin Coras1-175/+80
Type: refactor Switch from a wrapped byte space to a "continuous" one wherein fifo chunks are appended to the fifo as more data is enqueued and chunks are removed as data is dequeued. The fifo is still subject to a maximum size, i.e., maximum number of bytes that can be enqueued, so the max number of chunks associated to the fifo is also constrained. When enqueueing data, which must fit within the available free space, if not enough "supporting" chunk memory is available, the fifo asks the fifo segment for enough chunk memory to ensure that the write can succeed. To avoid allocating large amounts of small chunks due to small writes, if possible, the size of the chunks requested is lower capped by min_alloc. When dequeuing data, all the chunks that have been completely drained, i.e., head moved beyond the chunks’ end bytes, are unlinked from the fifo and returned to the fifo segment. The one exception to this is the last chunk which is never unlinked. Change-Id: I98c1dbd9135fb79650365c7e40c29238b96cd4ee Signed-off-by: Florin Coras <fcoras@cisco.com>
2020-02-25svm: refactor fifo chunk trackingFlorin Coras1-2/+4
Avoid tracking with rbtrees all of the chunks associated to a fifo. Instead, only track chunks when doing out-of-order operations (peek or ooo enqueue). Type: refactor Change-Id: I9f8bd266211746637d98e6a12ffc4b2d6346950a Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-12-12session svm: per thread fifo segment slicesFlorin Coras1-2/+3
Type: refactor Change-Id: I9be652e56cdb48b0aee3253f7ce8d9bed299d824 Signed-off-by: Florin Coras <fcoras@cisco.com> Signed-off-by: Dave Barach <dave@barachs.net>
2019-10-16svm: immediate fifo growth even when wrappedRyujiro Shibuya1-0/+9
Type: fix - when the fifo is wrapped, and if applicable, insert a new chunk after the tail-chunk and rebuild the rb_tree. - make sure that this new algorithm can be applied only when the fifo is used by a single thread (master-thread of the fifo). Signed-off-by: Ryujiro Shibuya <ryujiro.shibuya@owmobility.com> Change-Id: I3fc187bc496ea537ca24381e4abc08d2906c9e03
2019-08-08svm: fix fifo max writeable chunk computationFlorin Coras1-1/+1
Type:fix Change-Id: I60fdd7651a3d3fac505e7aec7a41f9d096022895 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-07-03svm: fix multi-chunk fifo alloc and add more testsFlorin Coras1-1/+15
Type: fix - make sure that chunks and the rbtree are initialized if fifo segment allocates multiple chunks for the fifo. - ensure head/tail chunks are updated on all enqueue/dequeue events, including when dropping data. - more unit tests Also fixes dequeue drop updates of head chunk. Change-Id: I77f3550bc4e8b4e077f80ea87fe82b83ed013aeb Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-06-29svm: rename fifo tx notifications to reflect useFlorin Coras1-42/+42
Type: refactor Change-Id: I651db44acdcb666a9c63e1037352cf88c68795b5 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-06-28quic : Use TX event for app read notificationNathan Skrzypczak1-3/+7
Type: feature Change-Id: I1846cdeb35f079249f66a0351aa244c540923a43 Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2019-05-07svm: decouple fifo and default chunk allocationFlorin Coras1-2/+0
Default chunk is no longer embedded into the fifo and on free is returned to its respective chunk list. Change-Id: Ifc5d214eaa6eca44356eb79dd75650fb8569113f Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-05-03svm: basic support for fifo shrinkingFlorin Coras1-24/+55
As opposed to growing, this is not a bulk operation, instead dependent on how the producer/consumer advance head and tail, the fifo will shrink in one or multiple steps. Only once the fifo's nitems and size are reduced to their appropriate values, equal or larger to what was requested, can the fifo chunks be collected by the owner. Chunk collection must be done with the segment heap pushed. Change-Id: Iae407ccf48d85320aa3c1e0304df56c5972c88c1 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-05-03svm: store normalized head/tail for fifoFlorin Coras1-28/+20
If head/tail are stored as "absolute" values that are normalized to [0, fifo_size] interval, when fifo is shrunk/grown the consumer and producer have to independently update to the new fifo size and fix head and tail, respectively. If the head and tail are stored as normalized values, under the right conditions, they don't need to be fixed when fifo size changes. This reverts one of the changes in gerrit 18223. Change-Id: I55a908828afe90925cf7c20186a940b25e5805f9 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-04-30svm: more fifo refactor/cleanupFlorin Coras1-217/+461
Change-Id: Ie76c69641c8598164d0d00fd498018037258fd86 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-04-27svm: fix fifo tail/head/ooo logic for u32 wrapFlorin Coras1-2/+2
These were introduced with the switch to unbound tail/head size, so they only affect master. Added unit tests to avoid future surprises. Change-Id: I83b6c9efbe31d8092ba59b8e2ed46f4da97f35db Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-04-26svm: fifo segment support for chunk allocationFlorin Coras1-0/+31
Change-Id: Ie96706b4d8bcb32d2d5f065bc765f95f4e9369e7 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-04-25session: cleanup segment manager and fifo segmentFlorin Coras1-4/+5
Change-Id: I984f347fb465c0c405cef668d8690457e81788e2 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-04-24svm: fifo ooo reads/writes with multiple chunksFlorin Coras1-6/+19
Change-Id: If23a04623a7138c9f6c98ee9ecfa587396618a60 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-04-20svm: reorganize fifo march codeFlorin Coras1-70/+9
- make only the chunk copying (memcpy) code march aware - cleanup dependencies Change-Id: I369378264cacfcdaf0823353b957876554eaa17c Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-04-19svm: support addition of chunks to fifosFlorin Coras1-5/+78
Change-Id: Ia56cad89b85b7a99ab4bfb85318a45a71381fb53 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-04-18svm: allow indirect fifo data chunksFlorin Coras1-6/+19
Fifos can use multiple memory chunks for simple read/write operations. Adding/removing chunks after assignment not yet supported. Change-Id: I2aceab6aea78059d74e0d3a9993c40d5196d077b Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-04-16svm_fifo rework to avoid contention on cursizeSirshak Das1-37/+158
Problems Addressed: - Contention of cursize by producer and consumer. - Reduce the no of modulo operations. Changes: - Synchronization between producer and consumer changed from cursize to head and tail indexes Implications: reduces the usable size of fifo by 1. - Using weaker memory ordering C++11 atomics to access head and tail based on producer and consumer role. - Head and tail indexes are unsigned 32 bit integers. Additions and subtraction on them are implicit 32 bit Modulo operation. - Adding weaker memory ordering variants of max_enq, max_deq, is_empty and is_full Using them appropriately in all places. Perfomance improvement (iperf3 via Hoststack): iperf3 Server: Marvell ThunderX2(AArch64) - iperf3 Client: Skylake(x86) ~6%(256 rxd/txd) - ~11%(2048 rxd/txd) Change-Id: I1d484e000e437430fdd5a819657d1c6b62443018 Signed-off-by: Sirshak Das <sirshak.das@arm.com> Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
2019-04-08host stack: update stale copyrightFlorin Coras1-1/+1
Change-Id: I33cd6e44d126c73c1f4c16b2041ea607b4d7f39f Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-03-22svm/atomics: add clib_atomic_swap_rel_nFlorin Coras1-2/+2
Change-Id: Iea2c173000570043beafef58ca923463ce76d872 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-03-21session/fifo: make event unset atomicFlorin Coras1-2/+2
Ensures that fifo cursize loads cannot be speculated to before the event unset. Change-Id: Ia7c20c510d58f26a8e9b82d3982c6d4143a3a4d6 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-02-21svm: fix mem ordering for enqueue nocopyFlorin Coras1-1/+1
Change-Id: Ibc852f88747125a895415570349975d3b379a6f4 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-01-29vls: multi-process and multi-threaded apps improvementsFlorin Coras1-1/+1
- More fine tuning for multi-process applications. - Experimental support for multi-thread apps. This is meant for app whose threads are not vcl workers and the sessions are shared between them. Change-Id: Ie07651da5f2cdcf39f5dead5431f50ad39cf3f74 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-01-17vcl/session: replicate events for shared sessionsFlorin Coras1-7/+20
Change-Id: I7fb5402d4a530b5f2ffd9bb5787632099f4b4189 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-01-10session/vcl: improve fifo tx notificationsFlorin Coras1-30/+80
Change-Id: Ied476c417c8ba5b4d7c615125c36eecd9ca64fbc Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-11-30session: segment handle in accept/connect notificationsFlorin Coras1-0/+2
Change-Id: I03884b6cde9d4c38ae13d1994fd8d37d44016ef0 Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-11-28Use acquire/release ordering when accessing svm_fifo shared variable cursizeSirshak Das1-3/+3
Improves TCP iperf3 performance by ~3% on AArch64. Change-Id: I1e51bd8403ba45ec6af4c2f96b95e884c1ae0d67 Signed-off-by: Sirshak Das <sirshak.das@arm.com> Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com> Reviewed-by: Ola Liljedahl <ola.liljedahl@arm.com>
2018-10-19vppinfra: add atomic macros for __sync builtinsSirshak Das1-1/+1
This is first part of addition of atomic macros with only macros for __sync builtins. - Based on earlier patch by Damjan (https://gerrit.fd.io/r/#/c/10729/) Additionally - clib_atomic_release macro added and used in the absence of any memory barrier. - clib_atomic_bool_cmp_and_swap added Change-Id: Ie4e48c1e184a652018d1d0d87c4be80ddd180a3b Original-patch-by: Damjan Marion <damarion@cisco.com> Signed-off-by: Sirshak Das <sirshak.das@arm.com> Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com> Reviewed-by: Ola Liljedahl <ola.liljedahl@arm.com> Reviewed-by: Steve Capper <steve.capper@arm.com>
2018-10-05session: fix local session tagging and cleanupFlorin Coras1-2/+4
Change-Id: I0d42a0c71fea7dd669fb1fe5ded7e6e944245c7d Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-09-20session/svm: add want_tx_event flag to fifoFlorin Coras1-0/+13
Have applications use explicit flag to request events from vpp when it transmits from a full fifo. Change-Id: I687c8f050a066bd5ce739d880eaec1f286038d95 Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-09-12vcl: add apis that expos fifo as bufferFlorin Coras1-2/+10
Change-Id: I4bd9c9f73499711e04b38d53daa5c917a4285bf5 Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-09-12vcl: improve read and fifo event handlingFlorin Coras1-3/+7
Change-Id: Ic1c51818b8aa8dbd164e70bb3b7471868e5af6f6 Signed-off-by: Florin Coras <fcoras@cisco.com>