summaryrefslogtreecommitdiffstats
path: root/src/vnet/classify
AgeCommit message (Collapse)AuthorFilesLines
2018-02-26Added u8x16,u32x4,u64x2 variants of _zero_byte_mask(x) for ARM/NEON ↵Adrian Oanca1-8/+4
platform. VPP-1129 Change-Id: I954acb56d901e42976e71534317f38d7c4359bcf Signed-off-by: Adrian Oanca <adrian.oanca@enea.com>
2018-02-20vppinfra: CLIB_HAVE_VEC128 mandates SSE4.2Damjan Marion1-1/+1
Change-Id: I6511110d0472203498a4f8741781eeeeb4f90844 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-02-08classifier-based ACL: testcases for L2 ACLs + fix the enabling of outbound ↵Andrew Yourtchenko1-3/+6
L2 ACL There was no test coverage for the L2 ACL (other than indirect by means of ACL plugin tests), so the enabling of the outbound ACL got fumbled throughout the revisions of the refactoring. Fix both issues - the error and the lack of test coverage for L2 ACL. Change-Id: Ib7f42780ef84b4a4f70bd88d7319aeeda866cf06 Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
2018-02-07classifier-based ACL: refactor + add output ACLAndrew Yourtchenko6-77/+230
For implementation of MACIP ACLs enhancement (VPP-1088), an outbound classifier-based ACL would be needed. There was an existing incomplete code for outbound ACLs, it looked almost exact copy of input ACLs, minus the various enhancements, trying to sync that code seemed error-prone and cumbersome to maintain in the longer run. This change refactors the input+output ACLs processing into a unified routine (thus any changes will have effect on both), and also adds the API to set the output interface ACL, with the same format and semantics as the existing input one (except working on output ACL of course). WARNING: IP outbound ACL in L3 mode clobbers the ip.* fields in the vnet_buffer_opaque_t, since the code is using l2_classify.* The net_buffer (p0)->ip.save_rewrite_length is rescued into l2_classify.pad.l2_len, and used to rewind the header in case of drop, so that ipX_drop prints something sensible. Change-Id: I62f814f1e3650e504474a3a5359edb8a0a8836ed Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
2018-01-23VPPAPIGEN: vppapigen replacement in Python PLY.Ole Troan1-1/+1
This is a version of the VPP API generator in Python PLY. It supports the existing language, and has a plugin architecture for generators. Currently C and JSON are supported. Changes: - vl_api_version to option version = "major.minor.patch" - enum support - Added error checking and reporting - import support (removed the C pre-processor) - services (tying request/reply together) Version: option version = "1.0.0"; Enum: enum colours { RED, BLUE = 50, }; define foo { vl_api_colours_t colours; }; Services: service { rpc foo returns foo_reply; rpc foo_dump returns stream foo_details; rpc want_stats returns want_stats_reply events ip4_counters, ip6_counters; }; Future planned features: - unions - bool, text - array support (including length) - proto3 output plugin - Refactor C/C++ generator as a plugin - Refactor Java generator as a plugin Change-Id: Ifa289966c790e1b1a8e2938a91e69331e3a58bdf Signed-off-by: Ole Troan <ot@cisco.com>
2018-01-11api: remove transport specific code from handlersFlorin Coras1-26/+28
This does not update api client code. In other words, if the client assumes the transport is shmem based, this patch does not change that. Furthermore, code that checks queue size, for tail dropping, is not updated. Done for the following apis: Plugins - acl - gtpu - memif - nat - pppoe VNET - bfd - bier - tapv2 - vhost user - dhcp - flow - geneve - ip - punt - ipsec/ipsec-gre - l2 - l2tp - lisp-cp/one-cp - lisp-gpe - map - mpls - policer - session - span - udp - tap - vxlan/vxlan-gpe - interface VPP - api/api.c OAM - oam_api.c Stats - stats.c Change-Id: I0e33ecefb2bdab0295698c0add948068a5a83345 Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-01-09api: refactor vlibmemoryFlorin Coras1-10/+8
- separate client/server code for both memory and socket apis - separate memory api code from generic vlib api code - move unix_shared_memory_fifo to svm and rename to svm_fifo_t - overall declutter Change-Id: I90cdd98ff74d0787d58825b914b0f1eafcfa4dc2 Signed-off-by: Florin Coras <fcoras@cisco.com>
2017-11-28VPP-246 Coding standards cleanup - vnet/vnet/classifykhemendra kumar10-1830/+2045
Signed-off-by: khemendra kumar <khemendra.kumar13@gmail.com> Change-Id: I0d94ef604d34da6981e7c2d2b4da5ec3ec5fb19a
2017-11-18Rename classifier ip6-sr metadata set actionDave Barach2-4/+4
There's nothing ip6-sr specific about it. Change-Id: I9e3710162bd81b535c46599c988557abf5a5003b Signed-off-by: Dave Barach <dave@barachs.net>
2017-11-10Break up vpe.apiNeale Ranns2-1/+150
- makes the VAPI generated file more consumable. - VOM build times improve. Change-Id: I838488930bd23a0d3818adfdffdbca3eead382df Signed-off-by: Neale Ranns <neale.ranns@cisco.com>
2017-11-10add classify session action set-sr-policy-indexGabriel Ganne3-1/+15
This allows to use the classifier to steer source routing packets instead of using the "sr steer" command. This way we can steer on anything instead of only the dst ip address. test: * add add_node_next function to the VppPapiProvider class. * add simple test scenario using the classifier to steer packets with dest ip addr == a7::/8 to the source routing insert node. * use new interface indexes (3,4) instead of (0,1) to prevent a cleanup conflict with the other tests which attach a specific fib to the interface. The test creates interfaces sepsrated from the other tests to prevent a conflict in the cleaning of the ip6 fib index 1 which causes vpp not to be able to find a default route on this table. Change-Id: Ibacb30fab3ce53f0dfe848ca6a8cdf0d111d8336 Signed-off-by: Gabriel Ganne <gabriel.ganne@enea.com>
2017-10-26Consolidate short_help for classify table with memory-sizeHongjun Ni1-0/+1
When creating 32K classify sessions, VPP crashes. Default heap size is 2MB. Need to configure it when requiring large number sessions. Change-Id: I16678ee4a9e0ba61cbd2d3b38c43d10c59325968 Signed-off-by: Hongjun Ni <hongjun.ni@intel.com>
2017-10-24Add extern to *_main global variable declarations in header files.Dave Wallace4-2/+6
- Global variables declared in header files without the use of the 'extern' keword will result in multiple instances of the variable to be created by the compiler -- one for each different source file in which the the header file is included. This results in wasted memory allocated in the BSS segments as well as potentially introducing bugs in the application. Change-Id: I6ef1790b60a0bd9dd3994f8510723decf258b0cc Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2017-10-09vppapigen: support per-file (major,minor,patch) version stampsDave Barach1-0/+2
Add one of these statements to foo.api: vl_api_version 1.2.3 to generate a version tuple stanza in foo.api.h: /****** Version tuple *****/ vl_api_version_tuple(foo, 1, 2, 3) Change-Id: Ic514439e4677999daa8463a94f948f76b132ff15 Signed-off-by: Dave Barach <dave@barachs.net> Signed-off-by: Ole Troan <ot@cisco.com>
2017-09-11FIB table add/delete APINeale Ranns1-6/+10
part 2; - this adds the code to create an IP and MPLS table via the API. - but the enforcement that the table must be created before it is used is still missing, this is so that CSIT can pass. Change-Id: Id124d884ade6cb7da947225200e3bb193454c555 Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-06-07VPP-874: fix coverity warning in vnet_classify.cDave Barach1-1/+1
Change-Id: Icffd2862eadbe9ddfb3ee34f3cb19c9324b3d9b4 Signed-off-by: Dave Barach <dave@barachs.net>
2017-05-24Leak locks and tables in the ClassifierNeale Ranns2-3/+45
Change-Id: Iae04c57bba87ab3665388eadd0805f75171636a5 Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-05-20VPP-849: improve vnet classifier memory allocator performanceDave Barach2-274/+450
Port the linear-scan bucket fix from bihash_template.c. Change-Id: Id8b2d1fe402401f098270ce6121c2f44f2f24c49 Signed-off-by: Dave Barach <dave@barachs.net>
2017-04-25"autoreply" flag: autogenerate standard xxx_reply_t messagesDave Barach1-33/+4
Change-Id: I72298aaae7d172082ece3a8edea4217c11b28d79 Signed-off-by: Dave Barach <dave@barachs.net>
2017-04-06Use thread local storage for thread indexDamjan Marion1-8/+8
This patch deprecates stack-based thread identification, Also removes requirement that thread stacks are adjacent. Finally, possibly annoying for some folks, it renames all occurences of cpu_index and cpu_number with thread index. Using word "cpu" is misleading here as thread can be migrated ti different CPU, and also it is not related to linux cpu index. Change-Id: I68cdaf661e701d2336fc953dcb9978d10a70f7c1 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-03-29Sub-net broadcast addresses for IPv4Neale Ranns1-1/+1
Change-Id: Ib2189d01e8bc61de57404159690fb70f89c47277 Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-03-02Clean up binary api message handler registration issuesDave Barach1-8/+0
Removed a fair number of "BUG" message handlers, due to conflicts with actual message handlers in api_format.c. Vpp itself had no business receiving certain messages, up to the point where we started building in relevant code from vpp_api_test. Eliminated all but one duplicate registration complaint. That one needs attention from the vxlan team since the duplicated handlers have diverged. Change-Id: Iafce5429d2f906270643b4ea5f0130e20beb4d1d Signed-off-by: Dave Barach <dave@barachs.net>
2017-03-01VPP-598: tcp stack initial commitDave Barach1-2/+2
Change-Id: I49e5ce0aae6e4ff634024387ceaf7dbc432a0351 Signed-off-by: Dave Barach <dave@barachs.net> Signed-off-by: Florin Coras <fcoras@cisco.com>
2017-02-23Fix vpp built-in version of api_unformat_sw_if_index(...)Dave Barach2-3/+2
Change-Id: I103fe19a1ecbaf3746ec6b957fa1010458cc9fae Signed-off-by: Dave Barach <dave@barachs.net>
2017-01-25API refactoring : classifyPavel Kotucek2-0/+911
Change-Id: Ib75197ef8e5057e7f0d9361a10705c3743d05333 Signed-off-by: Pavel Kotucek <pkotucek@cisco.com>
2016-12-28Reorganize source tree to use single autotools instanceDamjan Marion11-0/+4724
Change-Id: I7b51f88292e057c6443b12224486f2d0c9f8ae23 Signed-off-by: Damjan Marion <damarion@cisco.com>
d } /* Keyword.Constant */ .highlight .kd { color: #008800; font-weight: bold } /* Keyword.Declaration */ .highlight .kn { color: #008800; font-weight: bold } /* Keyword.Namespace */ .highlight .kp { color: #008800 } /* Keyword.Pseudo */ .highlight .kr { color: #008800; font-weight: bold } /* Keyword.Reserved */ .highlight .kt { color: #888888; font-weight: bold } /* Keyword.Type */ .highlight .m { color: #0000DD; font-weight: bold } /* Literal.Number */ .highlight .s { color: #dd2200; background-color: #fff0f0 } /* Literal.String */ .highlight .na { color: #336699 } /* Name.Attribute */ .highlight .nb { color: #003388 } /* Name.Builtin */ .highlight .nc { color: #bb0066; font-weight: bold } /* Name.Class */ .highlight .no { color: #003366; font-weight: bold } /* Name.Constant */ .highlight .nd { color: #555555 } /* Name.Decorator */ .highlight .ne { color: #bb0066; font-weight: bold } /* Name.Exception */ .highlight .nf { color: #0066bb; font-weight: bold } /* Name.Function */ .highlight .nl { color: #336699; font-style: italic } /* Name.Label */ .highlight .nn { color: #bb0066; font-weight: bold } /* Name.Namespace */ .highlight .py { color: #336699; font-weight: bold } /* Name.Property */ .highlight .nt { color: #bb0066; font-weight: bold } /* Name.Tag */ .highlight .nv { color: #336699 } /* Name.Variable */ .highlight .ow { color: #008800 } /* Operator.Word */ .highlight .w { color: #bbbbbb } /* Text.Whitespace */ .highlight .mb { color: #0000DD; font-weight: bold } /* Literal.Number.Bin */ .highlight .mf { color: #0000DD; font-weight: bold } /* Literal.Number.Float */ .highlight .mh { color: #0000DD; font-weight: bold } /* Literal.Number.Hex */ .highlight .mi { color: #0000DD; font-weight: bold } /* Literal.Number.Integer */ .highlight .mo { color: #0000DD; font-weight: bold } /* Literal.Number.Oct */ .highlight .sa { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Affix */ .highlight .sb { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Backtick */ .highlight .sc { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Char */ .highlight .dl { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Delimiter */ .highlight .sd { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Doc */ .highlight .s2 { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Double */ .highlight .se { color: #0044dd; background-color: #fff0f0 } /* Literal.String.Escape */ .highlight .sh { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Heredoc */ .highlight .si { color: #3333bb; background-color: #fff0f0 } /* Literal.String.Interpol */ .highlight .sx { color: #22bb22; background-color: #f0fff0 } /* Literal.String.Other */ .highlight .sr { color: #008800; background-color: #fff0ff } /* Literal.String.Regex */ .highlight .s1 { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Single */ .highlight .ss { color: #aa6600; background-color: #fff0f0 } /* Literal.String.Symbol */ .highlight .bp { color: #003388 } /* Name.Builtin.Pseudo */ .highlight .fm { color: #0066bb; font-weight: bold } /* Name.Function.Magic */ .highlight .vc { color: #336699 } /* Name.Variable.Class */ .highlight .vg { color: #dd7700 } /* Name.Variable.Global */ .highlight .vi { color: #3333bb } /* Name.Variable.Instance */ .highlight .vm { color: #336699 } /* Name.Variable.Magic */ .highlight .il { color: #0000DD; font-weight: bold } /* Literal.Number.Integer.Long */ }
/*
 * Copyright (c) 2015 Cisco and/or its affiliates.
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at:
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
/*
  Copyright (c) 2001-2005 Eliot Dresselhaus

  Permission is hereby granted, free of charge, to any person obtaining
  a copy of this software and associated documentation files (the
  "Software"), to deal in the Software without restriction, including
  without limitation the rights to use, copy, modify, merge, publish,
  distribute, sublicense, and/or sell copies of the Software, and to
  permit persons to whom the Software is furnished to do so, subject to
  the following conditions:

  The above copyright notice and this permission notice shall be
  included in all copies or substantial portions of the Software.

  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
  EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
  MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
  NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
  LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
  OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
  WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/

#ifndef included_hash_h
#define included_hash_h

#include <vppinfra/error.h>
#include <vppinfra/format.h>
#include <vppinfra/vec.h>
#include <vppinfra/vector.h>

struct hash_header;

typedef uword (hash_key_sum_function_t) (struct hash_header *, uword key);
typedef uword (hash_key_equal_function_t)
  (struct hash_header *, uword key1, uword key2);

/* Vector header for hash tables. */
typedef struct hash_header
{
  /* Number of elements in hash table. */
  uword elts;

  /* Flags as follows. */
  u32 flags;

  /* Set if user does not want table to auto-resize when sufficiently full. */
#define HASH_FLAG_NO_AUTO_GROW		(1 << 0)
  /* Set if user does not want table to auto-resize when sufficiently empty. */
#define HASH_FLAG_NO_AUTO_SHRINK	(1 << 1)
  /* Set when hash_next is in the process of iterating through this hash table. */
#define HASH_FLAG_HASH_NEXT_IN_PROGRESS (1 << 2)

  u32 log2_pair_size;

  /* Function to compute the "sum" of a hash key.
     Hash function is this sum modulo the prime size of
     the hash table (vec_len (v)). */
  hash_key_sum_function_t *key_sum;

  /* Special values for key_sum "function". */
#define KEY_FUNC_NONE		(0)	/*< sum = key */
#define KEY_FUNC_POINTER_UWORD	(1)	/*< sum = *(uword *) key */
#define KEY_FUNC_POINTER_U32	(2)	/*< sum = *(u32 *) key */
#define KEY_FUNC_STRING         (3)	/*< sum = string_key_sum, etc. */

  /* key comparison function */
  hash_key_equal_function_t *key_equal;

  /* Hook for user's data.  Used to parameterize sum/equal functions. */
  any user;

  /* Format a (k,v) pair */
  format_function_t *format_pair;

  /* Format function arg */
  void *format_pair_arg;

  /* Bit i is set if pair i is a user object (as opposed to being
     either zero or an indirect array of pairs). */
  uword is_user[0];
} hash_t;

/* Hash header size in bytes */
always_inline uword
hash_header_bytes (void *v)
{
  hash_t *h;
  uword is_user_bytes =
    (sizeof (h->is_user[0]) * vec_len (v)) / BITS (h->is_user[0]);
  return sizeof (h[0]) + is_user_bytes;
}

/* Returns a pointer to the hash header given the vector pointer */
always_inline hash_t *
hash_header (void *v)
{
  return vec_header (v, hash_header_bytes (v));
}

/* Number of elements in the hash table */
always_inline uword
hash_elts (void *v)
{
  hash_t *h = hash_header (v);
  return v ? h->elts : 0;
}

/* Number of elements the hash table can hold */
always_inline uword
hash_capacity (void *v)
{
  return vec_len (v);
}

/* Returns 1 if the hash pair contains user data */
always_inline uword
hash_is_user (void *v, uword i)
{
  hash_t *h = hash_header (v);
  uword i0 = i / BITS (h->is_user[0]);
  uword i1 = i % BITS (h->is_user[0]);
  return (h->is_user[i0] & ((uword) 1 << i1)) != 0;
}

/* Set the format function and format argument for a hash table */
always_inline void
hash_set_pair_format (void *v,
		      format_function_t * format_pair, void *format_pair_arg)
{
  hash_t *h = hash_header (v);
  h->format_pair = format_pair;
  h->format_pair_arg = format_pair_arg;
}

/* Set hash table flags */
always_inline void
hash_set_flags (void *v, uword flags)
{
  hash_header (v)->flags |= flags;
}

/* Key value pairs. */
typedef struct
{
  /* The Key */
  uword key;

  /* The Value. Length is 2^log2_pair_size - 1. */
  uword value[0];
} hash_pair_t;

/* The indirect pair structure

    If log2_pair_size > 0 we overload hash pairs
    with indirect pairs for buckets with more than one
    pair. */
typedef struct
{
  /* pair vector */
  hash_pair_t *pairs;
  /* padding */
  u8 pad[sizeof (uword) - sizeof (hash_pair_t *)];
  /* allocated length */
  uword alloc_len;
}
hash_pair_indirect_t;

/* Direct / Indirect pair union */
typedef union
{
  hash_pair_t direct;
  hash_pair_indirect_t indirect;
} hash_pair_union_t;

#define LOG2_ALLOC_BITS (5)
#define PAIR_BITS	(BITS (uword) - LOG2_ALLOC_BITS)

/* Log2 number of bytes allocated in pairs array. */
always_inline uword
indirect_pair_get_log2_bytes (hash_pair_indirect_t * p)
{
  return p->alloc_len >> PAIR_BITS;
}

/* Get the length of an indirect pair */
always_inline uword
indirect_pair_get_len (hash_pair_indirect_t * p)
{
  if (!p->pairs)
    return 0;
  else
    return p->alloc_len & (((uword) 1 << PAIR_BITS) - 1);
}

/* Set the length of an indirect pair */
always_inline void
indirect_pair_set (hash_pair_indirect_t * p, uword log2_alloc, uword len)
{
  ASSERT (len < ((uword) 1 << PAIR_BITS));
  ASSERT (log2_alloc < ((uword) 1 << LOG2_ALLOC_BITS));
  p->alloc_len = (log2_alloc << PAIR_BITS) | len;
}

/* internal routine to fetch value for given key */
uword *_hash_get (void *v, uword key);

/* internal routine to fetch value (key, value) pair for given key */
hash_pair_t *_hash_get_pair (void *v, uword key);

/* internal routine to unset a (key, value) pair */
void *_hash_unset (void *v, uword key, void *old_value);

/* internal routine to set a (key, value) pair, return the old value */
void *_hash_set3 (void *v, uword key, void *value, void *old_value);

/* Resize a hash table */
void *hash_resize (void *old, uword new_size);

/* duplicate a hash table */
void *hash_dup (void *old);

/* Returns the number of bytes used by a hash table */
uword hash_bytes (void *v);

/* Public macro to set a (key, value) pair, return the old value */
#define hash_set3(h,key,value,old_value)				\
({									\
  uword _v = (uword) (value);						\
  (h) = _hash_set3 ((h), (uword) (key), (void *) &_v, (old_value));	\
})

/* Public macro to fetch value for given key */
#define hash_get(h,key)		_hash_get ((h), (uword) (key))

/* Public macro to fetch value (key, value) pair for given key */
#define hash_get_pair(h,key)	_hash_get_pair ((h), (uword) (key))

/* Public macro to set a (key, value) pair */
#define hash_set(h,key,value)	hash_set3(h,key,value,0)

/* Public macro to set (key, 0) pair */
#define hash_set1(h,key)	(h) = _hash_set3(h,(uword) (key),0,0)

/* Public macro to unset a (key, value) pair */
#define hash_unset(h,key)	((h) = _hash_unset ((h), (uword) (key),0))

/* Public macro to unset a (key, value) pair, return the old value */
#define hash_unset3(h,key,old_value) ((h) = _hash_unset ((h), (uword) (key), (void *) (old_value)))

/* get/set/unset for pointer keys. */

/* Public macro to fetch value for given pointer key */
#define hash_get_mem(h,key)	_hash_get ((h), pointer_to_uword (key))

/* Public macro to fetch (key, value) for given pointer key */
#define hash_get_pair_mem(h,key) _hash_get_pair ((h), pointer_to_uword (key))

/* Public macro to set (key, value) for pointer key */
#define hash_set_mem(h,key,value) hash_set3 (h, pointer_to_uword (key), (value), 0)

/* Public macro to set (key, 0) for pointer key */
#define hash_set1_mem(h,key)     hash_set3 ((h), pointer_to_uword (key), 0, 0)

/* Public macro to unset (key, value) for pointer key */
#define hash_unset_mem(h,key)    ((h) = _hash_unset ((h), pointer_to_uword (key),0))

/* internal routine to free a hash table */
extern void *_hash_free (void *v);

/* Public macro to free a hash table */
#define hash_free(h) (h) = _hash_free ((h))

clib_error_t *hash_validate (void *v);

/* Public inline funcion to get the number of value bytes for a hash table */
always_inline uword
hash_value_bytes (hash_t * h)
{
  hash_pair_t *p;
  return (sizeof (p->value[0]) << h->log2_pair_size) - sizeof (p->key);
}

/* Public inline funcion to get log2(size of a (key,value) pair) */
always_inline uword
hash_pair_log2_bytes (hash_t * h)
{
  uword log2_bytes = h->log2_pair_size;
  ASSERT (BITS (hash_pair_t) == 32 || BITS (hash_pair_t) == 64);
  if (BITS (hash_pair_t) == 32)
    log2_bytes += 2;
  else if (BITS (hash_pair_t) == 64)
    log2_bytes += 3;
  return log2_bytes;
}

/* Public inline funcion to get size of a (key,value) pair */
always_inline uword
hash_pair_bytes (hash_t * h)
{
  return (uword) 1 << hash_pair_log2_bytes (h);
}

/* Public inline funcion to advance a pointer past one (key,value) pair */
always_inline void *
hash_forward1 (hash_t * h, void *v)
{
  return (u8 *) v + hash_pair_bytes (h);
}

/* Public inline funcion to advance a pointer past N (key,value) pairs */
always_inline void *
hash_forward (hash_t * h, void *v, uword n)
{
  return (u8 *) v + ((n * sizeof (hash_pair_t)) << h->log2_pair_size);
}

/** Iterate over hash pairs.

    @param p    The current (key,value) pair. This should be of type
                <code>(hash_pair_t *)</code>.
    @param v    The hash table to iterate.
    @param body The operation to perform on each (key,value) pair.

    Executes the expression or code block @c body with each active hash pair.
*/
/* A previous version of this macro made use of the hash_pair_union_t
 * structure; this version does not since that approach mightily upset
 * the static analysis tool. In the rare chance someone is reading this
 * code, pretend that _p below is of type hash_pair_union_t and that when
 * used as an rvalue it's really using one of the union members as the
 * rvalue. If you were confused before you might be marginally less
 * confused after.
 */
#define hash_foreach_pair(p,v,body)                                         \
do {                                                                        \
 __label__ _hash_foreach_done;                                              \
  hash_t * _h = hash_header (v);                                            \
  void * _p;                                                                \
  hash_pair_t * _q, * _q_end;                                               \
  uword _i, _i1, _id, _pair_increment;                                      \
                                                                            \
  _p = (v);                                                                 \
  _i = 0;                                                                   \
  _pair_increment = 1;                                                      \
  if ((v))                                                                  \
      _pair_increment = 1 << _h->log2_pair_size;                            \
  while (_i < hash_capacity (v))                                            \
    {                                                                       \
      _id = _h->is_user[_i / BITS (_h->is_user[0])];                        \
      _i1 = _i + BITS (_h->is_user[0]);                                     \
                                                                            \
      do {                                                                  \
        if (_id & 1)                                                        \
          {                                                                 \
            _q = _p;                                                        \
            _q_end = _q + _pair_increment;                                  \
          }                                                                 \
        else                                                                \
          {                                                                 \
            hash_pair_indirect_t * _pi = _p;                                \
            _q = _pi->pairs;                                                \
            if (_h->log2_pair_size > 0)                                     \
              _q_end = hash_forward (_h, _q, indirect_pair_get_len (_pi));  \
            else                                                            \
              _q_end = vec_end (_q);                                        \
          }                                                                 \
                                                                            \
        /* Loop through all elements in bucket.                             \
           Bucket may have 0 1 or more (indirect case) pairs. */            \
        while (_q < _q_end)                                                 \
          {                                                                 \
            uword _break_in_body = 1;                                       \
            (p) = _q;                                                       \
            do {                                                            \
              body;                                                         \
              _break_in_body = 0;                                           \
            } while (0);                                                    \
            if (_break_in_body)                                             \
              goto _hash_foreach_done;                                      \
            _q += _pair_increment;                                          \
          }                                                                 \
                                                                            \
        _p = (hash_pair_t *)_p + _pair_increment;                           \
        _id = _id / 2;                                                      \
        _i++;                                                               \
      } while (_i < _i1);                                                   \
    }                                                                       \
  _hash_foreach_done:                                                       \
  /* Be silent Mr. Compiler-Warning. */                                     \
  ;                                                                         \
 } while (0)

/* Iterate over key/value pairs

    @param key_var the current key
    @param value_var the current value
    @param h the hash table to iterate across
    @param body the operation to perform on each (key_var,value_var) pair.

    calls body with each active hash pair
*/
/* Iteratate over key/value pairs. */
#define hash_foreach(key_var,value_var,h,body)			\
do {								\
  hash_pair_t * _r;						\
  hash_foreach_pair (_r, (h), {					\
    (key_var) = (__typeof__ (key_var)) _r->key;			\
    (value_var) = (__typeof__ (value_var)) _r->value[0];	\
    do { body; } while (0);					\
  });								\
} while (0)

/* Iterate over key/value pairs for pointer key hash tables

    @param key_var the current key
    @param value_var the current value
    @param h the hash table to iterate across
    @param body the operation to perform on each (key_var,value_var) pair.

    calls body with each active hash pair
*/
#define hash_foreach_mem(key_var,value_var,h,body)			\
do {									\
  hash_pair_t * _r;							\
  hash_foreach_pair (_r, (h), {						\
    (key_var) = (__typeof__ (key_var)) uword_to_pointer (_r->key, void *); \
    (value_var) = (__typeof__ (value_var)) _r->value[0];		\
    do { body; } while (0);						\
  });									\
} while (0)

/* Support for iteration through hash table. */

/* This struct saves iteration state for hash_next.
   None of these fields are meant to be visible to the user.
   Hence, the cryptic short-hand names. */
typedef struct
{
  uword i, j, f;
} hash_next_t;

hash_pair_t *hash_next (void *v, hash_next_t * hn);

void *_hash_create (uword elts, hash_t * h);

always_inline void
hash_set_value_bytes (hash_t * h, uword value_bytes)
{
  hash_pair_t *p;
  h->log2_pair_size =
    max_log2 ((sizeof (p->key) + value_bytes + sizeof (p->key) -
	       1) / sizeof (p->key));
}

#define hash_create2(_elts,_user,_value_bytes,               \
                     _key_sum,_key_equal,                    \
                     _format_pair,_format_pair_arg)          \
({							     \
  hash_t _h;						     \
  memset (&_h, 0, sizeof (_h));				     \
  _h.user = (_user);				             \
  _h.key_sum   = (hash_key_sum_function_t *) (_key_sum);     \
  _h.key_equal = (_key_equal);				     \
  hash_set_value_bytes (&_h, (_value_bytes));		     \
  _h.format_pair = (format_function_t *) (_format_pair);     \
  _h.format_pair_arg = (_format_pair_arg);                   \
  _hash_create ((_elts), &_h);				     \
})

/* Hash function based on that of Bob Jenkins (bob_jenkins@compuserve.com).
   Public domain per: http://www.burtleburtle.net/bob/hash/doobs.html
   Thanks, Bob. */

#define hash_mix_step(a,b,c,s0,s1,s2)		\
do {						\
  (a) -= (b) + (c); (a) ^= (c) >> (s0);		\
  (b) -= (c) + (a); (b) ^= (a) << (s1);		\
  (c) -= (a) + (b); (c) ^= (b) >> (s2);		\
} while (0)

#define hash_mix32_step_1(a,b,c) hash_mix_step(a,b,c,13,8,13)
#define hash_mix32_step_2(a,b,c) hash_mix_step(a,b,c,12,16,5)
#define hash_mix32_step_3(a,b,c) hash_mix_step(a,b,c,3,10,15)

#define hash_mix64_step_1(a,b,c) hash_mix_step(a,b,c,43,9,8)
#define hash_mix64_step_2(a,b,c) hash_mix_step(a,b,c,38,23,5)
#define hash_mix64_step_3(a,b,c) hash_mix_step(a,b,c,35,49,11)
#define hash_mix64_step_4(a,b,c) hash_mix_step(a,b,c,12,18,22)

/* Hash function based on that of Bob Jenkins (bob_jenkins@compuserve.com).
   Thanks, Bob. */
#define hash_mix64(a0,b0,c0)			\
do {						\
  hash_mix64_step_1 (a0, b0, c0);		\
  hash_mix64_step_2 (a0, b0, c0);		\
  hash_mix64_step_3 (a0, b0, c0);		\
  hash_mix64_step_4 (a0, b0, c0);		\
} while (0)					\

#define hash_mix32(a0,b0,c0)			\
do {						\
  hash_mix32_step_1 (a0, b0, c0);		\
  hash_mix32_step_2 (a0, b0, c0);		\
  hash_mix32_step_3 (a0, b0, c0);		\
} while (0)					\

/* Finalize from Bob Jenkins lookup3.c */

always_inline uword
hash32_rotate_left (u32 x, u32 i)
{
  return (x << i) | (x >> (BITS (i) - i));
}

#define hash_v3_mix32(a,b,c)					\
do {								\
  (a) -= (c); (a) ^= hash32_rotate_left ((c), 4); (c) += (b);	\
  (b) -= (a); (b) ^= hash32_rotate_left ((a), 6); (a) += (c);	\
  (c) -= (b); (c) ^= hash32_rotate_left ((b), 8); (b) += (a);	\
  (a) -= (c); (a) ^= hash32_rotate_left ((c),16); (c) += (b);	\
  (b) -= (a); (b) ^= hash32_rotate_left ((a),19); (a) += (c);	\
  (c) -= (b); (c) ^= hash32_rotate_left ((b), 4); (b) += (a);	\
} while (0)

#define hash_v3_finalize32(a,b,c)			\
do {							\
  (c) ^= (b); (c) -= hash32_rotate_left ((b), 14);	\
  (a) ^= (c); (a) -= hash32_rotate_left ((c), 11);	\
  (b) ^= (a); (b) -= hash32_rotate_left ((a), 25);	\
  (c) ^= (b); (c) -= hash32_rotate_left ((b), 16);	\
  (a) ^= (c); (a) -= hash32_rotate_left ((c),  4);	\
  (b) ^= (a); (b) -= hash32_rotate_left ((a), 14);	\
  (c) ^= (b); (c) -= hash32_rotate_left ((b), 24);	\
} while (0)

/* 32 bit mixing/finalize in steps. */

#define hash_v3_mix32_step1(a,b,c)				\
do {								\
  (a) -= (c); (a) ^= hash32_rotate_left ((c), 4); (c) += (b);	\
  (b) -= (a); (b) ^= hash32_rotate_left ((a), 6); (a) += (c);	\
} while (0)

#define hash_v3_mix32_step2(a,b,c)				\
do {								\
  (c) -= (b); (c) ^= hash32_rotate_left ((b), 8); (b) += (a);	\
  (a) -= (c); (a) ^= hash32_rotate_left ((c),16); (c) += (b);	\
} while (0)

#define hash_v3_mix32_step3(a,b,c)				\
do {								\
  (b) -= (a); (b) ^= hash32_rotate_left ((a),19); (a) += (c);	\
  (c) -= (b); (c) ^= hash32_rotate_left ((b), 4); (b) += (a);	\
} while (0)

#define hash_v3_finalize32_step1(a,b,c)			\
do {							\
  (c) ^= (b); (c) -= hash32_rotate_left ((b), 14);	\
  (a) ^= (c); (a) -= hash32_rotate_left ((c), 11);	\
} while (0)

#define hash_v3_finalize32_step2(a,b,c)			\
do {							\
  (b) ^= (a); (b) -= hash32_rotate_left ((a), 25);	\
  (c) ^= (b); (c) -= hash32_rotate_left ((b), 16);	\
} while (0)

#define hash_v3_finalize32_step3(a,b,c)			\
do {							\
  (a) ^= (c); (a) -= hash32_rotate_left ((c),  4);	\
  (b) ^= (a); (b) -= hash32_rotate_left ((a), 14);	\
  (c) ^= (b); (c) -= hash32_rotate_left ((b), 24);	\
} while (0)

/* Vector v3 mixing/finalize. */
#define hash_v3_mix_step_1_u32x(a,b,c)				\
do {								\
  (a) -= (c); (a) ^= u32x_irotate_left ((c), 4); (c) += (b);	\
  (b) -= (a); (b) ^= u32x_irotate_left ((a), 6); (a) += (c);	\
  (c) -= (b); (c) ^= u32x_irotate_left ((b), 8); (b) += (a);	\
} while (0)

#define hash_v3_mix_step_2_u32x(a,b,c)				\
do {								\
  (a) -= (c); (a) ^= u32x_irotate_left ((c),16); (c) += (b);	\
  (b) -= (a); (b) ^= u32x_irotate_left ((a),19); (a) += (c);	\
  (c) -= (b); (c) ^= u32x_irotate_left ((b), 4); (b) += (a);	\
} while (0)

#define hash_v3_finalize_step_1_u32x(a,b,c)		\
do {							\
  (c) ^= (b); (c) -= u32x_irotate_left ((b), 14);	\
  (a) ^= (c); (a) -= u32x_irotate_left ((c), 11);	\
  (b) ^= (a); (b) -= u32x_irotate_left ((a), 25);	\
} while (0)

#define hash_v3_finalize_step_2_u32x(a,b,c)		\
do {							\
  (c) ^= (b); (c) -= u32x_irotate_left ((b), 16);	\
  (a) ^= (c); (a) -= u32x_irotate_left ((c),  4);	\
  (b) ^= (a); (b) -= u32x_irotate_left ((a), 14);	\
  (c) ^= (b); (c) -= u32x_irotate_left ((b), 24);	\
} while (0)

#define hash_v3_mix_u32x(a,b,c)			\
do {						\
  hash_v3_mix_step_1_u32x(a,b,c);		\
  hash_v3_mix_step_2_u32x(a,b,c);		\
} while (0)

#define hash_v3_finalize_u32x(a,b,c)		\
do {						\
  hash_v3_finalize_step_1_u32x(a,b,c);		\
  hash_v3_finalize_step_2_u32x(a,b,c);		\
} while (0)

extern uword hash_memory (void *p, word n_bytes, uword state);

extern uword mem_key_sum (hash_t * h, uword key);
extern uword mem_key_equal (hash_t * h, uword key1, uword key2);

#define hash_create_mem(elts,key_bytes,value_bytes)	\
  hash_create2((elts),(key_bytes),(value_bytes),mem_key_sum,mem_key_equal,0,0)

extern uword vec_key_sum (hash_t * h, uword key);
extern uword vec_key_equal (hash_t * h, uword key1, uword key2);
extern u8 *vec_key_format_pair (u8 * s, va_list * args);

#define hash_create_vec(elts,key_bytes,value_bytes)	\
  hash_create2((elts),(key_bytes),(value_bytes),\
               vec_key_sum,vec_key_equal,vec_key_format_pair,0)

extern uword string_key_sum (hash_t * h, uword key);
extern uword string_key_equal (hash_t * h, uword key1, uword key2);
extern u8 *string_key_format_pair (u8 * s, va_list * args);

#define hash_create_string(elts,value_bytes)                    \
  hash_create2((elts),0,(value_bytes),                          \
               (hash_key_sum_function_t *) KEY_FUNC_STRING,     \
               (hash_key_equal_function_t *)KEY_FUNC_STRING,    \
               0, 0)

#define hash_create(elts,value_bytes)				\
  hash_create2((elts),0,(value_bytes),				\
               (hash_key_sum_function_t *) KEY_FUNC_NONE,	\
               (hash_key_equal_function_t *) KEY_FUNC_NONE,	\
               0,0)

#define hash_create_uword(elts,value_bytes)				\
  hash_create2((elts),0,(value_bytes),					\
               (hash_key_sum_function_t *) KEY_FUNC_POINTER_UWORD,	\
               (hash_key_equal_function_t *) KEY_FUNC_POINTER_UWORD,	\
               0,0)

#define hash_create_u32(elts,value_bytes)				\
  hash_create2((elts),0,(value_bytes),					\
               (hash_key_sum_function_t *) KEY_FUNC_POINTER_U32,	\
               (hash_key_equal_function_t *) KEY_FUNC_POINTER_U32,	\
               0,0)

u8 *format_hash (u8 * s, va_list * va);

/* Looks up input in hash table indexed by either vec string or
   c string (null terminated). */
unformat_function_t unformat_hash_vec_string;
unformat_function_t unformat_hash_string;

/* Main test routine. */
int test_hash_main (unformat_input_t * input);

#endif /* included_hash_h */

/*
 * fd.io coding-style-patch-verification: ON
 *
 * Local Variables:
 * eval: (c-set-style "gnu")
 * End:
 */