aboutsummaryrefslogtreecommitdiffstats
path: root/src/vnet/l2
AgeCommit message (Expand)AuthorFilesLines
2017-11-10Break up vpe.apiNeale Ranns2-5/+273
2017-11-10Further fix to SHG handling for ARP/ICMPv6 from BVI in a BDJohn Lo1-6/+25
2017-11-08Fix SHG handling for ARP/ICMPv6 received from BVI in a BDJohn Lo1-0/+8
2017-11-03silence l2fib_init() invalid read of size 2 found by address-sanitizerGabriel Ganne1-1/+6
2017-10-31Fix "l2fib add" CLI to allow adding of filter MAC entriesJohn Lo1-25/+17
2017-10-31l2fib: MAC: Fix uint64 to u8 byte arrayMohsin Kazmi5-49/+44
2017-10-24Update L2 header offset after VLAN tag rewriteNeale Ranns1-1/+3
2017-10-24Add extern to *_main global variable declarations in header files.Dave Wallace3-2/+4
2017-10-09vppapigen: support per-file (major,minor,patch) version stampsDave Barach1-0/+2
2017-10-03Update L2FIB entry timestamp only if BD aging enabled (VPP-1002)John Lo2-3/+5
2017-10-03Repair vlib API socket serverDave Barach1-5/+7
2017-10-02L2-FIB:add mac learn events testEyal Bari1-56/+49
2017-09-15Update of free text tag patch for BDJerome Tollet3-4/+7
2017-09-07Support for bridge domain free text tagJerome Tollet4-2/+66
2017-09-03Improve L2FIB PDR/NDR performance (VPP-963)John Lo4-18/+30
2017-08-22Increase default MAC learn limit and check it in learn-update pathJohn Lo5-4/+8
2017-08-03Add support for API client to receive L2 MAC eventsJohn Lo8-160/+494
2017-07-31CLI:add l2 input/outut to "sh int features"Eyal Bari5-1/+51
2017-07-24SPAN:add l2 mirrorEyal Bari3-10/+16
2017-07-23Improve L2 Input/Output Feature Infrastructure and UsageJohn Lo13-319/+154
2017-07-20L2FWD:move vec_validate out of access functionEyal Bari2-2/+9
2017-07-19Add a bihash prefetchable bucket-level cacheDave Barach1-4/+4
2017-07-14Introduce l{2,3,4}_hdr_offset fields in the buffer metadataDamjan Marion1-1/+1
2017-07-13Fix crash with worker threads on 4K VXLAN/BD setup (VPP-907)John Lo7-167/+49
2017-07-11L2INPUT:fix features mask cailculationEyal Bari1-6/+10
2017-06-27L2-LEARN:fix l2fib entry seq num not updated on hit (VPP-888)Eyal Bari5-87/+64
2017-06-19L2FWD:fix seq_num overwritten + validate l2fib entries when forwardingEyal Bari7-53/+87
2017-06-14ETH:fix l2_len/vlan count mismatch for > 2 tagsEyal Bari2-15/+3
2017-06-12L2FIB:fix crash in show with deleted subif entriesEyal Bari1-4/+7
2017-05-20API: Cleaning up message naming that does not follow the conventionsOle Troan2-52/+41
2017-05-19Enforce Bridge Domain ID range to match 24-bit VNI rangeJohn Lo3-7/+20
2017-05-15L2BD/API:fix bd dump to ignore unknown bd_idEyal Bari1-19/+20
2017-05-08L2FIB:CLI/API to flush all non-static entriesEyal Bari6-86/+156
2017-05-05Fix L2FIB learn counter and memory cleanup of mac_by_ip6 hash tableJohn Lo2-5/+16
2017-05-03L2FIB:flush interface learned macs on downEyal Bari4-39/+42
2017-05-02Prevent Bridge Domain operations on BD 0.Jon Loeliger2-2/+62
2017-04-28Fix memory leak on deletion of BD (bridge domain)John Lo1-3/+9
2017-04-25"autoreply" flag: autogenerate standard xxx_reply_t messagesDave Barach1-88/+8
2017-04-24BD:unify bridge domain creation codeEyal Bari3-106/+86
2017-04-06VPP-686: create bridge-domainChoonho Son1-2/+4
2017-04-06Use thread local storage for thread indexDamjan Marion3-11/+11
2017-04-05L2FIB:add l2fib_flush_bd l2fib_flush_int apisEyal Bari2-0/+82
2017-04-04BD/API:add bridge_domain_set_mac_age apiEyal Bari4-10/+55
2017-04-01CLI: create/delete bridge-domainChoonho Son2-1/+201
2017-03-28Implement MAC Flush for BD or Interface from the L2FIBJohn Lo8-46/+253
2017-03-16vlib: make runtime_data thread-localDamjan Marion2-0/+31
2017-03-07fix gcc 5.4 warning: argument to 'sizeof' in 'memcpy' call is the same expres...Gabriel Ganne2-10/+8
2017-03-03VPP-651: Ensure sw_if_index to node mapping for L2 output path is only done v...Andrew Yourtchenko3-3/+17
2017-03-02Clean up binary api message handler registration issuesDave Barach1-22/+0
2017-03-02bridge domain: fix members reordered when removingEyal Bari1-1/+1
e sure the calling libraries have appropriate robot library scope. For usual testing, it means "GLOBAL" scope.""" def __init__( self, directory, fail_on_mismatch=Constants.FAIL_ON_CRC_MISMATCH): """Initialize empty state, then register known collections. This also scans directory for .api.json files and performs initial checks, but does not report the findings yet. :param directory: Root directory of the search for .api.json files. :type directory: str """ self.fail_on_mismatch = fail_on_mismatch """If True, mismatch leads to test failure, by raising exception. If False, the mismatch is logged, but the test is allowed to continue. """ self._expected = dict() """Mapping from collection name to mapping from API name to CRC string. Collection name should be something useful for logging. Order of addition reflects the order collections should be queried. If an incompatible CRC is found, affected collections are removed. A CRC that would remove all does not, added to _reported instead, while causing a failure in single test (if fail_on_mismatch).""" self._missing = dict() """Mapping from collection name to mapping from API name to CRC string. Starts the same as _expected, but each time an encountered api,crc pair fits the expectation, the pair is removed from all collections within this mapping. Ideally, the active mappings will become empty. If not, it is an error, VPP removed or renamed a message CSIT needs.""" self._found = dict() """Mapping from API name to CRC string. This gets populated with CRCs found in .api.json, to serve as a hint when reporting errors.""" self._options = dict() """Mapping from API name to options dictionary. This gets populated with options found in .api.json, to serve as a hint when reporting errors.""" self._reported = dict() """Mapping from API name to CRC string. This gets populated with APIs used, but not found in collections, just before the fact is reported in an exception. The CRC comes from _found mapping (otherwise left as None). The idea is to not report those next time, allowing the job to find more problems in a single run.""" self._initial_conflicts_reported = False self._register_all() self._check_dir(directory) def log_and_raise(self, exc_msg): """Log to console, on fail_on_mismatch also raise runtime exception. :param exc_msg: The message to include in log or exception. :type exc_msg: str :raises RuntimeError: With the message, if fail_on_mismatch. """ logger.console("RuntimeError:\n{m}".format(m=exc_msg)) if self.fail_on_mismatch: raise RuntimeError(exc_msg) def _register_collection(self, collection_name, name_to_crc_mapping): """Add a named (copy of) collection of CRCs. :param collection_name: Helpful string describing the collection. :param name_to_crc_mapping: Mapping from API names to CRCs. :type collection_name: str or unicode :type name_to_crc_mapping: dict from str/unicode to str/unicode :raises RuntimeError: If the name of a collection is registered already. """ collection_name = _str(collection_name) if collection_name in self._expected: raise RuntimeError( f"Collection {collection_name!r} already registered." ) mapping = {_str(k): _str(v) for k, v in name_to_crc_mapping.items()} self._expected[collection_name] = mapping self._missing[collection_name] = mapping.copy() def _register_all(self): """Add all collections this CSIT codebase is tested against.""" file_path = os.path.normpath(os.path.join( os.path.dirname(os.path.abspath(__file__)), u"..", u"..", u"api", u"vpp", u"supported_crcs.yaml")) with open(file_path, u"rt") as file_in: collections_dict = yaml.safe_load(file_in.read()) for collection_name, name_to_crc_mapping in collections_dict.items(): self._register_collection(collection_name, name_to_crc_mapping) @staticmethod def _get_name(msg_obj): """Utility function to extract API name from an intermediate json. :param msg_obj: Loaded json object, item of "messages" list. :type msg_obj: list of various types :returns: Name of the message. :rtype: str or unicode :raises RuntimeError: If no name is found. """ for item in msg_obj: if isinstance(item, (dict, list)): continue return _str(item) raise RuntimeError(f"No name found for message: {msg_obj!r}") @staticmethod def _get_crc(msg_obj): """Utility function to extract API CRC from an intermediate json. :param msg_obj: Loaded json object, item of "messages" list. :type msg_obj: list of various types :returns: CRC of the message. :rtype: str or unicode :raises RuntimeError: If no CRC is found. """ for item in reversed(msg_obj): if not isinstance(item, dict): continue crc = item.get(u"crc", None) if crc: return _str(crc) raise RuntimeError(f"No CRC found for message: {msg_obj!r}") @staticmethod def _get_options(msg_obj, version): """Utility function to extract API options from an intermediate json. Empty dict is returned if options are not found, so old VPP builds can be tested without spamming. If version starts with "0.", add a fake option, as the message is treated as "in-progress" by the API upgrade process. :param msg_obj: Loaded json object, item of "messages" list. :param version: Version string from the .api.json file. :type msg_obj: list of various types :type version: Optional[str] :returns: Object found as value for "options" key. :rtype: dict """ options = dict() for item in reversed(msg_obj): if not isinstance(item, dict): continue options = item.get(u"options", dict()) if not options: break if version is None or version.startswith(u"0."): options[u"version"] = version return options def _process_crc(self, api_name, crc, options): """Compare API to verified collections, update class state. Here, API stands for (message name, CRC) pair. Conflict is NOT when a collection does not recognize the API. Such APIs are merely added to _found for later reporting. Conflict is when a collection recognizes the API under a different CRC. If a partial match happens, only the matching collections are preserved. On no match, all current collections are preserved, but the offending API is added to _reported mapping. Note that it is expected that collections are incompatible with each other for some APIs. The removal of collections on partial match is there to help identify the intended collection for the VPP build under test. But if no collection fits perfectly, the last collections to determine the "known" flag depends on the order of api_name submitted, which tends to be fairly random (depends on order of .api.json files). Order of collection registrations does not help much in this regard. Attempts to overwrite value in _found or _reported should not happen, so the code does not check for that, simply overwriting. Options are stored, to be examined later. The intended usage is to call this method multiple times, and then raise exception listing all _reported. :param api_name: API name to check. :param crc: Discovered CRC to check for the name. :param options: Empty dict or options value for in .api.json :type api_name: str :type crc: str :type options: dict """ # Regardless of the result, remember as found. self._found[api_name] = crc self._options[api_name] = options old_expected = self._expected new_expected = old_expected.copy() for collection_name, name_to_crc_mapping in old_expected.items(): if api_name not in name_to_crc_mapping: continue if name_to_crc_mapping[api_name] == crc: self._missing[collection_name].pop(api_name, None) continue # Remove the offending collection. new_expected.pop(collection_name, None) if new_expected: # Some collections recognized the CRC. self._expected = new_expected self._missing = {name: self._missing[name] for name in new_expected} return # No new_expected means some collections knew the api_name, # but CRC does not match any. This has to be reported. self._reported[api_name] = crc def _check_dir(self, directory): """Parse every .api.json found under directory, remember conflicts. As several collections are supported, each conflict invalidates some of them, failure happens only when no collections would be left. In that case, set of collections just before the failure is preserved, the _reported mapping is filled with conflicting APIs. The _found mapping is filled with discovered api names and crcs. The exception is not thrown here, but from report_initial_conflicts. :param directory: Root directory of the search for .api.json files. :type directory: str """ for root, _, files in os.walk(directory): for filename in files: if not filename.endswith(u".api.json"): continue with open(f"{root}/{filename}", u"rt") as file_in: json_obj = json.load(file_in) version = json_obj[u"options"].get(u"version", None) msgs = json_obj[u"messages"] for msg_obj in msgs: msg_name = self._get_name(msg_obj) msg_crc = self._get_crc(msg_obj) msg_options = self._get_options(msg_obj, version) self._process_crc(msg_name, msg_crc, msg_options) logger.debug(f"Surviving collections: {self._expected.keys()!r}") def report_initial_conflicts(self, report_missing=False): """Report issues discovered by _check_dir, if not done that already. Intended use: Call once after init, at a time when throwing exception is convenient. Optionally, report also missing messages. Missing reporting is disabled by default, because some messages come from plugins that might not be enabled at runtime. After the report, clear _reported, so that test cases report them again, thus tracking which message is actually used (by which test). :param report_missing: Whether to raise on missing messages. :type report_missing: bool :raises RuntimeError: If CRC mismatch or missing messages are detected, and fail_on_mismatch is True. """ if self._initial_conflicts_reported: return self._initial_conflicts_reported = True if self._reported: reported_indented = json.dumps( self._reported, indent=1, sort_keys=True, separators=[u",", u":"] ) self._reported = dict() self.log_and_raise( f"Incompatible API CRCs found in .api.json files:\n" f"{reported_indented}" ) if not report_missing: return missing = {name: mapp for name, mapp in self._missing.items() if mapp} if missing: missing_indented = json.dumps( missing, indent=1, sort_keys=True, separators=[u",", u":"]) self.log_and_raise( f"API CRCs missing from .api.json:\n{missing_indented}" ) def check_api_name(self, api_name): """Fail if the api_name has no, or different from known CRC associated. Print warning if options contain anything more than vat_help. Do not fail if this particular failure has been already reported. Intended use: Call during test (not in initialization), every time an API call is queued or response received. :param api_name: VPP API message name to check. :type api_name: str or unicode :raises RuntimeError: If no verified CRC for the api_name is found. """ api_name = _str(api_name) if api_name in self._reported: return old_expected = self._expected new_expected = old_expected.copy() for collection_name, name_to_crc_mapping in old_expected.items(): if api_name in name_to_crc_mapping: continue # Remove the offending collection. new_expected.pop(collection_name, None) if new_expected: # Some collections recognized the message name. self._expected = new_expected crc = self._found.get(api_name, None) matching = False if crc is not None: # Regardless of how many collections are remaining, # verify the known CRC is on one of them. for name_to_crc_mapping in self._expected.values(): if api_name not in name_to_crc_mapping: continue if name_to_crc_mapping[api_name] == crc: matching = True break if not matching: self._reported[api_name] = crc self.log_and_raise( f"No active collection has API {api_name!r} with CRC {crc!r}" ) options = self._options[api_name] options.pop(u"vat_help", None) if options: self._reported[api_name] = crc logger.console(f"{api_name} used but has options {options}") def print_warnings(self): """Call check_api_name for every API name in surviving collections. Useful for VPP CRC checking job. Even though there usually is only one surviving collection, the implementation has to be prepared for multiple collections, and it should de-duplicate api names. """ api_name_to_crc_maps = self._expected.values() api_name_sets = [set(n2c.keys()) for n2c in api_name_to_crc_maps] api_names = set().union(*api_name_sets) for api_name in sorted(api_names): self.check_api_name(api_name)