# Release Notes {#release_notes} * @subpage release_notes_1810 * @subpage release_notes_1807 * @subpage release_notes_1804 * @subpage release_notes_18012 * @subpage release_notes_18011 * @subpage release_notes_1801 * @subpage release_notes_1710 * @subpage release_notes_1707 * @subpage release_notes_1704 * @subpage release_notes_17011 * @subpage release_notes_1701 * @subpage release_notes_1609 * @subpage release_notes_1606 @page release_notes_1810 Release notes for VPP 18.10 More than 632 commits since the 18.07 release. ## Features ### Infrastructure - DPDK 18.08 integration - New Stats infrastructure (interface, error, node performance counters) - Add configurable "Doug Lea malloc" support ### VNET & Plugins - Load balancing: support per-port VIP and all-port VIP - Port NSH plugin to VPP - NAT - Configurable port range - Virtual Fragmentation Reassembly for endpoint-dependent mode - Client-IP based session affinity for load-balancing - TCP MSS clamping - Session timeout - Bug-fixing and performance optimizations ### Host stack - Support for applications with multiple workers - Support for binds from multiple app workers to same ip:port - Switched to a message queue for io and control event notifications - Support for eventfd based notifications as alternative to mutext-condvar pair - VCL refactor to support async event notifications and multiple workers - TLS async support in client for HW accleration - Performance optimizations and bug-fixing - A number of binary APIs will be deprecated in favor of using the event message queue. Details in the API section. ## Known issues For the full list of issues please refer to fd.io [JIRA](https://jira.fd.io). ## Issues fixed For the full list of fixed issues please refer to: - fd.io [JIRA](https://jira.fd.io) - git [commit log](https://git.fd.io/vpp/log/?h=stable/1810) ## API changes Description of results: * _Definition changed_: indicates that the API file was modified between releases. * _Only in image_: indicates the API is new for this release. * _Only in file_: indicates the API has been removed in this release. Message Name Result api_versions_reply definition changed app_cut_through_registration_add definition changed app_worker_add_del definition changed application_attach_reply definition changed bd_ip_mac_details only in image bd_ip_mac_dump only in image bfd_udp_get_echo_source definition changed bier_imp_details definition changed bier_route_details definition changed bind_sock definition changed bridge_domain_details definition changed bridge_flags definition changed classify_add_del_session definition changed classify_add_del_table definition changed connect_sock definition changed create_vhost_user_if definition changed get_first_msg_id_reply definition changed gpe_add_del_fwd_entry_reply definition changed gpe_fwd_entry_path_details definition changed ip6_fib_details definition changed ip6nd_proxy_details definition changed ip_add_del_route_reply definition changed ip_address_details definition changed ip_details definition changed ip_fib_details definition changed ip_mfib_details definition changed ip_mroute_add_del_reply definition changed ip_neighbor_add_del_reply definition changed ip_neighbor_details definition changed ip_reassembly_get_reply definition changed ip_unnumbered_details definition changed ipip_6rd_add_tunnel definition changed ipip_add_tunnel definition changed ipsec_spds_details only in image ipsec_spds_dump only in image l2_interface_efp_filter definition changed lisp_eid_table_vni_details definition changed map_another_segment definition changed mfib_signal_details definition changed mpls_route_add_del_reply definition changed mpls_tunnel_add_del definition changed mpls_tunnel_add_del_reply definition changed mpls_tunnel_details definition changed mpls_tunnel_dump definition changed one_eid_table_vni_details definition changed qos_mark_enable_disable definition changed qos_record_enable_disable definition changed reset_session_reply definition changed rpc_call definition changed show_threads definition changed sockclnt_create_reply definition changed sockclnt_delete definition changed sockclnt_delete_reply definition changed sw_interface_rx_placement_details only in image sw_interface_rx_placement_dump only in image sw_interface_set_ip_directed_broadcast definition changed sw_interface_set_l2_bridge definition changed sw_interface_set_rx_placement definition changed sw_interface_set_vxlan_gbp_bypass definition changed udp_encap_add definition changed udp_encap_add_del_reply only in file udp_encap_add_reply only in image udp_encap_del definition changed udp_encap_details definition changed unbind_sock definition changed vxlan_gbp_tunnel_add_del definition changed vxlan_gbp_tunnel_details only in image vxlan_gbp_tunnel_dump only in image Found 68 api message signature differences ### Patches that changed API definitions | @c src/plugins/avf/avf.api || | ------- | ------- | | [149d0e28](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=149d0e28) | avf: RSS support | | [4e6014fc](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=4e6014fc) | avf: api fix | | @c src/plugins/gbp/gbp.api || | ------- | ------- | | [c0a93143](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=c0a93143) | GBP Endpoint Updates | | [61b94c6b](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=61b94c6b) | vxlan-gbp: Add support for vxlan gbp | | @c src/plugins/igmp/igmp.api || | ------- | ------- | | [bdc0e6b7](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=bdc0e6b7) | Trivial: Clean up some typos. | | @c src/plugins/lb/lb.api || | ------- | ------- | | [6a4375e0](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=6a4375e0) | LB: fix flush flow table issue | | [49ca2601](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=49ca2601) | Add flush flag on del as command | | [219cc90c](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=219cc90c) | Support lb on both vip and per-port-vip case | | @c src/plugins/nat/nat.api || | ------- | ------- | | [bb4e0225](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=bb4e0225) | NAT: TCP MSS clamping | | [5d28c7af](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=5d28c7af) | NAT: add support for configurable port range (VPP-1346) | | [ea5b5be4](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=ea5b5be4) | NAT44: client-IP based session affinity for load-balancing (VPP-1297) | | [878c646a](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=878c646a) | NAT44: add support for session timeout (VPP-1272) | | [69ce30d6](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=69ce30d6) | NAT: update nat_show_config_reply API (VPP-1403) | | [6bd197eb](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=6bd197eb) | Remove client_index field from replies in API | | [c6c0d2a0](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=c6c0d2a0) | NAT44: LB NAT - local backends in multiple VRFs (VPP-1345) | | @c src/plugins/vmxnet3/vmxnet3.api || | ------- | ------- | | [df7f8e8c](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=df7f8e8c) | vmxnet3 device driver | | @c src/plugins/nsh/nsh.api || | ------- | ------- | | [d313f9e6](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=d313f9e6) | Port NSH plugin to VPP | | @c src/plugins/nsim/nsim.api || | ------- | ------- | | [9e3252b5](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=9e3252b5) | Network delay simulator plugin | | @c src/plugins/svs/svs.api || | ------- | ------- | | [d1e68ab7](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=d1e68ab7) | Source VRF Select | | @c src/vlibmemory/memclnt.api || | ------- | ------- | | [94495f2a](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=94495f2a) | PAPI: Use UNIX domain sockets instead of shared memory | | [6bd197eb](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=6bd197eb) | Remove client_index field from replies in API | | [75282457](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=75282457) | Fix "Old Style VLA" build warnings | | @c src/vnet/interface.api || | ------- | ------- | | [f0b42f48](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=f0b42f48) | itf: dump interface rx-placement | | [bdc0e6b7](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=bdc0e6b7) | Trivial: Clean up some typos. | | [54f7c51f](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=54f7c51f) | rx-placement: Add API call for interface rx-placement | | [1855b8e4](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=1855b8e4) | IP directed broadcast | | @c src/vnet/bfd/bfd.api || | ------- | ------- | | [2d3c7b9c](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=2d3c7b9c) | BFD: add get echo source API (VPP-1367) | | @c src/vnet/bier/bier.api || | ------- | ------- | | [ef90ed08](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=ef90ed08) | BIER API and load-balancing fixes | | [6bd197eb](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=6bd197eb) | Remove client_index field from replies in API | | @c src/vnet/classify/classify.api || | ------- | ------- | | [34eb5d42](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=34eb5d42) | classify_add_del_session API: Use more descriptive docstring (VPP-1385) | | [75282457](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=75282457) | Fix "Old Style VLA" build warnings | | @c src/vnet/devices/pipe/pipe.api || | ------- | ------- | | [208c29aa](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=208c29aa) | VOM: support for pipes | | @c src/vnet/devices/virtio/vhost_user.api || | ------- | ------- | | [ee2e58f6](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=ee2e58f6) | vhost-user: Add disable feature support in api | | @c src/vnet/ethernet/ethernet_types.api || | ------- | ------- | | [de5b08fb](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=de5b08fb) | Introduce a mac_address_t on the API and in VPP | | @c src/vnet/ip/ip_types.api || | ------- | ------- | | [d0df49f2](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=d0df49f2) | Use IP address types on UDP encap API | | @c src/vnet/ip/ip.api || | ------- | ------- | | [412ecd32](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=412ecd32) | Improve ip_mroute_add_del documentation | | [14260393](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=14260393) | Add adjacency counters to the stats segment | | [28c142e3](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=28c142e3) | mroute routers in the stats segment | | [008dbe10](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=008dbe10) | Route counters in the stats segment | | [de5b08fb](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=de5b08fb) | Introduce a mac_address_t on the API and in VPP | | [6bd197eb](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=6bd197eb) | Remove client_index field from replies in API | | [b11f903a](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=b11f903a) | Fix context field position in API definition | | @c src/vnet/ipip/ipip.api || | ------- | ------- | | [61502115](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=61502115) | IPIP and SIXRD tunnels create API needs table-IDs not fib-indexes | | @c src/vnet/ipsec/ipsec.api || | ------- | ------- | | [a9a0b2ce](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=a9a0b2ce) | IPsec: add API for SPDs dump (VPP-1363) | | [bdc0e6b7](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=bdc0e6b7) | Trivial: Clean up some typos. | | @c src/vnet/l2/l2.api || | ------- | ------- | | [0a4e0063](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=0a4e0063) | Fix documentation about sw_interface_set_l2_bridge | | [b474380f](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=b474380f) | L2 BD: introduce a BD interface on which to send UU packets | | [bdc0e6b7](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=bdc0e6b7) | Trivial: Clean up some typos. | | [5c7c49d1](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=5c7c49d1) | Fix documentation for SHG in bridge domain | | [5d82d2f1](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=5d82d2f1) | l2: arp termination dump | | [6b9b41c8](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=6b9b41c8) | L2 EFP: byteswap sw_if_index, enable flag can be u8 on .api | | @c src/vnet/lisp-cp/lisp.api || | ------- | ------- | | [bdc0e6b7](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=bdc0e6b7) | Trivial: Clean up some typos. | | [6bd197eb](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=6bd197eb) | Remove client_index field from replies in API | | @c src/vnet/lisp-cp/one.api || | ------- | ------- | | [bdc0e6b7](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=bdc0e6b7) | Trivial: Clean up some typos. | | [6bd197eb](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=6bd197eb) | Remove client_index field from replies in API | | @c src/vnet/lisp-gpe/lisp_gpe.api || | ------- | ------- | | [6bd197eb](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=6bd197eb) | Remove client_index field from replies in API | | [b11f903a](https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=b11f903a)
/*
* Copyright (c) 2015 Cisco and/or its affiliates.
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at:
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include <vppinfra/bitmap.h>
#include <vppinfra/os.h>
#include <vppinfra/qhash.h>
#include <vppinfra/random.h>
#include <vppinfra/time.h>
typedef struct
{
u32 n_iter, seed, n_keys, n_hash_keys, verbose;
u32 max_vector;
uword *hash;
uword *keys_in_hash_bitmap;
u32 *qhash;
uword *keys;
uword *lookup_keys;
uword *lookup_key_indices;
u32 *lookup_results;
u32 *get_multiple_results;
clib_time_t time;
f64 overflow_fraction, ave_elts;
f64 get_time, hash_get_time;
f64 set_time, set_count;
f64 unset_time, unset_count;
f64 hash_set_time, hash_unset_time;
} test_qhash_main_t;
clib_error_t *
test_qhash_main (unformat_input_t * input)
{
clib_error_t *error = 0;
test_qhash_main_t _tm, *tm = &_tm;
uword i, iter;
memset (tm, 0, sizeof (tm[0]));
tm->n_iter = 10;
tm->seed = 1;
tm->n_keys = 10;
tm->max_vector = 1;
while (unformat_check_input (input) != UNFORMAT_END_OF_INPUT)
{
if (unformat (input, "iter %d", &tm->n_iter))
;
else if (unformat (input, "seed %d", &tm->seed))
;
else if (unformat (input, "keys %d", &tm->n_keys))
;
else if (unformat (input, "size %d", &tm->n_hash_keys))
;
else if (unformat (input, "vector %d", &tm->max_vector))
;
else if (unformat (input, "verbose"))
tm->verbose = 1;
else
{
error = clib_error_create ("unknown input `%U'\n",
format_unformat_error, input);
goto done;
}
}
if (!tm->seed)
tm->seed = random_default_seed ();
clib_time_init (&tm->time);
clib_warning ("iter %d, seed %u, keys %d, max vector %d, ",
tm->n_iter, tm->seed, tm->n_keys, tm->max_vector);
vec_resize (tm->keys, tm->n_keys);
vec_resize (tm->get_multiple_results, tm->n_keys);
for (i = 0; i < vec_len (tm->keys); i++)
tm->keys[i] = random_uword (&tm->seed);
if (!tm->n_hash_keys)
tm->n_hash_keys = 2 * max_pow2 (tm->n_keys);
tm->n_hash_keys = clib_max (tm->n_keys, tm->n_hash_keys);
qhash_resize (tm->qhash, tm->n_hash_keys);
{
qhash_t *h = qhash_header (tm->qhash);
int i;
for (i = 0; i < ARRAY_LEN (h->hash_seeds); i++)
h->hash_seeds[i] = random_uword (&tm->seed);
}
vec_resize (tm->lookup_keys, tm->max_vector);
vec_resize (tm->lookup_key_indices, tm->max_vector);
vec_resize (tm->lookup_results, tm->max_vector);
for (iter = 0; iter < tm->n_iter; iter++)
{
uword *p, j, n, is_set;
n = tm->max_vector;
is_set = random_u32 (&tm->seed) & 1;
is_set |= hash_elts (tm->hash) < (tm->n_keys / 4);
if (hash_elts (tm->hash) > (3 * tm->n_keys) / 4)
is_set = 0;
_vec_len (tm->lookup_keys) = n;
_vec_len (tm->lookup_key_indices) = n;
j = 0;
while (j < n)
{
i = random_u32 (&tm->seed) % vec_len (tm->keys);
if (clib_bitmap_get (tm->keys_in_hash_bitmap, i) != is_set)
{
f64 t[2];
tm->lookup_key_indices[j] = i;
tm->lookup_keys[j] = tm->keys[i];
t[0] = clib_time_now (&tm->time);
if (is_set)
hash_set (tm->hash, tm->keys[i], i);
else
hash_unset (tm->hash, tm->keys[i]);
t[1] = clib_time_now (&tm->time);
if (is_set)
tm->hash_set_time += t[1] - t[0];
else
tm->hash_unset_time += t[1] - t[0];
tm->keys_in_hash_bitmap
= clib_bitmap_set (tm->keys_in_hash_bitmap, i, is_set);
j++;
}
}
{
f64 t[2];
if (is_set)
{
t[0] = clib_time_now (&tm->time);
qhash_set_multiple (tm->qhash,
tm->lookup_keys,
vec_len (tm->lookup_keys),
tm->lookup_results);
t[1] = clib_time_now (&tm->time);
tm->set_time += t[1] - t[0];
tm->set_count += vec_len (tm->lookup_keys);
for (i = 0; i < vec_len (tm->lookup_keys); i++)
{
uword r = tm->lookup_results[i];
*vec_elt_at_index (tm->qhash, r) = tm->lookup_key_indices[i];
}
}
else
{
t[0] = clib_time_now (&tm->time);
qhash_unset_multiple (tm->qhash,
tm->lookup_keys,
vec_len (tm->lookup_keys),
tm->lookup_results);
t[1] = clib_time_now (&tm->time);
tm->unset_time += t[1] - t[0];
tm->unset_count += vec_len (tm->lookup_keys);
for (i = 0; i < vec_len (tm->lookup_keys); i++)
{
uword r = tm->lookup_results[i];
*vec_elt_at_index (tm->qhash, r) = ~0;
}
}
}
if (qhash_elts (tm->qhash) != hash_elts (tm->hash))
os_panic ();
{
qhash_t *h;
uword i, k, l, count;
h = qhash_header (tm->qhash);
for (i = k = 0; k < vec_len (h->hash_key_valid_bitmap); k++)
i += count_set_bits (h->hash_key_valid_bitmap[k]);
k = hash_elts (h->overflow_hash);
l = qhash_elts (tm->qhash);
if (i + k != l)
os_panic ();
count = hash_elts (h->overflow_hash);
for (i = 0; i < (1 << h->log2_hash_size); i++)
count += tm->qhash[i] != ~0;
if (count != qhash_elts (tm->qhash))
os_panic ();
{
u32 *tmp = 0;
/* *INDENT-OFF* */
hash_foreach (k, l, h->overflow_hash, ({
j = qhash_hash_mix (h, k) / QHASH_KEYS_PER_BUCKET;
vec_validate (tmp, j);
tmp[j] += 1;
}));
/* *INDENT-ON* */
for (k = 0; k < vec_len (tmp); k++)
{
if (k >= vec_len (h->overflow_counts))
os_panic ();
if (h->overflow_counts[k] != tmp[k])
os_panic ();
}
for (; k < vec_len (h->overflow_counts); k++)
if (h->overflow_counts[k] != 0)
os_panic ();
vec_free (tmp);
}
}
{
f64 t[2];
t[0] = clib_time_now (&tm->time);
qhash_get_multiple (tm->qhash, tm->keys, vec_len (tm->keys),
tm->get_multiple_results);
t[1] = clib_time_now (&tm->time);
tm->get_time += t[1] - t[0];
for (i = 0; i < vec_len (tm->keys); i++)
{
u32 r;
t[0] = clib_time_now (&tm->time);
p = hash_get (tm->hash, tm->keys[i]);
t[1] = clib_time_now (&tm->time);
tm->hash_get_time += t[1] - t[0];
r = qhash_get (tm->qhash, tm->keys[i]);
if (p)
{
if (p[0] != i)
os_panic ();
if (*vec_elt_at_index (tm->qhash, r) != i)
os_panic ();
}
else
{
if (r != ~0)
os_panic ();
}
if (r != tm->get_multiple_results[i])
os_panic ();
}
}
tm->overflow_fraction +=
((f64) qhash_n_overflow (tm->qhash) / qhash_elts (tm->qhash));
tm->ave_elts += qhash_elts (tm->qhash);
}
fformat (stderr, "%d iter %.6e overflow, %.4f ave. elts\n",
tm->n_iter,
tm->overflow_fraction / tm->n_iter, tm->ave_elts / tm->n_iter);
tm->get_time /= tm->n_iter * vec_len (tm->keys);
tm->hash_get_time /= tm->n_iter * vec_len (tm->keys);
tm->set_time /= tm->set_count;
tm->unset_time /= tm->unset_count;
tm->hash_set_time /= tm->set_count;
tm->hash_unset_time /= tm->unset_count;
fformat (stderr,
"get/set/unset clocks %.2e %.2e %.2e clib %.2e %.2e %.2e ratio %.2f %.2f %.2f\n",
tm->get_time * tm->time.clocks_per_second,
tm->set_time * tm->time.clocks_per_second,
tm->unset_time * tm->time.clocks_per_second,
tm->hash_get_time * tm->time.clocks_per_second,
tm->hash_set_time * tm->time.clocks_per_second,
tm->hash_unset_time * tm->time.clocks_per_second,
tm->hash_get_time / tm->get_time, tm->hash_set_time / tm->set_time,
tm->hash_unset_time / tm->unset_time);
done:
return error;
}
#ifdef CLIB_UNIX
int
main (int argc, char *argv[])
{
unformat_input_t i;
clib_error_t *error;
unformat_init_command_line (&i, argv);
error = test_qhash_main (&i);
unformat_free (&i);
if (error)
{
clib_error_report (error);
return 1;
}
else
return 0;
}
#endif /* CLIB_UNIX */
/*
* fd.io coding-style-patch-verification: ON
*
* Local Variables:
* eval: (c-set-style "gnu")
* End:
*/