aboutsummaryrefslogtreecommitdiffstats
path: root/src/plugins/crypto_native
AgeCommit message (Collapse)AuthorFilesLines
2020-11-13crypto-native: fix multi-arch variant initializationRay Kinsella1-4/+0
crypto_native/main.h is being built as default, and crypto_native_main is initialized with a size of 64 bytes. crypto_native/aes_gcm.c and crypto_native/aes_cbc.c are march variants, their ICL variants are expecting crypto_native_main to be 256 bytes. Type: fix Signed-off-by: Georgii Tkachuk <georgii.tkachuk@intel.com> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Signed-off-by: Ray Kinsella <mdr@ashroe.eu> Change-Id: I4cddb75b712ea83c9cfca621887605d7bae104ec
2020-09-15crypto-native: fix issues detected by coiverityDamjan Marion2-3/+11
Type: fix Change-Id: Id61aa407eeeb4d44cf47ed39283a0c79ed3abbee Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-08-06misc: harmonize namesDave Barach1-8/+8
Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Ibad744788e200ce012ad88ff59c2c34920742454
2020-07-21crypto: bails out early for unsupported key typeBenoît Ganne1-4/+4
Do not access data structures based on uninitialized key->alg. Type: fix Fixes: f539578bac8b64886b57c460c9d74273e6613f8b Change-Id: I6bfb7e7a51af2c131b8bdf3bca6a38fcf1094760 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-05-11crypto-native: properly deal with broken or outdated toolchainsDamjan Marion5-42/+54
Avoids crash due to missing symbol, when build system detects toolchain which is not able to produce binaries for all targets we need.... Type: fix Change-Id: I77ee63cb8dca3c9e4e83a6235c60f1439a472444 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-04-30crypto: introduce async crypto infraFan Zhang1-0/+4
Type: feature Signed-off-by: Damjan Marion <damarion@cisco.com> Signed-off-by: Filip Tehlar <ftehlar@cisco.com> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com> Signed-off-by: Dariusz Kazimierski <dariuszx.kazimierski@intel.com> Signed-off-by: Piotr Kleski <piotrx.kleski@intel.com> Change-Id: I4c3fcccf55c36842b7b48aed260fef2802b5c54b
2020-04-29misc: switch to clang-9Damjan Marion1-3/+3
Type: improvement Change-Id: Iebf77a63c0c19b130a3fbd26b5293304a9fed4c1 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-03-06crypto: align per thread data to cache lineFilip Tehlar1-0/+1
Type: improvement Change-Id: I6bad46403c07b211dfda7229aed1b5e19342865f Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
2020-02-25crypto-native: GCM implementation with vector AESNI instructionsDamjan Marion1-4/+479
Introduced on intel IceLake uarch. Type: feature Change-Id: I1514c76c34e53ce0577666caf32a50f95eb6548f Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-02-24crypto-native: introduce aes_gcm_flags_tDamjan Marion1-84/+107
Type: refactor Change-Id: I53b4a9c0b63e6e6c6a13c33c5baa4c3de562584b Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-02-24crypto-native: introduce counter structDamjan Marion1-44/+51
Type: refactor Change-Id: I9ecc18ba3ec5f11622ea225690fb1dc262ea6fc1 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-02-24crypto-native: inverse Hi so it naturally fits into 512-bit registerDamjan Marion2-30/+34
Type: refactor Change-Id: I0c6ca9356af179abd0a414b356dea7e3a3eb0dd6 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-02-24crypto-native: replace aesni with aesDamjan Marion1-137/+134
This code also works on ARM so let's not use intel term.... Type: refactor Change-Id: Ie51d4359a83f2bf7a61c4861d486b7d009fc8057 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-02-24crypto-native: do not overwrite data after partial tagDamjan Marion1-1/+1
Type: fix Change-Id: I01de5f8813faff5406ccf67c21c42393c8648af6 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-02-17crypto-native: calculate ghash using vpclmulqdq instructionsDamjan Marion1-0/+126
vpclmulqdq is introduced on intel icelake architecture and allows computing 4 carry-less multiplications in paralled by using 512-bit SIMD registers Type: feature Change-Id: Idb09d6f51ba6f116bba11649b2d99f649356d449 Signed-off-by: Damjan Marion <damjan.marion@gmail.com>
2020-02-14crypto-native: refactor CBC codeDamjan Marion4-606/+440
Type: refactor Change-Id: I61e25942de318d03fb3d75689259709d687479bc Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-02-13vppinfra: add 128-bit and 512-bit a ^ b ^ c shortcutDamjan Marion1-18/+6
This allows us to combine 2 XOR operations into signle instruction which makes difference in crypto op: - in x86, by using ternary logic instruction - on ARM, by using EOR3 instruction (available with sha3 feature) Type: refactor Change-Id: Ibdf9001840399d2f838d491ca81b57cbd8430433 Signed-off-by: Damjan Marion <damjan.marion@gmail.com>
2020-02-13crypto-native: add AArch64 AES-GCM native implementationDamjan Marion5-55/+60
Type: feature Change-Id: I4f96b0af13b875d491704b010328a1814e1dbda1 Signed-off-by: Damjan Marion <dmarion@me.com>
2020-02-13crypto-native: add AArch64 ghash supportDamjan Marion1-1/+23
Type: improvement Change-Id: I1f204685ea9374389fc24fc53184ce06806beed3 Signed-off-by: Damjan Marion <dmarion@me.com>
2020-02-12crypto-native: fix ghash function namingDamjan Marion1-6/+6
Type: refactor Change-Id: I1d594af6d7c0d065d5c2decc5b22d549189b1882 Signed-off-by: Damjan Marion <dmarion@me.com>
2020-02-12crypto-native: use and instead of blendDamjan Marion1-4/+1
Faster and more portable... Type: improvement Change-Id: I915f8d3c45d1ba76ddfdbfac98482b04c98d148b Signed-off-by: Damjan Marion <dmarion@me.com>
2020-02-12crypto-native: add missing static_always_inlineDamjan Marion1-6/+6
Type: improvement Change-Id: Ibfa756f5f489882bc2ac658ae4d238e98da73875 Signed-off-by: Damjan Marion <dmarion@me.com>
2020-02-12crypto-native: refactor GCM code to use generic typesDamjan Marion3-291/+325
Type: refactor Change-Id: I76733a9ed362ec60badd22c0fbc2a9c5749da88d Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-02-11crypto-native: fix ARM compilatiion flagsDamjan Marion1-1/+1
Type: fix Change-Id: I3b39c17d9cfadc1d383d9c4dba9029ea279d09d7 Signed-off-by: Damjan Marion <dmarion@me.com>
2020-01-31crypto-native: add ARMv8 AES-CBC implementationDamjan Marion7-367/+756
Type: feature Change-Id: I32256061b9509880eec843db2f918879cdafbe47 Signed-off-by: Damjan Marion <dmarion@me.com>
2020-01-30crypto-native: refactor AES codeDamjan Marion4-166/+169
- use neutral types in preparation for ARMv8 support - simplify x86 key extraction support Type: refactor Change-Id: I947eb37b8c9d9ee6909bb32ef14c4de192d40a46 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-01-30crypto-native: don't expand aes-cbc keys twiceDamjan Marion2-11/+7
Type: refactor Change-Id: If0d9ec70f9e8c228c39505864a4a73bf94b67479 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-01-28crypto-native: rename crypto_ia32 to crypto_nativeDamjan Marion8-0/+1993
Type: refactor Change-Id: I9f21b3bf669ff913ff50afe5459cf52ff987e701 Signed-off-by: Damjan Marion <damarion@cisco.com>
#960050; background-color: #1e0010 } /* Error */ .highlight .k { color: #66d9ef } /* Keyword */ .highlight .l { color: #ae81ff } /* Literal */ .highlight .n { color: #f8f8f2 } /* Name */ .highlight .o { color: #f92672 } /* Operator */ .highlight .p { color: #f8f8f2 } /* Punctuation */ .highlight .ch { color: #75715e } /* Comment.Hashbang */ .highlight .cm { color: #75715e } /* Comment.Multiline */ .highlight .cp { color: #75715e } /* Comment.Preproc */ .highlight .cpf { color: #75715e } /* Comment.PreprocFile */ .highlight .c1 { color: #75715e } /* Comment.Single */ .highlight .cs { color: #75715e } /* Comment.Special */ .highlight .gd { color: #f92672 } /* Generic.Deleted */ .highlight .ge { font-style: italic } /* Generic.Emph */ .highlight .gi { color: #a6e22e } /* Generic.Inserted */ .highlight .gs { font-weight: bold } /* Generic.Strong */ .highlight .gu { color: #75715e } /* Generic.Subheading */ .highlight .kc { color: #66d9ef } /* Keyword.Constant */ .highlight .kd { color: #66d9ef } /* Keyword.Declaration */ .highlight .kn { color: #f92672 } /* Keyword.Namespace */ .highlight .kp { color: #66d9ef } /* Keyword.Pseudo */ .highlight .kr { color: #66d9ef } /* Keyword.Reserved */ .highlight .kt { color: #66d9ef } /* Keyword.Type */ .highlight .ld { color: #e6db74 } /* Literal.Date */ .highlight .m { color: #ae81ff } /* Literal.Number */ .highlight .s { color: #e6db74 } /* Literal.String */ .highlight .na { color: #a6e22e } /* Name.Attribute */ .highlight .nb { color: #f8f8f2 } /* Name.Builtin */ .highlight .nc { color: #a6e22e } /* Name.Class */ .highlight .no { color: #66d9ef } /* Name.Constant */ .highlight .nd { color: #a6e22e } /* Name.Decorator */ .highlight .ni { color: #f8f8f2 } /* Name.Entity */ .highlight .ne { color: #a6e22e } /* Name.Exception */ .highlight .nf { color: #a6e22e } /* Name.Function */ .highlight .nl { color: #f8f8f2 } /* Name.Label */ .highlight .nn { color: #f8f8f2 } /* Name.Namespace */ .highlight .nx { color: #a6e22e } /* Name.Other */ .highlight .py { color: #f8f8f2 } /* Name.Property */ .highlight .nt { color: #f92672 } /* Name.Tag */ .highlight .nv { color: #f8f8f2 } /* Name.Variable */ .highlight .ow { color: #f92672 } /* Operator.Word */ .highlight .w { color: #f8f8f2 } /* Text.Whitespace */ .highlight .mb { color: #ae81ff } /* Literal.Number.Bin */ .highlight .mf { color: #ae81ff } /* Literal.Number.Float */ .highlight .mh { color: #ae81ff } /* Literal.Number.Hex */ .highlight .mi { color: #ae81ff } /* Literal.Number.Integer */ .highlight .mo { color: #ae81ff } /* Literal.Number.Oct */ .highlight .sa { color: #e6db74 } /* Literal.String.Affix */ .highlight .sb { color: #e6db74 } /* Literal.String.Backtick */ .highlight .sc { color: #e6db74 } /* Literal.String.Char */ .highlight .dl { color: #e6db74 } /* Literal.String.Delimiter */ .highlight .sd { color: #e6db74 } /* Literal.String.Doc */ .highlight .s2 { color: #e6db74 } /* Literal.String.Double */ .highlight .se { color: #ae81ff } /* Literal.String.Escape */ .highlight .sh { color: #e6db74 } /* Literal.String.Heredoc */ .highlight .si { color: #e6db74 } /* Literal.String.Interpol */ .highlight .sx { color: #e6db74 } /* Literal.String.Other */ .highlight .sr { color: #e6db74 } /* Literal.String.Regex */ .highlight .s1 { color: #e6db74 } /* Literal.String.Single */ .highlight .ss { color: #e6db74 } /* Literal.String.Symbol */ .highlight .bp { color: #f8f8f2 } /* Name.Builtin.Pseudo */ .highlight .fm { color: #a6e22e } /* Name.Function.Magic */ .highlight .vc { color: #f8f8f2 } /* Name.Variable.Class */ .highlight .vg { color: #f8f8f2 } /* Name.Variable.Global */ .highlight .vi { color: #f8f8f2 } /* Name.Variable.Instance */ .highlight .vm { color: #f8f8f2 } /* Name.Variable.Magic */ .highlight .il { color: #ae81ff } /* Literal.Number.Integer.Long */ } @media (prefers-color-scheme: light) { .highlight .hll { background-color: #ffffcc } .highlight .c { color: #888888 } /* Comment */ .highlight .err { color: #a61717; background-color: #e3d2d2 } /* Error */ .highlight .k { color: #008800; font-weight: bold } /* Keyword */ .highlight .ch { color: #888888 } /* Comment.Hashbang */ .highlight .cm { color: #888888 } /* Comment.Multiline */ .highlight .cp { color: #cc0000; font-weight: bold } /* Comment.Preproc */ .highlight .cpf { color: #888888 } /* Comment.PreprocFile */ .highlight .c1 { color: #888888 } /* Comment.Single */ .highlight .cs { color: #cc0000; font-weight: bold; background-color: #fff0f0 } /* Comment.Special */ .highlight .gd { color: #000000; background-color: #ffdddd } /* Generic.Deleted */ .highlight .ge { font-style: italic } /* Generic.Emph */ .highlight .gr { color: #aa0000 } /* Generic.Error */ .highlight .gh { color: #333333 } /* Generic.Heading */ .highlight .gi { color: #000000; background-color: #ddffdd } /* Generic.Inserted */ .highlight .go { color: #888888 } /* Generic.Output */ .highlight .gp { color: #555555 } /* Generic.Prompt */ .highlight .gs { font-weight: bold } /* Generic.Strong */ .highlight .gu { color: #666666 } /* Generic.Subheading */ .highlight .gt { color: #aa0000 } /* Generic.Traceback */ .highlight .kc { color: #008800; font-weight: bold } /* Keyword.Constant */ .highlight .kd { color: #008800; font-weight: bold } /* Keyword.Declaration */ .highlight .kn { color: #008800; font-weight: bold } /* Keyword.Namespace */ .highlight .kp { color: #008800 } /* Keyword.Pseudo */ .highlight .kr { color: #008800; font-weight: bold } /* Keyword.Reserved */ .highlight .kt { color: #888888; font-weight: bold } /* Keyword.Type */ .highlight .m { color: #0000DD; font-weight: bold } /* Literal.Number */ .highlight .s { color: #dd2200; background-color: #fff0f0 } /* Literal.String */ .highlight .na { color: #336699 } /* Name.Attribute */ .highlight .nb { color: #003388 } /* Name.Builtin */ .highlight .nc { color: #bb0066; font-weight: bold } /* Name.Class */ .highlight .no { color: #003366; font-weight: bold } /* Name.Constant */ .highlight .nd { color: #555555 } /* Name.Decorator */ .highlight .ne { color: #bb0066; font-weight: bold } /* Name.Exception */ .highlight .nf { color: #0066bb; font-weight: bold } /* Name.Function */ .highlight .nl { color: #336699; font-style: italic } /* Name.Label */ .highlight .nn { color: #bb0066; font-weight: bold } /* Name.Namespace */ .highlight .py { color: #336699; font-weight: bold } /* Name.Property */ .highlight .nt { color: #bb0066; font-weight: bold } /* Name.Tag */ .highlight .nv { color: #336699 } /* Name.Variable */ .highlight .ow { color: #008800 } /* Operator.Word */ .highlight .w { color: #bbbbbb } /* Text.Whitespace */ .highlight .mb { color: #0000DD; font-weight: bold } /* Literal.Number.Bin */ .highlight .mf { color: #0000DD; font-weight: bold } /* Literal.Number.Float */ .highlight .mh { color: #0000DD; font-weight: bold } /* Literal.Number.Hex */ .highlight .mi { color: #0000DD; font-weight: bold } /* Literal.Number.Integer */ .highlight .mo { color: #0000DD; font-weight: bold } /* Literal.Number.Oct */ .highlight .sa { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Affix */ .highlight .sb { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Backtick */ .highlight .sc { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Char */ .highlight .dl { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Delimiter */ .highlight .sd { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Doc */ .highlight .s2 { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Double */ .highlight .se { color: #0044dd; background-color: #fff0f0 } /* Literal.String.Escape */ .highlight .sh { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Heredoc */ .highlight .si { color: #3333bb; background-color: #fff0f0 } /* Literal.String.Interpol */ .highlight .sx { color: #22bb22; background-color: #f0fff0 } /* Literal.String.Other */ .highlight .sr { color: #008800; background-color: #fff0ff } /* Literal.String.Regex */ .highlight .s1 { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Single */ .highlight .ss { color: #aa6600; background-color: #fff0f0 } /* Literal.String.Symbol */ .highlight .bp { color: #003388 } /* Name.Builtin.Pseudo */ .highlight .fm { color: #0066bb; font-weight: bold } /* Name.Function.Magic */ .highlight .vc { color: #336699 } /* Name.Variable.Class */ .highlight .vg { color: #dd7700 } /* Name.Variable.Global */ .highlight .vi { color: #3333bb } /* Name.Variable.Instance */ .highlight .vm { color: #336699 } /* Name.Variable.Magic */ .highlight .il { color: #0000DD; font-weight: bold } /* Literal.Number.Integer.Long */ }
# C generation
import datetime
import os
import time
import sys
from io import StringIO
import shutil

datestring = datetime.datetime.utcfromtimestamp(
    int(os.environ.get('SOURCE_DATE_EPOCH', time.time())))
input_filename = 'inputfil'
top_boilerplate = '''\
/*
 * VLIB API definitions {datestring}
 * Input file: {input_filename}
 * Automatically generated: please edit the input file NOT this file!
 */

#include <stdbool.h>
#if defined(vl_msg_id)||defined(vl_union_id) \\
    || defined(vl_printfun) ||defined(vl_endianfun) \\
    || defined(vl_api_version)||defined(vl_typedefs) \\
    || defined(vl_msg_name)||defined(vl_msg_name_crc_list) \\
    || defined(vl_api_version_tuple)
/* ok, something was selected */
#else
#warning no content included from {input_filename}
#endif

#define VL_API_PACKED(x) x __attribute__ ((packed))
'''

bottom_boilerplate = '''\
/****** API CRC (whole file) *****/

#ifdef vl_api_version
vl_api_version({input_filename}, {file_crc:#08x})

#endif
'''


def msg_ids(s):
    output = '''\

/****** Message ID / handler enum ******/

#ifdef vl_msg_id
'''

    for t in s['Define']:
        output += "vl_msg_id(VL_API_%s, vl_api_%s_t_handler)\n" % \
                  (t.name.upper(), t.name)
    output += "#endif"

    return output


def msg_names(s):
    output = '''\

/****** Message names ******/

#ifdef vl_msg_name
'''

    for t in s['Define']:
        dont_trace = 0 if t.dont_trace else 1
        output += "vl_msg_name(vl_api_%s_t, %d)\n" % (t.name, dont_trace)
    output += "#endif"

    return output


def msg_name_crc_list(s, suffix):
    output = '''\

/****** Message name, crc list ******/

#ifdef vl_msg_name_crc_list
'''
    output += "#define foreach_vl_msg_name_crc_%s " % suffix

    for t in s['Define']:
        output += "\\\n_(VL_API_%s, %s, %08x) " % \
                   (t.name.upper(), t.name, t.crc)
    output += "\n#endif"

    return output


def api2c(fieldtype):
    mappingtable = {'string': 'vl_api_string_t', }
    if fieldtype in mappingtable:
        return mappingtable[fieldtype]
    return fieldtype


def typedefs(filename):

    output = '''\

/****** Typedefs ******/

#ifdef vl_typedefs
#include "{include}.api_types.h"
#endif
'''.format(include=filename)
    return output


format_strings = {'u8': '%u',
                  'bool': '%u',
                  'i8': '%d',
                  'u16': '%u',
                  'i16': '%d',
                  'u32': '%u',
                  'i32': '%ld',
                  'u64': '%llu',
                  'i64': '%lld',
                  'f64': '%.2f'}

noprint_fields = {'_vl_msg_id': None,
                  'client_index': None,
                  'context': None}


class Printfun():
    _dispatch = {}

    def __init__(self, stream):
        self.stream = stream

    def print_string(self, o, stream):
        write = stream.write
        if o.modern_vla:
            write('    if (vl_api_string_len(&a->{f}) > 0) {{\n'
                  .format(f=o.fieldname))
            write('        s = format(s, "\\n%U{f}: %.*s", '
                  'format_white_space, indent, '
                  'vl_api_string_len(&a->{f}) - 1, '
                  'vl_api_from_api_string(&a->{f}));\n'.format(f=o.fieldname))
            write('    } else {\n')
            write('        s = format(s, "\\n%U{f}:", '
                  'format_white_space, indent);\n'.format(f=o.fieldname))
            write('    }\n')
        else:
            write('    s = format(s, "\\n%U{f}: %s", '
                  'format_white_space, indent, a->{f});\n'
                  .format(f=o.fieldname))

    def print_field(self, o, stream):
        write = stream.write
        if o.fieldname in noprint_fields:
            return
        if o.fieldtype in format_strings:
            f = format_strings[o.fieldtype]
            write('    s = format(s, "\\n%U{n}: {f}", '
                  'format_white_space, indent, a->{n});\n'
                  .format(n=o.fieldname, f=f))
        else:
            write('    s = format(s, "\\n%U{n}: %U", '
                  'format_white_space, indent, '
                  'format_{t}, &a->{n}, indent);\n'
                  .format(n=o.fieldname, t=o.fieldtype))

    _dispatch['Field'] = print_field

    def print_array(self, o, stream):
        write = stream.write

        forloop = '''\
    for (i = 0; i < {lfield}; i++) {{
        s = format(s, "\\n%U{n}: %U",
                   format_white_space, indent, format_{t}, &a->{n}[i], indent);
    }}
'''

        forloop_format = '''\
    for (i = 0; i < {lfield}; i++) {{
        s = format(s, "\\n%U{n}: {t}",
                   format_white_space, indent, a->{n}[i]);
    }}
'''

        if o.fieldtype == 'string':
            return self.print_string(o, stream)

        if o.fieldtype == 'u8':
            if o.lengthfield:
                write('    s = format(s, "\\n%U{n}: %U", format_white_space, '
                      'indent, format_hex_bytes, a->{n}, a->{lfield});\n'
                      .format(n=o.fieldname, lfield=o.lengthfield))
            else:
                write('    s = format(s, "\\n%U{n}: %U", format_white_space, '
                      'indent, format_hex_bytes, a, {lfield});\n'
                      .format(n=o.fieldname, lfield=o.length))
            return

        lfield = 'a->' + o.lengthfield if o.lengthfield else o.length
        if o.fieldtype in format_strings:
            write(forloop_format.format(lfield=lfield,
                                        t=format_strings[o.fieldtype],
                                        n=o.fieldname))
        else:
            write(forloop.format(lfield=lfield, t=o.fieldtype, n=o.fieldname))

    _dispatch['Array'] = print_array

    def print_alias(self, k, v, stream):
        write = stream.write
        if ('length' in v.alias and v.alias['length'] and
                v.alias['type'] == 'u8'):
            write('    return format(s, "%U", format_hex_bytes, a, {});\n'
                  .format(v.alias['length']))
        elif v.alias['type'] in format_strings:
            write('    return format(s, "{}", *a);\n'
                  .format(format_strings[v.alias['type']]))
        else:
            write('    return format(s, "{} (print not implemented)");\n'
                  .format(k))

    def print_enum(self, o, stream):
        write = stream.write
        write("    switch(*a) {\n")
        for b in o:
            write("    case %s:\n" % b[1])
            write('        return format(s, "{}");\n'.format(b[0]))
        write('    }\n')

    _dispatch['Enum'] = print_enum

    def print_obj(self, o, stream):
        write = stream.write

        if o.type in self._dispatch:
            self._dispatch[o.type](self, o, stream)
        else:
            write('    s = format(s, "\\n{} {} {} (print not implemented");\n'
                  .format(o.type, o.fieldtype, o.fieldname))


def printfun(objs, stream, modulename):
    write = stream.write

    h = '''\
/****** Print functions *****/
#ifdef vl_printfun
#ifndef included_{module}_printfun
#define included_{module}_printfun

#ifdef LP64
#define _uword_fmt \"%lld\"
#define _uword_cast (long long)
#else
#define _uword_fmt \"%ld\"
#define _uword_cast long
#endif

'''

    signature = '''\
static inline void *vl_api_{name}_t_print (vl_api_{name}_t *a, void *handle)
{{
    u8 *s = 0;
    u32 indent __attribute__((unused)) = 2;
    int i __attribute__((unused));
'''

    h = h.format(module=modulename)
    write(h)

    pp = Printfun(stream)
    for t in objs:
        if t.manual_print:
            write("/***** manual: vl_api_%s_t_print  *****/\n\n" % t.name)
            continue
        write(signature.format(name=t.name))
        write('    /* Message definition: vl_api_{}_t: */\n'.format(t.name))
        write("    s = format(s, \"vl_api_%s_t:\");\n" % t.name)
        for o in t.block:
            pp.print_obj(o, stream)
        write('    vec_add1(s, 0);\n')
        write('    vl_print (handle, (char *)s);\n')
        write('    vec_free (s);\n')
        write('    return handle;\n')
        write('}\n\n')

    write("\n#endif")
    write("\n#endif /* vl_printfun */\n")

    return ''


def printfun_types(objs, stream, modulename):
    write = stream.write
    pp = Printfun(stream)

    h = '''\
/****** Print functions *****/
#ifdef vl_printfun
#ifndef included_{module}_printfun_types
#define included_{module}_printfun_types

'''
    h = h.format(module=modulename)
    write(h)

    signature = '''\
static inline u8 *format_vl_api_{name}_t (u8 *s, va_list * args)
{{
    vl_api_{name}_t *a = va_arg (*args, vl_api_{name}_t *);
    u32 indent __attribute__((unused)) = va_arg (*args, u32);
    int i __attribute__((unused));
    indent += 2;
'''

    for t in objs:
        if t.__class__.__name__ == 'Enum':
            write(signature.format(name=t.name))
            pp.print_enum(t.block, stream)
            write('    return s;\n')
            write('}\n\n')
            continue

        if t.manual_print:
            write("/***** manual: vl_api_%s_t_print  *****/\n\n" % t.name)
            continue

        if t.__class__.__name__ == 'Using':
            write(signature.format(name=t.name))
            pp.print_alias(t.name, t, stream)
            write('}\n\n')
            continue

        write(signature.format(name=t.name))
        for o in t.block:
            pp.print_obj(o, stream)

        write('    return s;\n')
        write('}\n\n')

    write("\n#endif")
    write("\n#endif /* vl_printfun_types */\n")


def imports(imports):
    output = '/* Imported API files */\n'
    output += '#ifndef vl_api_version\n'

    for i in imports:
        s = i.filename.replace('plugins/', '')
        output += '#include <{}.h>\n'.format(s)
    output += '#endif\n'
    return output


endian_strings = {
    'u16': 'clib_net_to_host_u16',
    'u32': 'clib_net_to_host_u32',
    'u64': 'clib_net_to_host_u64',
    'i16': 'clib_net_to_host_i16',
    'i32': 'clib_net_to_host_i32',
    'i64': 'clib_net_to_host_i64',
    'f64': 'clib_net_to_host_f64',
}


def endianfun_array(o):
    forloop = '''\
    for (i = 0; i < {length}; i++) {{
        a->{name}[i] = {format}(a->{name}[i]);
    }}
'''

    forloop_format = '''\
    for (i = 0; i < {length}; i++) {{
        {type}_endian(&a->{name}[i]);
    }}
'''

    output = ''
    if o.fieldtype == 'u8' or o.fieldtype == 'string':
        output += '    /* a->{n} = a->{n} (no-op) */\n'.format(n=o.fieldname)
    else:
        lfield = 'a->' + o.lengthfield if o.lengthfield else o.length
        if o.fieldtype in endian_strings:
            output += (forloop
                       .format(length=lfield,
                               format=endian_strings[o.fieldtype],
                               name=o.fieldname))
        else:
            output += (forloop_format
                       .format(length=lfield, type=o.fieldtype,
                               name=o.fieldname))
    return output


def endianfun_obj(o):
    output = ''
    if o.type == 'Array':
        return endianfun_array(o)
    elif o.type != 'Field':
        output += ('    s = format(s, "\\n{} {} {} (print not implemented");\n'
                   .format(o.type, o.fieldtype, o.fieldname))
        return output
    if o.fieldtype in endian_strings:
        output += ('    a->{name} = {format}(a->{name});\n'
                   .format(name=o.fieldname,
                           format=endian_strings[o.fieldtype]))
    elif o.fieldtype.startswith('vl_api_'):
        output += ('    {type}_endian(&a->{name});\n'
                   .format(type=o.fieldtype, name=o.fieldname))
    else:
        output += '    /* a->{n} = a->{n} (no-op) */\n'.format(n=o.fieldname)

    return output


def endianfun(objs, modulename):
    output = '''\

/****** Endian swap functions *****/\n\
#ifdef vl_endianfun
#ifndef included_{module}_endianfun
#define included_{module}_endianfun

#undef clib_net_to_host_uword
#ifdef LP64
#define clib_net_to_host_uword clib_net_to_host_u64
#else
#define clib_net_to_host_uword clib_net_to_host_u32
#endif

'''
    output = output.format(module=modulename)

    signature = '''\
static inline void vl_api_{name}_t_endian (vl_api_{name}_t *a)
{{
    int i __attribute__((unused));
'''

    for t in objs:
        if t.__class__.__name__ == 'Enum':
            output += signature.format(name=t.name)
            if t.enumtype in endian_strings:
                output += ('    *a = {}(*a);\n'
                           .format(endian_strings[t.enumtype]))
            else:
                output += ('    /* a->{name} = a->{name} (no-op) */\n'
                           .format(name=t.name))

            output += '}\n\n'
            continue

        if t.manual_endian:
            output += "/***** manual: vl_api_%s_t_endian  *****/\n\n" % t.name
            continue

        if t.__class__.__name__ == 'Using':
            output += signature.format(name=t.name)
            if ('length' in t.alias and t.alias['length'] and
                    t.alias['type'] == 'u8'):
                output += ('    /* a->{name} = a->{name} (no-op) */\n'
                           .format(name=t.name))
            elif t.alias['type'] in format_strings:
                output += ('    *a = {}(*a);\n'
                           .format(endian_strings[t.alias['type']]))
            else:
                output += '    /* Not Implemented yet {} */'.format(t.name)
            output += '}\n\n'
            continue

        output += signature.format(name=t.name)

        for o in t.block:
            output += endianfun_obj(o)
        output += '}\n\n'

    output += "\n#endif"
    output += "\n#endif /* vl_endianfun */\n\n"

    return output


def version_tuple(s, module):
    output = '''\
/****** Version tuple *****/

#ifdef vl_api_version_tuple

'''
    if 'version' in s['Option']:
        v = s['Option']['version']
        (major, minor, patch) = v.split('.')
        output += "vl_api_version_tuple(%s, %s, %s, %s)\n" % \
                  (module, major, minor, patch)

    output += "\n#endif /* vl_api_version_tuple */\n\n"

    return output


def generate_include_enum(s, module, stream):
    write = stream.write

    if len(s['Define']):
        write('typedef enum {\n')
        for t in s['Define']:
            write('   VL_API_{},\n'.format(t.name.upper()))
        write('   VL_MSG_FIRST_AVAILABLE\n')
        write('}} vl_api_{}_enum_t;\n'.format(module))


#
# Generate separate API _types file.
#
def generate_include_types(s, module, stream):
    write = stream.write

    write('#ifndef included_{module}_api_types_h\n'.format(module=module))
    write('#define included_{module}_api_types_h\n'.format(module=module))

    if len(s['Import']):
        write('/* Imported API files */\n')
        for i in s['Import']:
            filename = i.filename.replace('plugins/', '')
            write('#include <{}_types.h>\n'.format(filename))

    for o in s['types'] + s['Define']:
        tname = o.__class__.__name__
        if tname == 'Using':
            if 'length' in o.alias:
                write('typedef %s vl_api_%s_t[%s];\n' % (o.alias['type'], o.name, o.alias['length']))
            else:
                write('typedef %s vl_api_%s_t;\n' % (o.alias['type'], o.name))
        elif tname == 'Enum':
            if o.enumtype == 'u32':
                write("typedef enum {\n")
            else:
                write("typedef enum __attribute__((packed)) {\n")

            for b in o.block:
                write("    %s = %s,\n" % (b[0], b[1]))
            write('} vl_api_%s_t;\n' % o.name)
            if o.enumtype != 'u32':
                size1 = 'sizeof(vl_api_%s_t)' % o.name
                size2 = 'sizeof(%s)' % o.enumtype
                err_str = 'size of API enum %s is wrong' % o.name
                write('STATIC_ASSERT(%s == %s, "%s");\n'
                      % (size1, size2, err_str))
        else:
            if tname == 'Union':
                write("typedef union __attribute__ ((packed)) _vl_api_%s {\n" % o.name)
            else:
                write(("typedef struct __attribute__ ((packed)) _vl_api_%s {\n")
                           % o.name)
            for b in o.block:
                if b.type == 'Option':
                    continue
                if b.type == 'Field':
                      write("    %s %s;\n" % (api2c(b.fieldtype),
                                              b.fieldname))
                elif b.type == 'Array':
                    if b.lengthfield:
                      write("    %s %s[0];\n" % (api2c(b.fieldtype),
                                                 b.fieldname))
                    else:
                        # Fixed length strings decay to nul terminated u8
                        if b.fieldtype == 'string':
                            if b.modern_vla:
                                write('    {} {};\n'
                                      .format(api2c(b.fieldtype),
                                              b.fieldname))
                            else:
                                write('    u8 {}[{}];\n'
                                      .format(b.fieldname, b.length))
                        else:
                            write("    %s %s[%s];\n" %
                                  (api2c(b.fieldtype), b.fieldname,
                                   b.length))
                else:
                    raise ValueError("Error in processing type {} for {}"
                                     .format(b, o.name))

            write('} vl_api_%s_t;\n' % o.name)

    write("\n#endif\n")


def generate_c_boilerplate(services, defines, file_crc, module, stream):
    write = stream.write

    hdr = '''\
#define vl_endianfun		/* define message structures */
#include "{module}.api.h"
#undef vl_endianfun

/* instantiate all the print functions we know about */
#define vl_print(handle, ...) vlib_cli_output (handle, __VA_ARGS__)
#define vl_printfun
#include "{module}.api.h"
#undef vl_printfun

'''

    write(hdr.format(module=module))
    write('static u16\n')
    write('setup_message_id_table (void) {\n')
    write('   api_main_t *am = &api_main;\n')
    write('   u16 msg_id_base = vl_msg_api_get_msg_ids ("{}_{crc:08x}", VL_MSG_FIRST_AVAILABLE);\n'
          .format(module, crc=file_crc))


    for d in defines:
        write('   vl_msg_api_add_msg_name_crc (am, "{n}_{crc:08x}",\n'
              '                                VL_API_{ID} + msg_id_base);\n'
              .format(n=d.name, ID=d.name.upper(), crc=d.crc))
    for s in services:
        write('   vl_msg_api_set_handlers(VL_API_{ID} + msg_id_base, "{n}",\n'
              '                           vl_api_{n}_t_handler, vl_noop_handler,\n'
              '                           vl_api_{n}_t_endian, vl_api_{n}_t_print,\n'
              '                           sizeof(vl_api_{n}_t), 1);\n'
              .format(n=s.caller, ID=s.caller.upper()))

    write('   return msg_id_base;\n')
    write('}\n')


def generate_c_test_plugin_boilerplate(services, defines, file_crc, module, stream):
    write = stream.write

    define_hash = {d.name:d for d in defines}
    replies = {}

    hdr = '''\
#define vl_endianfun		/* define message structures */
#include "{module}.api.h"
#undef vl_endianfun

/* instantiate all the print functions we know about */
#define vl_print(handle, ...) vlib_cli_output (handle, __VA_ARGS__)
#define vl_printfun
#include "{module}.api.h"
#undef vl_printfun

'''

    write(hdr.format(module=module))
    for s in services:
        try:
            d = define_hash[s.reply]
        except:
            continue
        if d.manual_print:
            write('/* Manual definition requested for: vl_api_{n}_t_handler() */\n'
                  .format(n=s.reply))
            continue
        if not define_hash[s.caller].autoreply:
            write('/* Only autoreply is supported (vl_api_{n}_t_handler()) */\n'
                  .format(n=s.reply))
            continue
        write('#ifndef VL_API_{n}_T_HANDLER\n'.format(n=s.reply.upper()))
        write('static void\n')
        write('vl_api_{n}_t_handler (vl_api_{n}_t * mp) {{\n'.format(n=s.reply))
        write('   vat_main_t * vam = {}_test_main.vat_main;\n'.format(module))
        write('   i32 retval = ntohl(mp->retval);\n')
        write('   if (vam->async_mode) {\n')
        write('      vam->async_errors += (retval < 0);\n')
        write('   } else {\n')
        write('      vam->retval = retval;\n')
        write('      vam->result_ready = 1;\n')
        write('   }\n')
        write('}\n')
        write('#endif\n')

        for e in s.events:
            if define_hash[e].manual_print:
                continue
            write('static void\n')
            write('vl_api_{n}_t_handler (vl_api_{n}_t * mp) {{\n'.format(n=e))
            write('    vl_print(0, "{n} event called:");\n'.format(n=e))
            write('    vl_api_{n}_t_print(mp, 0);\n'.format(n=e))
            write('}\n')

    write('static void\n')
    write('setup_message_id_table (vat_main_t * vam, u16 msg_id_base) {\n')
    for s in services:
        write('   vl_msg_api_set_handlers(VL_API_{ID} + msg_id_base, "{n}",\n'
              '                           vl_api_{n}_t_handler, vl_noop_handler,\n'
              '                           vl_api_{n}_t_endian, vl_api_{n}_t_print,\n'
              '                           sizeof(vl_api_{n}_t), 1);\n'
              .format(n=s.reply, ID=s.reply.upper()))
        write('   hash_set_mem (vam->function_by_name, "{n}", api_{n});\n'.format(n=s.caller))
        try:
            write('   hash_set_mem (vam->help_by_name, "{n}", "{help}");\n'
                  .format(n=s.caller, help=define_hash[s.caller].options['vat_help']))
        except:
            pass

        # Events
        for e in s.events:
            write('   vl_msg_api_set_handlers(VL_API_{ID} + msg_id_base, "{n}",\n'
                  '                           vl_api_{n}_t_handler, vl_noop_handler,\n'
                  '                           vl_api_{n}_t_endian, vl_api_{n}_t_print,\n'
                  '                           sizeof(vl_api_{n}_t), 1);\n'
                  .format(n=e, ID=e.upper()))

    write('}\n')

    write('clib_error_t * vat_plugin_register (vat_main_t *vam)\n')
    write('{\n')
    write('   {n}_test_main_t * mainp = &{n}_test_main;\n'.format(n=module))
    write('   mainp->vat_main = vam;\n')
    write('   mainp->msg_id_base = vl_client_get_first_plugin_msg_id ("{n}_{crc:08x}");\n'
          .format(n=module, crc=file_crc))
    write('   if (mainp->msg_id_base == (u16) ~0)\n')
    write('      return clib_error_return (0, "{} plugin not loaded...");\n'.format(module))
    write('   setup_message_id_table (vam, mainp->msg_id_base);\n')
    write('#ifdef VL_API_LOCAL_SETUP_MESSAGE_ID_TABLE\n')
    write('    VL_API_LOCAL_SETUP_MESSAGE_ID_TABLE(vam);\n')
    write('#endif\n')
    write('   return 0;\n')
    write('}\n')


#
# Plugin entry point
#
def run(args, input_filename, s):
    stream = StringIO()

    if not args.outputdir:
        sys.stderr.write('Missing --outputdir argument')
        return None

    basename = os.path.basename(input_filename)
    filename, file_extension = os.path.splitext(basename)
    modulename = filename.replace('.', '_')
    filename_enum = os.path.join(args.outputdir + '/' + basename + '_enum.h')
    filename_types = os.path.join(args.outputdir + '/' + basename + '_types.h')
    filename_c = os.path.join(args.outputdir + '/' + basename + '.c')
    filename_c_test = os.path.join(args.outputdir + '/' + basename + '_test.c')

    # Generate separate types file
    st = StringIO()
    generate_include_types(s, modulename, st)
    with open (filename_types, 'w') as fd:
        st.seek (0)
        shutil.copyfileobj (st, fd)
    st.close()

    # Generate separate enum file
    st = StringIO()
    generate_include_enum(s, modulename, st)
    with open (filename_enum, 'w') as fd:
        st.seek (0)
        shutil.copyfileobj (st, fd)
    st.close()

    # Generate separate C file
    st = StringIO()
    generate_c_boilerplate(s['Service'], s['Define'], s['file_crc'],
                           modulename, st)
    with open (filename_c, 'w') as fd:
        st.seek (0)
        shutil.copyfileobj(st, fd)
    st.close()

    # Generate separate C test file
    # This is only supported for plugins at the moment
    st = StringIO()
    generate_c_test_plugin_boilerplate(s['Service'], s['Define'], s['file_crc'],
                                       modulename, st)
    with open (filename_c_test, 'w') as fd:
        st.seek (0)
        shutil.copyfileobj(st, fd)
    st.close()

    output = top_boilerplate.format(datestring=datestring,
                                    input_filename=basename)
    output += imports(s['Import'])
    output += msg_ids(s)
    output += msg_names(s)
    output += msg_name_crc_list(s, filename)
    output += typedefs(modulename)
    printfun_types(s['types'], stream, modulename)
    printfun(s['Define'], stream, modulename)
    output += stream.getvalue()
    stream.close()
    output += endianfun(s['types'] + s['Define'], modulename)
    output += version_tuple(s, basename)
    output += bottom_boilerplate.format(input_filename=basename,
                                        file_crc=s['file_crc'])

    return output