Packet Builder Language
=======================
:author: hhaim
:email: <hhaim@cisco.com>
:revnumber: 0.04
:quotes.++:
:numbered:
== change log
[options="header",cols="^1,^h,a"]
|=================
| Version | name | meaning
| 0.01 | hhaim |
- first version
| 0.02 | hhaim
|
- change the bool fields to properties
- add external/internal property
- add const property ( instead cant_change)
- change TLV property - now learn the prev header
- add choice of next protocol that is not base on a field ( TCP->IP->TCP)
| 0.03 | ybrustin
|
- add MAC address regexp
- add gui_representation class with data_type, form_type, combobox_values, data_type_regexp items to describe GUI view of field
- rename choice attribute to value_based_next_header
- fixed some typos
| 0.04 | ybrustin
|
- change value_based_next_header, combobox_values (to be consistent with value_based_next_header) to dictionary
- added value_based_next_class for options
- move 'help' attribute to gui_representation
- add link to headers.yaml (references at bottom of the page)
|=================
== A file format for GUI packet builder
=== Introduction
We would like a file that will be read by GUI and will give us the ability to build packets using GUI
The format should be *YAML*
=== High Level Requirement
* Define a YAML object format for dynamic building of packets and a program that change various fields
* Ability to *parse* back the same buffer that was created using this tool (reversibility)
** Ability to load packet from a pcap file and parse it
* Ability to save the packet to a pcap file
* Ability to save the packet and program in JSON format (same JSON-RPC format)
* Set a value for any field of any protocol
* Vary packet fields across packets at run time e.g. changing IP/MAC addresses
* Stack protocols in any arbitrary order define in YAML format
=== Header that should be supported (first phase)
==== L2
* Ethernet
* 802.3
* LLC SNAP
* VLAN (with QinQ) stack
* MPLS stack
==== L3
* ARP
* IPv4
* IPv6 (4x header)
* IP-in-IP a.k.a IP Tunnelling (6over4, 4over6, 4over4, 6over6)
==== L4
* TCP
* UDP
* ICMPv4
* ICMPv6
* IGMP
==== L7 anchor:Payload[]
* Any text based protocol (HTTP, SIP, RTSP, NNTP etc.)
** random string
** repeat string
* Pattern Binary
** repeat of value (e.g 0x55)
** random
** seq (1,2,3,3,4)
** User Hex Dump editor
=== YAML Format
==== Header section
.Default Types anchor:Types[]
[options="header",cols="1,2,3"]
|=================
| Field Name | meaning | size in bits
| bit | describe the header object e.g tcp | 1
| uint8 | describe the header object e.g tcp | 8
| uint16 | the name in the GUI | 16
| uint32 | sub fields of this header | 32
| uint64 | sub fields of this header | 64
| other class type | name of other class. for example, "c-mac-addr"; take fields from there, optionally overload them later | The size taken from that class
| Payload | xref:Payload[Payload] | total packet size - all header until now
| vlen_t | in case of varible size header this include the size to the end of varible size header see example xref:IpvOption[Ipv4Option] |total size of the object
|=================
.Default Data_Type anchor:Data_Type[]
[options="header",cols="1,2"]
|=================
| Field Name | meaning
| none | use Hex Editor as Types
| ipv4_t | 4 decimals 0-255 each
| mac_addr_t | ([0-9a-fA-F]\{2\}:)\{5\}[0-9a-fA-F]\{2\}
| ipv4_mask_t | should match uint32 type
| ipv6_t | should have 16 bytes field size 8x16
| ipv6_mask_t | should have 16 bytes field size 8x16
| another header class | sub fields of this header
| char_t | array of bytes , look into the array_size of cost string
| var_char_t | array based on a field value look into
| regexp_t | define a Java function that converts a reg exp string to a buffer see here xref:GenRegExp[RegExp]
|=================
.Default Form_Type anchor:Form_Type[]
[options="header",cols="1,3"]
|=================
| Field Name | meaning
| none | simple editing field
| combo_with_edit | combo box with predefined choices, can edit the field value manually
| combo_without_edit | combo box with predefined choices, can [underline]#not# edit the field value manually
| checkbox | toggle bits values, if item is array of bits, display several checkboxes per number of bits
|=================
.Default Gui_Representation anchor:Gui_Representation[]
[options="header",cols="1,^1,5,^1,10"]
|=================
| Field Name | value type | meaning | Link | Additional info
| help | string | the name in the GUI | |
| data_type | string | how to represent data | xref:Data_Type[Data_Type] | data_type could get data_type_regexp e.g data_type = "ipv4"; data_type = "regexp" data_type_regexp = "string that define regexp and Java function"
| form_type | string | which editing form to use | xref:Form_Type[Form_Type] | for example for ip address use combobox with option to edit value manually or choose: key "localhost" value "127.0.0.1" etc.
| combobox_values | dictionary | pairs of 'key - value' for combo_with/without_edit | |
| data_type_regexp | string | in case it is reg_exp the name of the function | xref:GenRegExp[GenRegExp] |
|=================
.Default Properties anchor:Properties[]
[options="header",cols="1,7"]
|=================
| Field Name | meaning
| ipv4_checksum | auto calculates checksum on this header Ipv4 type
| tcp_checksum | calculate TCP checksum
| udp_checksum | calculate UDP checksum
| ipv4_total_length | calculate ipv4 total length this pkt_size = header + reset of packet
| tlv | TLV length of the header (inlcudes the prev field length) example ip-option, tcp-option
| le | little endian. deault is big
| const | const field for example the 4 version of ipv4 header - this GUI won't give option to change this field
| external | marks the header as an external header for the GUI. for example IPv4 is external header and mac-addr is internal header ( compose external header)
|=================
.Field_Type anchor:Field_Type[]
[options="header",cols="1,^1,30,^1,^1,30"]
|=================
| Field Name | value type | meaning | Default Value | Link | Example
| class | string | describe the class type | in case class is defined no need to have name and vise versa | | class : tcp
| name | string | describe the instance name | in case class is defined no need to have name and vise versa | | name : tcp
| array_size | integer | how many objects of this type, default value is 1 | 1 | | array_size : 6 in case of mac-addr
| type | string | type, see Types define the size | "uint8_t" | xref:Types[Types] | type : "uint32_t" type : "mac_addr"
| gui_representation | dictionary | description of how to view/edit data in GUI | | xref:Gui_Representation[Gui_Representation] | xref:Gui_Representation_Example[Gui_Representation_Example]
| default | array/value | default value in the packets , you can override value for subfields in parent see example
| [0 ]x header size | | xref:Overide_Subfields_Example[Overide_Subfields_Example]
| properies | array of string like masks
| properies of this fields | [] | xref:Properties[Properties] | ["le","external"] , ["tlv","le","const"]
| value_based_next_header | dictionary | define the next protocol based on a field value | none | xref:Value_Based_Next_Header[Value_Based_Next_Header] |
| value_based_next_class | dictionary | define the next class based on a field value (useful for options) | none | xref:Value_Based_Next_Class[Value_Based_Next_Class] |
| next_headers | string or type | a name of class that define the next or just an array | "none" | xref:Next_headers[Next_headers] |
| fields | array | array of Field_Type | [] | | fields : [ ]
| offset | integer/string | offset into the packet in bits, in case of auto add base of prev fields | "auto" | |
| option | string | a java code that define a way to calculate varible size | "none" | | |
|=================
.Field_Type anchor:ConstHeadesClass[]
[options="header",cols="^1,^10"]
|=================
| Field Name | value type
| "root" | the root pointer to the start of blocks L2/802.3 etc
| "end" | end TLV headers
| "payload" | the rest of the packets as buffer/string etc
|=================
.Next_headers anchor:Next_headers[]
Example of Next_headers
[source,python]
----
- class : "next-example-t-1"
gui_representation:
help : "next-example-t-1"
next_headers : ["ipv4", "ipv6, "tcp"]
# option 1 define in the header itself
- class : "tcp"
gui_representation:
help : "TCP header"
properies : ["external"]
next_headers : ["ipv4", "ipv6, "tcp"]
fields :
- name : "ver"
# option 2 define throw a class
- class : "tcp"
gui_representation:
help : "TCP header"
properies : ["external"]
next_headers : "next-example-t-1" #
fields :
- name : "ver"
----
.Value_Based_Next_Header anchor:Value_Based_Next_Header[]
Example of value_based_next_header
[source,python]
----
value_based_next_header:
0x0800: 'ipv4'# name of an external or internal class , the GUI should distinct betwean internal and external
0x0806: 'arp'
0x86DD: 'ipv6'
0x8100: 'vlan'
0x8847: 'mpls unicast'
default: 'payload' # if no match for any of above
----
.Generic RegExp Edit Field anchor:GenRegExp[]
This will define a regexp that match for user input and how to converts it to buffer of bytes
[source,python]
----
class MyClass : public RegExpBase {
public:
string get_reg_exp_string( ) {
return ((\d){1-3})[.]((\d){1-3})[.]((\d){1-3})[.]((\d){1-3}))
}
# in case of match
buffer get_buffer(){
g= [get_group()[1].to_int()*256,get_group()[1].to_int()]
# return list
return (g)
}
}
----
==== Relations between object headers
There would be a root object to point to possible starting headers
[source,python]
----
- class : "root"
gui_representation:
help : "Root"
next_headers : [ "ethernet", "llc", "_802-3"]
----
So in a way you could define a tree like this
[source,python]
----
root -> L2 ( Ethernet , 802.3 , LLC SNAP )
|( by field )
|
------------------------------------- ( VLAN (with QinQ), MPLS , ipv4, ipv6, ARP , ICMP )
| | | |
| ipv4/ipv6 - -
| |
| |
[Possibility - Ethernet/802.3/LLC SNAP) | UDP/TCP/Pyload
Object | |
for each option there tree of all the option --- -
----
==== Rules
* The size of the header and offset is automatically defined in default by the order of the fields ( inc by type size multiply by array_size)
* It can be overrided by offset field ( put offset in the object ) and then an more advanced field can be shown earlier in the GUI
* The packet size is defined before the headers. Header Should not be allowed to be added if the size + header size is bigger than packet size
* "Payload" is predefined Fields that take the reset of the packet and user can edit it ( see xref:Payload[Payload] )
* There would be a spare field in the Stream object so GUI could add more metadata for reconstructing the builder types
for example in this example Ethrenet/IP/TCP/IP/TCP you can't extrac from buffer alone that Payload is IP/TCP only the builder known that in build time.
* Ip total length need to keep the total_pkt_size - this ip header . this should work for internal header too.
* When GUI add header ("external") the total size of this header should be calculated ( varible size should be given a default - ipv4)
=== Examples
==== TLV (Ip option) anchor:IpvOption[], value_based_next_class anchor:Value_Based_Next_Class[]
IP-option see link:http://tools.ietf.org/html/rfc791[ip_option]
0 : END
1 : Length 1
other : Byte : Length ( +first) |option
[source,python]
----
- class : "ip_option_131"
gui_representation:
help : "ip_option"
fields :
- name : "length" # tree with leaf of bits
gui_representation:
help : "length"
type : uint8
properties : ["tlv"] # the length include the prev field size (8 byte)
- name : "pointer" # tree with leaf of bits
type : uint8
- name : "buffer" # tree with leaf of bits
type : "tlv_reset"
- class : "default_ip4_option_tlv"
gui_representation:
help : "ip_option"
fields :
- name : "length" # tree with leaf of bits
gui_representation:
help : "length"
type : uint8
properties : "tlv" # the length include the prev field size (8 byte)
- name : "buffer" # tree with leaf of bits
type : "vlen_t"
- class : "ip_option"
gui_representation:
help : "ip_option"
type : uint8
default : [0x01]
value_based_next_class :
0x00 : "end" # reserve name for ending the loop
0x01 : "ip_option" # back to this header
0x131 : "ip_option_131"
0x0812: "gre"
default : "default_ip4_option_tlv"
----
* case of varible length field ip_option example
==== Example TCP/IP
[source,python]
----
- class : "c-mac-addr"
type : "uint8"
array_size : 6
default : [0x00, 0x00, 0x01, 0x00, 0x00, 0x00]
gui_representation:
data_type : "mac-addr_t" # format ([0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2}
help : "Mac addrees"
- class : "ethernet"
gui_representation:
help : "Ethernet-L2"
properties: ['external']
fields :
- name : "Dst"
gui_representation:
help : "destination mac"
type : "c-mac-addr"
- name : "Src"
gui_representation:
help : "source mac"
type : "c-mac-addr"
- name: "Ethertype"
gui_representation:
help: "Ethertype"
type: "uint16"
default: [0x0800]
value_based_next_header :
0x0800 : "ipv4"
0x86DD : "ipv6"
0x8100 : "vlan"
0x8847 : "mpls" #unicast
default : "payload"
- class : "ipv4"
gui_representation:
help : "Ipv4"
fields :
- name : "ver"
gui_representation:
help : "Version"
type : "bit"
array_size : 4
default : [0, 1, 0, 0]
properties : ["const"]
- name : "ihl"
type : "bit"
array_size : 4
default : [0, 1, 1, 1]
properties : ["ipv4_ihl"]
gui_representation:
help : "IHL"
form_type: "checkbox"
..
- name : "hdr_chsum"
gui_representation:
help : "Header Checksum"
default : [0x00,0x00]
properties : ["ipv4_check_sum", "const"]
- name : "total_len"
gui_representation:
help : "Total Length"
default : [0x00,0x00]
properties : ["ipv4_total_len", "const"] # auto calculate total_size-offset_header
- name : "protocol"
type : uint8
default : 0x06
value_based_next_header : &ipv4_next_header
0x06 : "tcp"
0x11 : "udp"
0x29 : "ipv6"
0x2F : "gre"
default : "payload"
gui_representation:
help : "IPv4 next Protocol"
form_type: "combo_without_edit"
combobox_values:
<<: *ipv4_next_header # take same choices as value_based_next_header
- name : "src_addr"
type : uint32
default : [16, 0, 0, 0]
gui_representation:
help : "Source Address"
data_type : "ipv4" # reserve
- name : "dst_addr"
default : [48, 0, 0, 0]
type : uint32
gui_representation:
help : "Destination Address"
data_type : "ipv4" # reserve
form_type : "combo_with_edit"
combobox_values:
[127, 0, 0, 1]: 'localhost'
[255, 255, 255, 255]: 'broadcast'
- class : "tcp"
gui_representation:
help : "TCP"
properties : ["external"]
fields :
- name : "src_port"
gui_representation:
help : "Source Port"
default : [0x30,0x00]
type : uint16
- name : "dest_port"
gui_representation:
help : "Source Port"
default : [0x30,0x00]
type : uint16
- name : "seq"
gui_representation:
help : "Seq Number"
type : uint32
default : [0x30,0x00,00,00]
- name : "ack"
gui_representation:
help : "Ack Number"
type : uint32
default : [0x30,0x00,00,00]
...
- name : "flags" # tree with leaf of bits
gui_representation:
help : "Ack Number"
type : uint8
default : [0x30]
fields :
- name : "urg"
help : "URG"
type : bit
default : [0x0]
- name : "ack"
help : "ACK"
type : bit
default : [0x1]
..
- name : "checksum"
gui_representation:
help : "TCP Checksum"
type : uint16
default : [0x00,0x00]
properties : ["tcp_checksum"] # auto calculate total_size-offset_header
- class : "root" # reserve
gui_representation:
help : "Root"
next_headers : [ "ethrenet" ,"llc","_802-3"]
---------------------------
==== Overide subfields example anchor:Overide_Subfields_Example[]
In this example parent class default value overrides default values of sub-fields ( 2 different mac-addr)
[source,python]
----
- class : "c-mac-addr"
type : "uint8"
array_size : 6
gui_representation:
help : "Mac addrees"
data_type : "mac-addr_t" # format ([0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2}
default : [0x00,0x00,0x01,0x00,0x00,0x00]
- class : "ethernet"
gui_representation:
help : "Ethernet-L2"
properties : ["external"]
default : [0x00,0x01,0x01,0x00,0x00,0x00, 0x00,0x02,0x02,0x00,0x00,0x00 ,0x08,00] # change the default of sub-fields . it is const size
fields :
- name : "Dst"
gui_representation:
help : "destination mac"
type : "c-mac-addr"
- name : "Src"
gui_representation:
help : "source mac"
type : "c-mac-addr"
- name : "ip_protocol"
type : "uint16_t"
default : [0x08,0x00]
value_based_next_header :
0x0800 : "ipv4"
0x86DD : "ipv6"
0x8100 : "vlan"
0x8847 : "mpls unicast"
default : "payload"
----
==== Gui Representation example anchor:Gui_Representation_Example[]
[underline]#In YAML:#
[source,python]
----
- name: 'Flags'
type: 'bit'
array_size: 3
gui_representation:
help: 'IPv4 Flags'
form_type: 'checkbox' # can check each bit
- name: 'dst_addr'
default: [48, 0, 0, 0]
type: uint32
gui_representation:
help: 'IPv4 Destination Address'
data_type: 'ipv4_t' # special representation case, show as 4 decimal numbers
form_type: 'combo_with_edit' # can choose from pre-defined values or edit manually
combobox_values:
[127, 0, 0, 1]: 'localhost'
[255, 255, 255, 255]: 'broadcast'
- name: 'protocol'
type: uint8
default: 0x06
value_based_next_header: &ipv4_next_header
0x06: 'tcp'
0x11: 'udp'
default : "payload"
gui_representation:
help: 'IPv4 Protocol Field'
form_type: 'combo_without_edit' # choose from supported protocols, no manual edit
combobox_values:
<<: *ipv4_next_header # take same choices as value_based_next_header
----
[underline]#In GUI:#
checkbox for bits:
image:images/checkbox.jpg[]
editing in combo-box:
image:images/combo_button_editing.jpg[]
choosing from predefined values:
image:images/combo_button_choosing.jpg[]
==== Union base
TBD
=== Resource
* link:yaml/headers.yaml[headers.yaml]
* link:https://wireedit.com/[WireEdit]
* link:http://ostinato.org/[ostinato]
* link:http://www.slideshare.net/nlekh/ixiaexplorer[IxExplorer]