diff options
-rw-r--r-- | .gitignore | 24 | ||||
-rw-r--r-- | 1 | 0 | ||||
-rwxr-xr-x | images/Thumbs.db | bin | 84992 -> 106496 bytes | |||
-rwxr-xr-x | images/checkbox.jpg | bin | 0 -> 2653 bytes | |||
-rwxr-xr-x | images/combo_button_choosing.jpg | bin | 0 -> 17359 bytes | |||
-rwxr-xr-x | images/combo_button_editing.jpg | bin | 0 -> 3029 bytes | |||
-rw-r--r-- | images/rpc_server_big_picture.png | bin | 0 -> 54566 bytes | |||
-rw-r--r-- | images/rpc_states.png | bin | 0 -> 58390 bytes | |||
-rw-r--r-- | packet_builder_yaml.asciidoc | 670 | ||||
-rwxr-xr-x | release_notes.asciidoc | 15 | ||||
-rwxr-xr-x | trex_book.asciidoc | 2 | ||||
-rwxr-xr-x | trex_book_basic.asciidoc | 4 | ||||
-rwxr-xr-x | trex_control_plane_design_phase1.asciidoc | 28 | ||||
-rw-r--r-- | trex_rpc_server_spec.asciidoc | 1086 | ||||
-rwxr-xr-x | wscript | 12 | ||||
-rwxr-xr-x | yaml/headers.yaml | 429 |
16 files changed, 2230 insertions, 40 deletions
@@ -1,21 +1,11 @@ # Compiled source # ################### -*.com -*.class -*.dll -*.exe -*.o -*.so -.lock* -linux_dpdk* +.lock-waf* +.waf* linux* -scripts/_t-rex-* -scripts/bp-sim-* scripts/doc/* -scripts/mock-* - +build/ *.pyc - # Packages # ############ @@ -29,12 +19,7 @@ scripts/mock-* *.rar *.tar *.zip - -# Logs and databases # -###################### -*.log -*.sql -*.sqlite +*.vpj # OS generated files # ###################### @@ -45,7 +30,6 @@ scripts/mock-* ehthumbs.db Thumbs.db - # IDE/ Editors files # ###################### .idea/ diff --git a/images/Thumbs.db b/images/Thumbs.db Binary files differindex b775040b..98a8ffc0 100755 --- a/images/Thumbs.db +++ b/images/Thumbs.db diff --git a/images/checkbox.jpg b/images/checkbox.jpg Binary files differnew file mode 100755 index 00000000..38fa30e1 --- /dev/null +++ b/images/checkbox.jpg diff --git a/images/combo_button_choosing.jpg b/images/combo_button_choosing.jpg Binary files differnew file mode 100755 index 00000000..11483ed2 --- /dev/null +++ b/images/combo_button_choosing.jpg diff --git a/images/combo_button_editing.jpg b/images/combo_button_editing.jpg Binary files differnew file mode 100755 index 00000000..ff8d3c49 --- /dev/null +++ b/images/combo_button_editing.jpg diff --git a/images/rpc_server_big_picture.png b/images/rpc_server_big_picture.png Binary files differnew file mode 100644 index 00000000..dae6976d --- /dev/null +++ b/images/rpc_server_big_picture.png diff --git a/images/rpc_states.png b/images/rpc_states.png Binary files differnew file mode 100644 index 00000000..cdbf1c51 --- /dev/null +++ b/images/rpc_states.png diff --git a/packet_builder_yaml.asciidoc b/packet_builder_yaml.asciidoc new file mode 100644 index 00000000..5d824a58 --- /dev/null +++ b/packet_builder_yaml.asciidoc @@ -0,0 +1,670 @@ +Packet Builder Language
+=======================
+:author: hhaim
+:email: <hhaim@cisco.com>
+:revnumber: 0.04
+:quotes.++:
+:numbered:
+
+== change log
+
+[options="header",cols="^1,^h,a"]
+|=================
+| Version | name | meaning
+| 0.01 | hhaim |
+- first version
+| 0.02 | hhaim
+|
+- change the bool fields to properties
+- add external/internal property
+- add const property ( instead cant_change)
+- change TLV property - now learn the prev header
+- add choice of next protocol that is not base on a field ( TCP->IP->TCP)
+| 0.03 | ybrustin
+|
+- add MAC address regexp
+- add gui_representation class with data_type, form_type, combobox_values, data_type_regexp items to describe GUI view of field
+- rename choice attribute to value_based_next_header
+- fixed some typos
+| 0.04 | ybrustin
+|
+- change value_based_next_header, combobox_values (to be consistent with value_based_next_header) to dictionary
+- added value_based_next_class for options
+- move 'help' attribute to gui_representation
+- add link to headers.yaml (references at bottom of the page)
+
+|=================
+
+
+== A file format for GUI packet builder
+
+=== Introduction
+
+We would like a file that will be read by GUI and will give us the ability to build packets using GUI
+
+The format should be *YAML*
+
+
+=== High Level Requirement
+
+* Define a YAML object format for dynamic building of packets and a program that change various fields
+* Ability to *parse* back the same buffer that was created using this tool (reversibility)
+** Ability to load packet from a pcap file and parse it
+* Ability to save the packet to a pcap file
+* Ability to save the packet and program in JSON format (same JSON-RPC format)
+* Set a value for any field of any protocol
+* Vary packet fields across packets at run time e.g. changing IP/MAC addresses
+* Stack protocols in any arbitrary order define in YAML format
+
+=== Header that should be supported (first phase)
+
+==== L2
+
+* Ethernet
+* 802.3
+* LLC SNAP
+* VLAN (with QinQ) stack
+* MPLS stack
+
+==== L3
+
+* ARP
+* IPv4
+* IPv6 (4x header)
+* IP-in-IP a.k.a IP Tunnelling (6over4, 4over6, 4over4, 6over6)
+
+==== L4
+
+* TCP
+* UDP
+* ICMPv4
+* ICMPv6
+* IGMP
+
+==== L7 anchor:Payload[]
+
+* Any text based protocol (HTTP, SIP, RTSP, NNTP etc.)
+** random string
+** repeat string
+
+* Pattern Binary
+** repeat of value (e.g 0x55)
+** random
+** seq (1,2,3,3,4)
+** User Hex Dump editor
+
+
+=== YAML Format
+
+==== Header section
+
+.Default Types anchor:Types[]
+[options="header",cols="1,2,3"]
+|=================
+| Field Name | meaning | size in bits
+| bit | describe the header object e.g tcp | 1
+| uint8 | describe the header object e.g tcp | 8
+| uint16 | the name in the GUI | 16
+| uint32 | sub fields of this header | 32
+| uint64 | sub fields of this header | 64
+| other class type | name of other class. for example, "c-mac-addr"; take fields from there, optionally overload them later | The size taken from that class
+| Payload | xref:Payload[Payload] | total packet size - all header until now
+| vlen_t | in case of varible size header this include the size to the end of varible size header see example xref:IpvOption[Ipv4Option] |total size of the object
+|=================
+
+
+.Default Data_Type anchor:Data_Type[]
+[options="header",cols="1,2"]
+|=================
+| Field Name | meaning
+| none | use Hex Editor as Types
+| ipv4_t | 4 decimals 0-255 each
+| mac_addr_t | ([0-9a-fA-F]\{2\}:)\{5\}[0-9a-fA-F]\{2\}
+| ipv4_mask_t | should match uint32 type
+| ipv6_t | should have 16 bytes field size 8x16
+| ipv6_mask_t | should have 16 bytes field size 8x16
+| another header class | sub fields of this header
+| char_t | array of bytes , look into the array_size of cost string
+| var_char_t | array based on a field value look into
+| regexp_t | define a Java function that converts a reg exp string to a buffer see here xref:GenRegExp[RegExp]
+|=================
+
+.Default Form_Type anchor:Form_Type[]
+[options="header",cols="1,3"]
+|=================
+| Field Name | meaning
+| none | simple editing field
+| combo_with_edit | combo box with predefined choices, can edit the field value manually
+| combo_without_edit | combo box with predefined choices, can [underline]#not# edit the field value manually
+| checkbox | toggle bits values, if item is array of bits, display several checkboxes per number of bits
+|=================
+
+
+.Default Gui_Representation anchor:Gui_Representation[]
+[options="header",cols="1,^1,5,^1,10"]
+|=================
+| Field Name | value type | meaning | Link | Additional info
+| help | string | the name in the GUI | |
+| data_type | string | how to represent data | xref:Data_Type[Data_Type] | data_type could get data_type_regexp e.g data_type = "ipv4"; data_type = "regexp" data_type_regexp = "string that define regexp and Java function"
+| form_type | string | which editing form to use | xref:Form_Type[Form_Type] | for example for ip address use combobox with option to edit value manually or choose: key "localhost" value "127.0.0.1" etc.
+| combobox_values | dictionary | pairs of 'key - value' for combo_with/without_edit | |
+| data_type_regexp | string | in case it is reg_exp the name of the function | xref:GenRegExp[GenRegExp] |
+|=================
+
+
+.Default Properties anchor:Properties[]
+[options="header",cols="1,7"]
+|=================
+| Field Name | meaning
+| ipv4_checksum | auto calculates checksum on this header Ipv4 type
+| tcp_checksum | calculate TCP checksum
+| udp_checksum | calculate UDP checksum
+| ipv4_total_length | calculate ipv4 total length this pkt_size = header + reset of packet
+| tlv | TLV length of the header (inlcudes the prev field length) example ip-option, tcp-option
+| le | little endian. deault is big
+| const | const field for example the 4 version of ipv4 header - this GUI won't give option to change this field
+| external | marks the header as an external header for the GUI. for example IPv4 is external header and mac-addr is internal header ( compose external header)
+|=================
+
+
+.Field_Type anchor:Field_Type[]
+[options="header",cols="1,^1,30,^1,^1,30"]
+|=================
+| Field Name | value type | meaning | Default Value | Link | Example
+| class | string | describe the class type | in case class is defined no need to have name and vise versa | | class : tcp
+| name | string | describe the instance name | in case class is defined no need to have name and vise versa | | name : tcp
+| array_size | integer | how many objects of this type, default value is 1 | 1 | | array_size : 6 in case of mac-addr
+| type | string | type, see Types define the size | "uint8_t" | xref:Types[Types] | type : "uint32_t" type : "mac_addr"
+| gui_representation | dictionary | description of how to view/edit data in GUI | | xref:Gui_Representation[Gui_Representation] | xref:Gui_Representation_Example[Gui_Representation_Example]
+| default | array/value | default value in the packets , you can override value for subfields in parent see example
+| [0 ]x header size | | xref:Overide_Subfields_Example[Overide_Subfields_Example]
+| properies | array of string like masks
+| properies of this fields | [] | xref:Properties[Properties] | ["le","external"] , ["tlv","le","const"]
+| value_based_next_header | dictionary | define the next protocol based on a field value | none | xref:Value_Based_Next_Header[Value_Based_Next_Header] |
+| value_based_next_class | dictionary | define the next class based on a field value (useful for options) | none | xref:Value_Based_Next_Class[Value_Based_Next_Class] |
+| next_headers | string or type | a name of class that define the next or just an array | "none" | xref:Next_headers[Next_headers] |
+| fields | array | array of Field_Type | [] | | fields : [ ]
+| offset | integer/string | offset into the packet in bits, in case of auto add base of prev fields | "auto" | |
+| option | string | a java code that define a way to calculate varible size | "none" | | |
+|=================
+
+
+.Field_Type anchor:ConstHeadesClass[]
+[options="header",cols="^1,^10"]
+|=================
+| Field Name | value type
+| "root" | the root pointer to the start of blocks L2/802.3 etc
+| "end" | end TLV headers
+| "payload" | the rest of the packets as buffer/string etc
+|=================
+
+
+.Next_headers anchor:Next_headers[]
+Example of Next_headers
+[source,python]
+----
+
+ - class : "next-example-t-1"
+ gui_representation:
+ help : "next-example-t-1"
+ next_headers : ["ipv4", "ipv6, "tcp"]
+
+# option 1 define in the header itself
+ - class : "tcp"
+ gui_representation:
+ help : "TCP header"
+ properies : ["external"]
+ next_headers : ["ipv4", "ipv6, "tcp"]
+ fields :
+ - name : "ver"
+
+# option 2 define throw a class
+ - class : "tcp"
+ gui_representation:
+ help : "TCP header"
+ properies : ["external"]
+ next_headers : "next-example-t-1" #
+ fields :
+ - name : "ver"
+----
+
+
+.Value_Based_Next_Header anchor:Value_Based_Next_Header[]
+Example of value_based_next_header
+[source,python]
+----
+ value_based_next_header:
+ 0x0800: 'ipv4'# name of an external or internal class , the GUI should distinct betwean internal and external
+ 0x0806: 'arp'
+ 0x86DD: 'ipv6'
+ 0x8100: 'vlan'
+ 0x8847: 'mpls unicast'
+ default: 'payload' # if no match for any of above
+
+----
+
+
+.Generic RegExp Edit Field anchor:GenRegExp[]
+
+This will define a regexp that match for user input and how to converts it to buffer of bytes
+
+[source,python]
+----
+
+class MyClass : public RegExpBase {
+ public:
+
+
+ string get_reg_exp_string( ) {
+ return ((\d){1-3})[.]((\d){1-3})[.]((\d){1-3})[.]((\d){1-3}))
+ }
+
+ # in case of match
+ buffer get_buffer(){
+ g= [get_group()[1].to_int()*256,get_group()[1].to_int()]
+ # return list
+ return (g)
+ }
+
+}
+
+----
+
+
+
+==== Relations between object headers
+
+There would be a root object to point to possible starting headers
+
+
+[source,python]
+----
+
+- class : "root"
+ gui_representation:
+ help : "Root"
+ next_headers : [ "ethernet", "llc", "_802-3"]
+----
+
+So in a way you could define a tree like this
+
+[source,python]
+----
+
+root -> L2 ( Ethernet , 802.3 , LLC SNAP )
+ |( by field )
+ |
+ ------------------------------------- ( VLAN (with QinQ), MPLS , ipv4, ipv6, ARP , ICMP )
+ | | | |
+ | ipv4/ipv6 - -
+ | |
+ | |
+ [Possibility - Ethernet/802.3/LLC SNAP) | UDP/TCP/Pyload
+ Object | |
+ for each option there tree of all the option --- -
+----
+
+
+==== Rules
+
+* The size of the header and offset is automatically defined in default by the order of the fields ( inc by type size multiply by array_size)
+* It can be overrided by offset field ( put offset in the object ) and then an more advanced field can be shown earlier in the GUI
+* The packet size is defined before the headers. Header Should not be allowed to be added if the size + header size is bigger than packet size
+* "Payload" is predefined Fields that take the reset of the packet and user can edit it ( see xref:Payload[Payload] )
+* There would be a spare field in the Stream object so GUI could add more metadata for reconstructing the builder types
+ for example in this example Ethrenet/IP/TCP/IP/TCP you can't extrac from buffer alone that Payload is IP/TCP only the builder known that in build time.
+* Ip total length need to keep the total_pkt_size - this ip header . this should work for internal header too.
+* When GUI add header ("external") the total size of this header should be calculated ( varible size should be given a default - ipv4)
+
+
+=== Examples
+
+
+==== TLV (Ip option) anchor:IpvOption[], value_based_next_class anchor:Value_Based_Next_Class[]
+
+
+IP-option see link:http://tools.ietf.org/html/rfc791[ip_option]
+
+0 : END
+
+1 : Length 1
+
+other : Byte : Length ( +first) |option
+
+
+
+[source,python]
+----
+
+ - class : "ip_option_131"
+ gui_representation:
+ help : "ip_option"
+ fields :
+ - name : "length" # tree with leaf of bits
+ gui_representation:
+ help : "length"
+ type : uint8
+ properties : ["tlv"] # the length include the prev field size (8 byte)
+
+ - name : "pointer" # tree with leaf of bits
+ type : uint8
+
+ - name : "buffer" # tree with leaf of bits
+ type : "tlv_reset"
+
+ - class : "default_ip4_option_tlv"
+ gui_representation:
+ help : "ip_option"
+ fields :
+ - name : "length" # tree with leaf of bits
+ gui_representation:
+ help : "length"
+ type : uint8
+ properties : "tlv" # the length include the prev field size (8 byte)
+
+ - name : "buffer" # tree with leaf of bits
+ type : "vlen_t"
+
+
+ - class : "ip_option"
+ gui_representation:
+ help : "ip_option"
+ type : uint8
+ default : [0x01]
+ value_based_next_class :
+ 0x00 : "end" # reserve name for ending the loop
+ 0x01 : "ip_option" # back to this header
+ 0x131 : "ip_option_131"
+ 0x0812: "gre"
+ default : "default_ip4_option_tlv"
+
+
+----
+
+* case of varible length field ip_option example
+
+
+
+==== Example TCP/IP
+
+
+[source,python]
+----
+
+ - class : "c-mac-addr"
+ type : "uint8"
+ array_size : 6
+ default : [0x00, 0x00, 0x01, 0x00, 0x00, 0x00]
+ gui_representation:
+ data_type : "mac-addr_t" # format ([0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2}
+ help : "Mac addrees"
+
+
+ - class : "ethernet"
+ gui_representation:
+ help : "Ethernet-L2"
+ properties: ['external']
+ fields :
+ - name : "Dst"
+ gui_representation:
+ help : "destination mac"
+ type : "c-mac-addr"
+
+ - name : "Src"
+ gui_representation:
+ help : "source mac"
+ type : "c-mac-addr"
+
+ - name: "Ethertype"
+ gui_representation:
+ help: "Ethertype"
+ type: "uint16"
+ default: [0x0800]
+ value_based_next_header :
+ 0x0800 : "ipv4"
+ 0x86DD : "ipv6"
+ 0x8100 : "vlan"
+ 0x8847 : "mpls" #unicast
+ default : "payload"
+
+
+ - class : "ipv4"
+ gui_representation:
+ help : "Ipv4"
+ fields :
+ - name : "ver"
+ gui_representation:
+ help : "Version"
+ type : "bit"
+ array_size : 4
+ default : [0, 1, 0, 0]
+ properties : ["const"]
+
+ - name : "ihl"
+ type : "bit"
+ array_size : 4
+ default : [0, 1, 1, 1]
+ properties : ["ipv4_ihl"]
+ gui_representation:
+ help : "IHL"
+ form_type: "checkbox"
+
+ ..
+
+ - name : "hdr_chsum"
+ gui_representation:
+ help : "Header Checksum"
+ default : [0x00,0x00]
+ properties : ["ipv4_check_sum", "const"]
+
+ - name : "total_len"
+ gui_representation:
+ help : "Total Length"
+ default : [0x00,0x00]
+ properties : ["ipv4_total_len", "const"] # auto calculate total_size-offset_header
+
+ - name : "protocol"
+ type : uint8
+ default : 0x06
+ value_based_next_header : &ipv4_next_header
+ 0x06 : "tcp"
+ 0x11 : "udp"
+ 0x29 : "ipv6"
+ 0x2F : "gre"
+ default : "payload"
+ gui_representation:
+ help : "IPv4 next Protocol"
+ form_type: "combo_without_edit"
+ combobox_values:
+ <<: *ipv4_next_header # take same choices as value_based_next_header
+
+ - name : "src_addr"
+ type : uint32
+ default : [16, 0, 0, 0]
+ gui_representation:
+ help : "Source Address"
+ data_type : "ipv4" # reserve
+
+ - name : "dst_addr"
+ default : [48, 0, 0, 0]
+ type : uint32
+ gui_representation:
+ help : "Destination Address"
+ data_type : "ipv4" # reserve
+ form_type : "combo_with_edit"
+ combobox_values:
+ [127, 0, 0, 1]: 'localhost'
+ [255, 255, 255, 255]: 'broadcast'
+
+
+ - class : "tcp"
+ gui_representation:
+ help : "TCP"
+ properties : ["external"]
+ fields :
+ - name : "src_port"
+ gui_representation:
+ help : "Source Port"
+ default : [0x30,0x00]
+ type : uint16
+
+ - name : "dest_port"
+ gui_representation:
+ help : "Source Port"
+ default : [0x30,0x00]
+ type : uint16
+
+ - name : "seq"
+ gui_representation:
+ help : "Seq Number"
+ type : uint32
+ default : [0x30,0x00,00,00]
+
+ - name : "ack"
+ gui_representation:
+ help : "Ack Number"
+ type : uint32
+ default : [0x30,0x00,00,00]
+
+ ...
+
+ - name : "flags" # tree with leaf of bits
+ gui_representation:
+ help : "Ack Number"
+ type : uint8
+ default : [0x30]
+ fields :
+ - name : "urg"
+ help : "URG"
+ type : bit
+ default : [0x0]
+
+ - name : "ack"
+ help : "ACK"
+ type : bit
+ default : [0x1]
+ ..
+
+ - name : "checksum"
+ gui_representation:
+ help : "TCP Checksum"
+ type : uint16
+ default : [0x00,0x00]
+ properties : ["tcp_checksum"] # auto calculate total_size-offset_header
+
+
+- class : "root" # reserve
+ gui_representation:
+ help : "Root"
+ next_headers : [ "ethrenet" ,"llc","_802-3"]
+---------------------------
+
+
+==== Overide subfields example anchor:Overide_Subfields_Example[]
+
+In this example parent class default value overrides default values of sub-fields ( 2 different mac-addr)
+
+[source,python]
+----
+
+ - class : "c-mac-addr"
+ type : "uint8"
+ array_size : 6
+ gui_representation:
+ help : "Mac addrees"
+ data_type : "mac-addr_t" # format ([0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2}
+ default : [0x00,0x00,0x01,0x00,0x00,0x00]
+
+
+ - class : "ethernet"
+ gui_representation:
+ help : "Ethernet-L2"
+ properties : ["external"]
+ default : [0x00,0x01,0x01,0x00,0x00,0x00, 0x00,0x02,0x02,0x00,0x00,0x00 ,0x08,00] # change the default of sub-fields . it is const size
+ fields :
+ - name : "Dst"
+ gui_representation:
+ help : "destination mac"
+ type : "c-mac-addr"
+
+ - name : "Src"
+ gui_representation:
+ help : "source mac"
+ type : "c-mac-addr"
+
+ - name : "ip_protocol"
+ type : "uint16_t"
+ default : [0x08,0x00]
+ value_based_next_header :
+ 0x0800 : "ipv4"
+ 0x86DD : "ipv6"
+ 0x8100 : "vlan"
+ 0x8847 : "mpls unicast"
+ default : "payload"
+----
+
+==== Gui Representation example anchor:Gui_Representation_Example[]
+[underline]#In YAML:#
+[source,python]
+----
+ - name: 'Flags'
+ type: 'bit'
+ array_size: 3
+ gui_representation:
+ help: 'IPv4 Flags'
+ form_type: 'checkbox' # can check each bit
+
+
+ - name: 'dst_addr'
+ default: [48, 0, 0, 0]
+ type: uint32
+ gui_representation:
+ help: 'IPv4 Destination Address'
+ data_type: 'ipv4_t' # special representation case, show as 4 decimal numbers
+ form_type: 'combo_with_edit' # can choose from pre-defined values or edit manually
+ combobox_values:
+ [127, 0, 0, 1]: 'localhost'
+ [255, 255, 255, 255]: 'broadcast'
+
+
+ - name: 'protocol'
+ type: uint8
+ default: 0x06
+ value_based_next_header: &ipv4_next_header
+ 0x06: 'tcp'
+ 0x11: 'udp'
+ default : "payload"
+ gui_representation:
+ help: 'IPv4 Protocol Field'
+ form_type: 'combo_without_edit' # choose from supported protocols, no manual edit
+ combobox_values:
+ <<: *ipv4_next_header # take same choices as value_based_next_header
+----
+
+[underline]#In GUI:#
+
+checkbox for bits:
+
+image:images/checkbox.jpg[]
+
+editing in combo-box:
+
+image:images/combo_button_editing.jpg[]
+
+choosing from predefined values:
+
+image:images/combo_button_choosing.jpg[]
+
+==== Union base
+
+TBD
+
+
+
+=== Resource
+* link:yaml/headers.yaml[headers.yaml]
+* link:https://wireedit.com/[WireEdit]
+* link:http://ostinato.org/[ostinato]
+* link:http://www.slideshare.net/nlekh/ixiaexplorer[IxExplorer]
+
+
diff --git a/release_notes.asciidoc b/release_notes.asciidoc index 1bc0ba88..b4496213 100755 --- a/release_notes.asciidoc +++ b/release_notes.asciidoc @@ -20,15 +20,24 @@ ifdef::backend-docbook[] endif::backend-docbook[] -== Release 1.75 == + +== Release 1.77 == + +* improve tuple generator capability now it is more flexiable see more link:trex_manual.html#_clients_servers_ip_allocation_scheme[here] + +== Release 1.76 == === fix issues: === -* First version that works from GitHub/Git - init script are in the output package +* minor pcap loader issues +* plugin cleanup + == Release 1.75 == -* This version does not work, no init script +=== fix issues: === + +* First version that works from GitHub/Git - init script are in the output package == Release 1.72 == diff --git a/trex_book.asciidoc b/trex_book.asciidoc index 3d5bd449..8f3e4ac5 100755 --- a/trex_book.asciidoc +++ b/trex_book.asciidoc @@ -2,7 +2,7 @@ TRex ==== :author: hhaim :email: <hhaim@cisco.com> -:revnumber: 1.70-0.0 +:revnumber: 1.77-0.0 :quotes.++: :numbered: :web_server_url: http://trex-tgn.cisco.com/trex diff --git a/trex_book_basic.asciidoc b/trex_book_basic.asciidoc index d64cea43..ec0c9cf5 100755 --- a/trex_book_basic.asciidoc +++ b/trex_book_basic.asciidoc @@ -3190,7 +3190,7 @@ So if the m is set as 1, the total PPS is : 102*2+50*20 = 1204 PPS. The BPS depends on the packet size. You can refer to your packet size and get the BPS = PPS*Packet_size. -==== Roadmap of Client/Server IP allocation +==== Client/Server IP allocation We have several features under development for IP allocation. @@ -3247,7 +3247,7 @@ The YAML configuration is something like this: - *2) More distributions will be supported (normal distribution, random distribution, etc)* -Currently, only sequcence is supported. +Currently, only sequcence and random are supported. - *3) Histogram of tuple pool will be supported* diff --git a/trex_control_plane_design_phase1.asciidoc b/trex_control_plane_design_phase1.asciidoc index cbd02670..cdbd9def 100755 --- a/trex_control_plane_design_phase1.asciidoc +++ b/trex_control_plane_design_phase1.asciidoc @@ -154,17 +154,17 @@ Using class methods, the client interacts with T-Rex server, and enable it to pe 5. Get custom T-Rex stats based on a window of saved history of latest 'N' polling results. -The clients is also based on Python 2.7, however unlike ther server, it can run on any machine who wishes to. + -In fact, the client side is simply a python library that interact with the werver using JSON-RPC (v2), hence if needed, anyone can write a library on any other language that will interact with the server ins the very same way. +The clients is also based on Python 2.7, however unlike the server, it can run on any machine who wishes to. + +In fact, the client side is simply a python library that interact with the server using JSON-RPC (v2), hence if needed, anyone can write a library on any other language that will interact with the server ins the very same way. ==== `CTRexClient` module initialization As explained, `CTRexClient` is the main module to use when writing an T-Rex test-plan. + -This module holds the entire interaction with T-Rex server, and result contiaing via `result_obj`, which is an instance of `CTRexResult` class. + +This module holds the entire interaction with T-Rex server, and result containing via `result_obj`, which is an instance of `CTRexResult` class. + The `CTRexClient` instance is initialized in the following way: - 1. **T-Rex hostname**: represents the hostname on which the server is listening. Both hostname or IPv4 address will be a valid input. + 1. **T-Rex hostname**: represents the hostname on which the server is listening. Either hostname or IPv4 address will be a valid input. 2. **Server port**: the port on which the server listens to incoming client requests. This parameter value must be identical to `port` option configured in the server. @@ -174,7 +174,7 @@ The `CTRexClient` instance is initialized in the following way: This option is especially useful for developers who wishes to imitate the functionality of this client using other programming languages. **That's it!** + -Once these parameter has been passed, you're ready to interact woth T-Rex. +Once these parameter has been passed, you're ready to interact with T-Rex. [NOTE] The most common initialization will simply use the hostname, such that common initilization lookes like: + @@ -187,7 +187,7 @@ This section covers with great detail the usage of the client module. Each of th - **`start_trex (f, d, block_to_success, timeout, trex_cmd_options)`** + Issue a request to start T-Rex with certain configuration. The server will only handle the request if the T-Rex is in `Idle` status. + Once the status has been confirmed, T-Rex server will issue for this single client a token, so that only that client may abort running T-Rex session. + - `f` and `d` parameters are mandatory, as they are crutial parameter in setting T-Rex bahviour. Also, `d` parameter must be at least 30 seconds or larger. + `f` and `d` parameters are mandatory, as they are crucial parameter in setting T-Rex behaviour. Also, `d` parameter must be at least 30 seconds or larger. By default (and by design) this method **blocks** until T-Rex status changes to either 'Running' or back to 'Idle'. - **`stop_trex()`** + @@ -195,7 +195,7 @@ This section covers with great detail the usage of the client module. Each of th This option is very useful especially when the real-time data from the T-Rex are utilized. - **`wait_until_kickoff_finish(timeout = 40)`** + - This method blocks until T-Rex T-Rex status changes to 'Running'. In case of error an exception will be thrown. + + This method blocks until T-Rex status changes to 'Running'. In case of error an exception will be thrown. + The 'timeout' parameter sets the maximum waiting time. + This method is especially useful when `block_to_success` was set to false in order to utilize the time to configure other things, such as DUT. @@ -229,7 +229,7 @@ endif::backend-xhtml11[] This method performs single poll of T-Rex running data and process it into the result object (named `result_obj`). + The method returns the most updated data dump from T-Rex in the form of Python dictionary. + + - Behind the scenes, running that method will trigger inner-client process over the saved window, and produce window-relevant inforamtion, as well as get the most important data moe accessible. + + Behind the scenes, running that method will trigger inner-client process over the saved window, and produce window-relevant information, as well as get the most important data more accessible. + Once the data has been fetched (at sample rate the satisfies the user), a custom data manipulation can be done in various forms and techniques footnote:[See `CTRexResult` module usage for more details]. + **Note: ** the sampling rate is bounded from buttom to 2 samples/sec. @@ -270,7 +270,7 @@ These methods are: Fetch, out of all data dumps stored in history a value. - **History data access API** + - Since (as metioned earlier) the data dump is a JSON-RPC string, which is decoded into Python dictionaries and lists, nested within each other. + + Since (as mentioned earlier) the data dump is a JSON-RPC string, which is decoded into Python dictionaries and lists, nested within each other. + This "Mini API" is used by both `get_last_value` and `get_value_list` methods, and receives in both cases two arguments: `tree_path_to_key, regex` footnote:[By default, `regex` argument is set to None]. + The user may choose whatever value he wishes to extract, using the `tree_path_to_key` argument. @@ -291,7 +291,7 @@ These methods are: == Usage Examples -=== Exmaple #1: Checking T-Rex status and Launching T-Rex +=== Example #1: Checking T-Rex status and Launching T-Rex The following program checks T-Rex status, and later on launches it, querying its status along different time slots. @@ -345,7 +345,7 @@ Is T-Rex running? True {u'state': <TRexStatus.Running: 3>, u'verbose': u'T-Rex <1> When looking at T-Rex status, both an enum status (`Idle, Starting, Running`) and verbose output are available. -=== Exmaple #2: Checking T-Rex status and Launching T-Rex with 'BAD PARAMETERS' +=== Example #2: Checking T-Rex status and Launching T-Rex with 'BAD PARAMETERS' The following program checks T-Rex status, and later on launches it with wrong input ('mdf' is not legal option), hence T-Rex run will not start and a message will be available. @@ -402,7 +402,7 @@ Is T-Rex running? False {u'state': <TRexStatus.Idle: 1>, u'verbose': u'T-Rex ru <2> After T-Rex lanuching failed, a message indicating the failure reason. However, T-Rex is back Idle, ready to handle another launching request. -=== Exmaple #3: Launching T-Rex, let it run until custom condition is satisfied +=== Example #3: Launching T-Rex, let it run until custom condition is satisfied The following program will launch T-Rex, and poll its result data until custom condition function returns `True`. + In this case, the condition function is simply named `condition`. + Once the condition is met, T-Rex run will be terminated. @@ -439,7 +439,7 @@ print "Before Running, T-Rex status is: ", trex.get_running_status() <<< -=== Exmaple #4: Launching T-Rex, monitor live data and stopping on demand +=== Example #4: Launching T-Rex, monitor live data and stopping on demand The following program will launch T-Rex, and while it runs poll the server (every 5 seconds) for running inforamtion, such as latency, drops, and other extractable parameters. + Then, after some criteria was met, T-Rex execution is terminated, enabeling others to use the resource instead of waiting for the entire execution to finish. @@ -484,7 +484,7 @@ print "Before Running, T-Rex status is: ", trex.get_running_status() <<< -=== Exmaple #5: Launching T-Rex, let it run until finished +=== Example #5: Launching T-Rex, let it run until finished The following program will launch T-Rex, and poll it automatically until run finishes. The polling rate is customisable (in this case, every 10 seconds) using `time_between_samples` argument. diff --git a/trex_rpc_server_spec.asciidoc b/trex_rpc_server_spec.asciidoc new file mode 100644 index 00000000..94a95bfb --- /dev/null +++ b/trex_rpc_server_spec.asciidoc @@ -0,0 +1,1086 @@ +The TRex RPC Server +=================== +:author: Itay Marom +:email: <imarom@cisco.com> +:revnumber: 1.70-0.0 +:quotes.++: +:numbered: +:web_server_url: http://trex-tgn.cisco.com/trex +:local_web_server_url: csi-wiki-01:8181/trex + + + +== RPC Support On TRex + +TRex implements a RPC protocol in order to config, view and +in general execute remote calls on TRex + +In this document we will provide information on +how a client can implement the protocol used to communicate with TRex + +In general, we will describe the following: + +* *Transport Layer* - The transport layer used to communicate with TRex server +* *RPC Reprensentation Protocol* - The format in which remote procedures are carried + +=== Transport Layer + +TRex server transport layer is implemented using ZMQ. + +The default configuration is TCP on port 5555, however this is configurable. + +{zwsp} + +The communication model is based on the request-reply ZMQ model: + +http://zguide.zeromq.org/page:all#Ask-and-Ye-Shall-Receive + +{zwsp} + + +for more on ZMQ and implementation please refer to: +{zwsp} + +http://zeromq.org/intro:read-the-manual + +=== RPC Reprensentation Protocol + +The RPC reprensentation protocol is JSON RPC v2.0. +Every request and response will be encoded in a JSON RPC v2.0 format. + +{zwsp} + + +For more info on JSON RPC v2.0 spec please refer to: +{zwsp} + + +http://www.jsonrpc.org/specification + +{zwsp} + + +Later on in the document we will describe all the supported commands. + +=== TRex RPC Mock Server +Before we get into the commands, it's worth mentioning that TRex has a mock RPC server +designed to allow playing around with the server in order to understand the response +and perform adjustments to the request. + +TRex also provides a Python based console that can connect to the server (mock or real) and +send various commands to the server. + +==== Building The Mock Server +Building the mock server is performed like this: +[source,bash] +---- +trex-core/linux> ./b configure +trex-core/linux> ./b --target=mock-rpc-server-64-debug +---- + +==== Running The Mock Server +Launching the mock server is performed like this: + +[source,bash] +---- + +trex-core/scripts> ./mock-rpc-server-64-debug + +-= Starting RPC Server Mock =- + +Listening on tcp://localhost:5050 [ZMQ] + +Setting Server To Full Verbose + +Server Started + +---- + +==== Using The TRex Console To Interact +When the mock server is up, you can already send commands to the server. +{zwsp} + +{zwsp} + + +Let's demonstrate the operation with the Python based TRex console: + +{zwsp} + + +[source,bash] +---- +trex-core/scripts> ./trex-console + +Connecting To RPC Server On tcp://localhost:5050 +[SUCCESS] + + +-=TRex Console V1.0=- + +Type 'help' or '?' for supported actions + +TRex > + +---- +As we will see later on, a basic RPC command supported by the server is 'ping'. +{zwsp} + +Let's issue a ping command to the server and see what happens on both sides: + +{zwsp} + +{zwsp} + + +On the 'client' side: + +[source,bash] +---- +TRex > verbose on + +verbose set to on + +TRex > ping + +-> Pinging RPC server +[verbose] Sending Request To Server: + +{ + "id": "l0tog11a", + "jsonrpc": "2.0", + "method": "ping", + "params": null +} + +[verbose] Server Response: + +{ + "id": "l0tog11a", + "jsonrpc": "2.0", + "result": "ACK" +} + +[SUCCESS] + +---- +On the 'server' side: + +[source,bash] +---- + +trex-core/scripts> ./mock-rpc-server-64-debug + +-= Starting RPC Server Mock =- + +Listening on tcp://localhost:5050 [ZMQ] + +Setting Server To Full Verbose + +Server Started + + +[verbose][req resp] Server Received: + +{ + "id" : "maa5a3g1", + "jsonrpc" : "2.0", + "method" : "ping", + "params" : null +} + +[verbose][req resp] Server Replied: + +{ + "id" : "maa5a3g1", + "jsonrpc" : "2.0", + "result" : "ACK" +} + +---- + +== RPC Server Component Position Illustration + +The following diagram illustres the RPC server component's place: + +image::images/rpc_server_big_picture.png[title="RPC Server Position",align="left",width=800, link="images/rpc_server_big_picture.png"] + +== RPC Server Port State Machine +Any port on the server can be in numbered of states, each state provides other subset of the commands +that are allowed to be executed. + +We define the following possible states: + +* *unowned* - The specific port is either unowned or another user is owning the port +* *owned* - The specific port has been acquired by the client +* *active* - The specific port is in the middle of injecting traffic - currently active + +Each port command will specify on which states it is possible to execute it. + +For port related commands valid only on 'owned' or 'active', a field called ''handler'' 'MUST' be passed +along with the rest of the parameters. + + +This will identify the connection: + +image::images/rpc_states.png[title="Port States",align="left",width=150, link="images/rpc_states.png"] + +== RPC Commands +The following RPC commands are supported + +=== Ping +* *Name* - 'ping' +* *Valid States* - 'not relevant' +* *Description* - Pings the TRex server +* *Paramters* - None +* *Result* ['string'] - "ACK" On Sucess + +Example: + +[source,bash] +---- +'Request': + +{ + "jsonrpc": "2.0", + "id": 1, + "method": "ping", + "params": null +} + +'Response': + +{ + "jsonrpc" : "2.0", + "id" : 1, + "result" : "ACK" +} + +---- + +=== Get Server Supported Commands +* *Name* - 'get_supported_cmds' +* *Valid States* - 'not relevant' +* *Description* - Queries the server for all the supported commands +* *Paramters* - None +* *Result* ['array'] - A list of all the supported commands by the server + +Example: + +[source,bash] +---- +'Request': + +{ + "jsonrpc": "2.0", + "id": 1, + "method": "get_supported_cmds", + "params": null +} + + +'Response': + +{ + "jsonrpc": "2.0", + "id": 1, + "result": [ + "remove_all_streams", + "remove_stream", + "add_stream", + "get_reg_cmds", + "ping", + "test_sub", + "get_version", + "test_add" + ] +} + +---- + + +=== Get Version +* *Name* - 'get_version' +* *Valid States* - 'not relevant' +* *Description* - Queries the server for version information +* *Paramters* - None +* *Result* ['object'] - See table below + +.Object type 'return values for get_version' +[options="header",cols="1,1,3"] +|================= +| Field | Type | Description +| version | string | TRex version +| build_date | string | build date +| build_time | string | build time +| built_by | string | who built this version +|================= + +[source,bash] +---- + +'Request': + +{ + "id": "wapkk8m6", + "jsonrpc": "2.0", + "method": "get_version", + "params": null +} + + +'Response': + +{ + "id": "wapkk8m6", + "jsonrpc": "2.0", + "result": { + "build_date": "Sep 16 2015", + "build_time": "12:33:01", + "built_by": "imarom", + "version": "v0.0" + } +} + +---- + +=== Get System Info +* *Name* - 'get_system_info' +* *Description* - Queries the server for system properties +* *Paramters* - None +* *Result* ['object'] - See table below + +.return value: 'get_system_info' +[options="header",cols="1,1,3"] +|================= +| Field | Type | Description +| dp_core_count | int | DP core count +| core_type | string | DP core type +| hostname | string | machine host name +| uptime | string | uptime of the server +| port_count | int | number of ports on the machine +| ports | array | arary of object ''port'' - see below +|================= + +.return value: 'get_system_info'.'port' +[options="header",cols="1,1,3"] +|================= +| Field | Type | Description +| driver | string | driver type +| speed | string | speed of the port (1g, 10g, 40g, 100g) +| status | string | 'down', 'idle' or 'transmitting' +|================= + + +[source,bash] +---- + +'Request': + +{ + "id": "zweuldlh", + "jsonrpc": "2.0", + "method": "get_system_info", + "params": null +} + +'Response': + +{ + "id": "zweuldlh", + "jsonrpc": "2.0", + "result": { + "core_type": "Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz", + "dp_core_count": 1, + "hostname": "csi-kiwi-03.cisco.com", + "port_count": 4, + "ports": [ + { + "driver": "E1000", + "index": 0, + "speed": "1g", + "status": "down" + }, + { + "driver": "E1000", + "index": 1, + "speed": "1g", + "status": "down" + }, + { + "driver": "E1000", + "index": 2, + "speed": "1g", + "status": "down" + }, + { + "driver": "E1000", + "index": 3, + "speed": "1g", + "status": "down" + } + ] + } +} + +---- + +=== Get Owner +* *Name* - 'get_owner' +* *Valid States* - 'all' +* *Description* - Queries the server for a specific port current owner +* *Paramters* - +** *port_id* ['int'] - port id to query for owner +* *Result* ['string'] - owner name if exists, otherwise 'none' + +[source,bash] +---- + +'Request': + +{ + "id": "hxjkuwj9", + "jsonrpc": "2.0", + "method": "get_owner", + "params": { + "port_id": 1 + } +} + +'Response': + +{ + "id": "hxjkuwj9", + "jsonrpc": "2.0", + "result": { + "owner": "itay" + } +} + +---- + +=== Acquire +* *Name* - 'Acquire' +* *Valid States* - 'all' +* *Description* - Takes ownership over the port +* *Paramters* - +** *port_id* ['int'] - port id to take ownership +** *user* ['string'] - User name aquiring the system +** *force* ['boolean'] - force action even if another user is holding the port +* *Result* ['string'] - 'unique' connection handler for future requests for that port + +[source,bash] +---- + +'Request': + +{ + "id": "b1tr56yz", + "jsonrpc": "2.0", + "method": "Acquire", + "params": { + "user": "itay" + "port_id": 1 + "force": false, + } +} + + +'Response': + +{ + "id": "b1tr56yz", + "jsonrpc": "2.0", + "result": "4cBWDxS2" +} +---- + + +=== Release +* *Name* - 'release' +* *Valid States* - 'owned' +* *Description* - Release owernship over the device +* *Paramters* - +** *handler* ['string'] - unique connection handler +** *port_id* ['int'] - port id to release +* *Result* ['string'] - "ACK" on success + +[source,bash] +---- + +'Request': + +{ + "id": "m785dxwd", + "jsonrpc": "2.0", + "method": "release", + "params": { + "handler": "37JncCHr" + "port_id": 1 + } +} + + +'Response': + +{ + "id": "m785dxwd", + "jsonrpc": "2.0", + "result": "ACK" +} +---- + + +=== Add Stream +* *Name* - 'add_stream' +* *Valid States* - 'owned' +* *Description* - Adds a stream to a port +* *Paramters* +** *handler* ['string'] - unique connection handler +** *port_id* ['int'] - port id associated with this stream +** *stream_id* ['int'] - stream id associated with the stream object +** *stream* - object of type xref:stream_obj['stream'] +* *Result* ['string'] - "ACK" in case of success + +==== Object type 'stream' anchor:stream_obj[] + +Add_stream gets a single parameter of type object. + +The format of that object is as follows: + +.Object type 'stream' +[options="header",cols="1,1,3"] +|================= +| Field | Type | Description +| enabled | boolean | is this stream enabled +| self_start | boolean | is this stream triggered by starting injection or triggered by another stream +| isg | double | ['usec'] inter stream gap - delay time in usec until the stream is started +| next_stream_id | int | next stream to start after this stream. -1 means stop after this stream +| packet | object | object of type xref:packet_obj['packet'] +| mode | object | object of type xref:mode_obj['mode'] +| vm | array | array of objects of type xref:vm_obj['vm'] +| rx_stats | object | object of type xref:rx_stats_obj['rx_stats'] +|================= + +===== Object type 'packet' anchor:packet_obj[] +packet contains binary and meta data + +.Object type 'packet' +[options="header",cols="1,1,3"] +|================= +| Field | Type | Description +| binary | byte array | binary dump of the packet to be used in the stream as array of bytes +| meta | string | meta data object. opaque to the RPC server. will be passed on queries +|================= + +===== Object type 'mode' anchor:mode_obj[] +mode object can be 'one' of the following objects: + +.Object type 'mode - continuous' +[options="header",cols="1,1,3"] +|================= +| Field | Type | Description +| type | string | ''continuous'' +| pps | double | rate in packets per second +|================= + +.Object type 'mode - single_burst' +[options="header",cols="1,1,3"] +|================= +| Field | Type | Description +| type | string | ''single_burst'' +| pps | double | rate in packets per second +| total pkts | int | total packets in the burst +|================= + +.Object type 'mode - multi_burst' +[options="header",cols="1,1,3"] +|================= +| Field | Type | Description +| type | string | ''multi_burst'' +| pps | int | rate in packets per second +| pkts_per_burst | int | packets in a single burst +| ibg | double | ['usec'] inter burst gap. delay between bursts in usec +| count | int | number of bursts. ''0'' means loop forever, ''1'' will fall back to single burst +|================= + +===== Object type 'vm' anchor:vm_obj[] +Array of VM instruction objects to be used with this stream + +Any element in the array can be one of the following object types: + +.Object type 'vm - fix_checksum_ipv4' +[options="header",cols="1,1,3"] +|================= +| Field | Type | Description +| type | string | ''fix_checksum_ipv4'' +| pkt_offset | uint16 | offset of the field to fix +|================= + +.Object type 'vm - flow_var' +[options="header",cols="1,1,3"] +|================= +| Field | Type | Description +| type | string | ''flow_var''' +| name | string | flow var name - this should be a unique identifier +| size | [1,2,4,8] | size of the flow var in bytes +| op | ['inc', 'dec', 'random'] | operation type to perform on the field +| init value | uint64_t as string | init value for the field +| min value | uint64_t as string | minimum value for the field +| max value | uint64_t as string | maximum value for the field +|================= + +.Object type 'vm - write_flow_var' +[options="header",cols="1,1,3"] +|================= +| Field | Type | Description +| type | string | ''write_flow_var'' +| name | string | flow var name to write +| pkt_offset | uint16 | offset at the packet to perform the write +| add_value | int | delta to add to the field prior to writing - can be negative +| is_big_endian | boolean | should write as big endian or little +|================= + +TIP: For more information and examples on VM objects please refer to: +link:vm_doc.html[VM examples] + +===== Object type 'rx_stats' anchor:rx_stats_obj[] +Describes rx stats for the stream + +{zwsp} + + +IMPORTANT: In case rx_stats is enabled, meta data will be written in the end of the packet. +please also consider the following constraints: + +==== Constrains +* *performance* - this will have performance impact as rx packets will be examined +* *override* - up to 10 bytes at the end of the packet will be overidden by the meta data required + +==== The bytes needed for activating 'rx_stats': + +* *rx_stream_id* consumes 2 bytes +* *seq_enabled* consumes 4 bytes +* *latency_enabled* consumes 4 bytes + +so if no seq or latency are enabled 2 bytes will be used. + + +if seq or latency alone are enabled, 6 bytes will be used. + + +if both are enabled then 10 bytes will be used. + + +.Object type 'rx_stats' +[options="header",cols="1,1,3"] +|================= +| Field | Type | Description +| enabled | boolean | is rx_stats enabled for this stream +| seq_enabled | boolean | should write 32 bit sequence +| latency_enabled | boolean | should write 32 bit latency +|================= + +[source,bash] +---- + +'Request': + +{ + "id": 1, + "jsonrpc": "2.0", + "method": "add_stream", + "params": { + "handler": "37JncCHr", + "port_id": 1, + "stream_id": 502 + "stream": { + "enabled": true, + "isg": 4.3, + "mode": { + "pps": 3, + "total_pkts": 5000, + "type": "single_burst" + }, + "next_stream_id": -1, + "packet": { + "binary": [ + 4, + 1, + 255 + ], + "meta": "" + }, + "rx_stats": { + "enabled": false + }, + "self_start": true, + } + } +} + +'Response': + +{ + "id": 1, + "jsonrpc": "2.0", + "result": "ACK" +} + + +---- + + +=== Remove Stream +* *Name* - 'remove_stream' +* *Valid States* - 'owned' +* *Description* - Removes a stream from a port +* *Paramters* +** *handler* ['string'] - unique connection handler +** *port_id* ['int'] - port assosicated with the stream. +** *stream_id* ['int'] - stream to remove + +* *Result* ['string'] - "ACK" in case of success + +[source,bash] +---- + +'Request': + +{ + "id": 1 + "jsonrpc": "2.0", + "method": "remove_stream", + "params": { + "handler": "37JncCHr", + "port_id": 1, + "stream_id": 502 + } +} + + +'Response': + +{ + "id": 1 + "jsonrpc": "2.0", + "result": "ACK" +} + +---- + +=== Get Stream ID List +* *Name* - 'get_stream_list' +* *Valid States* - 'owned', 'active' +* *Description* - fetch all the assoicated streams for a port +* *Paramters* +** *handler* ['string'] - unique connection handler +** *port_id* ['int'] - port to query for registered streams + +* *Result* ['array'] - array of 'stream_id' + +[source,bash] +---- + +'Request': + +{ + "id": 1, + "jsonrpc": "2.0", + "method": "get_stream_list", + "params": { + "handler": "37JncCHr", + "port_id": 1 + } +} + +'Response': + +{ + "id": 1, + "jsonrpc": "2.0", + "result": [ + 502, + 18 + ] +} + + +---- + +=== Get Stream +* *Name* - 'get_stream' +* *Valid States* - 'owned', 'active' +* *Description* - get a specific stream object +* *Paramters* +** *handler* ['string'] - unique connection handler +** *port_id* ['int'] - port for the associated stream +** *stream_id* ['int'] - the requested stream id + +* *Result* ['object'] - object xref:stream_obj['stream'] + +[source,bash] +---- + +'Request': + +{ + "id": 1, + "jsonrpc": "2.0", + "method": "get_stream", + "params": { + "handler": "37JncCHr", + "port_id": 1, + "stream_id": 7 + } +} + + +'Response': + +{ + "id": 1, + "jsonrpc": "2.0", + "result": { + "stream": { + "enabled": true, + "isg": 4.3, + "mode": { + "pps": 3, + "type": "continuous" + }, + "next_stream_id": -1, + "packet": { + "binary": [ + 4, + 1, + 255 + ], + "meta": "" + }, + "self_start": true + } + } +} + +---- + + +=== Remove All Streams +* *Name* - 'remove_all_streams' +* *Valid States* - 'owned' +* *Description* - remove all streams from a port +* *Paramters* +** *handler* ['string'] - unique connection handler +** *port_id* ['int'] - port for the associated stream + +* *Result* ['string'] - "ACK" on success + + +[source,bash] +---- + +'Request': + +{ + "id": 1, + "jsonrpc": "2.0", + "method": "remove_all_streams", + "params": { + "handler": "37JncCHr", + "port_id": 2 + } +} + +'Response': + +{ + "id": 1, + "jsonrpc": "2.0", + "result": "ACK" +} + + +---- + + +=== Start Traffic +* *Name* - 'start_traffic' +* *Valid States* - 'owned' +* *Description* - Starts the traffic on a specific port. if traffic has already started an error will be returned +* *Paramters* +** *handler* ['string'] - unique connection handler +** *port_id* ['int'] - port id on which to start traffic + +* *Result* ['string'] - "ACK" on success + +[source,bash] +---- + +'Request': + +{ + "id": "b3llt8hs", + "jsonrpc": "2.0", + "method": "start_traffic", + "params": { + "handler": "37JncCHr", + "port_id": 3 + } + +'Response': + +{ + "id": "b3llt8hs", + "jsonrpc": "2.0", + "result": "ACK" +} + + +---- + +=== Stop Traffic +* *Name* - 'stop_traffic' +* *Valid States* - 'active' +* *Description* - Stops the traffic on a specific port. if the port has already started nothing will happen +* *Paramters* +** *handler* ['string'] - unique connection handler +** *port_id* ['int'] - port id on which to stop traffic + +* *Result* ['string'] - "ACK" on success + +[source,bash] +---- + +'Request': + +{ + "id": "h2fyhni7", + "jsonrpc": "2.0", + "method": "stop_traffic", + "params": { + "handler": "37JncCHr", + "port_id": 3 + } +} + +'Response': + +{ + "id": "h2fyhni7", + "jsonrpc": "2.0", + "result": "ACK" +} + + +---- + +=== Get Global Stats +* *Name* - 'get_global_stats' +* *Valid States* - 'owned', 'active' +* *Description* - Get machine global stats +* *Paramters* - None + +* *Result* ['object'] - See Below + +.Return value of 'get_global_stats' +[options="header",cols="1,1,3"] +|================= +| Field | Type | Description +| state | string | server state: can be 'unowned', 'owned' or 'active' +| cpu_util | double | DP CPU util. in % +| tx_bps | double | total TX bits per second +| rx_bps | double | total RX bits per second +| tx_pps | double | total TX packets per second +| rx_pps | double | total RX packets per second +| total_tx_pkts | int | total TX packets +| total_rx_pkts | int | total RX packets +| total_tx_bytes | int | total TX bytes +| total_rx_bytes | int | total RX bytes +| tx_rx_error | int | total Tx/Rx errors +|================= + +=== Get Port Stats +* *Name* - 'get_port_stats' +* *Valid States* - 'owned', 'active' +* *Description* - Get port stats +* *Paramters* +** *port_id* [int] - The port id for query + +* *Result* ['object'] - See Below + +.Return value of 'get_port_stats' +[options="header",cols="1,1,3"] +|================= +| Field | Type | Description +| status | string | 'down', 'idle' or 'transmitting' +| tx_bps | double | total TX bits per second +| rx_bps | double | total RX bits per second +| tx_pps | double | total TX packets per second +| rx_pps | double | total RX packets per second +| total_tx_pkts | int | total TX packets +| total_rx_pkts | int | total RX packets +| total_rx_bytes | int | total TX bytes +| total_tx_bytes | int | total RX bytes +| tx_rx_error | int | total Tx/Rx errors +|================= + +=== Get Stream Stats +* *Name* - 'get_steram_stats' +* *Valid States* - 'owned', 'active' +* *Description* - Get port stats +* *Paramters* +** *port_id* [int] - The port id for query +** *stream_id* [int] - The stream id for query + +* *Result* ['object'] - See Below + +.Return value of 'get_stream_stats' +[options="header",cols="1,1,3"] +|================= +| Field | Type | Description +| tx_bps | double | total TX bits per second +| tx_pps | double | total TX packets per second +| total_tx_pkts | int | total TX packets +| total_tx_bytes | int | total TX bytes +| rx_bps | double | total RX bits per second (if 'rx_stats' enabled) +| rx_pps | double | total RX packets per second (if 'rx_stats' enabled) +| total_rx_pkts | int | total RX packets (if 'rx_stats' enabled) +| total_rx_bytes | int | total RX bytes (if 'rx_stats' enabled) +| latency | array | array of 2 ordered elements average, maximum (if 'rx_stats' enabled) +|================= + + +== Typical Transactions Examples +the following examples represents common scenarios. +commands in [...] represents 'meta commands' +and not real RPC commands such as 'repeat', 'wait' and etc. + +=== Init/Boot +This sequence represents a client implementing the protocol taking ownership +over the server and preparing to perform work + +==== Commands Flow +* *ping* - Ping the server to verify the server is up +* *get_owner* - if owner is not me or 'none' prompt to the user if he wants to force it +* *acquire* - Ask or force for exclusive control over the server. save the 'handler' given for future commands +* *get_version* - Verify the server is compatible with the GUI +* *get_system_info* - Get the installed ports and cores +* *get_stream_list* - for every port, get the list and sync the GUI +* *get_stream* - for every stream in a port list, get the stream info and sync the GUI + +=== Simple Traffic With Adding / Editing Streams + +describes a simple scenario where a user wants to +add or edit one or more streams to one or more ports + +==== Commands Flow +* *[init]* - perform the init procedure from above +* *[GUI add/edit streams]* - GUI provides the user a way to add or edit streams and sync them +* *remove_all_streams* ['optional'] - remove all previous streams to start from scratch +* *add_stream* - configure a specific port with a stream. +* *['repeat previous']* - 'repeat' the above for how many ports and streams desired +* *get_stream_list* ['optional'] - sanity - verify the server is synced with the GUI +* *start_traffic* - start traffic on the specific port / all the ports +* *get_global_stats* ['optional'] - make sure the machine is transmiting traffic +* *['perfrom test']* - perform the required test +* *stop_traffic* - when done, stop the traffic on the specific port / all the ports +* *get_global_stats* ['optional'] - make sure the machine has stopped + +=== Logout + +Describes the log off from the machine + +==== Commands Flow +* *stop_traffic* ['optional'] - if traffic has started - stop it +* *get_global_stats* ['optional'] - make sure the machine has stopped +* *remove_all_streams* ['optional'] - if you want to clear all the previous streams - use this +* *release* - release the ownership over the device + @@ -139,6 +139,12 @@ def build(bld): bld(rule=my_copy, target=x) bld.add_group() + + for x in bld.path.ant_glob('yaml\\**\**.yaml'): + bld(rule=my_copy, target=x) + bld.add_group() + + for x in bld.path.ant_glob('video\\**\**.mp4'): bld(rule=my_copy, target=x) bld.add_group() @@ -186,6 +192,12 @@ def build(bld): source='vm_doc.asciidoc waf.css', target='vm_doc.html', scan=ascii_doc_scan) bld(rule='${ASCIIDOC} -a stylesheet=${SRC[1].abspath()} -a icons=true -a toc2 -a max-width=55em -o ${TGT} ${SRC[0].abspath()}', + source='packet_builder_yaml.asciidoc waf.css', target='packet_builder_yaml.html', scan=ascii_doc_scan) + + bld(rule='${ASCIIDOC} -a stylesheet=${SRC[1].abspath()} -a icons=true -a toc2 -a max-width=55em -o ${TGT} ${SRC[0].abspath()}', + source='trex_rpc_server_spec.asciidoc waf.css', target='trex_rpc_server_spec.html', scan=ascii_doc_scan) + + bld(rule='${ASCIIDOC} -a stylesheet=${SRC[1].abspath()} -a icons=true -a toc2 -a max-width=55em -o ${TGT} ${SRC[0].abspath()}', source='trex_control_plane_design_phase1.asciidoc waf.css', target='trex_control_plane_design_phase1.html', scan=ascii_doc_scan) bld(rule='${ASCIIDOC} -a stylesheet=${SRC[1].abspath()} -a icons=true -a toc2 -a max-width=55em -o ${TGT} ${SRC[0].abspath()}', diff --git a/yaml/headers.yaml b/yaml/headers.yaml new file mode 100755 index 00000000..fd217dc6 --- /dev/null +++ b/yaml/headers.yaml @@ -0,0 +1,429 @@ +# This yaml describes packet headers and their fields + + +####################################################################### +# Root, reserved starting point + + + - class: 'root' + gui_representation: + help: 'Root' + next_headers: ['ethernet', 'llc', '_802-3'] + + +####################################################################### +# Protocols + + +# L2 + + + - class: 'ethernet' + gui_representation: + help: 'Ethernet-L2' + properties: ['external'] + default: [0x00, 0x01, 0x01, 0x00, 0x00, 0x00, 0x00, 0x02, 0x02, 0x00, 0x00, 0x00, 0x08, 0x00] + fields: + - name: 'Dst' + gui_representation: + help: 'Destination MAC' + type: 'c-mac-addr' + default: [0x77, 0x55, 0x01, 0x00, 0x00, 0x01] + + - name: 'Src' + gui_representation: + help: 'Source MAC' + type: 'c-mac-addr' + + - name: 'Ethertype' + gui_representation: + help: 'Ethertype' + type: 'uint16' + default: 0x0800 + value_based_next_header: + 0x0800: 'ipv4' + # not implemented + # 0x0806: 'arp' + # 0x86DD: 'ipv6' + # 0x8100: 'vlan' # field of 4 bytes added in this case, what to do? + # 0x8847: 'mpls unicast' # unicast or multicast? multicast is 0x8848 + default: 'payload' + + +# L3 + + + - class: 'ipv4' + gui_representation: + help: 'IPv4' + properties: ['external'] + fields: + - name: 'ver' + gui_representation: + help: 'Version' + type: 'c-bit' + array_size: 4 + default: [0, 1, 0, 0] + properties: ['const'] + + - name: 'ihl' + gui_representation: + help: 'IHL' + type: 'c-bit' + array_size: 4 + default: [0, 1, 0, 1] + properties: ['ipv4_ihl'] + + - name: 'DSCP' + gui_representation: + help: 'Differentiated Services Code Point' + type: 'c-bit' + array_size: 6 + default: [0, 0, 0, 0, 0, 0] + + - name: 'ECN' + gui_representation: + help: 'Explicit Congestion Notification' + type: 'c-bit' + array_size: 2 + default: [0, 0] + + - name: 'total_len' + gui_representation: + help: 'Total Length' + type: 'uint8' + array_size: 2 + default: [0, 57] + properties: ['total_length'] + + - name: 'Identification' + gui_representation: + help: 'Identification' + type: 'uint8' + array_size: 2 + default: [0x00, 0x00] + + - name: 'Flags' + gui_representation: + help: 'IPv4 Flags' + type: 'c-bit' + array_size: 3 + default: [0, 0, 0] + fields: + - name: 'Reserved' + gui_representation: + help: 'Reserved' + type: 'bit' + default: 0 + properties: ['const'] + + - name: 'DF' + gui_representation: + help: "Don't Fragment" + type: 'bit' + default: 0 + + - name: 'MF' + gui_representation: + help: 'More Fragments' + type: 'bit' + default: 0 + + - name: 'Fragment Offset' + gui_representation: + help: 'Fragment Offset' + type: 'c-bit' + array_size: 13 + default: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] + + - name: 'TTL' + gui_representation: + help: 'Time To Live' + type: 'uint8' + default: 0x80 + + - name: 'protocol' + type: 'uint8' + default: 0x06 + value_based_next_header: &ipv4_next_header + 0x06: 'tcp' + 0x11: 'udp' + # not implemented + # 0x29: 'ipv6' + # 0x2F: 'gre' + default: 'payload' + gui_representation: + help: 'IPv4 next protocol field' + combobox_values: + <<: *ipv4_next_header + 0x11: 'Next header is UDP' # overrides the description for combo-box + + - name: 'ipv4_chksum' + gui_representation: + help: 'IPv4 Header Checksum' + type: 'uint16' + default: [0x0000] + properties: ['ipv4_checksum', 'const'] + + - name: 'Src' + gui_representation: + help: 'Source IPv4' + type: 'c-ipv4-addr' + + - name: 'Dst' + gui_representation: + help: 'Destination IPv4' + type: 'c-ipv4-addr' + + # not implemented + # - name: 'Opts' # presence of this field and it's size should be determined in python/java + # help: 'IPv4 options' + # type: 'c-ipv4-options' + + +# L4 + + + - class: 'tcp' + gui_representation: + help: 'TCP' + properties: ['external'] + next_headers: ['ipv6', 'ipv4'] + fields: + - name: 'src_port' + gui_representation: + help: 'Source Port' + type: 'c-port-16bit' + + - name: 'dest_port' + gui_representation: + help: 'Destination Port' + type: 'c-port-16bit' + + - name: 'Seq' + gui_representation: + help: 'Sequence number' + type: uint32 + default: 0x30000000 + + - name: 'Ack' + gui_representation: + help: 'Acknowledgment number' + type: uint32 + default: 0x30000000 + + - name: 'Data offset' + gui_representation: + help: 'Data offset' + type: 'c-bit' + array_size: 4 + default: [0, 1, 0, 1] + + - name: 'Reserved' + gui_representation: + help: 'TCP Reserved' + type: 'bit' + array_size: 3 + default: [0, 0, 0] + properties: ['const'] + + - name: 'flags' # tree with leaf of bits + gui_representation: + help: 'TCP flags' + type: 'c-bit' + array_size: 9 + default: [0, 0, 0, 0, 0, 0, 0, 0, 0] + fields: + - name: 'ns' + gui_representation: + help: 'NS flag' + type: 'bit' + default: 0 + + - name: 'cwr' + gui_representation: + help: 'CWR flag' + type: 'bit' + default: 0 + + - name: 'ece' + gui_representation: + help: 'ECE flag' + type: 'bit' + default: 0 + + - name: 'urg' + gui_representation: + help: 'URG flag' + type: 'bit' + default: 0 + + - name: 'ack' + gui_representation: + help: 'ACK flag' + type: 'bit' + default: 0 + + - name: 'psh' + gui_representation: + help: 'PSH flag' + type: 'bit' + default: 0 + + - name: 'rst' + gui_representation: + help: 'RST flag' + type: 'bit' + default: 0 + + - name: 'syn' + gui_representation: + help: 'SYN flag' + type: 'bit' + default: 0 + + - name: 'window_size' + gui_representation: + help: 'Window size' + type: uint16 + default: 0x0000 + + - name: 'checksum' + gui_representation: + help: 'Checksum' + type: uint16 + default: 0x0000 + properties: ['tcp_checksum'] + + - name: 'urgent_pointer' + gui_representation: + help: 'Urgent pointer' + type: uint16 + default: 0x0000 + + # not implemented + # - name: 'tcp options' + # type: 'c-tcp-options' + + + - class: 'udp' + gui_representation: + help: 'UDP' + properties: ['external'] + next_headers: ['ipv6', 'ipv4'] + fields: + - name: 'src_port' + gui_representation: + help: 'Source Port' + type: 'c-port-16bit' + + - name: 'dest_port' + gui_representation: + help: 'Destination Port' + type: 'c-port-16bit' + + - name: 'length' + gui_representation: + help: 'Length' + type: 'uint16' + default: 0x0020 + properties: ['total_length'] + + - name: 'checksum' + gui_representation: + help: 'UDP Checksum' + type: uint16 + default: 0x0000 + properties: ['udp_checksum'] + + +####################################################################### +# Fields + + + - class: 'c-mac-addr' + type: 'uint8' + array_size: 6 + default: [0x00, 0x00, 0x01, 0x00, 0x00, 0x00] + gui_representation: + help: 'MAC address' + data_type: 'mac-addr_t' # format ([0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2} + form_type: 'combo_with_edit' + + + - class: 'c-ipv4-addr' + type: 'uint8' + array_size: 4 + default: [1, 1, 1, 1] + gui_representation: + help: 'IPv4 address' + data_type: 'ipv4_t' + form_type: 'combo_with_edit' + + + - class: 'c-port-16bit' + type: 'uint16' + default: 0x1234 + gui_representation: + help: 'ports of TCP/UDP etc.' + form_type: 'combo_with_edit' + + + - class: 'c-bit' + type: 'bit' + gui_representation: + help: 'bit with checkbox representation' + form_type: 'checkbox' + + +###################### +# incomplete + + - class: 'c-ipv4-option' + type: 'vlen_t' + default: 0x01 + fields: + - name: 'copied flag' + help: 'Copy the option to all fragments flag' + type: 'c-bit' + default: 0 + + - name: 'option class' + help: '0 = controls, 2 = debugging' + type: 'bit' + array_size: 2 + default: 0 + + - name: 'option number' + help: 'Option Number' + type: 'bit' + array_size: 5 + value_based_next_class: + 0: 'end' # reserved name for ending options + 1: 'ip_option' # back to itself + 2: 'c-ipv4opt-security' + 3: 'c-ipv4opt-loose-source-routing' + 4: 'c-ipv4opt-internet-timestamp' + 7: 'c-ipv4opt-record-route' + 8: 'c-ipv4opt-stream-id' + 9: 'c-ipv4opt-strict-source-routing' + default: 'end' + + + - class: 'c-ipv4opt-security' + help: 'ipv4 option security' + type: 'bit' + array_size: 11 + default: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] + fields: + value_based_next_class: + default: 'c-ipv4-option' + + - class: 'c-ipv4opt-loose-source-routing' + help: 'ipv4 option loose source routing' + type: 'bit' + default: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] + fields: + value_based_next_class: + default: 'c-ipv4-option' + |