diff options
Diffstat (limited to 'docs/csit_python2_to_python3_migration.md')
| -rw-r--r-- | docs/csit_python2_to_python3_migration.md | 182 |
1 files changed, 0 insertions, 182 deletions
diff --git a/docs/csit_python2_to_python3_migration.md b/docs/csit_python2_to_python3_migration.md deleted file mode 100644 index 8263f7aabe..0000000000 --- a/docs/csit_python2_to_python3_migration.md +++ /dev/null @@ -1,182 +0,0 @@ -# FD.io CSIT migration Python 2.7 to Python 3 - -## Python 3 version - -There is a pre-agreement to migrate to Python 3 version used by -Ubuntu 18.04-LTS - currently it is version [3.6.8](https://docs.python.org/3.6/whatsnew/changelog.html#python-3-6-8-final). - -CentOS7 version 1810 that is used in [FD.io](https://fd.io/) also contains -Python 3.6. - -## Dependency libs - -There was used *[caniusepython3](https://pypi.org/project/caniusepython3/)* -tool to check readiness of current version of csit external libraries for -Python 3. It identified one external library that needs to be updated to -support Python 3: - ``` - (env) vpp@vpp-VirtualBox:~/Documents/csit$ caniusepython3 -r requirements.txt - Finding and checking dependencies ... - You need 1 project to transition to Python 3. - Of that 1 project, 1 has no direct dependencies blocking its transition: - pypcap - (env) vpp@vpp-VirtualBox:~/Documents/csit$ caniusepython3 -r tox-requirements.txt - Finding and checking dependencies ... - You have 0 projects blocking you from using Python 3! - (env) vpp@vpp-VirtualBox:~/Documents/csit$ - ``` - -The latest released version of *[pypcap](https://pypi.org/project/pypcap/)* is -version 1.2.3 (Python 3 support implemented in version 1.2.0). - -Packages were checked for Python 3.6.8 support too and here are proposed -package versions: - -- directly needed packages - - ecdsa==0.13.3 - - paramiko==2.6.0 - - pycrypto==2.6.1 - - pypcap==1.2.3 # min. v1.2.0 for Python 3.6 support - - PyYAML==5.1 - - requests==2.22.0 # min. v2.14.0 for Python 3.6 support - - robotframework==3.1.2 - - scapy==2.4.3 # min. v2.4.0 for Python 3.6 support - - scp==0.13.2 - -- directly needed packages for PLRSearch - - dill==0.3.1.1 - - numpy==1.17.3 # v1.14.5 - compatibility with Python 3.6.2, possible - incompatibility with Python 3.6.8; v1.14.6 should be compatible with - Python 3.6.8 - - scipy==1.3.1 - -- directly needed packages for PAL - - hdrhistogram==0.6.1 - - pandas==0.25.3 - - plotly==4.1.1 - - PTable==0.9.2 - - Sphinx==2.2.1 - - sphinx-rtd-theme==0.4.0 - - sphinxcontrib-programoutput==0.15 - -- packages needed by paramiko package - - bcrypt==3.1.7 - - cffi==1.13.1 - - cryptography==2.8 - - pycparser==2.19 - - PyNaCl==1.3.0 - - six==1.12.0 - -- packages needed by request package - - certifi==2019.9.11 - - chardet==3.0.4 - - idna==2.8 - - urllib3==1.25.6 - -- not needed anymore - - aenum - enum module in Python 3.6 already contains needed enum types - - ipaddress - module already included in Python 3.6 - - pexpect - can be removed when corresponding unused code is removed from - ssh.py - - pykwalify + docpot + python-dateutil - can be removed if virl not used - anymore - -After discussion there is an agreement to use pip freeze for indirect -dependencies when all direct dependency versions are resolved - see example of -*[requirements.txt](https://gerrit.fd.io/r/c/csit/+/23207/17/requirements.txt)* -file in CSIT gerrit commit -[Python3: PIP requirement](https://gerrit.fd.io/r/c/csit/+/23207). - -## Required CSIT code changes - -There were identified following code changes that need to be addressed during -Python 2.7 to Python 3 migration in CSIT: -- imports relative to package - - `import submodul1` => `from . import submodule1` - - `from csv import my_csv` => `from .csv import my_csv` -- StringIO - - `import StringIO` => `from io import StringIO` -- `StandardError` -=> `Exception` -- raising exceptions - should be ready - - `raise ValueError, "wrong value"` => `raise ValueError("wrong value")` -- catching exceptions - should be ready - - `except ValueError, e:` => `except ValueError as e:` -- integers - - `long` => `int` -- strings and bytes - - `unicode` => `str` - - `basestring` => `str` - - `str` => `bytes` - not generally, only if bytes type really required - - use following string style conventions: - - `u"a unicode string literal"` - - `b"a bytes string literal"` - - `f"a formatted unicode string literal"` - `f"He said his name is {name}"` - instead of `"He said his name is {n}".format(n=name)` -- integer division with rounding down - - `2 / 3` => `2 // 3` -- metaclasses - use only new style - - `class Form(BaseForm, metaclass=FormType):` -- for-loop variables and the global namespace leak - - for-loop variables don't leak into the global namespace anymore -- returning iterable objects instead of lists - - `xrange` => `range` - - `range` => `list(range())` - - `map` => `list(map())` - - `zip` => `list(zip())` - - `filter` => `list(filter())` - - dictionaries - - `.iteritems()` => `.items()` - - `.iterkeys()` => `.keys()` - - `.itervalues()` => `.values()` - - `.viewitems()` => `.items()` - - `.viewkeys()` => `.keys()` - - `.viewvalues()` => `.values()` - - `.items()`=> `list(.items())` - - `.keys()` => `list(.keys())` - - `.values()` => `list(.values())` - - `dict.has_key(key)` => `key in dict` - - lists - - `L = list(some_iterable); L.sort()` => `L = sorted(some_iterable)` - - parenthesis in list comprehensions - - `[... for var in item1, item2, ...]` => `[... for var in (item1, item2, ...)]` -- file IO with `open` - - `f = open('myfile.txt') # f.read() returns byte string` => - `from io import open` plus - - `f = open('myfile.txt', 'rb') # f.read() should return bytes` - - `f = open('myfile.txt', 'rt') # f.read() should return unicode text` -- reduce() - - `reduce()` => `from functools import reduce; reduce()` - -- python files in following directories: - - resources/libraries/python - - resources/tools - - resources/traffic_profiles/trex - - resources/traffic_scripts - -- check python calls in bash files: - - resources/libraries/bash/ - - csit root directory - -## Migration steps - -1. Update all external libraries - week(s) before the week W -1. Install agreed Python 3 version to all servers used by CSIT for test - execution - week(s) before the week W - 1. vpp device servers - already done - 1. performance testbeds - already done - 1. jenkins executors - already done -1. Freeze the CSIT master branch for one week for commits other then Python 2 to - Python 3 migration - week W - 1. Create back up branch of actual master - 1. Migrate libraries - work split between all available CSIT developers. Each - one will submit separate commit for review - csit-vpp-xxx verify jobs will - be failing at this phase so committers will need to overwrite verify - voting to be able to merged these commits. - - TODO: provide separate spread sheet with listed libraries to be migrated - with the name of CSIT developer responsible for the migration of this - library. - 1. Run jobs and tests of all of types when all libraries migrated to confirm - functionality or to catch bugs that needs to be fixed - iterate until - successful execution of all tests. -1. Unfreeze the CSIT master branch. |