This article discusses the coexistence and invocation of multiple versions of Python modules, and attempts to provide a refreshing solution.

Multiple versions of the Python module?

With Python becoming the main development language in data science, more and more machine learning libraries are releasing Python versions, including the famous scikit-learn, tensorflow, keras, etc. It is due to the rapid development of data science in recent years in terms of underlying algorithms and computational architectures that new versions of these Python modules have been released frequently.

The most disgusting thing about Tensorflow was that before the stable version 1.0.0 was released, almost every sub-version of Tensorflow had a different API. Of course this is mostly due to the endless array of algorithms in the deep learning space, on the other hand I personally suspect it is due to the Tensorflow code constantly making evolutions based on changes in google's computing platform.

The problem then becomes that the tf code for Seq2Seq, which you may have written yourself three months ago, no longer works in the new version of Tensorflow, which is caused by the API changes brought about by the version change. So some say, as long as I don't update TF all the time, won't it be fine?

The ideal is rich, but the reality is bleak!

• The Ubuntu machine used by the author is shared by the lab, and the server administrator's decision whether to update TF is important!
• The computational efficiency of the latest version of the TF is always improving in relative terms: the new version of the TF can use the newer CUDA version as well as the CUDNN version.
• New developers are often willing to go to the latest stable version of the TF to develop their algorithms, and if they want to use the algorithms they develop, then they will inevitably need to be in the corresponding TF version to run. With this in mind, in order to run both old and new code, we usually choose to run it on the same machine Configure co-existing multi-version Modules.

How do I configure and use multiple versions of the mod?

A simple idea is to create, for each development task, a separate environment that has modules independent of the outside world. As soon as we get into this separate environment, we can install and use the mission-corresponding version of the module in it.

Fortunately, there are many ways to do this, such as docker. But here we don't use such a high level technique for now, we just need to use the virtualenv command.

1. Create a standalone Python environment with the virtualenv command

Installation of virtualenv

# Install virtualenv under Python 2.7

sudo pip install virtualenv

Create a separate Python environment with the virtualenv command

# Create a space for as many environments as I may create

mkdir ~/my_lib/py_env cd ~/my_lib/py_env

# Assuming that the system default Python is Python 2.7

# Create a standalone environment for Python 2.7

virtualenv py27_env

How do you get into that standalone Python environment?

Notice that by entering the standalone Python environment here you don't necessarily have to cd to the folder where the Python standalone environment was created in the previous step, you can start the environment almost anywhere.

A boot environment means that in this environment, commands such as cd, ls, etc. are no different from the outside world except for Python related commands, and only Python related commands such as pip, python, etc. are independent of the outside world.

cd ~

The Python environment can be accessed anywhere

source ~/my_lib/py_env/py27_env/bin/activate

# In fact, the above command can be interpreted as adding ~/my_lib/py_env/py27_env/bin to the system path with the highest priority. However, the ~/my_lib/py_env/py27_env/bin folder only contains commands related to Python, so only the execution of Python related commands will be affected by this environment

How to exit the standalone Python environment

deactivate

2. Install any version of the module in the corresponding standalone environment

Our task here is to create a Python 3.5 environment and then install the 1.0.1 gpu version of tensorflow using pip

Create the Python 3.5 environment first and activate it

# Create a Python 3.5 environment

# -p PYTHON_EXE_PATH argument, which can also be replaced with --python=PYTHON_EXE_PATH

# represents the system Python command from which the standalone environment is created, change the parameter to get a different version of Python standalone environment virtualenv -p /usr/bin/python3.5 ~/my_lib/py_env/py35_env

# Activate it source ~/my_lib/py_env/py35_env/bin/activate

Install tensorflow 1.0.1 gpu version

pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.0.1-cp34-cp34m-linux_x86_64.whl

# Here the command pip is replaced by pip3 or pip3.5, all three commands are in ~/my_lib/py_env/py35_env/bin/

# It's worth noting that you can run pip2 here, which is the system command, /usr/bin/pip2. You can use which pip2 to verify which pip2 which pip which pip3 which pip3.5

You can go into the Python interpreter and see

# The following three commands all access the standalone Python 3.5 environment python python3 python3.5

After entering the interpreter

import tensorflow as tf# See where this tensorflow is installed print(tf.__path__)

More options for virtualenv

The system Python environment here is the -p parameter that was used to create the standalone environment

1) The --no-site-packages option means that all third-party packages installed into the system Python environment will not be copied over, currently I see this implemented by default

2) The --system-site-packages option means that the virtual environment can access third-party packages in the system Python environment

3) More options can be viewed directly from the shell by typing virtualenv

Recommended>>
1、The detailed tips for reading the literature are awesome to say the least
2、Central city cadastre management system goes live
3、Unmanned car sharing is coming are you ready
4、with respect tonbspReolink Webcam remote code execution vulnerability briefing
5、The niche culture blockchain is here and the niche culture blockchain will bring seven major changes to niche culture

已发送

发送中