Package management in Python 2 or 3 (Linux and Mac) with virtualenv or venv
TL;DR starting and ending package manager
- First time only:
$ python3 -m venv ENV
to create environment files. $ source ENV/bin/activate
Run environment.$ pip install [package-name]
Install packages inside running environment.$ deactivate
or close terminal to turn off environment.
Using this tutorial
- As of November 30, 2017, Python.org now recommends Pipenv for managing packages. But this tutorial is still accurate. Use whatever works for you.
- This is the basic guide I wish I had starting out with Python package management.
- It’s meant to be read beginning to end, but feel free to skip non-applicable sub-bullets and the optional section.
- Starts basic and gets you to a spot where you should be comfortable with the most common features.
- This tutorial covers Python 2 and 3, Virtualenv and venv, on Linux or Mac.
- Written May 1st, 2016 on an HP Stream 11 running Lubuntu 14.04.4 LTS and a MacBook Air 11 mid 2011. Software is always changing, this guide will likely become out-of-date at some point. The software referenced in this tutorial is: pip 8.1.1, virtualenv 15.0.0, Python 2.7.6 and 3.4.3. My terminals run Terminator 0.98 (Lubuntu) and iTerm2 (Mac) both running the Zsh shell with Oh-My-Zsh, but you should be able to run whatever terminal you prefer.
- When you see
[sudo]
in brackets like that, it means that it’s optional depending on your computer’s permissions. If you run a command withoutsudo
but the command requiressudo
, you’ll get a permission error. If you need to redo a command withsudo
, don’t retype the whole thing aftersudo
, just typesudo !!
. The double exclamation points represent the last terminal command. - Leave out the
$
at the beginning of the terminal command examples when you type or copy them into your own terminal. It’s just there to let you know the command is for the terminal. - Periods end most sentences in this guide even if the last part of a sentence is code. Be sure not to copy the period, it is outside the code sample:
like this
. - I use the words “directory” and “folder” interchangeably.
- Basic terminal navigation knowledge assumed.
- Written by Aaron Snitzer.
- Tweet me @snitzr or email me (first name last name gmail) if you have a question or comment. Don’t get stuck.
Basic Q & A
- What is a Python package? Python files that you install to give greater functionality to your Python projects.
- What is a package manager? A package manager lets you keep separate installed Python libraries for each Python program you’re working on.
- What is Virtualenv? Virtualenv is a package manager for the Python programming language. It is not a virtual machine running inside your computer (like VirtualBox or Vagrant) and it’s not Docker.
- Why do I need to keep packages for different projects separate? First, you may not need the exact same packages or package versions for each project. Second, if you collaborate, a package manager helps all team members use the exact same packages and versions.
- How do I install Python packages? There is a program called pip, which installs Python packages with:
$ [sudo] pip install [package-name]
. I’ll talk more about installing packages soon. - Is Virtualenv required for development? No, but it should help development go better.
- What package manager should I use? Doesn’t Python 3 come with its own package manager? Python 3 has venv. Virtualenv and venv are very similar. I think if you learn one, you will know the other. You can substitute
python3 -m venv
wherevirtualenv
is mentioned in the tutorial and it should be the same.python3 -m venv
can be activated with just,pyvenv
, butpyvenv
will likely be deprecated in Python 3.6. In the Python 3 install on my Linux machine, thepyvenv
script wasn’t even there. Note: I’m talking about pyvenv and not pyenv. Those are two different things with confusingly similar names. Note the “v” in the middle of pyvenv. - Does a virtual environment also hold Python or just Python packages? Just Python packages.
Setup virtualenv for the first time
- Install or upgrade pip.
- Virtualenv installs with a program called pip.
- Check for pip with
pip -V
. - If you don’t have pip installed, there are pip installation instructions on the pip documentation website.
- Even if pip is already installed, it might be an older version. Run
$ [sudo] pip install --upgrade pip
for the newest version.
- Install or upgrade virtualenv.
- Check for virtualenv with
$ virtualenv --version
. $ [sudo] pip install virtualenv
or$ [sudo] pip install --upgrade virtualenv
, if needed.
- Check for virtualenv with
- In the same directory as your project, type
$ virtualenv ENV
. This creates the ENV folder in that directory.- Running
$ virtualenv ENV
overwrites any existing directory namedENV
in the directory you are in. - If you have Python 2 and Python 3 installed, virtualenv defaults to Python 2. If you have Python 3 installed and want to use it in the virtual environment, do the following:
$ which python3
, add this output to the next command.$ virtualenv -p [output from which python3] ENV
. The-p
flag means “select the following Python version to use within the virtual environment.”- If the above command doesn’t work, substitute
python3 -m venv
where you seevirtualenv
in this tutorial.
- There is no requirement that dictates the name or capitalization for the virtualenv directory, other than individual style. You can call the directory almost any name you like.
ENV
andvenv
seem the most common. - If you use version control software (like Git), you might like to add “
ENV/
” to your.gitignore
or other similar ignore file.
- Running
- The virtual environment is created but not yet active. Wait to install Python packages specific to your project until you start the virtual environment.
Start the virtual environment
- While in the directory with
ENV
(not inENV
, but the directory withENV
), run$ source ENV/bin/activate
. This activates the virtual environment.- You’ll probably see the name of the virtual environment in parentheses in your terminal. For example:
(ENV) foo:~/$
, or something like that, it varies with the terminal you use. This helps you know the environment is active. The name in parenthesis is the name of the folder you created using$ virtualenv ENV
. - It would be more intuitive if activating the environment was something like,
$ virtualenv activate
, but that’s not the case (even though deactivating a virtual environment is just the worddeactivate
, as you’ll see in a moment). If you like, you could add an alias in your terminal setup dotfile like:alias activate="source ENV/bin/activate"
. source
is sometimes written as just.
in other tutorials. They do the same thing.
- You’ll probably see the name of the virtual environment in parentheses in your terminal. For example:
Install packages
- While the virtualenv is running (because you ran
$ source ENV/bin/activate
), use pip to install packages.- Example:
(ENV) $ pip install ipython
. - Adding
sudo
is not needed or recommended for pip inside an active virtual environment. - While virtualenv is running, you will only have access to the packages you installed while running the virtual environment (and vice versa).
- Example:
- Now you can work on your project in this environment, you’re all set.
- Most packages you install with pip while running the virtual machine are available for import in your project. For example: if you run
pip install fibo
, for Fibonacci math functionality, you add the lineimport fibo
into your Python script and now you have access to that module’s functionality and namespace in your program. Here’s Python’s more detailed explanation of importing packages and modules.
- Most packages you install with pip while running the virtual machine are available for import in your project. For example: if you run
Stop the virtual environment
Deactivate the virtualenv with (ENV) $ deactivate
. Virtualenv also stops if you close the terminal window or session. Remember to reactivate it with $ source ENV/bin/activate
when you want to use it again.
Sharing environments with a friend
$ source ENV/bin/activate
in your project directory.(ENV) $ pip freeze
, this is a list of what you’ve installed with pip.- If you installed ipython from the example in this tutorial, you will notice more packages listed than just ipython, this is OK and covered in the next section.
(ENV) $ pip freeze > requirements.txt
, create a file namedrequirements.txt
with a list of what you’ve installed with pip.- The name
requirements.txt
is just the commonly used name, it’s not a rule to name it that. - This is a manual process. Packages installed after this will not appear automatically.
- The name
- Send a copy of your
requirements.txt
file to a friend. - Have your friend install pip and virtualenv, if not already installed and up-to-date.
- Have your friend run the following commands on his or her machine:
$ virtualenv ENV
, creates the virtual environment.$ source ENV/bin/activate
, starts the virtual environment.(ENV) $ pip install -r requirements.txt
, installs all the packages inrequirements.txt
.
- Now you and your friend should have the same Python packages installed for the project you’re working on together.
How did this get in my requirements.txt?
Some packages need to install other packages called “dependencies.” Dependencies are Python packages and show up in the requirements.txt
file. But the dependencies might have a name nothing to do with the package they came from. There is no indication in requirements.txt
to show what package installed the dependency.
Example: the ipython package installs nine dependencies with names like: backports.shutil-get-terminal-size, pexpect, and traitlets. Suddenly your requirements.txt
file is cluttered. If you need to know what package installed the dependencies, the only solution I know is to use (ENV) $ pip show ipython
to get a list of what ipython requires. Or, if you’re OK not installing dependencies, install packages with pip’s --no-deps
flag: (ENV) $ pip install --no-deps ipython
. But --no-deps
won’t install a usable ipython without dependencies. I’m only using ipython as an example because it’s a package that installs dependencies, you don’t need to install ipython unless you need it.
Optional: if you need to move the ENV location
This breaks links to ENV and prevents the virtual environment from working. You have three options:
- Move
ENV
back to where it was created. Use it there. - Install from
requirements.txt
- Move
ENV
to its original location. $ pip freeze > requirements.txt
- Delete or remove the current ENV.
- In new virtual environment location, run
$ virtualenv ENV
. - Move
requirements.txt
to the same directory as the new ENV. $ source ENV/bin/activate
, starts the virtual environment.(ENV) $ pip install -r requirements.txt
, installs all the packages inrequirements.txt
.
- Move
- Move
ENV
back to where it was created. Run$ virtualenv --relocatable ENV
. Move the ENV to where you want it. Note:--relocatable
officially has issues and might not work.
References and other guides
- A non-magical introduction to Pip and Virtualenv for Python beginners (somewhat outdated).
- Kenneth Reitz’s Virtual Environments section in his Hitchhiker’s Guide to Python.
- Python Virtual Environments - a Primer.
- Pip documentation.
- Virtualenv documentation.
- Venv documentation (Python 3).
- Python Packaging User Guide: Creating and using virtual environments.
Changelog
- May 8, 2016:
- Capitalization correction, iPython to ipython.
- Add note for
--relocatable
issues. - Replace soon-to-be deprecated
pyvenv
withpython3 -m venv
. - Add link to Virtual Environment Primer.
- Add link to Python Packaging User Guide.
- May 27, 2016:
- Add description and link describing how to import and use installed packages.
- January 08, 2018
- Add legacy warning.
- Add TL;DR section.
License
Package management in Python 2 or 3 (Linux and Mac) with virtualenv or venv by Aaron Snitzer has a Creative Commons Attribution-ShareAlike 4.0 International License.