Here is how to upgrade a Home Assistant instance to python 3.8.6 using python virtual environments. This is my own experience, with some minor problems that I will explain later, yours could be different. I encourage to take a full backup of your Home Assistant, configuration and database before starting. Anyway I have tried to do the upgrade in the least possible intrusive way, so we can rollback to our last state if something goes wrong.
With the announce of the 0.116 version we have seen that python 3.7 will be deprecated, probably in version 0.118. This procedure is for those that use Home Assistant core installed in a virtual environment and do not have python >= 3.8 provided by their distribution, Raspberry PI OS and Debian Buster included. Those distributions provide python 3.7.3 at a maxim. This procedure has been tested with both. You could upgrade to python 3.9 as well only changing the sources to download.
Python 3.8.6 install
The idea is install our new python version using altinstall, so we maintain our main python version to not broke any dependencies on our system. All commands are meant to be executed as a regular user except those executed with sudo.
First we will install some required packages. For Debian use this command:
apt-get install -y build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libsqlite3-dev libreadline-dev libffi-dev curl libbz2-dev libmysqlclient-dev libjpeg-dev libtiff5 libopenjp2-7
If you are on Raspberry PI OS, use this one:
apt-get install -y build-essential tk-dev libncurses5-dev libncursesw5-dev libreadline6-dev libdb5.3-dev libgdbm-dev libsqlite3-dev libssl-dev libbz2-dev libexpat1-dev liblzma-dev zlib1g-dev libffi-dev libjpeg-dev default-libmysqlclient-dev libtiff5 libopenjp2-7
Then, download, compile and install python 3.8.6 from source.
wget https://www.python.org/ftp/python/3.8.6/Python-3.8.6.tar.xz tar xf Python-3.8.6.tar.xz cd Python-3.8.6 ./configure --enable-optimizations # compile using number of cpus available, ex. 2 make -j 2 sudo make altinstall
New virtual environment
Next we will create a new Home Assistant virtual environment with the upgraded python 3.8 version we have just compiled from source. Then we will install home assistant and check its logs while it is booting, searching for errors. This assumes a standard HA installation with the virtual environment located in ‘/srv/homeassistant’ and configuration in the home of the homeassistant user, adapt this to your settings. Commented commands I used are below.
# stop home assistant and related services sudo service hass stop # cd into /srv directory cd /srv/ # move your old virtualenv folder and create an empty new one sudo mv homeassistant homeassistant.3.7 sudo mkdir homeassistant # assign necessary permissions sudo chown homeassistant:homeassistant homeassistant # become homeassistant user and cd into your new created directory sudo -u homeassistant -H -s cd /srv/homeassistant # create your new virtual environmet in the current folder # with the new python version you compiled before python3.8 -m venv . # activate the new viltual environment source bin/activate # install necesary python module and homeassistant python3 -m pip install wheel pip3 install homeassistant # exit the virtual environment and # start home assistant and check logs for errors deactivate sudo service hass start # check logs
These are some of the problems I faced while updating/reinstalling. This is an error collection I got from different HA installs, you can find some or hopefully none of them.
- Modules not installed: During Home Assistant boot, some errors about missing modules that were not installed as part of homeassistant. You should reinstall them from inside the new virtual environment. That should solve the problem.
- Pillow installation: During pillow module installation it complains about some necessary files not found. In mi case installing ‘zlib1g-dev’ and ‘libjpeg8-dev’ with apt solved the issue.
- Permission problems: Some ‘Permission denied’ errors arose during installation. The error lines were like this:
error: [Errno 13] Permission denied: '/usr/local/lib/python3.8/lib2to3/Grammar3.8.6.final.0.pickle' error: [Errno 13] Permission denied: '/usr/local/lib/python3.8/lib2to3/PatternGrammar3.8.6.final.0.pickle'
This was because those files were created with 640 permissions. Chmoding them to 644 was the fix.
- Package versions: some packages complain about not satisfied dependencies like
aiohttp 3.6.2 requires multidict<5.0,>=4.5, but you'll have multidict 5.0.0 which is incompatible.
installing the proper package within the venv fixed it, in this case
pip install --upgrade --force-reinstall multidict==4.7.6
- Other modules installation issues: Depending which modules you had previously installed, some of them throw errors, ej. mysqllclient, pymysql, broadlink, homeassistant-pyozw, … I installed some of them without issues from within the virtual environment. Others needed some dev libraries installed system-wide with apt, ej. libudev-dev, libopenzwave1.5-dev, libjpeg-dev, zlib1g-dev, libmysqlclient-dev, default-libmysqlclient-dev, … You should google it to find the required ones for your system.
In case things go very bad and your Home Assistant become unstable or broken, you should go back to your old virtual environment. Find the commands below to rollback to your old setup.
# stop home assistant and related services sudo service hass stop # cd into /srv directory cd /srv/ # move the failed venv and restore the old one sudo mv homeassistant homeassistant.bad sudo mv homeassistant.3.7 homeassistant # start home assistant sudo service hass start
We have seen how to upgrade our Home Assistant virtual environment to python 3.8.6. Even though this is not a very difficult procedure, depending on your system and your Home Assistant installation you can face from none to several problems. I encourage you to do it and tell me how it was. I hope this post has been useful to you.