Install python3 and pandas on raspberry pi 3

Hi,

I am having troubles installing python3 and pandas on my raspberry 3 with the latest Kodi, as I want to do some data crawling with the most of the time unused MediaCenter setup. I am a noob concerning linux, so please be patient with me :slight_smile: I already googled a lot, but nothing worked so far. Here’s what i tried:

1)Installing pandas via pip (didnt work, some missing dependency)
2) Using these commands Scientific Python for Raspberry Pi – Geoff Boeing (didnt work, unmet dependcy, see snipped below)
3) Then I read here in this forum, that apt-get upgrade is bad because it messes up the system, so I formatted my SD card again and tried just plain sudo apt-get update, sudo apt-get install python3 (worked) and then sudo apt-get install python-pandas (same error as above for step 2)

Can someone give me hint :slight_smile:
I am pretty lost

Best regards,
Stephan

osmc@osmc:~$ sudo apt-get install python-pandas
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 python-pandas : Depends: python-pandas-lib (>= 0.19.2-5.1) but it is not installable
E: Unable to correct problems, you have held broken packages.

I played with this a little, and without to much digging it looks like the pandas package is badly broken. I’d suggest you contact the pandas developers for help on this.

I did try this on Linux Mint, and

sudo apt install python3-pandas

does work. It looks like the problem is that python3-pandas-lib is missing in the debian repositories for ARM.

found this debian package: python3-pandas-lib

python3-pandas-lib:arm64 (0.19.2-5.1 Debian:9.9/stable [arm64])

That’s the ARM64 version. From a Pi2:

osmc@dalek:~$ sudo apt install python3-pandas-lib
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Package python3-pandas-lib is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'python3-pandas-lib' has no installation candidate

The available packages are listed here: Debian -- Details of package python-pandas-lib in stretch – and no armhf.

Right now, I’m building it using the command pip3 install pandas. It’s taking a surprisingly long time to build – and hasn’t yet failed.

Update. Seems to have worked, but took ages to build.

osmc@osmc:~$ pip3 install pandas
Collecting pandas
  Using cached https://files.pythonhosted.org/packages/b2/4c/b6f966ac91c5670ba4ef0b0b5613b5379e3c7abdfad4e7b89a87d73bae13/pandas-0.24.2.tar.gz
Collecting python-dateutil>=2.5.0 (from pandas)
  Using cached https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl
Collecting pytz>=2011k (from pandas)
  Using cached https://files.pythonhosted.org/packages/3d/73/fe30c2daaaa0713420d0382b16fbb761409f532c56bdcc514bf7b6262bb6/pytz-2019.1-py2.py3-none-any.whl
Collecting numpy>=1.12.0 (from pandas)
  Using cached https://files.pythonhosted.org/packages/93/48/956b9dcdddfcedb1705839280e02cbfeb2861ed5d7f59241210530867d5b/numpy-1.16.3.zip
Collecting six>=1.5 (from python-dateutil>=2.5.0->pandas)
  Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Building wheels for collected packages: pandas, numpy
  Running setup.py bdist_wheel for pandas ... done
  Stored in directory: /home/osmc/.cache/pip/wheels/47/0a/bf/fd3647f5bc74698ed95dbdbb3d8d120cfe0280969c212cdb6c
  Running setup.py bdist_wheel for numpy ... done
  Stored in directory: /home/osmc/.cache/pip/wheels/8c/49/27/48d3f185a57ffcaabdb366bd1d05e86d587222a85de3b48ec2
Successfully built pandas numpy
Installing collected packages: six, python-dateutil, pytz, numpy, pandas
Successfully installed numpy-1.16.3 pandas-0.24.2 python-dateutil-2.8.0 pytz-2019.1 six-1.12.0
osmc@osmc:~$ pip3 show pandas
Name: pandas
Version: 0.24.2
Summary: Powerful data structures for data analysis, time series, and statistics
Home-page: http://pandas.pydata.org
Author: None
Author-email: None
License: BSD
Location: /home/osmc/.local/lib/python3.5/site-packages
Requires: numpy, pytz, python-dateutil

Needless to say, I haven’t tested it.

You’ll need to have installed (at least) the Debian packages build-essential python3-pip python3-dev and have run pip3 install setuptool

It failed for me building with pip3. It will be interesting to see if it works for you.

Hey together,

thanks for your fast response and sorry for getting back to you so late, I was busy working… :frowning:
I tried your hints with the following results:

$ sudo apt-get install build-essential python3-pip python3-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
build-essential is already the newest version (12.3).
python3-pip is already the newest version (9.0.1-2+deb9u1).
python3-dev is already the newest version (3.5.3-1).
0 upgraded, 0 newly installed, 0 to remove and 19 not upgraded.

Fine so far :slight_smile: Then it stopped working:

sudo pip3 install setuptool
Collecting setuptool
Exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python3/dist-packages/pip/commands/install.py", line 353, in run
    wb.build(autobuilding=True)
  File "/usr/lib/python3/dist-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 554, in _prepare_file
    require_hashes
  File "/usr/lib/python3/dist-packages/pip/req/req_install.py", line 278, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 465, in find_requirement
    all_candidates = self.find_all_candidates(req.name)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 423, in find_all_candidates
    for page in self._get_pages(url_locations, project_name):
  File "/usr/lib/python3/dist-packages/pip/index.py", line 568, in _get_pages
    page = self._get_page(location)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 683, in _get_page
    return HTMLPage.get_page(link, session=self.session)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 795, in get_page
    resp.raise_for_status()
  File "/usr/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/models.py", line 893, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://pypi.org/simple/setuptool/
osmc@osmc:~$

But this worked again :slight_smile:

 sudo apt-get install python3-setuptools
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  python3-pkg-resources
Suggested packages:
  python-setuptools-doc
The following NEW packages will be installed:
  python3-pkg-resources python3-setuptools
0 upgraded, 2 newly installed, 0 to remove and 19 not upgraded.
Need to get 352 kB of archives.
After this operation, 1467 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://ftp.debian.org/debian stretch/main armhf python3-pkg-resources all 33.1.1-1 [137 kB]
Get:2 http://ftp.debian.org/debian stretch/main armhf python3-setuptools all 33.1.1-1 [215 kB]
Fetched 352 kB in 0s (751 kB/s)
Selecting previously unselected package python3-pkg-resources.
(Reading database ... 47955 files and directories currently installed.)
Preparing to unpack .../python3-pkg-resources_33.1.1-1_all.deb ...
Unpacking python3-pkg-resources (33.1.1-1) ...
Selecting previously unselected package python3-setuptools.
Preparing to unpack .../python3-setuptools_33.1.1-1_all.deb ...
Unpacking python3-setuptools (33.1.1-1) ...
Setting up python3-pkg-resources (33.1.1-1) ...
Setting up python3-setuptools (33.1.1-1) ...
osmc@osmc:~$

Finally the last command is still running, it failed yesterday but back then setuptools didnt work yet… I’ll report back as soon as the pip3 install pandas is finished…

No luck…

osmc@osmc:~$ pip3 install pandas
Collecting pandas
  Using cached https://files.pythonhosted.org/packages/b2/4c/b6f966ac91c5670ba4ef0b0b5613b5379e3c7abdfad4e7b89a87d73bae13/pandas-0.24.2.tar.gz
Collecting numpy>=1.12.0 (from pandas)
Exception:
Traceback (most recent call last):
  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 594, in urlopen
    chunked=chunked)
  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 391, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 387, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.5/http/client.py", line 1198, in getresponse
    response.begin()
  File "/usr/lib/python3.5/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.5/http/client.py", line 258, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.5/socket.py", line 576, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.5/ssl.py", line 937, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.5/ssl.py", line 799, in read
    return self._sslobj.read(len, buffer)
  File "/usr/lib/python3.5/ssl.py", line 583, in read
    v = self._sslobj.read(len, buffer)
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python3/dist-packages/pip/commands/install.py", line 353, in run
    wb.build(autobuilding=True)
  File "/usr/lib/python3/dist-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 554, in _prepare_file
    require_hashes
  File "/usr/lib/python3/dist-packages/pip/req/req_install.py", line 278, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 465, in find_requirement
    all_candidates = self.find_all_candidates(req.name)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 423, in find_all_candidates
    for page in self._get_pages(url_locations, project_name):
  File "/usr/lib/python3/dist-packages/pip/index.py", line 568, in _get_pages
    page = self._get_page(location)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 683, in _get_page
    return HTMLPage.get_page(link, session=self.session)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 792, in get_page
    "Cache-Control": "max-age=600",
  File "/usr/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 501, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python3/dist-packages/pip/download.py", line 386, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/usr/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 630, in send
    history = [resp for resp in gen] if allow_redirects else []
  File "/usr/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 630, in <listcomp>
    history = [resp for resp in gen] if allow_redirects else []
  File "/usr/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 190, in resolve_redirects
    **adapter_kwargs
  File "/usr/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/usr/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/adapter.py", line 47, in send
    resp = super(CacheControlAdapter, self).send(request, **kw)
  File "/usr/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/adapters.py", line 423, in send
    timeout=timeout
  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 643, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/util/retry.py", line 315, in increment
    total -= 1
TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'
osmc@osmc:~$

Sorry, it should have been setuptools, not setuptool.

Please provide the output from running the following commands:

dpkg -l "python3*" | grep ^ii
pip3 list
osmc@osmc:~$ dpkg -l "python3*" | grep ^ii
ii  python3               3.5.3-1        armhf        interactive high-level object-oriented language (default python3 version)
ii  python3-dev           3.5.3-1        armhf        header files and a static library for Python (default)
ii  python3-minimal       3.5.3-1        armhf        minimal subset of the Python language (default python3 version)
ii  python3-numpy         1:1.12.1-3     armhf        Fast array facility to the Python 3 language
ii  python3-pip           9.0.1-2+deb9u1 all          Python package installer
ii  python3-pkg-resources 33.1.1-1       all          Package Discovery and Resource Access using pkg_resources
ii  python3-setuptools    33.1.1-1       all          Python3 Distutils Enhancements
ii  python3.5             3.5.3-1+deb9u1 armhf        Interactive high-level object-oriented language (version 3.5)
ii  python3.5-dev         3.5.3-1+deb9u1 armhf        Header files and a static library for Python (v3.5)
ii  python3.5-minimal     3.5.3-1+deb9u1 armhf        Minimal subset of the Python language (version 3.5)

And here for pip

    osmc@osmc:~$ pip3 list
    DEPRECATION: The default format will switch to columns in the future [... blablabla]
    numpy (1.16.4)
    pip (9.0.1)
    python-dateutil (2.8.0)
    pytz (2019.1)
    setuptools (33.1.1)
    six (1.12.0)

Your build seems to fail at “numpy” for some reason. You have a Debian numpy package,
python3-numpy 1:1.12.1-3, which I assume is version 1.12.1-3, and a version installed via pip3 that is version 1.16.4. I would hope it uses the later version but it might be better to remove the Debian package (which I don’t have installed, BTW):

sudo apt-get purge python3-numpy

Second, as you can see from my own experience above, building pandas also builds numpy, python-dateutil, pytz and six, which have (already) successfully been installed. Do you know how they got there?

Finally, you probably need to install wheel (sudo apt-get install python3-wheel) for the job to complete.

Yeeeah, dillthedog thank you so much! It works now!! The numpy and all the other packages were not the problem, installing python3-wheel did the trick. Perfect!

One further question: To run a python script periodically (scheduled), I should use crontab right?

1 Like

Cron is the easiest way to schedule things.