Skip to content

Must be in a specific directory to use modified module in a Python environment; attempts to install using pip complains about missing setuptools.

Summary

Must be in a specific directory to use modified module in a Python environment; attempts to install using pip complains about missing setuptools.

Steps to reproduce

  1. Clone https://git.ecdf.ed.ac.uk/swain-lab/aliby/aliby (676953be4d789ab7ab1d885301e12741e539c99f current of this writing) into $(my-path)/aliby and https://git.ecdf.ed.ac.uk/swain-lab/aliby/postprocessor (498cc952 on master branch and c49df131 on autoreg branch current of this writing) into $(my-path)/postprocessor

  2. Add/modify a module in a local installation of this repository

  • In this example, add an autoreg.py to $(my-path)/postprocessor/postprocessor/core/processes/ in the autoreg branch (c49df131). Or checkout to this branch -- it achieves the same thing.

  1. Use pyenv to create a new virtual environment based on Python 3.7.9

  2. Install aliby in this virtual environment

I attempted:

  • $ pip install aliby : takes a long time because of pip's dependency resolver, aborted
  • $ cd $(my-path)/aliby && pip install -e . : also takes a long time because of pip's dependency resolver, aborted
  • $ cd $(my-path)/aliby poetry install : installs based on existing pyproject.toml and it takes less time (about 10 minutes, zeroc-ice being the usual rate-limiting step). FYI I was using poetry version 1.1.12 (https://python-poetry.org/docs/#installation)

Ran python from shell and tested installation by importing (i.e import aliby) -- module was imported without errors.

  1. Install postprocessor in this virtual environment
  • $ cd $(my-path)/postprocessor && poetry install installs quickly without issues. I was then able to run python from shell and import postprocessor and postprocessor.core.processes.autoreg or any other process without issue.
  • However, $ pip install -e . returns a ModuleNotFoundError complaining that setuptools is not installed.
  1. Attempt to import new module
  • Run python from shell then import postprocessor.core.processes.autoreg: import fails when in $(my-path) but succeeds when in $(my-path)/postprocessor

What is the current bug behavior?

  • I have to be in the repository directory to access the new module. This might affect testing.
  • pip install complains about setuptools despite it already being installed.

What is the expected correct behavior?

Installing a module modified locally should be directory-independent.

Logs/Traceback

Obtaining file:///home/arin/phd/aliby-all/postprocessor
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: scipy>=1.4.1 in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from aliby-post==0.1.1) (1.7.3)
Requirement already satisfied: catch22<0.3.0,>=0.2.0 in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from aliby-post==0.1.1) (0.2.0)
Requirement already satisfied: pandas<2.0.0,>=1.3.5 in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from aliby-post==0.1.1) (1.3.5)
Requirement already satisfied: PyYAML<7.0,>=6.0 in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from aliby-post==0.1.1) (6.0)
Requirement already satisfied: more-itertools<9.0.0,>=8.12.0 in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from aliby-post==0.1.1) (8.12.0)
Requirement already satisfied: leidenalg<0.9.0,>=0.8.8 in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from aliby-post==0.1.1) (0.8.8)
Requirement already satisfied: numpy>=1.17.3 in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from aliby-post==0.1.1) (1.21.5)
Requirement already satisfied: aliby-agora>=0.2.0 in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from aliby-post==0.1.1) (0.2.6)
Requirement already satisfied: omni-gaussian in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from aliby-post==0.1.1) (0.1.1)
Requirement already satisfied: python-igraph>=0.9.0 in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from leidenalg<0.9.0,>=0.8.8->aliby-post==0.1.1) (0.9.9)
Requirement already satisfied: python-dateutil>=2.7.3 in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from pandas<2.0.0,>=1.3.5->aliby-post==0.1.1) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from pandas<2.0.0,>=1.3.5->aliby-post==0.1.1) (2021.3)
Requirement already satisfied: matplotlib<4.0.0,>=3.5.1 in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from omni-gaussian->aliby-post==0.1.1) (3.5.1)
Requirement already satisfied: pyparsing>=2.2.1 in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from matplotlib<4.0.0,>=3.5.1->omni-gaussian->aliby-post==0.1.1) (3.0.6)
Requirement already satisfied: cycler>=0.10 in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from matplotlib<4.0.0,>=3.5.1->omni-gaussian->aliby-post==0.1.1) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from matplotlib<4.0.0,>=3.5.1->omni-gaussian->aliby-post==0.1.1) (4.28.5)
Requirement already satisfied: kiwisolver>=1.0.1 in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from matplotlib<4.0.0,>=3.5.1->omni-gaussian->aliby-post==0.1.1) (1.3.2)
Requirement already satisfied: pillow>=6.2.0 in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from matplotlib<4.0.0,>=3.5.1->omni-gaussian->aliby-post==0.1.1) (9.0.0)
Requirement already satisfied: packaging>=20.0 in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from matplotlib<4.0.0,>=3.5.1->omni-gaussian->aliby-post==0.1.1) (21.3)
Requirement already satisfied: six>=1.5 in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from python-dateutil>=2.7.3->pandas<2.0.0,>=1.3.5->aliby-post==0.1.1) (1.16.0)
Requirement already satisfied: igraph==0.9.9 in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from python-igraph>=0.9.0->leidenalg<0.9.0,>=0.8.8->aliby-post==0.1.1) (0.9.9)
Requirement already satisfied: texttable>=1.6.2 in /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages (from igraph==0.9.9->python-igraph>=0.9.0->leidenalg<0.9.0,>=0.8.8->aliby-post==0.1.1) (1.6.4)
Installing collected packages: aliby-post
  Attempting uninstall: aliby-post
    Found existing installation: aliby-post 0.1.1
    Uninstalling aliby-post-0.1.1:
      Successfully uninstalled aliby-post-0.1.1
  Running setup.py develop for aliby-post
    ERROR: Command errored out with exit status 1:
     command: /home/arin/.pyenv/versions/3.7.9/envs/aliby/bin/python3.7 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/arin/phd/aliby-all/postprocessor/setup.py'"'"'; __file__='"'"'/home/arin/phd/aliby-all/postprocessor/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: /home/arin/phd/aliby-all/postprocessor/
    Complete output (3 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    ModuleNotFoundError: No module named 'setuptools'
    ----------------------------------------
  Rolling back uninstall of aliby-post
  Moving to /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages/aliby_post-0.1.1.dist-info/
   from /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages/~liby_post-0.1.1.dist-info
  Moving to /home/arin/.pyenv/versions/3.7.9/envs/aliby/lib/python3.7/site-packages/aliby_post.pth
   from /tmp/pip-uninstall-ntv4sqsp/aliby_post.pth
ERROR: Command errored out with exit status 1: /home/arin/.pyenv/versions/3.7.9/envs/aliby/bin/python3.7 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/arin/phd/aliby-all/postprocessor/setup.py'"'"'; __file__='"'"'/home/arin/phd/aliby-all/postprocessor/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

Possible fixes

Regarding setuptools:

  • pip freeze suggests that it is already installed:
setuptools-scm @ file:///home/arin/.cache/pypoetry/artifacts/47/37/e2/b0359a9f019df81ea7317e945b8c21e71e1b59721d7118cb5c10399514/setuptools_scm-6.3.2-py3-none-any.whl
  • pip install setuptools does not solve the problem.
  • Adding setuptools to $(my-path)/postprocessor/setup.py does not solve the problem.

Alternatively, using a newer version of poetry (e.g. 1.2, https://python-poetry.org/docs/master/) could fix the issue...

Edited by Arin Wongprommoon