pythonpackage.yml workflow doesn't install my package before running its tests?

So I tried Github Actions for the first time, with the Python packageworkflow option. The tests fail because it can’t import my package. 

___________________ERROR collecting tests/test_something.py____________________

ImportError while importing test module '/home/runner/work/my_package/my_package/tests/test_something.py'.

Hint: make sure your test modules/packages have valid Python names.

Traceback:

tests/test_something.py:7: in <module>

from my_package import something

E ModuleNotFoundError: No module named 'my_package'

Either I have to

  1. Tell the pythonpackage.yml to install my package before running the tests?  pip install . or something?  But if this is correct, why isn’t it already there?

  2. Change my package so it doesn’t need to be installed in order for tests to run?  Am I not following best practices?

1 Like

When you want to use Python in the workflow runs on GitHub-hosted runner, you should use the setup-python action to set up a Python environment at first. And before building and testing your code, you may also need to install some dependencies from the PyPI package registry by using the pip install.
More details about using Python with GitHub Actions, you can reference here: https://help.github.com/en/actions/automating-your-workflow-with-github-actions/using-python-with-github-actions#starting-with-the-python-workflow-template

If you still have questions for this ticket, can you share more details of your workflow definition and the completed debug logs of the workflow run?

1 Like

I’m just using the default pythonpackage.yml.  I think it’s ultimately from https://github.com/actions/starter-workflows/blob/master/ci/python-package.yml ?

Yes, it installs python and dependencies from requirements.txt correctly, but doesn’t install the actual package itself.  I’m not sure if it’s supposed to, or if my tests should run without installing the package.

In this article they run tests that just import from a module that is in the same folder, which wouldn’t require installing first: https://realpython.com/python-continuous-integration/

Is that what it expects we’re doing?

In scipy the tests actually import scipy, though, which would require it to be installed:

https://github.com/scipy/scipy/blob/master/scipy/signal/tests/test_dltisys.py

Hi @endolith 

Do you have an example run/yaml file or repo where you’re runing into this issue? I can quickly take a look and try to figure out why your package isn’t being installed. The error message you have might be somehow related:

make sure your test modules/packages have valid Python names

The basic pythonpackage.yml sample doesn’t take into account any extra packages that your project or repository requires. It’s meant to serve as a template/example to get you up and running. I would make sure that any extra package or dependency that you need gets installed either through pip or by adding it to requirements.txt

There are a lot of ways to setup tests with Python, but generally what I’ve seen is you first install all depedencies which include any packages, and then you run tests. If a specific package isn’t installed and a test requires that package, then the test will fail.

Ok I made it public: https://github.com/endolith/elsim

https://github.com/endolith/elsim/blob/master/.github/workflows/pythonpackage.yml

There’s no line in the default workflow that installs the package itself, which makes me wonder if I’m doing the tests wrong.

https://github.com/endolith/elsim/runs/390504822

This one had python setup.py install and then changed it to pip install .:

https://github.com/havakv/pycox/commit/9a8fd8e9f93ee73f514515c899b32f721798a722#diff-fd3c00174dcaf7b8a36765f1a3de6791

It has a similar tests folder structure and imports as mine.

This one added pip install -e .:

https://github.com/hofstee/shale/commit/7a7f4f245d020522c5c3892511768d222f77e303#diff-fd3c00174dcaf7b8a36765f1a3de6791

This one has pip install -e .[test] and similar folders and imports:

https://github.com/Zsailer/nbclassic/blob/master/.github/workflows/pythonpackage.yml

This one doesn’t install the package either, I’m not sure how it is passing:

https://github.com/mattblaha/readability-selenium/blob/master/.github/workflows/pythonpackage.yml

unittest works differently from pytest?

Ok I got it to pass by adding that:

https://github.com/endolith/elsim/commit/6646ccb25c019e5accfd0429951a81a2f9356b18

It still has problems, though.  numba doesn’t work in the Github Actions environment even though it works fine locally:

/opt/hostedtoolcache/Python/3.7.6/x64/lib/python3.7/site-packages/llvmlite/ir/instructions.py:84: in __init__
    raise TypeError(msg)
E TypeError: Failed in nopython mode pipeline (step: nopython mode backend)
E Type of #4 arg mismatch: i1 != i32

Possibly related to https://github.com/numba/numba/issues/5035

It also doesn’t understand tests_require?

Run pip install -e .[test]

Obtaining file:///home/runner/work/elsim/elsim

WARNING: elsim 0.1.0 does not provide the extra 'test'

Also WTF does this forum not use Markdown like every other place on Github?

1 Like

Glad you got it mostly working :grinning:

Seems to me like you’re mostly having configuration issues. Python can have a lot of annoying little configuration issues that can be hard/difficult to find.

I would suggest trying to add an empty __init__.py file in your tests/ directory, some information about what this file is: [https://pythontips.com/2013/07/28/what-is-__init__-py/](https://pythontips.com/2013/07/28/what-is- init -py/)

https://stackoverflow.com/questions/448271/what-is-init-py-for

Not including this has the potential for certain things not to be found and installed. If you have this file, you may not need to do pip install -e :crossed_fingers:

I’m honestly not all that familiar with unittest (there are ALOT of different testing frameworks/tools for Python) but I’m sure it has some differences when compared to pytest.

Regarding markdown not being used here, I’ll try to pass on the feedback to the appropriate team! Different teams work on different products and github.community is separate from the base github.com

1 Like

I got this to work styled after the example you posted for readability-selenium.
You can see what I did at https://github.com/swajime/ansi
I think the key that you were missing was that init.py has content in their code.
Following that example, I did not need to add pip install commands to the workflow.
Thanks for the question and the links. They helped me solve my issue.

Can you explain this in more detail?

Why? This does work for me, but pytest explicitly says not to do this…

avoid “init.py” files in your test directories.
This way your tests can run easily against an installed version
of mypkg, independently from the installed package if it contains
the tests or not.

https://docs.pytest.org/en/3.0.2/goodpractices.html

This file: https://github.com/mattblaha/readability-selenium/blob/eaac47e683e5b8ecb6818cca5c24ac6a9ce93811/readability/init.py

This file tells what is where. If it has the correct contents then you should not need to add code for installing the package to enable testing.

Contents: from .Reader import Reader

Well I think my project has the same thing: https://github.com/endolith/elsim/blob/95662e1b00914b822351f857b1a91d553d682b82/elsim/init.py

That doesn’t make the tests work without installing my package, though

So I think it is because you have from . import elections, methods instead of from .elections import … and from .methods import …

I do not have time right now to experiment to see exactly what you need but am fairly certain that this file is the culprit.

I don’t understand why that would make any difference if the package is not installed, though. I had to add this install line for it to work at all:

and I don’t understand why changing the imports within the package itself would make this unnecessary.