Compiling against Python with setup-python

My mostly C++ project uses GitHub Actions for CI and to build release binaries. We target Windows, Ubuntu, and MacOS.

We use CMake as a meta-build system and SWIG to wrap our C++ to generate a Python API.

We use setup-python to control the version of Python in each Actions instance. Unfortunately, it seems that setup-python is focused on the ‘running Python’ use case instead of the ‘building with Python as a library’ use case.

In CMake, we use FIND_PACKAGE( PythonLibs ) to search for Python. According to the CMake documentation, we should set PYTHON_LIBRARY and PYTHON_INCLUDE_DIR as needed to specify a specific Python to find instead of searching for the main system one.

What is the best way to set these variables in Actions using setup-python?

We found the undocumented pythonLocation, but that seems to be only part of the answer.

Does anyone have an example of an Actions/CMake/SWIG/Python/C++ system that they can point to?

Thanks in advance.

python -c "from distutils import sysconfig; print(sysconfig.get_python_inc())" from:

appears to get one of them.

Per:
https://cmake.org/cmake/help/latest/module/FindPythonLibs.html
The thing you’re using is deprecated, if you use the replacement:
https://cmake.org/cmake/help/latest/module/FindPython3.html#module:FindPython3

do you still hit the same problem?

(I don’t use any of these things, I’m just browsing)

The feature request for PYTHON_INCLUDE_DIR and PYTHON_LIBRARY variables would solve my problem.

Yes FindPythonLibs is deprecated, but there are a lot of machines out there with older CMake. The new FindPython3 was introduced in 3.12, which is a big step for a lot of projects.

I read the source of FindPython3 and it will not do any better. FindPython3 will search certain standard locations for Python – and will return with the first one it finds that satisfies certain criteria (you can require a certain version number for example). However, setup_python does not install Python into a standard location – so it is very unlikely that FindPython3 will ever find it.

Constraining FindPython3 with version number constraints is challenging when you’re trying to use a build matrix – and avoid finding the system-installed version of Python instead of the setup-python installed version. To me, explicit identification of the setup-python version is the way to go.

I found FindPythonLibsNew from pybind and it has allowed me to get past my problem. On Line 82, it looks for pythonLocation from GHA to serve as a hint when searching for Python. So, it is ‘aware’ of GHA’s behavior. It uses that information to find the python executable and hten it runs a command similar to the above suggestion (line 115-126).

IMHO, the best solution is still for setup_python to define a bunch as many useful variables as possible. You never know what someone might need.

I think it’s totally reasonable to request PYTHON_INCLUDE_DIR and PYTHON_LIBRARY be exported, or at least made available…

In principle, the way to request that would be via: