How to install project package to run on another computer?

In order to get the best help, it is suggested to answer the following questions:

What is the goal you are trying to achieve?
I want my pipeline to run on a remote desktop so that colleagues can run it by double-clicking a .bat file. I packaged the project, pip installed the wheel file (egg file throws an error) on the remote computer and added the necessary folders (config, logs, data). However, I dont get a Kedro project. All I see is that it installed the necessary packages. None of the codes or usual folders are there. Furthermore, when I run the project using python -m project.run, it throws me an error.

Is this an error or am I misunderstanding what packaging a project does? Should the codes and folders appear in the parent folder? Does anyone have an example wheel file or egg file to share that I could test to see what pip installing a Kedro project should look like?

Many thanks!

What version of Kedro are you using? (Use kedro -V)
0.16.6

What is the full stack trace of the error (if applicable)

When I install the egg file, I get:

ERROR: Could not find a version that satisfies the requirement project-0.1-py3.8.egg (from versions: none)
ERROR: No matching distribution found for project-0.1-py3.8.egg

When I run python -m project.run, it tells me that the .kedro.yml is missing:

kedro.framework.context.context.KedroContextError: Could not find any of configuration files '.kedro.yml, pyproject.toml
2 Likes

Interesting. This sounds like an issue with the original packaging. How are you doing the packaging of the project? What version are you using to package?

Im doing the packaging by using Kedro’s kedro package command:

Im using kedro 0.16.6, python 3.8 and pip 20.2.4.

Im not sure if I understood your question “How are you doing the packaging of the project?” but I didnt have the impression anything else was needed to package a project. Am I missing something?

1 Like

Very strange; I’m under the impression it should work as is. Can you replicate the issue using the example project? Is there something special you’ve added to the context of your current project?

@ljam Unfortunately at some point between 0.16.0 and 0.16.6 we broke the package deployment and it requires the .kedro.yml file alongside with your config. Fortunately we’re fixing this in 0.17 and it will become the recommended way to deploy Kedro projects. 0.17 is supposed to be released in two weeks time, so stay tuned!

1 Like

Hi @idanov! Many thanks for the info. Unfortunately adding just the .kedro.yml doesnt suffice in my case. I really need to add all files manually. So what I have do for now is just to clone my git project and run kedro install. Looking forwards to Kedro 0.17!

1 Like

@ljam Probably you’ve seen that already, but just to give you heads up that Kedro 0.17 is out now, so this should hopefully be resolved for you.

.kedro.yml is not a python file, so you need to do something to tell the package that this file should be included as well. I typically have the most success by adding it to the MANIFEST.in

image

Also is your .kedro.yml inside or your package our outside? Mine is within the projecect directory. To be a bit clearer, it sits in a directory with a __init__.py and one directory up is my setup.py


I feel your pain, its quite tricky to debug these packaging issues. The best thing I have found is to set yourself up a script to reinstall. I cannot remember everything that needs to happen, but using manifest may put files in places that do not uninstall well, especially if you are constantly changing it.

pip uninstall my_package
rm -rf dist/
python setup.py sdist bdist_wheel
pip install dist/my_package*.whl

After you think it works make a fresh virtual environment far away from your project, and pip install and try it there to make sure that there was nothing from being in the project directory, or left over in your venv that was causing a false success. I’ve thought I fixed packaging before only to find that I didn’t, its tricky.

Hi,

Yes I did and now packaging in kedro >= 0.17.0 works without a glitch. Makes packaging very straight forward. I had a second look at my project that was in 0.16.6 and I managed to get it working this time (also by adding the .kedro.yml). Dont know what I did wrong last time but I am happy to see that it works now.