(SOLVED) Storing Catalogs inside project package breaks `kedro pipeline create`

TLDR, can I move my CONF_DIR into my project and still use the create functionality


Everything as a package

My workflow for any python development has been creating packages for most of my career. Whatever small thing I am doing is treated as a package that I can pip install -e . during development.

I setup my own kedro templates to behave this way, but with each major release it’s requiring more maintenance to keep up with and there are more features that I would rather work as intended.

In the past I moved my catalogs inside the package, added a glob to the MANIFEST.in and I am good, but some of the cli functionality is now broken.


What’s not working

:x: I am loosing out on things such as kedro pipeline create

:white_check_mark: kedro catalog list still works

:x: kedro catalog create does not work

:white_check_mark: kedro run still works


Spent some time in the source code.

After digging through the source code I quite a large refactor has been done between the 0.17.0 tag and the latest master branch.

@limdauto it appears that my settings are not being picked up by kedro pipeline create. Is this already fixed in 0.17.1?

my settings.py

image

It’s running settings.py, but pipeline create is not picking it up.

Even the “current” _get_project_settings function gets it right.

It just came to me to try from the bleeding edge version

pip install git+https://github.com/quantumblacklabs/kedro/

Move conf/ into the package, adjust CONF_ROOT in settings.py and we are off and running.

mv conf src/<proj_name>

Thanks everyone for being my rubber duck.