I’ve experimented with Kedro for a week now, and absolutely love it. I have also been learning more about hooks (watched DE1’s video of it, and read docs) and find that to be very interesting to my needs.
Simplified, I have the following situation and problem:
You can asume that I have the latest version of Kedro. I have also a succesful implementation of the Spaceflights-tutorial. I can run the pipeline from terminal. Easy peasy.
However, what I now would like is the ability to supply additional input when running the pipeline from terminal. That input should then, before the pipeline is executed, alter the filepaths of my registered datasets in config/base/catalog.
More specifically, I would like to be able to change the filepaths below for every run.
It would be fantastic if there was a way to make this work something like this:
kedro run companies_filepath==‘xxx’ reviews_filepath==‘yyy’
I suspect that there actually is a fairly smart, perhaps even straightforward, way to implement this, but I just havn’t been able to solve it yet…