With Pipeline registry in Kedro 0.18, what's the best way to handle dynamic pipeline or many pipelines that have small difference?

I have encountered a lot of situation like this. I may have pipelines that looks like this

a -> b -> c1 -> d
a -> b -> c2 -> d
a -> b -> c3 -> d
a -> b -> c4 -> d
a -> b -> c5 -> d
a -> b -> c6 -> d

Most of the nodes are the same, only 1(or few) nodes are different. So, I do this instead

a -> b  -> c(parameter) ->d    where now before create_pipeline(), it takes a parameter to generate the node on the fly.

However, since pipeline is refactored a few times, from KedroContexthooks and now pipeline_registry, is it still possible to pass parameters to create_pipeline()? Or is there a better way to organize these pipelines?

2 Likes