I have encountered a lot of situation like this. I may have pipelines that looks like this
a -> b -> c1 -> d a -> b -> c2 -> d a -> b -> c3 -> d a -> b -> c4 -> d a -> b -> c5 -> d a -> b -> c6 -> d
Most of the nodes are the same, only 1(or few) nodes are different. So, I do this instead
a -> b -> c(parameter) ->d where now before create_pipeline(), it takes a parameter to generate the node on the fly.
However, since pipeline is refactored a few times, from
hooks and now
pipeline_registry, is it still possible to pass parameters to
create_pipeline()? Or is there a better way to organize these pipelines?