I have a use case that my pipeline did a few steps.
- Get data (For development use)
- Train a Model
- Save the output
Let say if I package up this pipeline and want to pass it to another team. I would like to pass in an actual python object to the pipeline instead of relying on catalog only (Since it would mean they must first write the data to some kind of file)
data = release_team_get_data() context.run(pipeline="my_publish_pipeline", input=data) # I want to pass in data that is already loaded in memory