In order to get the best help, it is suggested to answer the following questions:
What is the goal you are trying to achieve?
Hello!!! I am new to Kedro, but I have been hours reading today and now I am a believer!! I’m very excited for all the good things it has.
I am a very immature data scientist that has a decent knowledge of ML concepts and algorithms, some grasp of Python but nothing of MLOps and DevOps. I work for a big company that is starting its ML way, and for the moment everything we do is manual and poorly standarized.
So I will be very grateful to know some other tools that might be recommended, at least in your experience, in order to set up good MLOps practices. For example, frlm what I have read:
Prefect/Luigi/Airflow for orchestration. Any recomendation?
MLflow for experiment tracking and model registry. By the way, the plugin between kedro and MLFlow is recommended?
Feast for feature store
Great expectations for test on data products
Dask/PySpark for parallel executions. Any recomendation?
Do you think that this libraries can be enough (apart from pandas, sklearn and the other obvious choices…) for setting up the standards that we seek? Of course I didn’t mention Kedro, which will be like in the middle of everything, as the framework for our project and code.
I’m sorry if these are too many questions… But I have been reading 8-10 hours a day about MLOps in blogs and books, libraries comparisons and this is the first time that I see the light at the end of tje tunnel… Thanks in advance!!