Kedro example: Deep Learning model using breast cancer dataset

Our super awesome contributor from PyCon India Sprints @NeurlAP just finished his deep learning kedro project using the Breast Cancer dataset and Tensorflow.

You can have a look at the GitHub repo here.

I would really love to know more details about the project, though. :slight_smile:

Here is the viz of the pipelines.

3 Likes

The viz itself tells everything about the project but lemme explain a little bit.
I’ve used wdbc dataset, in this dataset we’ve to predict whether a person is going to diagnose with Breast Cancer or Not based on certain Features. And the dataset is of CSV fromat.

  1. I’ve put my dataset(raw data) inside data/01 folder.
    Then I’ve to create two pipeline one is Data engineering pipeline and other one is Data Science.
  2. I’ve created two funtions, i.e creating_features ` what it does it select some of the features from the dataset and save it as a seperate CSV file(Intermediate data) and ``` creating_lables`` pop the labels from the dataset(Intermidiate data) and then save it to separate csv file.
  3. I’ve defined my parameters value in parameter’s .yml file. Later this parameter is going to be used in specifying train test split.
  4. Then with the help of scikit learn train test split I’ve splitted the data based on the ratio saved in parameter value And the split data function is returning np.array not the pandas dataframe
    And the Data engineering part is finished here.
  5. Now to inside the train function I’ve defined my Tensorflow model, compile it and then called model.fit(). ANd the train function is going to return Tf.keras.model trained model instance. And in catalog .yml file I’ve register it as TensorflowModelDataset type. So, that I can use this trained model for evaluation.
  6. Then I used the trainde model and then evaluate it on the test data.
    This was the summary of whole project.
2 Likes

Great work! Perhaps I’ll use this pipeline as an example pipeline to point to, when I give kedro trainings :slight_smile:

1 Like

Thanks, @DataEngineerOne, it’s not know well documented but I’m going to work with @laisbsc for this. (After my end semester xam).

2 Likes

Indeed. Thanks for going into more detail :slight_smile:

1 Like