Supplements to kedro.extras for Computer Vision & GPU benchmarking

Supplements to kedro.extras

pipelinex.extras provides features not available in kedro.extras. Contributors who are willing to help preparing the test code and send pull request to Kedro following Kedro’s CONTRIBUTING.md are welcomed.

Additional Kedro datasets (data interface sets)

pipelinex.extras.datasets provides the following Kedro Datasets (data interface sets) mainly for Computer Vision applications using PyTorch/torchvision, OpenCV, and Scikit-image.

Additional function decorators for benchmarking

Open In Colab

pipelinex.extras.decorators provides Python decorators for benchmarking.

  • log_time
    • logs the duration time of a function (difference of timestamp before and after running the function).
    • Slightly modified version of Kedro’s log_time
  • mem_profile
    • logs the peak memory usage during running the function.
    • memory_profiler needs to be installed.
    • Slightly modified version of Kedro’s mem_profile
  • nvml_profile
    • logs the difference of NVIDIA GPU usage before and after running the function.
    • pynvml or py3nvml needs to be installed.
1 Like

Such a full-featured release set!

1 Like