Whats your favorite part of kedro 0.17.x?

I have just started really diving in and playing with the latest version of kedro (0.17.0) and have really been enjoying the composability of the different pieces. Today I was playing with the KedroSession.

I discovered that you can easily load up your project from any directory with a simple function. If you have data stored locally the path will still be relative to the project.

# Added to my __init__.py

from kedro.framework.session import KedroSession 
from kedro.framework.project import configure_project 


def get_session():
    cur_path = os.getcwd()
    os.chdir(Path(__file__).parents[2])
    configure_project(package_name)
    session = KedroSession.create(Path(__file__).resolve().parent.name)
    os.chdir(cur_path)

    return session

If I add this to my projects __init__.py file I can now access all of its context from anywhere.

import my_proj
session = my_proj.get_session()
session.run()
session.load_context().catalog
session.load_context().pipelines

edit added configure step ahead of create_session

2 Likes

Hey there,
this seems quite cool. I have a few questions, though.

  • Which __init__.py are you referring to? Is this the one inside ./src/<project_name> ?
  • Also, I get that the method get_session() goes inside the init file but the second code snippet, where is that supposed to go to?

I’m currently having a bit of an issue and thought this workaround could help.
Trying to update a project from 0.16.5 but the runner can’t find the function I’m using for one of the nodes. :frowning_face:

2 Likes

There’s also the get_current_session function, for when your session has already started! You can use that to get whatever session you are currently using :slight_smile:

1 Like

I’ve seen that, it looks fantastic for use in hooks.

I have been doing a dive into how pluggy works and various implementations of it. How all of these things hook together is starting to make a lot of sense.

@DataEngineerOne What’s your favorite 0.17.x feature?

Haha, obviously Session is the biggest and best thing to come out… still trying to harness its power, though, lol

I think 0.17.1 will be so much better. I’m genuinely excited. In a lot of ways, in my personal opinion, 0.17.0 is a necessary evil. The most exciting change we are working on is to get rid of the registration hooks and make Kedro library components configurable in settings.py, similar to Django’s settings.py. This will allows you to retrieve pipeline, catalog, etc. without hopefully worrying about things like context and session. It will be tricky to implement but should be a lot more intuitive for end users.

1 Like

I can say that 0.17.0 did not make much sense to me until I made my own (toy) framework that was hooks driven, a s studying how pytest works.

I cannot wait to see 0.17.1, I can see see 0.17.0 was a lot of setup for something great.