Hey! Introduce Yourself!

#1

Introduce Yourself

There’s data pipeliners all over the globe, walking all sorts of walks, beating all sorts of drums, and pipelining all sorts of pipelines, and we all are coming from different backgrounds, different experiences, and different places. Let’s celebrate those unique qualities! Tell us about yourself.

Tell Us:

What is your experience with data pipeline? Ex. Newbie, Journeyman, Practitioner, Grizzled Master

What was the last data pipeline that you built?

How did you get to know about kedro?

What was the most thrilling thing you’ve ever done?

How did you find us, here?

Where else can others connect with you online?

5 Likes
About the Português category
Speeding up pipeline processing with change detection
#2
#3

Hello! You can call me DE1.

What is your experience with data pipeling? Ex. Newbie, Journeyman, Practitioner, Grizzled Master

I’ve been doing data pipeline for the majority of my computer science career. While I wouldn’t necessarily say I’m a Grizzled Master, I’ve certainly seen (and perhaps written) my fair share of gnarly pipes.

What was the last data pipeline that you built?

The last one I built was an example pipeline, showcasing how information can get from several federated sources, into a single postgres database. Then, I connected that postgres database to hasura, and made a simple web app to demonstrate how graphql supports real-time updates (though kedro wasn’t running in real-time, the updates were happening in real time).

How did you get to know about kedro?

I was lucky enough to attend Kiyo’s small introductory talk on kedro, whilst he and I were both in Tokyo, last year. At that instant, I knew the technology was an incredible asset.

What was the most thrilling thing you’ve ever done?

It’s probably pretty basic, but I think the most thrilling thing I’ve ever done was drop out of a plane from 10k feet in the air. There’s probably plenty of people with a similar experience, but from my memory I don’t think I’ve ever encountered something more exhilarating than that!

How did you find us, here?

I helped found this website! It’s my sincere wish that its existence benefits everyone who comes across it.

5 Likes
#4

What is your experience with data pipeline? Ex. Newbie, Journeyman, Practitioner, Grizzled Master

I have been developing data pipeline for Machine Learning projects including a Kaggle competition to predict the results of American Football plays:

I also implemented features missing in Kedro including integrations with MLflow and released as PipelineX Python package:

What was the last data pipeline that you built?

A demo ML project using Titanic data based on Kedro, MLflow, and PipelineX:

How did you get to know about kedro?

Quantum Black Singapore team introduced Kedro at a meetup I participated in summer 2019.

What was the most thrilling thing you’ve ever done?

I programmed to play Mega Man (known as Rockman in Japan) using Ring Fit Adventure controller for exercise probably as the first of its kind:

How did you find us, here?

DE1 talked to me after we met in Zoom.

2 Likes
#5

LOL Holy moly, dude. That’s amazing!!!

If I ever get a Ring Fit Adventure controller, I totally want to try this out. I wonder if people would be interested in seeing that? haha

1 Like
#6

I uploaded a demo video to YouTube , but I look forward to DE1’s version :sunglasses:

2 Likes
#7

Ahahaha. 最高！Did you get fit??

2 Likes
#8

Hello there!

What is your experience with data pipelining? Ex. Newbie, Journeyman, Practitioner, Grizzled Master

I am a complete newbie on data pipelines. I’m a bit of a newbie with Python, actually. You can expect to see many newbie questions from me around here.

What was the last data pipeline that you built?

The last data pipeline I put together was inspired by one of DEOne’s video tutorials on how to convert [Jupyter] Notebooks into Kedro projects. It was a basic data pipeline that used data from a Weather station in Singapore and plotted timeseries graphs using Pandas. Simple but cool to demo how Kedro can also be used for simpler projects and still be super useful.

How did you get to know about kedro?

I started working as QuantumBlack’s Developer Advocate and learned about Kedro on my job interview. It was an incredible experience to have the opportunity to contribute to such a powerful framework with such a wonderful story and team behind it.

What was the most thrilling thing you’ve ever done?

It might sound silly and mundane but I think the most thrilling thing I’ve ever done was to leave everything and go on a 6 months trip around South America by myself. The only rule I had was ‘no planes’ so all the hiking, hitchhiking, vans and buses on the way contributed to the ‘trilling’ part.

How did you find us, here?

I kinda helped to found this website! I mean, if ‘help’ means give unsolicited opinions and complain about unimportant visual details. DEOne is a very patient man!
Jokes apart, I just hope this platform brings together everyone out there looking for a bit of a community on the Kedroverse.

Glad you’re joining us :slight_smile:

2 Likes
#9

Oh this is rad!! It’s so easy to under appreciate transportation technology, which is so foundational to all societal infrastructure (just like how data pipelines are pivotal for a working data science project).

Was there any time you ever felt unsafe hitchhiking?? That stuff can be super scary, I feel! I’ve never done it, but I have friends who have gone on trips like that.

Ahahaha, oh stop. You’re an incredibly valuable member and trusted advisor. You are most definitely a founder, by all stretches of the imagination. Thank you for all your help, great ideas, and determined spirit! They have been indispensable, and will continue to be.

1 Like
#10

Not sure about my weight as I don’t have a weight scale, but at least I can sleep way better after I started RingFit MegaMan after dinner. :sleeping:

3 Likes
#11

Holy moly that is quite cool indeed! I wasn’t even aware of the existence of RingFit!

1 Like
#13

I was gonna ask the same thing! :stuck_out_tongue:

@Minyus I can see you found a cool way to exercise through the lockdown. Awesome!

2 Likes
#14

You are too kind, my friend. :star2:

Now, let’s build this community up! :smiley:

2 Likes
#15

What is your experience with data pipeline? Ex. Newbie, Journeyman, Practitioner, Grizzled Master
I’ve actually got introduced to data pipelines when browsing the awesome production machine learning repo on GitHub. But my first tool for creating data pipelines was Kedro.

I can consider myself as a person trying to learn from other people in order to become a better software engineer.

What was the last data pipeline that you built?

Just today actually. I was using Kedro to create a data science pipeline to train a network on medical images.

How did you get to know about kedro?

I came across it on the awesome production machine learning github repo .

What was the most thrilling thing you’ve ever done?
Haha. Don’t know. Maybe refactoring code to the newest versions of the libraries I’m using?

How did you find us, here?

DE1’s Tweet !

2 Likes
#16

What is your experience with data pipeline? Ex. Newbie, Journeyman, Practitioner, Grizzled Master

I am quite a newbie. I am studying AI but my day-to-day work is more about building data pipelines.

What was the last data pipeline that you built?

I am building a data pipeline for a sales forecasting project.

How did you get to know about kedro?

I was looking into Prefect and Dagster. lol.

What was the most thrilling thing you’ve ever done?

Nothing worth mentioning. :stuck_out_tongue:

How did you find us, here?

Recommendation from the one and only DE1.

3 Likes
#17

I love this! Truly, the power of sotware engineering comes from sharing and learning from each other. There’s so much that I’ve been taught, by colleagues and anonymous people on stackoverflow; there’s no way I can repay them fr their kind efforts, so instead I pay it forward.

Awesome! You should chat with @Minyus, he also does image training pipelines. What are you trying to train your models to do?

Wow, you really were browsing, haha. You got lucky to find kedro this early, cause we’re about to blow this up! :grin:

LOL ok, that is somewhat thrilling. “Will it work? Dunno!” :slight_smile:

1 Like
#18

Nice! When you pipeline daily, that’s when skills improve quickest. Are you using kedro for it?

Oooo, what’s some of input data you’re using to train the forecast models?

You are probably the foremost expert of these two technologies, on this board, haha. Would love to get your thoughts on them :slight_smile:

Oh stop, surely you’ve got something interesting :smiley:

:sunglasses: glad to have you, sir. :grin:

2 Likes
#19

@facepalm, I’m calling you out! Give us an into, haha

2 Likes
#20

Hello! Let me humbly introduce myself :wink: . My name is Max and I’m currently a CS student from Russia, so to say…
What is your experience with data pipeline?
I used to work as a fullstack developer for a while, when I realized, that data aggregation and new information extraction are the most enjoying parts of my job, so I “rolled” into Data Science field. Now I do some kind of research & development for myself as long as studying various new stuff.
What was the last data pipeline that you built?
I’m currently working on my first Kedro pipeline, and I’m far from having it done… Just exploring, what I can do. Like a child playing lego, but instead of getting houses and cars, I get surprises, errors, meaning of life, dark secrets, some catnip or rainy seasons. I really have to say it: Data Science just doesn’t stop to amaze me - so many opportunities, so many unexpected applications, connections, approaches! SO MUCH POWER!!! devilish laughter
How did you get to know about kedro?
Actually it was a lucky accident. I was so upset with my spaghetti code in my notebook, so I really was going to create some library to at least manage my data catalogue and keep track of my experiments. And suddenly I found some tiny channel on YouTube about Graph ANN, which also had a mention of kedro. And then shallow searching of YouTube brought me to DE1 channel, so I am here !
What was the most thrilling thing you’ve ever done?
I’m still actively learning and it really takes me. I adore the moments when I’m just feeling curious about some topic and it suddenly have some fine application to some unexpected field, like ,my recent finding: Topology -> Time Series anomaly detection. So many things awaiting to be learned of and absorbed by eternal flow of ideas.
How did you find us, here?
We all know how :wink:

3 Likes
#21

What is your experience with data pipeline? Ex. Newbie, Journeyman, Practitioner, Grizzled Master

Practitioner. I started writing and deploying pipelines with kedro starting March 2019

How did you get to know about kedro?

Through an engagement with McKinsey.

What was the most thrilling thing you’ve ever done?

I raced motocross when I was in high school. The most exciting race I lined up in was on the famous Red Bud National in Michigan.

How did you find us, here?

@DataEngineerOne of course!

Where else can others connect with you online?




https://twitter.com/_waylonwalker
https://www.linkedin.com/in/waylonwalker/
2 Likes