Are pipelines the correct analogy to use?
Reddit r/dataengineering6d4 min read
I was looking at posts in this subreddit the other day. There were quite a few talking about pipelines and all the things that go along with them (forking, merging and version control). This is typical source controls stuff that has been around since the 70s. The tools change but the concepts and model are the same. I wonder if we have been so concerned about the minutea that we quit looking at better options than pipelines. I have a couple of thoughts about better ways of doing data repositories/warehouses/ingestion beyond pipelines, three layers and fabrics. I'd rather see if anyone else out
