The Rise of Data Modeling Data modeling has been one of the hot topics in Data LinkedIn. Hadoop put forward the schema-on-read strategy that leads to the disruption of data modeling techniques as we know until then. …

I’m thrilled to write about Schemata, a decentralized schema modeling framework for data contracts. Oh, wait, all the jargon, what is it? Let me take you all on the Schemata journey. You can find the source code and the documentation here. GitHub Repo: https://github.com/ananthdurai/schemata Why do we need Schemata? The TV Remote Under The Couch Problem: Let’s first admit it, The era…

The best conversation happens at home, especially when you have a different domain of expertise. Different perceptive and life experience always brings innovative thoughts and help to refine your ideas. I capture my lunch conversation with my wife about data contracts and hope it will be helpful for you. She…

Not too long ago, I worked for a famous messaging platform; You may be typing on it as you read this blog. I took a break from data engineering and focused on system monitoring and observability for some time. …

The article is an extraction of my thoughts on the recent data mesh discussions over Twitter. You can subscribe to my weekly data engineering newsletter to follow the latest updates on the data community. Data Engineering Weekly Data Engineering Weekly Newsletterwww.dataengineeringweekly.com Last week Data Twitter had some interesting discussions around Data Mesh. …

An Introduction To Omicron Paradigm Many enterprises are increasingly relying on vertical & horizontal SAAS applications to operate their business. …

The story is a cross-posting from the Data Engineering Weekly. Please subscribe to the Data Engineering Newsletter for the latest update. www.dataengineeringweekly.com Welcome to the 34th edition of the data engineering newsletter. This week’s release is a new set of articles that focus on Google’s massively parallel graph computation, Uber’s data journey, Hyperight’s is data mesh right for your organization, Lyft’s ML feature infrastructure, Flyte joins LF Data & AI, PayPal’s…

The story is a cross-posting from the Data Engineering Weekly. Please subscribe to the Data Engineering Newsletter for the latest update. www.dataengineeringweekly.com Welcome to the 33rd edition of the data engineering newsletter. This week’s release is a new set of articles that focus on Michael Stonebraker’s Top 10 Big Data Blunders, Stanford University’s AI index report 2021, Maxime’s The future of the Business Intelligence is open source, Mehdi’s data engineering skills…

The story is a cross-posting from the Data Engineering Weekly. Please subscribe to the Data Engineering Newsletter for the latest update. www.dataengineeringweekly.com Welcome to the 32nd edition of the data engineering newsletter. This week’s release is a new set of articles that focus on Picnic’s Data Vault modeling, Mihaileric’s why we need more data engineers, Microsoft’s onboarding data scientist checklist, Netflix’s data movement with Google Services, Redpoint Venture’s data feedback loop…