Book Metadata and Cover Retrieval Using OCR and Google Books API
KDnuggets
NOVEMBER 17, 2021
With KNIME extracting critical pieces of information from images becomes as easy as ABC.
Metadata Management And Integration At LinkedIn With DataHub
Data Engineering Podcast
AUGUST 24, 2020
The key to those solutions is a robust and flexible metadata management system. LinkedIn has gone through several iterations on the most maintainable and scalable approach to metadata, leading them to their current work on DataHub. If you hand a book to a new data engineer, what wisdom would you add to it?
Metadata 
100
100 Join 37,000+
Insiders
Sign Up for our Newsletter
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Webinars
Agent Tooling: Connecting AI to Your Tools, Systems & Data
How to Modernize Manufacturing Without Losing Control
Mastering Apache Airflow® 3.0: What’s New (and What’s Next) for Data Orchestration
Trending Sources
Writing The Book That Offers A Single Reference For The Fundamentals Of Data Engineering
Data Engineering Podcast
JULY 24, 2022
Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Can you explain what possessed you to write such an ambitious book? What are your goals with this book?
Data Engineer 100
100 
Book Metadata and Cover Retrieval Using OCR and Google Books API
KDnuggets
NOVEMBER 17, 2021
With KNIME extracting critical pieces of information from images becomes as easy as ABC.
Metadata 83
83 
Data Engineering Weekly #198
Data Engineering Weekly
NOVEMBER 24, 2024
Canva writes about its custom solution using dbt and metadata capturing to attribute costs, monitor performance, and enable data-driven decision-making, significantly enhancing its Snowflake environment management. link] JBarti: Write Manageable Queries With The BigQuery Pipe Syntax Our quest to simplify SQL is always an adventure.
Metadata: What Is It and Why it Matters
Ascend.io
JULY 11, 2024
Metadata is the information that provides context and meaning to data, ensuring it’s easily discoverable, organized, and actionable. Imagine a library with millions of books but no catalog system to organize them. This is what managing data without metadata feels like. What is Metadata? Chaos, right?
Metadata 52
52 
The Data Discovery Team
Jesse Anderson
NOVEMBER 14, 2023
It’s a team that connects naturally into the constellation of the three data teams Operations team Data engineering team Data Science team as described in Jesse Anderson’s book Data Teams (2020) Before I explain what the data discovery team should do, it is necessary to add a bit of context on the concept of data discovery itself.
Metadata 147
147 
Build your data pipelines like the Toyota Way
François Nguyen
FEBRUARY 28, 2021
If there is one only book to read about lean manufacturing, this is the one. This is the kind of book you can read again and again and still learn something about your current context. It is also a book you can read whatever your industry, you will always find situations covered by this book.
Data Pipeline 130
130 
Combining The Simplicity Of Spreadsheets With The Power Of Modern Data Infrastructure At Canvas
Data Engineering Podcast
JUNE 19, 2022
Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold. Atlan is the metadata hub for your data ecosystem.
Metadata 130
130 
Bring Geospatial Analytics Across Disparate Datasets Into Your Toolkit With The Unfolded Platform
Data Engineering Podcast
JUNE 26, 2022
Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold. Atlan is the metadata hub for your data ecosystem.
Datasets 130
130 
Making The Total Cost Of Ownership For External Data Manageable With Crux
Data Engineering Podcast
JULY 17, 2022
Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold. Atlan is the metadata hub for your data ecosystem.
Data Management 130
130 
Building A Data Mesh Platform At PayPal
Data Engineering Podcast
FEBRUARY 26, 2023
TimeXtender Logo]([link] TimeXtender is a holistic, metadata-driven solution for data integration, optimized for agility. TimeXtender Logo]([link] TimeXtender is a holistic, metadata-driven solution for data integration, optimized for agility. Email hosts@dataengineeringpodcast.com ) with your story.
Building 147
147 
Toward a Data Mesh (part 2) : Architecture & Technologies
François Nguyen
MARCH 22, 2021
and he/she has different actions to execute (reading, calling a vision API, transform, create metadata, store them, etc…). It is a huge shift in skills needed (will talk about that in part 3) but this is the only way to fully “accelerate” ( like the title of this book where devops is a key part).
Technology 130
130 
Data Engineering Weekly #213
Data Engineering Weekly
MARCH 23, 2025
link] LinkedIn: Journey of next-generation control plane for data systems LinkedIn writes about the evolution of Nuage, its internal control plane framework for managing data infrastructure resources. link] Grab: Improving Hugo's stability and addressing oncall challenges through automation.
Towards a Data Mesh (part 1) : Data Domains and Teams Topologies.
François Nguyen
MARCH 7, 2021
In this article, Juan Sequada gives maybe one of the best definition of Data Mesh ” It is paradigm shift towards a distributed architecture that attempts to find an ideal balance between centralization and decentralization of metadata and data management.” ” I would have added data teams to metadata and data management.
Government 130
130 
An Introduction To Data And Analytics Engineering For Non-Programmers
Data Engineering Podcast
JANUARY 15, 2022
You can observe your pipelines with built in metadata search and column level lineage. Can you describe what motivated you to write a book about the work of building data products? What are the main goals that you are trying to achieve through the book? What are the main goals that you are trying to achieve through the book?
Engineering 100
100 
Hire And Scale Your Data Team With Intention
Data Engineering Podcast
JUNE 12, 2022
Atlan is the metadata hub for your data ecosystem. Instead of locking all of that information into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Go to dataengineeringpodcast.com/atlan today to learn more about how you can take advantage of active metadata and escape the chaos.
Metadata 100
100 
Charting the Path of Riskified's Data Platform Journey
Data Engineering Podcast
JULY 10, 2022
Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold. Atlan is the metadata hub for your data ecosystem.
Metadata 100
100 
The View From The Lakehouse Of Architectural Patterns For Your Data Platform
Data Engineering Podcast
JULY 3, 2022
Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold. Atlan is the metadata hub for your data ecosystem.
Architecture 100
100 A Reflection On Data Observability As It Reaches Broader Adoption
Data Engineering Podcast
SEPTEMBER 4, 2022
Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Atlan is the metadata hub for your data ecosystem. And don’t forget to thank them for their continued support of this show!
IT 100
100 
Tame The Entropy In Your Data Stack And Prevent Failures With Sifflet
Data Engineering Podcast
NOVEMBER 20, 2022
Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold. What are some of the data modeling considerations that need to be considered when pushing metadata to Sifflet? Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.
Data Lake 130
130 
Simplify Data Security For Sensitive Information With The Skyflow Data Privacy Vault
Data Engineering Podcast
JUNE 5, 2022
Atlan is the metadata hub for your data ecosystem. Instead of locking all of that information into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Go to dataengineeringpodcast.com/atlan today to learn more about how you can take advantage of active metadata and escape the chaos.
Data Security 100
100 
Supporting content decision makers with machine learning
Netflix Tech
DECEMBER 10, 2020
For each source task, we learn a model on a large set of historical titles, leveraging information such as title metadata (e.g., From this title-level information, we devise the following supervised learning tasks: {metadata, tags, summaries} ? content category {metadata, tags, summaries, country} ?
How to learn data engineering
Christophe Blefari
JANUARY 20, 2024
This book, 📘 Data Pipelines Pocket Reference , defines everything related to data pipelines and how to treat data movement from source to target. new concepts — in today's data engineering a lot of new concepts enter the field every year like quality, lineage, metadata management, governance, privacy, sharing, etc.
Data Engineer 130
130 
Effective Pandas Patterns For Data Engineering
Data Engineering Podcast
JANUARY 30, 2022
You can observe your pipelines with built in metadata search and column level lineage. You can observe your pipelines with built in metadata search and column level lineage. As a result it has become a standard tool for data engineers for a wide range of applications. __init__ Episode pytest Podcast.__init__
Data Engineer 100
100 
Data News — Week 24.28
Christophe Blefari
JULY 13, 2024
DevOps for data science — An open-source and free book covering what data scientists need to know about DevOps. It provides abstractions and tools for the translation of lakehouse table format metadata. It has been written by someone at Posit (the company behind RStudio). It covers general knowledge about infra + code snippets.
Kafka 130
130 
Cleaning And Curating Open Data For Archaeology
Data Engineering Podcast
FEBRUARY 3, 2019
It started because we need better ways of dissminating structured data and digital media than is possible with conventional articles, books and reports. What pieces of metadata do you track for a given data set? What pieces of metadata do you track for a given data set?
Digital Media 100
100 
Distributed In Memory Processing And Streaming With Hazelcast
Data Engineering Podcast
SEPTEMBER 14, 2020
If you hand a book to a new data engineer, what wisdom would you add to it? Tree Schema is a data catalog that is making metadata management accessible to everyone. If you hand a book to a new data engineer, what wisdom would you add to it? Tree Schema is a data catalog that is making metadata management accessible to everyone.
Process 100
100 
Data Collection And Management To Power Sound Recognition At Audio Analytic
Data Engineering Podcast
JUNE 29, 2020
If you hand a book to a new data engineer, what wisdom would you add to it? If you hand a book to a new data engineer, what wisdom would you add to it? This was a great conversation about the complexities of working in a niche domain of data analysis and how to build a pipeline of high quality data from collection to analysis.
Data Collection 100
100 Building A Cost Effective Data Catalog With Tree Schema
Data Engineering Podcast
NOVEMBER 9, 2020
He also shares the internal architecture, how he approached the design to make it accessible and easy to use, and how it autodiscovers the schemas and metadata for your source systems. If you hand a book to a new data engineer, what wisdom would you add to it?
Building 100
100 
Put Your Whole Data Team On The Same Page With Atlan
Data Engineering Podcast
APRIL 5, 2021
What makes Atlan stand out from other systems for data cataloguing, metadata management, or data governance? What makes Atlan stand out from other systems for data cataloguing, metadata management, or data governance? What components of the data stack might Atlan replace? What types of data assets (e.g. is Atlan designed to understand?
Data Warehouse 100
100 
Adding Context And Comprehension To Your Analytics Through Data Discovery With SelectStar
Data Engineering Podcast
JULY 30, 2021
Now there’s a book that captures the foundational lessons and principles that underly everything that you hear about here. There has been a surge in tools and services for metadata collection, data catalogs, and data collaboration. Can you describe what SelectStar is and the story behind it?
BI 100
100 
Production Media Management: Transforming Media Workflows by leveraging the Cloud
Netflix Tech
MARCH 12, 2021
For a more comprehensive overview, please refer to Scott Arundale and Tashi Trieu’s book, Modern Post: Workflows and Techniques for Digital Filmmakers. Media Workflows: Content Hub (Netflix UI) Import: Imports footage media, which is inspected and, with the help of the metadata, categorized into assets. Workflows are louder than words.
Media 72
72 
Mainframe Optimization: 5 Best Practices to Implement Now
Precisely
JANUARY 25, 2024
You must carefully consider various mainframe functions, including security, system logs, metadata, and COBOL copybooks when moving to the new cloud platform. Mind Your Metadata When you move data from one system to another, it’s important to maintain metadata regarding that data’s lineage. Best Practice 2. Best Practice 3.
Metadata 68
68 
Drive Your Retail Media Strategy with Data Clean Rooms
Snowflake
DECEMBER 7, 2023
At this point, we’re all just waiting for the day when Google fully blocks third-party cookies on Chrome and closes the book on an interesting era of advertising. Known data can serve as metadata around customers’ purchasing history, affinities and buying intent.
Media 105
105 Beyond Legacy Detection: How AI-Driven Data Governance Surpasses Traditional Methods
Striim
MARCH 4, 2025
It logs each event with AI-enhanced metadata for effective tracking and auditing, while its adaptive design accommodates evolving data sources through schema evolution. Try Striim today with a free trial or book a demo to see it in action. Ready to take your data governance efforts to the next level?
Data Governance Series: Part 2 – The Importance of Data Taxonomy
The Modern Data Company
MARCH 7, 2023
A number such as 516.375 above identifies a book or other resource specifically as dealing with Finsler Geometry. That number also shows how that book relates to others above and below it in the hierarchy. For various reasons, the data sometimes can’t be re-tagged to provide consistent metadata.
Data Governance Trends for 2023
Precisely
JANUARY 11, 2023
Read our eBook Data Governance: How to Get Started If your organization is embarking on a new data governance initiative, or if you’re looking to build momentum behind a nascent data governance project, download our free e-book. Metadata is often referred to as “data about data.”
97 things every data engineer should know
Grouparoo
OCTOBER 6, 2021
Last month, we decided that we should all read a book and talk about it as a company. This was the first book I have read in this series and I liked the format. For example, grouping the ones about metadata, discoverability, and column naming might have made a lot of sense. The articles are in alphabetical order.
Large Scale Ad Data Systems at Booking.com using the Public Cloud
Booking.com Engineering
DECEMBER 2, 2022
Booking Holdings, as a whole, spent $4.7 We make extensive use of Google BigQuery in PPC due to the scale of our business (up to 2 million room nights booked per day). As an example, in one of our first BigQuery aggregations, we had a large query that joined statistics data with metadata, then aggregated over it.
Systems 52
52 
5 Use Cases for Vector Search
Rockset
MAY 8, 2023
Spotify builds the vector embeddings with the query text being the input embedding and a concatenation of textual metadata fields including title and description for the podcast episode embeddings. One of the reasons that Vespa was chosen is that it can also incorporate metadata filtering post-search on features like episode popularity.
Metadata 52
52 
Data Engineering Weekly #138
Data Engineering Weekly
JULY 9, 2023
Andrew Jones: Data Contracts - the book. Out now Congrats Andrew Jones for the brand new book on Data Contracts - The topic is very close to my heart. I briefly reviewed the book, and it looks like a solid one. Thank you for the reference mention of Schemata in the book. We will reach you ASAP. I liked the chapter.
The Good and the Bad of Apache Kafka Streaming Platform
AltexSoft
OCTOBER 21, 2022
How Apache Kafka streams relate to Franz Kafka’s books. It also provides books , academic papers, and educational videos to explore the technology in more detail. The tool takes care of storing metadata about partitions and brokers. Books and papers. Practically, nothing. ZooKeeper issue. Best Kafka Summit videos.
Kafka 93
93 
What is Data Fabric: Architecture, Principles, Advantages, and Ways to Implement
AltexSoft
AUGUST 22, 2022
It is also driven by metadata, which it uses to recognize patterns, map data, and perform continuous analysis. It is a metadata integration layer that can be built on top of a data lake or an architectural ensemble that includes it. Data and metadata. Basic metadata can be structural, descriptive, and administrative.
Architecture 52
52 
Let's personalize your content