Data Engineering Podcast

JULY 10, 2022

Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Metadata 100

Metadata MongoDB MySQL Data Pipeline 100

Snowflake

JANUARY 23, 2024

Snowpark ML Operations: Model management The path to production from model development starts with model management, which is the ability to track versioned model artifacts and metadata in a scalable, governed manner. The Snowpark Model Registry API provides simple catalog and retrieval operations on models.

Machine Learning 121

Machine Learning Metadata Python Telecommunication 121

Monte Carlo

JULY 22, 2024

While it shares similarities with software versioning, data versioning has unique characteristics specific to your data management needs. Maintaining metadata about each version. By implementing data versioning, you can create a systematic approach to managing the evolution of your data.

Metadata 52

Metadata Datasets ETL Tools Data 52

Snowflake

JUNE 5, 2024

To help organizations better govern AI, Snowflake Horizon is also advancing security, lineage and sharing capabilities for models. Additional built-in UI’s and privacy enhancements make it even easier to understand and manage sensitive data.

Government 103

Government Accessible Accessibility Cloud 103

Data Engineering Weekly

OCTOBER 6, 2024

link] Sponsored: Monte Carlo Demo - October You want to deliver reliable business insights to your organization, but that probably means you’re spending a whole lotta time writing a whole lotta of rules. It updates the popular Pirate metrics (AARRR) framework by adding concepts like Attention and Adoption.

Data Engineering 62

Data Engineering Data Engineer Engineering Data Warehouse 62

Christophe Blefari

JUNE 16, 2023

The power of pre-commit and SQLFluff —SQL is a query programming language used to retrieve information from data storages, and like any other programming language, you need to enforce checks at all times. Malloy's Near Term Roadmap — I've shared recently Malloy demo , which was awesome. but I missed it).

Programming Language 130

Programming Language SQL PostgreSQL Data 130

Precisely

MAY 31, 2023

Users cannot find the data they need, gain access in a timely fashion, rely on data being in a useable format, or understand its impact on the business – meaning that it is unusable to them. The end result is that users can’t trust their data because it lacks integrity.

Data Integration 52

Data Integration Metadata Data Governance Data 52

Monte Carlo

MAY 25, 2023

That’s why we’re excited to announce that uniting diverse data team personas to collectively ensure data quality just got easier, thanks to a new integration between Monte Carlo’s data observability platform and Atlan’s active metadata platform.

Metadata 52

Metadata Data Governance Government Business Intelligence 52

Monte Carlo

MAY 20, 2024

It’s an essential resource for database administrators, data engineers, and developers who need to ensure that the data within a specific system or database is used consistently and correctly. On the other hand, a data catalog is used more broadly within an organization and is instrumental in data governance and business analysis.

Datasets 52

Datasets Data Government Business Analyst 52

The Modern Data Company

JANUARY 17, 2023

These include internal data lakes, apps (1st or 3rd party), web SDK, internal or 3rd-party databases, and even spreadsheets generated within the marketing department. Without lineage or governance metadata, the quality of this data is anyone’s guess. New data is added to the catalog automatically upon ingestion.

Metadata 59

Metadata Government Data Lake Data 59

Data Engineering Podcast

SEPTEMBER 23, 2018

Request a demo at dataengineeringpodcast.com/metis-machine to learn more about how Metis Machine is operationalizing data science. Request a demo at dataengineeringpodcast.com/metis-machine to learn more about how Metis Machine is operationalizing data science.

Data Lake 100

Data Lake Data Warehouse Data Architecture Architecture 100

Silectis

FEBRUARY 3, 2021

Backed by a robust metadata repository, Magpie leverages activity tracking and data governance to offer a comprehensive suite of automated data lineage features. With an abundance of tooling items, Magpie’s features include metadata object lineage, read and write event tracking, and table lineage graphs, among others.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Lake 52

Silectis

DECEMBER 15, 2020

Integrated Security & Governance Features Ensuring that your data is protected and secure is one of our top priorities. The time to identify more insights out of huge sets of data. Request your demo below and we’ll be in touch to get you set up. Learn more about the technology behind Magpie here.

Data Engineering 52

Data Engineering Data Engineer Engineering ETL Tools 52

The Modern Data Company

APRIL 15, 2024

The term “Data Product” has become a buzzword, often misused or overstretched. But at its core, a Data Product is much more than just data. This blend ensures that a Data Product is informative, actionable, and adaptable to various needs.

Government 52

Government Metadata Data Data Management 52

Monte Carlo

APRIL 24, 2023

Traditionally, after being stored in a data lake, raw data was then often moved to various destinations like a data warehouse for further processing, analysis, and consumption. Databricks Data Catalog and AWS Lake Formation are examples in this vein. AWS is one of the most popular data lake vendors.

Data Lake 52

Data Lake Google Cloud Data Warehouse AWS 52

phData: Data Engineering

DECEMBER 16, 2021

As a result, it is important to know what is happening to your data. Is your data just being scanned, or is it being moved? Where is your metadata stored? How easy is it to offboard your data if you choose another tool? It can let the user know about the quality and age of the data as well as any policies attached to it.

Metadata 52

Metadata Datasets Cloud ETL Tools 52

Monte Carlo

JULY 10, 2023

Data sampling If you’re working with large data sets where it’s impractical to evaluate every attribute or record, you can systematically sample your data set to estimate completeness. Be sure to use random sampling to select representative subsets of your data.

Data Collection 40

Data Collection Data Governance Government Data 40

AltexSoft

OCTOBER 8, 2021

To make a more informed decision, enquire about demos and do your own in-depth research. Oracle Data Integrator, IBM InfoSphere, Snaplogic, Xplenty, and. The prevailing part of users claim that it is quite easy to configure and manage data flows with Oracle’s graphical tools. Data profiling and cleansing. Source: G2.

Data Integration 52

Data Integration Hadoop Data Warehouse Data Lake 52

Monte Carlo

JANUARY 23, 2023

Data lineage , an automated visualization of the relationships for how data flows across tables and other data assets, is a must-have in the data engineering toolbox. Not only is it helpful for data governance and compliance use cases, it also plays a starring role as one of the 5 pillars of data observability.

IT 52

IT BI Government Data Governance 52

The Modern Data Company

JANUARY 22, 2024

The result—no matter the final configuration or what a company’s data landscape looks like—is a data ecosystem characterized by consistency, reliability, and absolute adherence to governance principles. Operational data integration leadership DataOS supports a broad spectrum of use cases.

Data Integration 98

Data Integration Metadata Government Unstructured Data 98

Precisely

JULY 15, 2024

The move to productize data also requires a way to package data products so they are easily and uniformly discoverable and consumed. Data products must be properly designed and organized to be reused across the organization. Data catalogs or data marketplaces can be used to enhance the value of your data mesh strategy.

Data Governance 72

Data Governance Data Integration Government Data 72

The Modern Data Company

JANUARY 22, 2024

The result—no matter the final configuration or what a company’s data landscape looks like—is a data ecosystem characterized by consistency, reliability, and absolute adherence to governance principles. Operational data integration leadership DataOS supports a broad spectrum of use cases.

Data Integration 87

Data Integration Metadata Government Unstructured Data 87

Ascend.io

APRIL 7, 2023

Data fabric is a centralized platform architecture originating from a curated metadata layer that sits on top of an organization’s data infrastructure. Every time a new data source is added, the metadata layer is updated to define how and when that data should be used. Increasing speed.

Metadata 52

Metadata Datasets Data Governance Government 52

The Modern Data Company

MARCH 1, 2024

Instead of forcing all data through a centralized bottleneck, they break it down into modular, self-contained units. Each Data Product is designed for a specific purpose, equipped with the necessary data, transformations, and metadata. Data Products simplify access and processing, empowering faster decision-making.

Database-centric 40

Database-centric Data Lake Government Retail 40

Data Engineering Weekly

JULY 23, 2023

Find the right, trusted data by simply asking a question. Atlan AI leverages metadata that Atlan captures across the data stack to make AI part of your data stack. And you won’t have to ask 3 different people in your team just to find the right, trusted data — you just need your AI copilot.

Data Engineering 52

Data Engineering Data Engineer Engineering Deep Learning 52

Monte Carlo

APRIL 20, 2023

I can surface ownership metadata and alert the relevant owners to make sure the appropriate changes are made so these breakages never happen. Data lineage helps ensure you have clear lines of accountability that data team structure–centralized, center of excellence , data mesh– you use or however your data pipelines evolve.

Data Warehouse 52

Data Warehouse BI Data Government 52

AltexSoft

MARCH 30, 2023

Source: The Data Team’s Guide to the Databricks Lakehouse Platform Integrating with Apache Spark and other analytics engines, Delta Lake supports both batch and stream data processing. Besides that, it’s fully compatible with various data ingestion and ETL tools.

Scala 64

Scala Data Lake BI Google Cloud 64