Data Engineering Podcast

AUGUST 10, 2021

Summary All of the fancy data platform tools and shiny dashboards that you use are pointless if the consumers of your analysis don’t have trust in the answers. Stemma helps you establish and maintain that trust by giving visibility into who is using what data, annotating the reports with useful context, and understanding who is responsible for keeping it up to date.

IT 130

IT Building Data Warehouse Python 130

Cloudera

AUGUST 11, 2021

Once upon an IT time, everything was a “point product,” a specific application designed to do a single job inside a desktop PC, server, storage array, network, or mobile device. Point solutions are still used every day in many enterprise systems, but as IT continues to evolve, the platform approach beats point solutions in almost every use case.

Cloud 121

Cloud Big Data Cloud Computing Government 121

DataKitchen

AUGUST 13, 2021

A drug company tests 50,000 molecules and spends a billion dollars or more to find a single safe and effective medicine that addresses a substantial market. Figure 1 shows the 15-year cycle from screening to government agency approval and phase IV trials. Drug companies desperately look for ways to compress this lengthy time frame and to demonstrate the competitive advantage of their intellectual property.

Pharmaceutical 98

Pharmaceutical Amazon Web Services Government Google Cloud 98

Teradata

AUGUST 12, 2021

For both analysts and data scientists, identifying paths and patterns in data is a valuable way to gain insight into the occurrences leading to or from any event of interest. Read more.

Data 98

Data 98

Speaker: Tamara Fingerlin, Developer Advocate

Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.

Data

Confluent

AUGUST 10, 2021

Today, Confluent is announcing the general availability (GA) of the fully managed Azure Cosmos DB Sink Connector within Confluent Cloud. Now, with just a few simple clicks, you can link […].

Cloud 98

Cloud Management Data Pipeline Kafka 98

Cloudera

AUGUST 12, 2021

In my previous blog post, I shared examples of how data provides the foundation for a modern organization to understand and exceed customers’ expectations. However, the important role data occupies extends beyond customer experience and revenue, as it becomes increasingly central in optimizing internal processes for the long-term growth of an organization.

Manufacturing 111

Manufacturing Insurance Consulting Education 111

AltexSoft

AUGUST 10, 2021

B2B sales strategies can be roughly divided into two activities: lead generation and lead conversion. It’s clear how each works. The former, attracting visitors to your website and then helping them take certain actions, is almost automated and works through carefully placed calls to action. The latter, supporting a lead to make the purchasing decision, is done by professional sales people with their arsenal of personalized tactics.

Machine Learning 72

Machine Learning Data Mining Algorithm Datasets 72

DareData

AUGUST 12, 2021

Introduction "Data is the 21st century oil". If you work anywhere in the vicinity of data, odds are you've heard some variation of this statement at least once. But while the value of data and data-driven decision making is becoming increasingly more apparent, it is not immediately obvious how to build and maintain a healthy data ecosystem. In fact, this not a trivial endeavor at all.

Transportation 52

Transportation Data Lake Data Warehouse Data 52

Cloudera

AUGUST 13, 2021

Every enterprise is trying to collect and analyze data to get better insights into their business. Whether it is consuming log files, sensor metrics, and other unstructured data, most enterprises manage and deliver data to the data lake and leverage various applications like ETL tools, search engines, and databases for analysis. This whole architecture made a lot of sense when there was a consistent and predictable flow of data to process.

Data Pipeline 111

Data Pipeline Data Lake ETL Tools Unstructured Data 111

DataKitchen

AUGUST 12, 2021

The post DBTA Readers’ Choice Awards, 2021 first appeared on DataKitchen.

52

52

Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage

There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.

Systems

ProjectPro

AUGUST 11, 2021

“Data Lake vs Data Warehouse = Load First, Think Later vs Think First, Load Later” The terms data lake and data warehouse are frequently stumbled upon when it comes to storing large volumes of data. Often they are used interchangeably but they are totally different on how the data is structured and processed. If you’re a big data engineer and finding it difficult to decide whether to use a data lake or a data warehouse for your organizational needs then we’ve got you cove

Data Lake 52

Data Lake Data Warehouse Cloud Hadoop 52

Datakin

AUGUST 10, 2021

Blog An instant demo of data lineage is worth a thousand words Written by Ross Turk on August 10, 2021 They say that a picture is worth a thousand words. If you’ve ever tried to describe how all the jobs in your data pipeline are interrelated using just words, I am sure it wasn’t easy. I bet you used way more than a thousand of them. But you probably never got past a hundred words before looking for something to draw with – it’s far easier to explain data lineage on a whiteboard in t

Data Pipeline 52

Data Pipeline Metadata Data IT 52

Cloudera

AUGUST 10, 2021

Follow your data in object storage on-premises. As businesses look to scale-out storage, they need a storage layer that is performant, reliable and scalable. With Apache Ozone on the Cloudera Data Platform (CDP) , they can implement a scale-out model and build out their next generation storage architecture without sacrificing security, governance and lineage.

Hadoop 105

Hadoop Kafka Datasets Government 105

Teradata

AUGUST 10, 2021

As medical claims become more complex, automation will be crucial to insurers’ longevity. How can insurers manage the demand to automate without sacrificing customer experience or payment integrity?

Medical 52

Medical Insurance Management 52

Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives

Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri

Manufacturing

ProjectPro

AUGUST 9, 2021

Table of Contents Why Learn Python for Data Science? Top 20 Python Projects for Data Science Getting Started with Python for Data Science FAQs about data science projects Why Learn Python for Data Science? Python has come to command a celebrity status in data science over the years. It is loved by all data enthusiasts and provides an easy introduction to data science and machine learning.

Data Science 52

Data Science Python Project Datasets 52

RudderStack

AUGUST 13, 2021

This update includes a few of our most requested features like volume reporting and sync retry which our customers are happy to see in production.

40

40

Cloudera

AUGUST 10, 2021

With the release of CDP Private Cloud (PvC) Base 7.1.7, you can look forward to new features, enhanced security, and better platform performance to help your business drive faster insights and value. We understand that migrating your data platform to the latest version can be an intricate task, and at Cloudera we’ve worked hard to simplify this process for all our customers. .

Cloud 97

Cloud Kafka Metadata SQL 97

Teradata

AUGUST 8, 2021

Digital hubs for supply chains enable resiliency in the operation of the supply chain & in the underlying data analytics. Learn more about its main components and benefits.

Data Analytics 52

Data Analytics IT Data 52

Advertisement

With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to: Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to Write DAGs that adapt to your data at runtime and set up alerts and notifications Scale you

Data Engineer

ProjectPro

AUGUST 12, 2021

Global data generation will expand to 63 zettabytes (ZB) by 2025. Business Intelligence (BI) offers excellent ways to gain data insights and use them in data-driven decision-making. BI market will grow to $39.35 billion in the next five years. It is essential to pick the right BI tools to obtain the most out of the BI technologies. We have compared the two most popular BI tools viz.

BI 40

BI Business Intelligence Data Security Accessibility 40

RudderStack

AUGUST 10, 2021

Check out this guide to learn how you can switch from Segment to RudderStack in four steps with minimal engineering work and no data loss.

Engineering 40

Engineering Data 40

DareData

AUGUST 13, 2021

About me I've recruited, hired, and worked directly with ~50 technical workers over the last 8 years. And by "worked with" I mean actually worked with. Gone to client meetings with them, made project plans with them, written code with them, released bugs into production with them, fixed said bugs with them, watched them overcome challenges, watched them fail challenges, etc.

Recruitment 52

Recruitment Process Management Programming 52

ProjectPro

AUGUST 13, 2021

If you are a beginner searching for Machine Learning GitHub Projects, you are on the right page. Below you will find a list of Machine Learning projects on Github that are beginner-friendly and popular among Data Science enthusiasts. Table of Contents 15 Sample GitHub Machine Learning Projects Python Machine Learning Projects on GitHub 1. Predictive Analytics 2.

Machine Learning 40

Machine Learning Project Deep Learning Algorithm 40

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data