Data Engineering Podcast

APRIL 14, 2018

Summary The rate of change in the data engineering industry is alternately exciting and exhausting. Joe Crobak found his way into the work of data management by accident as so many of us do. This led to his creation of the Hadoop Weekly newsletter, which he recently rebranded as the Data Engineering Weekly newsletter.

Data Engineering 100

Data Engineering Data Engineer Engineering Hadoop 100

Data Engineering Podcast

JANUARY 21, 2024

In this episode Tasso Argyros, CEO of ActionIQ, gives a summary of the major epochs in database technologies and how he is applying the capabilities of cloud data warehouses to the challenge of building more comprehensive experiences for end-users through a modern customer data platform (CDP).

Data Lake 147

Data Lake High Quality Data NoSQL Data Warehouse 147

Data Engineering Podcast

FEBRUARY 19, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Hey there podcast listener, are you tired of dealing with the headache that is the 'Modern Data Stack'? It's supposed to make building smarter, faster, and more flexible data infrastructures a breeze.

IT 147

IT Data Lake Metadata Data Warehouse 147

ProjectPro

MAY 2, 2016

News on Hadoop-April 2016 Cutting says Hadoop is not at its peak but at its starting stages. Datanami.com At his keynote address in San Jose, Strata+Hadoop World 2016, Doug Cutting said that Hadoop is not at its peak and not going to phase out. Source: [link] ) Dr. Elephant will now solve your Hadoop flow problems.

Hadoop 52

Hadoop NoSQL Hospitality Big Data 52

ProjectPro

APRIL 1, 2016

News on Hadoop- March 2016 Hortonworks makes its core more stable for Hadoop users. PCWorld.com Hortonworks is going a step further in making Hadoop more reliable when it comes to enterprise adoption. Hortonworks Data Platform 2.4, Source: [link] ) Syncsort makes Hadoop and Spark available in native Mainframe.

Hadoop 52

Hadoop BI Big Data Big Data Tools 52

Data Engineering Podcast

JANUARY 6, 2019

Summary The Hadoop platform is purpose built for processing large, slow moving data in long-running batch jobs. As the ecosystem around it has grown, so has the need for fast data analytics on fast moving data. How does it fit into the Hadoop ecosystem? What was the reasoning for using Raft in Kudu?

Data Analytics 100

Data Analytics Hadoop Kafka Media 100

Data Engineering Podcast

APRIL 2, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Businesses that adapt well to change grow 3 times faster than the industry average. As your business adapts, so should your data. As your business adapts, so should your data.

Hadoop 130

Hadoop Machine Learning Python Architecture 130

ProjectPro

MAY 2, 2017

News on Hadoop-April 2017 AI Will Eclipse Hadoop, Says Forrester, So Cloudera Files For IPO As A Machine Learning Platform. Apache Hadoop was one of the revolutionary technology in the big data space but now it is buried deep by Deep Learning. Forbes.com, April 3, 2017. Hortonworks HDP 2.6

Hadoop 52

Hadoop Entertainment Data Lake Big Data 52

Data Engineering Podcast

AUGUST 19, 2019

Summary Managing big data projects at scale is a perennial problem, with a wide variety of solutions that have evolved over the past 20 years. One of the early entrants that predates Hadoop and has since been open sourced is the HPCC (High Performance Computing Cluster) system.

Big Data 100

Big Data Hadoop Data Lake Media 100

ProjectPro

NOVEMBER 14, 2015

Big data has taken over many aspects of our lives and as it continues to grow and expand, big data is creating the need for better and faster data storage and analysis. These Apache Hadoop projects are mostly into migration, integration, scalability, data analytics, and streaming analysis. Data Migration 2.

Hadoop 52

Hadoop Project Big Data Healthcare 52

ProjectPro

FEBRUARY 1, 2018

News on Hadoop - Janaury 2018 Apache Hadoop 3.0 The latest update to the 11 year old big data framework Hadoop 3.0 The latest update to the 11 year old big data framework Hadoop 3.0 This new feature of YARN federation in Hadoop 3.0 This new feature of YARN federation in Hadoop 3.0

Hadoop 52

Hadoop Food Healthcare Cloud Computing 52

ProjectPro

SEPTEMBER 30, 2014

Hadoop’s significance in data warehousing is progressing rapidly as a transitory platform for extract, transform, and load (ETL) processing. Mention about ETL and eyes glaze over Hadoop as a logical platform for data preparation and transformation as it allows them to manage huge volume, variety, and velocity of data flawlessly.

Hadoop 52

Hadoop ETL Tools Unstructured Data ETL System 52

Data Engineering Podcast

OCTOBER 14, 2018

Summary With the growth of the Hadoop ecosystem came a proliferation of implementations for the Hive table format. The Hive format is also built with the assumptions of a local filesystem which results in painful edge cases when leveraging cloud object storage for a data lake.

Data Lake 100

Data Lake Big Data Cloud Hadoop 100

Data Engineering Podcast

AUGUST 3, 2021

In this episode Vinoth shares the history of the project, how its architecture allows for building more frequently updated analytical queries, and the work being done to add a more polished experience to the data lake paradigm. Interview Introduction How did you get involved in the area of data management?

Data Lake 130

Data Lake Data Warehouse Hadoop Architecture 130

ProjectPro

JUNE 4, 2015

All the components of the Hadoop ecosystem, as explicit entities are evident. All the components of the Hadoop ecosystem, as explicit entities are evident. The holistic view of Hadoop architecture gives prominence to Hadoop common, Hadoop YARN, Hadoop Distributed File Systems (HDFS ) and Hadoop MapReduce of the Hadoop Ecosystem.

Hadoop 52

Hadoop Architecture IT Java 52

Data Engineering Podcast

MARCH 27, 2022

Summary Data governance is a practice that requires a high degree of flexibility and collaboration at the organizational and technical levels. The growing prominence of cloud and hybrid environments in data management adds additional stress to an already complex endeavor. What do you have planned for the future of Privacera?

Data Governance 130

Data Governance Government Cloud Building 130

Data Engineering Podcast

NOVEMBER 20, 2021

In this episode Ori Rafael shares his experiences from Upsolver and building scalable stream processing for integrating and analyzing data, and what the tradeoffs are when coming from a batch oriented mindset. Can you start by giving an overview of the state of the market for data lakes today?

Data Lake 100

Data Lake Data Integration Lambda Architecture Process 100

Knowledge Hut

DECEMBER 28, 2023

Imagine having a framework capable of handling large amounts of data with reliability, scalability, and cost-effectiveness. That's where Hadoop comes into the picture. Hadoop is a popular open-source framework that stores and processes large datasets in a distributed manner. Why Are Hadoop Projects So Important?

Hadoop 52

Hadoop Project Big Data Datasets 52

ProjectPro

JANUARY 12, 2016

Choosing the right Hadoop Distribution for your enterprise is a very important decision, whether you have been using Hadoop for a while or you are a newbie to the framework. Different Classes of Users who require Hadoop- Professionals who are learning Hadoop might need a temporary Hadoop deployment.

Hadoop 52

Hadoop Big Data Java Metadata 52

Data Engineering Podcast

JANUARY 21, 2019

This is a useful conversation for engineers, managers, and leadership who are interested in building enterprise big data systems. My understanding is that the big data group at LEGO is a fairly recent development. What are some of the most critical sources and types of data that you are managing?

Big Data 100

Big Data Systems Building Hadoop 100

Ripple Engineering

JULY 9, 2024

Introduction: Embracing the Future with Ripple's Data Platform Migration Welcome to a pivotal moment in Ripple's data journey. As leaders at the intersection of blockchain technology and financial services, we're excited to share a transformative step in our data management evolution.

Hadoop 96

Hadoop Data Lake Machine Learning Raw Data 96

Monte Carlo

OCTOBER 31, 2024

Both traditional and AI data engineers should be fluent in SQL for managing structured data, but AI data engineers should be proficient in NoSQL databases as well for unstructured data management.

Data Engineering 52

Data Engineering Data Engineer Engineering Unstructured Data 52

Data Engineering Podcast

MAY 20, 2018

In this episode he describes how Presto is architected, how you can use it for your analytics, and the work that he is doing at Starburst Data. With private networking, shared block storage, node balancers, and a 40Gbit network, all controlled by a brand new API you’ve got everything you need to run a bullet-proof data platform.

PostgreSQL 100

PostgreSQL Hadoop SQL Kafka 100

Data Engineering Podcast

APRIL 29, 2018

Preamble Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline you’ll need somewhere to deploy it, so check out Linode. The current goal for most companies is to be “data driven” How would you define that concept?

Business Intelligence 100

Business Intelligence Scala Hadoop Machine Learning 100

Data Engineering Podcast

NOVEMBER 4, 2018

Preamble Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline you’ll need somewhere to deploy it, so check out Linode. Can you start by describing what Looker is and the problem that it is aiming to solve?

Business Intelligence 100

Business Intelligence Hadoop BI Data Warehouse 100

ProjectPro

AUGUST 22, 2016

Capgemini, a leading provider of consulting, technology and outsourcing services, helps companies identify, design and develop technology programs to sharpen their competitive edge. Capgemini’s Big Data Service Centre framework lets organizations implement next generation data management architecture that uses Hadoop.

Hadoop 40

Hadoop Big Data Cloud Computing Consulting 40

Data Engineering Podcast

FEBRUARY 9, 2020

If you’re struggling with unwieldy dimensional models, slow moving projects, or challenges integrating new data sources then listen in on this conversation and then give data vault a try for yourself. We have partnered with organizations such as O’Reilly Media, Corinium Global Intelligence, ODSC, and Data Council.

Data Lake 100

Data Lake Data Warehouse Hadoop NoSQL 100

ProjectPro

SEPTEMBER 9, 2016

billion USD, 95000 professionals across diverse nationalities in 31 countries- India’s original IT garage startup, HCL, uses a data driven methodology to migrate ETL jobs into corresponding hadoop jobs. HCL has adopted hadoop as a viable alternative to reduce cost and speed up processing. With an annual revenue of $6.5

Hadoop 40

Hadoop Data Lake Big Data Cloud Computing 40

ProjectPro

FEBRUARY 10, 2016

It is difficult to believe that the first Hadoop cluster was put into production at Yahoo, 10 years ago, on January 28 th , 2006. Ten years ago nobody was aware that an open source technology, like Apache Hadoop will fire a revolution in the world of big data. Happy Birthday Hadoop With more than 1.7

Hadoop 40

Hadoop Big Data Project Programming 40

Data Engineering Podcast

JANUARY 13, 2019

release, how the use cases for timeseries data have proliferated, and how they are continuing to simplify the task of processing your time oriented events. Links TimescaleDB Original Appearance on the Data Engineering Podcast 1.0 Links TimescaleDB Original Appearance on the Data Engineering Podcast 1.0

Database 100

Database PostgreSQL SQL MongoDB 100

ProjectPro

MARCH 15, 2016

Hadoop is beginning to live up to its promise of being the backbone technology for Big Data storage and analytics. Companies across the globe have started to migrate their data into Hadoop to join the stalwarts who already adopted Hadoop a while ago. Hadoop runs on clusters of commodity servers.

Hadoop 40

Hadoop Retail Healthcare Banking 40

Data Engineering Podcast

DECEMBER 9, 2018

He also discusses what you need to know to get it deployed and keep it running in a production environment and how it fits into the overall data ecosystem. What are some of the common ways that Spark is deployed, in terms of the cluster topology and the supporting technologies? Can you start by explaining what Spark is? Who uses Spark?

MySQL 100

MySQL Scala Kafka Hadoop 100

Data Engineering Podcast

AUGUST 27, 2018

In this episode Ellison Anny Williams, CEO of Enveil, describes how her company uses homomorphic encryption to ensure that your analytical queries can be executed without ever having to decrypt your data. identifying individuals based on geographic data, or purchase patterns) What do you have planned for the future of Enveil?

Transportation 100

Transportation Hadoop Data Security Architecture 100

ProjectPro

NOVEMBER 1, 2016

News on Hadoop-October 2016 Microsoft upgrades Azure HDInsight, its Hadoop Big Data offering.SiliconAngle.com,October 2, 2016. product Azure HDInsight is a managed Hadoop service that gives users access to deploy and manage hadoop clusters on the Azure Cloud. Microsoft and Hortonworks Inc.

Hadoop 40

Hadoop NoSQL Big Data SQL 40

ProjectPro

MARCH 23, 2016

And so spawned from this research paper, the big data legend - Hadoop and its capabilities for processing enormous amount of data. Same is the story, of the elephant in the big data room- “Hadoop” Surprised? Yes, Doug Cutting named Hadoop framework after his son’s tiny toy elephant.

Hadoop 40

Hadoop IT Big Data Portfolio 40

Data Engineering Podcast

SEPTEMBER 16, 2018

Preamble Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline you’ll need somewhere to deploy it, so check out Linode. What is unique about customer event data from an ingestion and processing perspective?

Google Cloud 100

Google Cloud Hadoop Consulting Business Intelligence 100

Data Engineering Podcast

NOVEMBER 18, 2018

Preamble Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out Linode. What are some of the primary ways that Flink is used?

Process 100

Process Google Cloud Scala Kafka 100