Start Data Engineering
AUGUST 17, 2021
Introduction Patterns 1. Batch Data Pipelines 1.1 Process => Data Warehouse 1.2 Process => Cloud Storage => Data Warehouse 2. Near Real-Time Data pipelines 2.1 Data Stream => Consumer => Data Warehouse 2.2 Cloud Storage => process => Data Warehouse Conclusion Further Reading Introduction Loading data into a data warehouse is a key component of most data pipelines.
Cloudera
AUGUST 17, 2021
Dear Parents and Educators and Friends of Cloudera, If you are reading this blog, you know us at Cloudera as a group of self-described data geeks and data analysts. We believe data drives better decisions and moves businesses forward and for us, that’s exciting. We are innovating and helping Fortune 500 transform and grow because they can make better data-driven decisions at the accelerated pace we live and work in today.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Data Engineering Podcast
AUGUST 20, 2021
Summary Data lakes have been gaining popularity alongside an increase in their sophistication and usability. Despite improvements in performance and data architecture they still require significant knowledge and experience to deploy and manage. In this episode Vikrant Dubey discusses his work on the Cuelake project which allows data analysts to build a lakehouse with SQL queries.
Confluent
AUGUST 17, 2021
The Confluent Q3 ‘21 release is here and packed full of new features that enable the world’s most innovative businesses to continue building what keeps them on top: real-time, mission-critical […].
Advertisement
Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?
DataKitchen
AUGUST 19, 2021
Below is our fourth post (4 of 5) on combining data mesh with DataOps to foster innovation while addressing the challenges of a decentralized architecture. We’ve covered the basic ideas behind data mesh and some of the difficulties that must be managed. Below is a discussion of a data mesh implementation in the pharmaceutical space. For those embarking on the data mesh journey, it may be helpful to discuss a real-world example and the lessons learned from an actual data mesh implementation.
Cloudera
AUGUST 16, 2021
We just announced Cloudera DataFlow for the Public Cloud (CDF-PC), the first cloud-native runtime for Apache NiFi data flows. CDF-PC enables Apache NiFi users to run their existing data flows on a managed, auto-scaling platform with a streamlined way to deploy NiFi data flows and a central monitoring dashboard making it easier than ever before to operate NiFi data flows at scale in the public cloud.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Confluent
AUGUST 20, 2021
We’re pleased to announce ksqlDB 0.20.0! The 0.20 ksqlDB release includes support for the DATE and TIME data types, along with functionality for working with these types. The DATE type […].
Teradata
AUGUST 17, 2021
Vantage on AWS supports Next Best Action efforts - adding new supplemental coverage on policy renewals at a rate of 250%.
Cloudera
AUGUST 17, 2021
When we announced the GA of Cloudera Data Engineering back in September of last year, a key vision we had was to simplify the automation of data transformation pipelines at scale. By leveraging Spark on Kubernetes as the foundation along with a first class job management API many of our customers have been able to quickly deploy, monitor and manage the life cycle of their spark jobs with ease.
Data Engineering Podcast
AUGUST 14, 2021
Summary The vast majority of data tools and platforms that you hear about are designed for working with structured, text-based data. What do you do when you need to manage unstructured information, or build a computer vision model? Activeloop was created for exactly that purpose. In this episode Davit Buniatyan, founder and CEO of Activeloop, explains why he is spending his time and energy on building a platform to simplify the work of getting your unstructured data ready for machine learning.
Speaker: Jay Allardyce, Deepak Vittal, and Terrence Sheflin
As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.
DataKitchen
AUGUST 19, 2021
The post 4 Ways Conversational AI Is Improving the Customer Experience first appeared on DataKitchen.
AltexSoft
AUGUST 18, 2021
Pricing in the airline industry is often compared to a brain game between carriers and passengers where each party pursues the best rates. Carriers aim at selling tickets as expensive as possible — while still not losing consumers to competitors. Passengers want to buy flights at the lowest cost — while not missing the chance to get on board. All this makes flight prices fluctuant and hard to predict.
Cloudera
AUGUST 16, 2021
Are you ready to turbo-charge your data flows on the cloud for maximum speed and efficiency? We are excited to announce the general availability of Cloudera DataFlow for the Public Cloud (CDF-PC) – a brand new experience on the Cloudera Data Platform (CDP) to address some of the key operational and monitoring challenges of standard Apache NiFi clusters that are overloaded with high-performant flows.
Ripple Engineering
AUGUST 19, 2021
One of the best ways to make software more accessible is to reduce the hardware resources needed to run it. Blockchain software is no exception. The XRP Ledger is already one of the greenest blockchains due to its pioneering consensus protocol, but its ecosystem can still benefit from more efficient resource usage. Reduced inefficiencies benefit businesses, developers, and enthusiasts alike.
Speaker: Nikhil Joshi, Founder & President of Snic Solutions
Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.
DataKitchen
AUGUST 20, 2021
The post AIOps Benefits All Aspects of the Enterprise first appeared on DataKitchen.
Teradata
AUGUST 17, 2021
Consumers continue to place emphasis on the sustainability credentials of those they choose to shop with, & what products they buy. Find out how retailers & CPGs should respond.
Cloudera
AUGUST 20, 2021
Introduction. In the first part of this series , I outlined the prerequisites for a modern Enterprise Data Platform to enable complex data product strategies that address the needs of multiple target segments and deliver strong profit margins as the data product portfolio expands in scope and complexity: With this article, I will dive into the specific capabilities of the Cloudera Data Platform (CDP) that has helped organizations to meet the aforementioned prerequisite capabilities and fulfill a
Ripple Engineering
AUGUST 19, 2021
Public key cryptography is one of the fundamental technologies that enables the XRP Ledger and other blockchain systems to operate. It uses a pair of keys: a public key and a private key. Anyone can create a new account and have authority to sign transactions from that account. In order to generate these keys, you can use a software library like ripple-keypairs.
Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage
When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.
DataKitchen
AUGUST 20, 2021
The post DataOps engineers run toward error and automate it away first appeared on DataKitchen.
Grouparoo
AUGUST 15, 2021
We want to make Grouparoo as easy as possible to run, which means considering many different server environments. We recently had a customer who wanted to run Grouparoo in a Docker cluster that only had IPv6 addresses enabled. There are lots of reasons why IPv6 might be better (including the fact that we are running out of public IPv4 Addresses ), but it’s rare to find a deployment environment that only has IPv6 addresses by default.
Cloudera
AUGUST 18, 2021
Data Lifecycle Management: The Key to AI-Driven Innovation. In digital transformation projects, it’s easy to imagine the benefits of cloud, hybrid, artificial intelligence (AI), and machine learning (ML) models. The hard part is to turn aspiration into reality by creating an organization that is truly data-driven. ML models powering AI use cases are becoming more and more ubiquitous in a variety of environments, especially at industrial organizations adopting Industry 4.0 technologies.
Ripple Engineering
AUGUST 19, 2021
Hello, XRP In early October, Xpring launched Xpring SDK , a set of language specific libraries which made it easy to interact with XRP. As the creator of Xpring SDK, I wanted to take an opportunity to provide some insight into what Xpring has released, our future plans, and the technical architecture of our SDKs. First, a bit of background. The XRP Ledger is a sophisticated, yet complex, piece of software that runs in the context of a distributed system.
Speaker: Donna Laquidara-Carr, PhD, LEED AP, Industry Insights Research Director at Dodge Construction Network
In today’s construction market, owners, construction managers, and contractors must navigate increasing challenges, from cost management to project delays. Fortunately, digital tools now offer valuable insights to help mitigate these risks. However, the sheer volume of tools and the complexity of leveraging their data effectively can be daunting. That’s where data-driven construction comes in.
DataKitchen
AUGUST 18, 2021
DataKitchen's DataOps Engineers Priyanjna Sharma & Chip Bloche discuss what DataOps Engineering entails, key skills required & when to add one to your data team. The post A Day in the Life of a DataOps Engineer first appeared on DataKitchen.
Rock the JVM
AUGUST 18, 2021
Discover how to leverage ZIO to seamlessly interact with Apache Kafka: the proven, scalable solution for reliable communication between distributed application components
Teradata
AUGUST 19, 2021
The rich treasure trove of Teclo-derived data, specifically digital payments data, can be utilized to influence and predict business outcomes. Find out more.
Preset
AUGUST 17, 2021
Preset Cloud is now generally available! Preset Cloud is a modern data exploration and visualization platform powered by Apache Superset.
Speaker: Evelyn Chou
Choosing the right business intelligence (BI) platform can feel like navigating a maze of features, promises, and technical jargon. With so many options available, how can you ensure you’re making the right decision for your organization’s unique needs? 🤔 This webinar brings together expert insights to break down the complexities of BI solution vetting.
ProjectPro
AUGUST 18, 2021
Making a career transition into data analysis and visualization? Ace your next data analyst interview with these Tableau interview questions and answers that cover all the important topics and concepts in Tableau. Tableau is one of the most significant data visualization and business intelligence tools used by organizations across industries. Almost all fortune 500 companies use this tool to get better insights and work according to the market demands.
Monte Carlo
AUGUST 17, 2021
It would be an understatement to say your company is bullish on data. Your CEO can’t stop talking about her new Tableau dashboard, a report that tells which of your products are “stickiest” with customers. It didn’t take much convincing to sell your CTO on Snowflake. And your entire data engineering team is all in on this “data as code” movement. The flip side of this data-driven coin: your stakeholders (CEO and CTO included) ping you nearly every other hour to ask you: “is my data up-to-date?
Teradata
AUGUST 15, 2021
Tableau Server is now integrated out-of-the-box with Vantage Trial as part of the free 30-day experience. Find out more!
RudderStack
AUGUST 18, 2021
We recently helped a customer migrate from Segment to RudderStack, and the project included transitioning Personas functionality to RudderStack Reverse ETL.
Speaker: Aindra Misra, Senior Manager, Product Management (Data, ML, and Cloud Infrastructure) at BILL
Join us for an insightful webinar that explores the critical intersection of data privacy and AI governance. In today’s rapidly evolving tech landscape, building robust governance frameworks is essential to fostering innovation while staying compliant with regulations. Our expert speaker, Aindra Misra, will guide you through best practices for ensuring data protection while leveraging AI capabilities.
Expert insights. Personalized for you.
130 
Let's personalize your content