Uber Engineering
APRIL 12, 2021
Introduction. Uber’s many software systems require a high volume of changes every day. Because of our systems’ size and complexity, it is a significant challenge to implement these changes without unintended consequences, ultimately slowing down developer productivity. Flipr is a … The post Flipr: Making Changes Quickly and Safely at Scale appeared first on Uber Engineering Blog.
Confluent
APRIL 14, 2021
Updating a fundamental paradigm in your React app can be as easy as search and replace, or at other times, as difficult as convincing your entire frontend engineering to buy […].
DataKitchen
APRIL 13, 2021
Download the 2021 DataOps Vendor Landscape here. Read the complete blog below for a more detailed description of the vendors and their capabilities. DataOps is a hot topic in 2021. This is not surprising given that DataOps enables enterprise data teams to generate significant business value from their data. Companies that implement DataOps find that they are able to reduce cycle times from weeks (or months) to days, virtually eliminate data errors, increase collaboration, and dramatically imp
Data Engineering Podcast
APRIL 12, 2021
Summary "Business as usual" is changing, with more companies investing in data as a first class concern. As a result, the data team is growing and introducing more specialized roles. In this episode Josh Benamram, CEO and co-founder of Databand, describes the motivations for these emerging roles, how these positions affect the team dynamics, and the types of visibility that they need into the data platform to do their jobs effectively.
Advertisement
Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.
Cloudera
APRIL 13, 2021
You’ve probably heard it more than once: Machine learning (ML) can take your digital transformation to another level. It’s a pie-in-the-sky statement that sounds great, right? And while you’d be forgiven for thinking that it might sound too good to be true, operational ML is , in fact, achievable and sustainable. You can get the very kind of ML you need to increase revenue and lower costs.
Confluent
APRIL 13, 2021
Morgan Stanley uses Apache Kafka® to publish market data to internal clients and to persist it for replay purposes. We started out using librdkafka’s C++ API, which maintains C++98 compatibility. […].
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Retail Insight
APRIL 15, 2021
The last year has seen dramatic shifts in food and grocery retail. And the rate of change show s no signs of slowing down. However, there are some common truths that exist no matter the time or channel – features that usually indicate success.
Cloudera
APRIL 14, 2021
What is Cloudera Data Engineering (CDE) ? Cloudera Data Engineering is a serverless service for Cloudera Data Platform (CDP) that allows you to submit jobs to auto-scaling virtual clusters. CDE enables you to spend more time on your applications, and less time on infrastructure. CDE allows you to create, manage, and schedule Apache Spark jobs without the overhead of creating and maintaining Spark clusters.
Confluent
APRIL 15, 2021
The transition from a passive event stream to an active component like a workflow engine is very interesting. It raises a lot of questions about idempotency, scalability, and the capability […].
Teradata
APRIL 15, 2021
Many Teradata customers are interested in integrating Vantage with AWS First Party Services. This guide will help you to connect Vantage to Salesforce using Amazon Appflow.
Speaker: Tamara Fingerlin, Developer Advocate
In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!
Airbnb Tech
APRIL 13, 2021
The path to cloud efficiency begins with a cost data foundation by Anna Matlin and Tamar Eterman Introduction Business profitability and sustainability are powerful reasons to invest in infrastructure efficiency, but it is easy to feel lost about how to actually reduce costs. A foundation of robust and actionable data is essential for a successful efficiency program.
Cloudera
APRIL 13, 2021
What’s the fastest and easiest path towards powerful cloud-native analytics that are secure and cost-efficient? In our humble opinion, we believe that’s Cloudera Data Platform (CDP). And sure, we’re a little biased—but only because we’ve seen firsthand how CDP helps our customers realize the full benefits of public cloud. . Organizations from across the globe and virtually every industry have used CDP to generate new revenue streams, decrease operational costs, and mitigate risks.
Grouparoo
APRIL 12, 2021
Grouparoo supports syncing data to an ever-growing number of destinations. While building these integrations and talking to our users, we have found it's important to be intentional about how exactly data syncing to these destinations is performed. For example, our Salesforce data integration has a "Sync Mode" option that allows you to control whether contacts will be created, deleted or only updated.
Teradata
APRIL 13, 2021
As finance teams evolve into the providers of strategic insights, leveraging analytics will result in a new user base, new insights & reposition the CFO to a predictor of the future.
Advertisement
Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.
Dataquest
APRIL 12, 2021
Data engineer, data analyst, and data scientist — these are job titles you’ll often hear mentioned together when people are talking about the fast-growing field of data science. There are plenty of other job titles in data science and data analytics too. But here, we’re going to talk about: The “big three” roles (data analyst, data scientist, and data engineer) How they differ from each other Which role is best for you Although precisely how these roles are defined can va
Cloudera
APRIL 15, 2021
According to IDG, when customers consider updating to the latest release of a product, they expect new features, enhanced security, and better performance, but increasingly want a more streamlined upgrade process. With each new release of CDP Private Cloud, this is exactly what we strive to deliver. Along with a host of new features and capabilities, we are improving the upgrade process to be as painless as possible.
ProjectPro
APRIL 14, 2021
We decided to write this content piece because, in the past few months, many aspiring data science professionals asked our project advisors these questions on getting started with a data science career - “I want to learn data science but I don’t know where to start.” “I know python for web development but how to learn python for data science.” “What is the best way to learn data science?
Teradata
APRIL 11, 2021
Data mesh is gaining popularity as an approach to enterprise data architecture. While it embodies great ideas, there are a few principles that could easily be lost when applying it in real life.
Advertisement
Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?
Rockset
APRIL 15, 2021
Relational databases today are widely known to be suboptimal for supporting high-scale analytical use cases, and are all but certain to run into issues as your production data size and query volume grow. This has been by far one of the most well-known weaknesses of relational databases for much of the past decade, and has led to surges in popularity of several new classes of databases such as NoSQL and NewSQL – each with their own sets of tradeoffs and drawbacks.
Cloudera
APRIL 10, 2021
When working on complex, or rigorous enterprise machine learning projects, Data Scientists and Machine Learning Engineers experience various degrees of processing lag training models at scale. While model training on small data can typically take minutes, doing the same on large volumes of data can take hours or even weeks. To overcome this, practitioners often turn to NVIDIA GPUs to accelerate machine learning and deep learning workloads. .
ProjectPro
APRIL 12, 2021
Deep learning job interviews. A necessary evil. Most beginners in the industry break out in a cold sweat at the mere thought of a machine learning or a deep learning job interview. How do I prepare for my upcoming deep learning job interview? What kind of deep learning interview questions they are going to ask me? What questions should I ask them? These are just a few thoughts that run through the mind of any interviewee.
Grouparoo
APRIL 12, 2021
Grouparoo is a self-hosted product, so we are always looking for the simplest ways to help our customers run the application. A new member of the Google Cloud Platform (GCP) family is Google Cloud Run - which is the closest Google has come yet to a Heroku-like "Git-Ops" way to deploy your applications. It handles load balancing, scaling, and more for you and is a really compelling product.
Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali
As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.
Zalando Engineering
APRIL 12, 2021
GraphQL Errors GraphQL is an excellent language for writing data requirements in a declarative fashion. It gives us a clear and well-defined concept of nullability constraints and error propagation. In this post, let's discuss how GraphQL lacks in certain places regarding errors and how we can model those errors to fit some of our use-cases. Before we dive into the topic, let's understand how GraphQL currently treats and handles errors.
Cloudera
APRIL 13, 2021
Meet Gavin Welch, Cloudera’s Partner Manager of the Year! Gavin leads the hardware alliances team and has made big strides in Cloudera’s relationship with Dell/EMC , launched a new strategic partnership with NVIDIA to help GPU acceleration for CDP-PvC Base, and recently became a first-time dad. Gavin’s the all-star quarterback everyone wants on their team, the guy you always see active on Slack in the middle of the night, and the same guy that’s chipper after a long call with legal, trying to ge
Rockset
APRIL 12, 2021
Construction projects are hives of constant activity, sustained by steady incoming streams of building materials. Think construction logistics and one pictures a flow of trucks transporting concrete and other necessary materials from suppliers to construction sites. Yet for every physical delivery made, many more exchanges of data occur in the background in order to seamlessly orchestrate supply chain operations.
Cloudera
APRIL 12, 2021
The United States Veterans Administration (VA) over the last decade underwent a massive enterprise-wide IT transformation, eliminating its fragmented shadow IT and adopting a centralized system capable of supporting the agency’s 400,000 employees and more effectively utilizing its $240 billion-plus annual budget. The result: A more reliable and modern IT environment that improves access, availability, and user experience -ultimately supporting the VA mission more effectively.
Speaker: Nikhil Joshi, Founder & President of Snic Solutions
Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.
Cloudera
APRIL 14, 2021
The blog “ Migrating Apache NiFi Flows from HDF to CFM with Zero Downtime ” detailed how many common NiFi dataflows can be easily migrated when the Hortonworks DataFlow and Cloudera Flow Management clusters are running side-by-side. But what if you lack the resources to run multiple NiFi clusters concurrently? Not a problem. Rolling migration — decommissioning your HDF NiFi nodes and recycling them for use in a CFM NiFi cluster — is an alternative solution to migrate these NiFi dataflows with t
Expert insights. Personalized for you.
142 
Let's personalize your content