Confluent
SEPTEMBER 16, 2020
Near-real-time insights have become a de facto requirement for Azure use cases involving scalable log analytics, time series analytics, and IoT/telemetry analytics. Azure Data Explorer (also called Kusto) is the […].
Cloudera
SEPTEMBER 14, 2020
Everybody needs more data and more analytics, with so many different and sometimes often conflicting needs. Data engineers need batch resources, while data scientists need to quickly onboard ephemeral users. Data architects deal with constantly evolving workloads and business analysts must balance the urgency and importance of a concurrent user population that continues to grow.
Data Engineering Podcast
SEPTEMBER 14, 2020
Summary In memory computing provides significant performance benefits, but brings along challenges for managing failures and scaling up. Hazelcast is a platform for managing stateful in-memory storage and computation across a distributed cluster of commodity hardware. On top of this foundation, the Hazelcast team has also built a streaming platform for reliable high throughput data transmission.
Netflix Tech
SEPTEMBER 18, 2020
Analytics at Netflix: Who We Are and What We Do An Introduction to Analytics and Visualization Engineering at Netflix by Molly Jackman & Meghana Reddy Explained: Season 1 (Photo Credit: Netflix) Across nearly every industry, there is recognition that data analytics is key to driving informed business decision-making. But there is far less agreement on what that term “data analytics” actually means?
Advertisement
Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.
Confluent
SEPTEMBER 18, 2020
Since the MongoDB Atlas source and sink became available in Confluent Cloud, we’ve received many questions around how to set up these connectors in a secure environment. By default, MongoDB […].
Cloudera
SEPTEMBER 17, 2020
For enterprise organizations, managing and operationalizing increasingly complex data across the business has presented a significant challenge for staying competitive in analytic and data science driven markets. With growing disparate data across everything from edge devices to individual lines of business needing to be consolidated, curated, and delivered for downstream consumption, it’s no wonder that data engineering has become the most in-demand role across businesses — growing at an estima
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Netflix Tech
SEPTEMBER 18, 2020
Part of our series on who works in Analytics at Netflix?—?and what the role entails by Julie Beckley & Chris Pham This Q&A provides insights into the diverse set of skills, projects, and culture within Data Science and Engineering (DSE) at Netflix through the eyes of two team members: Chris Pham and Julie Beckley. Photo from a team curling offsite?
Confluent
SEPTEMBER 15, 2020
Are you looking for a way to run AWS services on premises in your own datacenter? I am excited to share today that we have completed validation of support for […].
Cloudera
SEPTEMBER 17, 2020
Announcing the finalists of the Data Impact Awards is always a highlight in our annual Cloudera calendar, and this year is no different. The 2020 entrants have shown incredible data-driven innovation, problem-solving ability and have proven real-world impact. . Our independent judges certainly had their jobs cut out for them, as they were faced with an overwhelming number of outstanding entries.
Data Science Blog: Data Engineering
SEPTEMBER 18, 2020
The Data Science and Engineering Process in PLM. Huge opportunities for digital products are accompanied by huge risks Digitalization is about to profoundly change the way we live and work. The increasing availability of data combined with growing storage capacities and computing power make it possible to create data-based products, services, and customer specific solutions to create insight with value for the business.
Speaker: Tamara Fingerlin, Developer Advocate
In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!
Rock the JVM
SEPTEMBER 17, 2020
Inspect, extract, and modify deeply nested data structures in Scala with ease: discover a powerful method to handle complex data effortlessly
Teradata
SEPTEMBER 17, 2020
Hispanic Heritage Month not only promotes the rich culture & heritage that so many Americans share, it sheds a distinct light on colleagues & friends. Read more from our colleague, Crystal Diaz.
Cloudera
SEPTEMBER 18, 2020
Supply chain redesign has become an area of critical focus as businesses try to ensure supply chains aren’t overly reliant on any one constituent part. . In Part One of our round-table discussion with Michael Ger, Managing Director of Manufacturing and Automotive, and Brent Biddulph, Global Managing Director, Retail and Consumer Goods at Cloudera they talked about the direct impact of COVID 19 on the retail and manufacturing sectors with Vijay Raja, Director of Industry & Solutions Marketing
Grouparoo
SEPTEMBER 16, 2020
Many businesses have built great analytics products to help with tracking the actions your users are taking in your product ( Mixpanel , Pendo , and Amplitude , to name a few). These products use an events-based data model where they track user behavior, usually client-side, so you can understand and visualize behavior like page views and button clicks.
Advertisement
Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.
Monte Carlo
SEPTEMBER 16, 2020
To keep pace with data’s lightning innovation speed, data engineers need to invest not only in the latest data modeling and analytics tools, but also technologies that can increase data accuracy and prevent broken ETL pipelines. The solution? Data observability tools , the next frontier of data engineering and a pillar of the emerging data reliability category.
Teradata
SEPTEMBER 16, 2020
If you’re responsible for data strategy in a large organization, there’s a good chance you’ve got a data mess on your hands. So what do you do with it? Read more.
Cloudera
SEPTEMBER 17, 2020
Today’s data tool challenges. A North American telecom company struggled for years trying to react quickly enough to new categories and new levels of spam texts and calls. They also did not have a good way to know when and why they would need additional capacity on their own, or any other telecom company’s networks. By enabling their event analysts to monitor and analyze events in real time, as well as directly in their data visualization tool, and also rate and give feedback to the system
Rock the JVM
SEPTEMBER 15, 2020
Akka HTTP needs JSON like humans need water: discover how to integrate Spray-Json, circe, and Jackson into Akka HTTP
Advertisement
Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?
Monte Carlo
SEPTEMBER 16, 2020
We’re excited to share that Monte Carlo has raised $16M in funding to pioneer the Data Reliability category. Our Series A was led by Accel , with participation from GGV Capital , and enables us to pursue our mission of accelerating the world’s adoption of data by reducing Data Downtime. Other angel investors include DJ Patil , the former Chief Data Scientist for the U.S. as well as top executives from Cloudera, eBay, Google and VMWare.
A Cloud Guru: Data Engineering
SEPTEMBER 14, 2020
You can complete the project requirements by yourself or in collaboration with others. Feel free to ask questions in the discussion forum or on social media using the #CloudGuruChallenge hashtag! The post #CloudGuruChallenge – Event-Driven Python on AWS appeared first on A Cloud Guru.
Cloudera
SEPTEMBER 15, 2020
Cloudera Data Platform 7.2.1 introduces fine-grained authorization for access to Azure Data Lake Storage using Apache Ranger policies. Cloudera and Microsoft have been working together closely on this integration, which greatly simplifies the security administration of access to ADLS-Gen2 cloud storage. Apache Ranger provides a centralized console to manage authorization and view audits of access to resources in a large number of services including Apache Hadoop’s HDFS, Apache Hive, Apache HBase
Teradata
SEPTEMBER 14, 2020
Learn how Teradata Vantage was used for a leading Turkish bank to predict credit scores for customers in real time and to make near-immediate decisions on their loan applications.
Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali
As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.
Grouparoo
SEPTEMBER 13, 2020
In Brian’s post, Building a Sync Engine , he talks about the value of using a High Water Mark to keep track of the latest bit of data you’ve imported. This approach is often a better pattern than using Limit and Offset , especially when the underlying data might be changing. In this post, I’m gong to dive even deeper into this topic, and suggest that you should be storing you High Water Marks as strings whenever possible.
Rockset
SEPTEMBER 15, 2020
YADB (Yet Another Database Benchmark) The world has a plethora of database benchmarks, starting with the Wisconsin Benchmark which is my favorite. Firstly, that benchmark was from Dr David Dewitt, who taught me Database Internals when I was a graduate student at University of Wisconsin. Secondly, it is probably the earliest conference paper (circa 1983) that I ever read.
Cloudera
SEPTEMBER 15, 2020
For some, this may look like a new category at this year’s Data Impact Awards. However, the Enterprise Data Cloud category marks the evolution of what was once the Data Anywhere category. The main reason for this change is that this title better represents the move that our customers are making; away from acknowledging the ability to have data ‘anywhere’.
Expert insights. Personalized for you.
139 
Let's personalize your content