Confluent
FEBRUARY 18, 2021
Soon, Apache Kafka® will no longer need ZooKeeper! With KIP-500, Kafka will include its own built-in consensus layer, removing the ZooKeeper dependency altogether. The next big milestone in this effort […].
François Nguyen
FEBRUARY 14, 2021
Let’s start with maybe the best definition you can find on Devops (credit to AWS ) : “DevOps is the combination of cultural philosophies , practices , and tools that increases an organization’s ability to deliver applications and services at high velocity : evolving and improving products at a faster pace than organizations using traditional software development and infrastructure management processes.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Cloudera
FEBRUARY 18, 2021
Almost a year into the pandemic, the accelerated digital transformation has begun to feel less abrupt and more sustained. 2021 looks likely to be defined by a new phase: Thriving on digital transformation, rather than just surviving through it. . We’ve written about the changes forced on the traditionally risk-averse insurance industry by COVID-19. In 2021, with the crisis hopefully fading, insurance will have time to evaluate the changes made in 2020, assessing what worked and what didn’t
Data Engineering Podcast
FEBRUARY 15, 2021
Summary Every business aims to be data driven, but not all of them succeed in that effort. In order to be able to truly derive insights from the data that an organization collects, there are certain foundational capabilities that they need to have capacity for. In order to help more businesses build those foundations, Tarush Aggarwal created 5xData, offering collaborative workshops to assist in setting up the technical and organizational systems that are necessary to succeed.
Speaker: Tamara Fingerlin, Developer Advocate
Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.
Confluent
FEBRUARY 16, 2021
One of the most common relational database systems that connects to Apache Kafka® is Oracle, which often holds highly critical enterprise transaction workloads. While Oracle Database (DB) excels at many […].
Start Data Engineering
FEBRUARY 13, 2021
Why data exploration Apache Superset architecture Setup Prerequisites Seed data Using Apache Superset 1. Connecting to a data warehouse 2. Querying data in SQL Lab 3. Creating a chart 4. Creating a dashboard Pros and Cons Pros Cons Conclusion Why data exploration In most companies the end users of a data warehouse are analysts, data scientists and business people.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
|
Cloudera
FEBRUARY 16, 2021
In the previous blog post , we looked at some of the application development concepts for the Cloudera Operational Database (COD). In this blog post, we’ll see how you can use other CDP services with COD. COD is an operational database-as-a-service that brings ease of use and flexibility to Apache HBase. Cloudera Operational Database enables developers to quickly build future-proof applications that are architected to handle data evolution.
Confluent
FEBRUARY 19, 2021
One of the most highly requested enhancements to ksqlDB is here! Apache Kafka® messages may contain data in message keys as well as message values. Until now, ksqlDB could only […].
Rock the JVM
FEBRUARY 18, 2021
Discover a powerful technique for eliminating hard-to-trace bugs with ad-hoc type definitions: learn how Scala 2's newtypes and Scala 3's opaque types can enhance your code's safety and maintainability
Rockset
FEBRUARY 18, 2021
Rockset released new numbers for the Star Schema Benchmark in April 2022. Learn how Rockset is 1.67 times faster than ClickHouse and 1.12 times faster than Druid in the latest performance blog post. Real-time analytics is all about deriving insights and taking actions as soon as data is produced. When broken down into its core requirements, real-time analytics means two things: access to fresh data and fast responses to queries.
Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage
There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.
Cloudera
FEBRUARY 17, 2021
Needless to say, 2020 was an unforgettable year in a lot of ways and we were all happy to say goodbye to it. The pandemic has ushered in new ways of how we conduct businesses, remote work cultures, telehealth, grocery/food deliveries, etc. While certain industries were hard-hit by this change, most of the businesses were able to adapt, pivot, and take on this adversity in their stride.
Confluent
FEBRUARY 17, 2021
We’re pleased to announce ksqlDB 0.15, our first release of 2021! This version adds rich support for message key columns and long-awaited improvement to interactive development with the command line […].
Rock the JVM
FEBRUARY 18, 2021
Discover a powerful technique for eliminating hard-to-trace bugs with ad-hoc type definitions: learn how Scala 2's newtypes and Scala 3's opaque types can enhance your code's safety and maintainability
Teradata
FEBRUARY 18, 2021
Cloud tech can be empowering for end users, but without effective data governance, one risks sliding into a morass of inconsistent data, excessive rework & slow projects.
Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives
Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri
Data Council
FEBRUARY 17, 2021
Here's our February 2021 roundup of links from across the web that we picked for you: 1. dbt at Shopify (Data Engineering Podcast) The Data Engineering Podcast recently featured a very interesting discussion about dbt at Shopify. Engineering manager Zeeshan Qureshi and senior data engineer Michelle Ark explained how dbt answered Shopify’s need for an SQL-based solution that its data scientists could use autonomously.
Grouparoo
FEBRUARY 17, 2021
We have a feature on this site that is using sessionStorage to send analytics data we want to capture. Being that it's an important feature, we should write test(s) to cover the use case(s), right? Okay, fine. Let's do it! This website is a Next.js application that uses Jest as our test runner and Selenium WebDriver for integration test help.
Silectis
FEBRUARY 17, 2021
In this blog post, we’ll walk you through how data science and data engineering are complementary disciplines. We’ll also delineate a third category: data analysis. We’ll explore how both data engineering and data science should be marshaled to make better decisions. Organizations often struggle to strike the right balance between engineering, analysis, and data science skills within data teams.
DataKitchen
FEBRUARY 17, 2021
The post 2021: The Year Real Time Gets Real first appeared on DataKitchen.
Advertisement
With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to: Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to Write DAGs that adapt to your data at runtime and set up alerts and notifications Scale you
Afterpay Tech
FEBRUARY 16, 2021
By: Maggie Luo I think we can all agree that 2020 was a year of many firsts. Maybe for you, it was your first time spending most of your time at home with family in years. Or maybe it was your first time voting in the election, downloading TikTok, or making Dalgona coffee (we all remember that phase of quarantine, don’t we?) For me, last year was filled with many milestones: graduating from UC Berkeley as a first-generation college student, moving into an apartment in San Francisco with m
Grouparoo
FEBRUARY 16, 2021
We have a form on our meet page (which, BTW, we'd love you to fill out because we like meeting new people). In addition to the data input from the user, we also wanted to capture how that user got to the page. That helps us determine which of our content is most effective in getting website visitors to take action. The document.referrer Attempt My gut was to start with document.referrer.
Teradata
FEBRUARY 16, 2021
Addressing the rapid evolution of fraud and risk is an imperative for payments players. Machine learning and advanced analytics can help. Find out more.
DataKitchen
FEBRUARY 19, 2021
The post Tame DataOps System Complexity with a DataOps Platform first appeared on DataKitchen.
Speaker: Tamara Fingerlin, Developer Advocate
In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!
Rock the JVM
FEBRUARY 17, 2021
Although frowned upon by FP purists, creating and managing mutable data structures is important in any language: Explore Scala's first-class mutability features
Zalando Engineering
FEBRUARY 15, 2021
Customers love the freedom to try the clothes first and pay later. We’d love to offer everyone the convenience of deferred payment. However, fraudsters exploit this to acquire goods they never pay for. The better we know the probability of an order defaulting, the better we can steer the risk and offer the convenience of deferred payment to more customers.
ProjectPro
FEBRUARY 19, 2021
Regression analysis is the favorite of data science and machine learning practitioners as it provides a great level of flexibility and reliability making it an ideal choice for analyzing different situations like - Do educational degrees and IQ affect salary? Is consuming caffeine and smoking-related to mortality risk? Do regular workouts and a dietary plan affect weight?
Expert insights. Personalized for you.
145 
Let's personalize your content