KDnuggets
JANUARY 24, 2022
What does the data reveal if we ask: "What are the 10 Best Python Courses?". Collecting almost all of the courses from top platforms shows there are plenty to choose from, with over 3000 offerings. This article summarizes my analysis and presents the top three courses.
Simon Späti
JANUARY 22, 2022
Image by Mohammad Bagher Adib Behrooz on Unsplash Why GraphQL for data engineers, you might ask? GraphQL solved the problem of providing a distinct interface for each client by unifying it to a single API for all clients such as web, mobile, web apps. The same challenge we’re now facing in the data world, where we integrate multiple clients with numerous backend systems.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Confluent
JANUARY 11, 2022
Whether you’re a seasoned Apache Kafka® developer or just getting started you’re likely to hit a snag at some point or another—either in configuring and understanding your clients or setting […].
Marc Lamberti
JANUARY 9, 2022
Airflow TaskGroups have been introduced to make your DAG visually cleaner and easier to read. They are meant to replace SubDAGs which was the historic way of grouping your tasks. The problem with SubDAGs is that they are much more than that. They bring a lot of complexity as you need to create a DAG in a DAG, import the SubDagOperator which is in fact a sensor, define the parameters properly, and so on.
Speaker: Tamara Fingerlin, Developer Advocate
Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.
Cloudera
JANUARY 28, 2022
As I meet with our customers, there are always a range of discussions regarding the use of the cloud for financial services data and analytics. Customers vary widely on the topic of public cloud – what data sources, what use cases are right for public cloud deployments – beyond sandbox, experimentation efforts. Private cloud continues to gain traction with firms realizing the benefits of greater flexibility and dynamic scalability.
DataKitchen
JANUARY 17, 2022
The post A busy year ahead in low-code and no-code development first appeared on DataKitchen.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Teradata
JANUARY 25, 2022
The use of integrated data to restore customer confidence will be big in 2022. Building a customer insights foundation should be high on the to-do list for retail & CPG businesses this year.
Confluent
JANUARY 5, 2022
Chances are your business is migrating to the cloud. But if you operate business applications in an on-premises datacenter, you know firsthand that the journey to the cloud is fraught […].
Data Engineering Podcast
JANUARY 30, 2022
Summary Pandas is a powerful tool for cleaning, transforming, manipulating, or enriching data, among many other potential uses. As a result it has become a standard tool for data engineers for a wide range of applications. Matt Harrison is a Python expert with a long history of working with data who now spends his time on consulting and training. He recently wrote a book on effective patterns for Pandas code, and in this episode he shares advice on how to write efficient data processing routines
Cloudera
JANUARY 21, 2022
This blog will summarise the security architecture of a CDP Private Cloud Base cluster. The architecture reflects the four pillars of security engineering best practice, Perimeter, Data, Access and Visibility. The release of CDP Private Cloud Base has seen a number of significant enhancements to the security architecture including: Apache Ranger for security policy management.
Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage
There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.
U-Next
JANUARY 17, 2022
At a recent training for fresher hire as part of an MNC’s analytics training program, my colleague Dr. Chetana highlighted that only 10% of the hires were women. TrustRadius reported that in 2021, 72% of women in tech are outnumbered by men in business meetings by at least a 2:1 ratio. Women are less than 1/3rd of the employees in many tech companies.
KDnuggets
JANUARY 19, 2022
The landscape of programming languages is rich and expanding, which can make it tricky to focus on just one or another for your career. We highlight some of the most popular languages that are modern, widely used, and come with loads of packages or libraries that will help you be more productive and efficient in your work.
DataKitchen
JANUARY 7, 2022
The post Trend-Setting Products in Data and Information Management for 2022 first appeared on DataKitchen.
Confluent
JANUARY 24, 2022
On behalf of the Apache Kafka® community, it is my pleasure to announce the release of Apache Kafka 3.1.0. The 3.1.0 release contains many improvements and new features. We’ll highlight […].
Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives
Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri
Data Engineering Podcast
JANUARY 30, 2022
Summary The Data Engineering Podcast has been going for five years now and has included conversations and interviews with a huge number of guests, covering a broad range of topics. In addition to that, the host curated the essays contained in the book "97 Things Every Data Engineer Should Know", using the knowledge and context gained from running the show to inform the selection process.
Netflix Tech
JANUARY 25, 2022
Martin Tingley with Wenjing Zheng , Simon Ejdemyr , Stephanie Lane , Colin McFarland , Mihir Tendulkar , and Travis Brooks This is the last post in an overview series on experimentation at Netflix. Need to catch up? Earlier posts covered the basics of A/B tests ( Part 1 and Part 2 ), core statistical concepts ( Part 3 and Part 4 ), how to build confidence in a decision ( Part 5 ), and the the role of Experimentation and A/B testing within the larger Data Science and Engineering organization at N
U-Next
JANUARY 17, 2022
A couple of decades ago, ‘Data’ was analyzed manually. With the advent of data management tools, we were able to computerize the same ‘Data’ for deeper analysis. Thus the trend of driving business decisions via insights drawn from data sets has never been old. However, with the availability of tools to manage and analyze data, the quantity and the quality of data analyzed have improved drastically, thereby increasing the accuracy and the efficacy of data-driven decisions.
KDnuggets
JANUARY 19, 2022
Here are some of the data science technologies needed to build a comprehensive and smooth supply chain pipeline.
Advertisement
With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to: Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to Write DAGs that adapt to your data at runtime and set up alerts and notifications Scale you
DataKitchen
JANUARY 3, 2022
The post Data Science and AI Predictions for 2022 first appeared on DataKitchen.
Confluent
JANUARY 6, 2022
While Self-Balancing Clusters (SBC) perform effectively in balancing Apache Kafka® clusters, one of the common themes we hear from our users is that they would love some visibility into the […].
Data Engineering Podcast
JANUARY 23, 2022
Summary Data platforms are exemplified by a complex set of connections that are subject to a set of constantly evolving requirements. In order to make this a tractable problem it is necessary to define boundaries for communication between concerns, which brings with it the need to establish interface contracts for communicating across those boundaries.
Netflix Tech
JANUARY 11, 2022
Martin Tingley with Wenjing Zheng , Simon Ejdemyr , Stephanie Lane , Colin McFarland , Andy Rhines , Sophia Liu , Mihir Tendulkar , Kevin Mercurio , Veronica Hannan , Ting-Po Lee Earlier posts in this series covered the basics of A/B tests ( Part 1 and Part 2 ), core statistical concepts ( Part 3 and Part 4 ), and how to build confidence in decisions based on A/B test results ( Part 5 ).
Advertisement
Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.
Teradata
JANUARY 10, 2022
There is a lot of hype today around data sharing and the value it brings to your business. But what exactly is data sharing, and why should you and your company care? Find out more.
KDnuggets
JANUARY 24, 2022
There are many great boosting Python libraries for data scientists to reap the benefits of. In this article, the author discusses LightGBM benefits and how they are specific to your data science job.
DataKitchen
JANUARY 3, 2022
Business analysts often find themselves in a no-win situation with constraints imposed from all sides. Their business unit colleagues ask an endless stream of urgent questions that require analytic insights. Business analysts must rapidly deliver value and simultaneously manage fragile and error-prone analytics production pipelines. Data tables from IT and other data sources require a large amount of repetitive, manual work to be used in analytics.
Confluent
JANUARY 12, 2022
We’re pleased to announce ksqlDB 0.23.1! This release allows you to now perform pull queries on streams, which makes it much easier to find a given record in a topic. […].
Advertisement
Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?
Data Engineering Podcast
JANUARY 23, 2022
Summary Data engineering is a relatively young and rapidly expanding field, with practitioners having a wide array of experiences as they navigate their careers. Ashish Mrig currently leads the data analytics platform for Wayfair, as well as running a local data engineering meetup. In this episode he shares his career journey, the challenges related to management of data professionals, and the platform design that he and his team have built to power analytics at a large company.
Netflix Tech
JANUARY 13, 2022
By Vikram Srivastava and Marcelo Mayworm Netflix has one of the most complex data platforms in the cloud on which our data scientists and engineers run batch and streaming workloads. As our subscribers grow worldwide and Netflix enters the world of gaming , the number of batch workflows and real-time data pipelines increases rapidly. The data platform is built on top of several distributed systems, and due to the inherent nature of these systems, it is inevitable that these workloads run into fa
Cloudera
JANUARY 18, 2022
Each year, the Cloudera Data Impact Awards recognize organizations that have accomplished amazing things with innovative data solutions. . For 2021, the awards will include a new category: People First. Entrants in this category were asked to demonstrate how they have addressed the world’s “most difficult workplace and societal challenges” with solutions aimed at transforming work culture and society as a whole.
KDnuggets
JANUARY 13, 2022
Learn how to query your Pandas DataFrames using the standard SQL SELECT statement, seamlessly from within your Python code.
Speaker: Tamara Fingerlin, Developer Advocate
In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!
Expert insights. Personalized for you.
160 
Let's personalize your content