Confluent
SEPTEMBER 1, 2020
If you are taking your first steps with Apache Kafka®, looking at a test environment for your client application, or building a Kafka demo, there are two “easy button” paths […].
Team Data Science
SEPTEMBER 2, 2020
"Where can I start if I want to become a Data Engineer?" This is a question I have heard many times before. My answer to it is actually always the same: Start doing a Data Engineering project! Choose a tool Your first step here should be to select a tool. Then start with that tool and then build the whole thing up. So you get some data and then start with a tool.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Cloudera
AUGUST 31, 2020
From leading banks, and insurance organizations to some of the largest telcos, manufacturers, retailers, healthcare and pharma, organizations across diverse verticals lead the way with real-time data and streaming analytics. These businesses use data-fueled insights to enhance the customer experience, reduce costs, and increase revenues. And Cloudera is at the heart of enabling these real-time data driven transformations. .
Data Engineering Podcast
AUGUST 31, 2020
Summary Data warehouse technology has been around for decades and has gone through several generational shifts in that time. The current trends in data warehousing are oriented around cloud native architectures that take advantage of dynamic scaling and the separation of compute and storage. Firebolt is taking that a step further with a core focus on speed and interactivity.
Advertisement
In Airflow, DAGs (your data pipelines) support nearly every use case. As these workflows grow in complexity and scale, efficiently identifying and resolving issues becomes a critical skill for every data engineer. This is a comprehensive guide with best practices and examples to debugging Airflow DAGs. You’ll learn how to: Create a standardized process for debugging to quickly diagnose errors in your DAGs Identify common issues with DAGs, tasks, and connections Distinguish between Airflow-relate
Teradata
SEPTEMBER 3, 2020
Taking the definition of physical gravity & extending it to data analytics, we explore the opportunities to combine data gravity with analytic processing, at scale with Vantage.
Team Data Science
SEPTEMBER 1, 2020
In which regions or countries is there a boom in the field of Data Sciences and thus a large number of jobs? This is a very interesting question, which newcomers or graduates often ask themselves. Maybe you have already asked yourself this question? The USA as an advanced country Companies in the USA are obviously very, very advanced with Data Science.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
|
Confluent
SEPTEMBER 3, 2020
Hybrid cloud architecture and accelerated cloud migrations are becoming the norm rather than the exception, as our increasingly digital world introduces certain challenges along the way, including modernizing existing application/architecture, […].
Netflix Tech
SEPTEMBER 1, 2020
Kamer Toker-Yildiz , Colin McFarland , Julia Glick At Netflix, when we can’t run A/B experiments we run quasi experiments ! We run quasi experiments with various objectives such as non-member experiments focusing on acquisition, member experiments focusing on member engagement, or video streaming experiments focusing on content delivery. Consolidating on one methodology could be a challenge, as we may face different design or data constraints or optimization goals.
Team Data Science
SEPTEMBER 1, 2020
"Since I started exploring Data Engineering, it has been overwhelming. In the end I have the feeling of giving up." This is a message that reached me from a viewer on YouTube. And that's exactly how I feel sometimes! Sometimes I feel a bit overwhelmed by the whole thing. Because there is so much going on. All the technology and Data Science hype. There is always something new on the horizon.
Cloudera
SEPTEMBER 1, 2020
Live data-streaming offers businesses exciting new opportunities to transform the way they operate, leveraging real-time insights to drive better decision making and enhance operational efficiency. To find out more about how streaming data might impact the financial services sector I sat down for a chat with Dinesh Chandrasekhar, Head of Product Marketing in Cloudera’s Data-in-Motion Business Unit.
Speaker: Tamara Fingerlin, Developer Advocate
Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.
Confluent
AUGUST 31, 2020
As a South African, who grew up during the era of apartheid, I’ve witnessed firsthand the negative and long-lasting impact of discrimination, bias, and inequity, and I have a strong […].
Netflix Tech
SEPTEMBER 4, 2020
HDMI?—?Scaling Netflix Certification Scott Bolter , Matthew Lehman , Akshay Garg ¹ At Netflix, we take the task of preserving the creative vision of our content all the way to a subscriber TV screen very seriously. This significantly increases the scope of our application integration and certification processes for streaming devices like set-top-boxes (STBs) and TVs.
Team Data Science
AUGUST 30, 2020
You want to become a data engineer, but don't know how to set up a data engineering project? I will show you! Do not make this mistake! First of all you should not make the mistake that unfortunately many people make! Often people want to build the whole thing from the beginning. They say: "Okay I need to do a project. I need to make a big thing. I don't even know what data and what tools I want to use.
Cloudera
SEPTEMBER 4, 2020
Article 3. The Future Of The Telco Industry And Impact Of 5G & IoT – Part 3. In the final installment in the series, Vijay Raja, Director of Industry & Solutions Marketing at Cloudera shares his views on how the telecom sector is changing and where it goes next. Hi Vijay, thank you so much for joining us again. To continue where we left off, how are ML and IoT influencing the Telecom sector, and how is Cloudera supporting this industry evolution?
Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage
There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.
Rock the JVM
SEPTEMBER 4, 2020
Scala syntax can be confusing: discover almost all uses of underscores and why understanding their inconsistent philosophy is worthwhile
Confluent
SEPTEMBER 3, 2020
ksqlDB, the event streaming database, is becoming one of the most popular ways to work with Apache Kafka®.
Team Data Science
AUGUST 30, 2020
Data Science , Artificial Intelligence and Machine Learning. These topics are currently the hype in the field of Data Science. Everyone wants to become a Data Scientist. But isn't the work being done in the field of Data Engineereing the real MVP? Isn't it important to have Data Scientists AND Data Engineers on board to make a project successful? Yes, it is!
Cloudera
SEPTEMBER 1, 2020
From a-z in 10 minutes! It is hard to believe if you have had previous experience with setting up, sizing, and deploying a distributed search engine service that this is possible. Imagine how many times IT has lost valuable time spending hours trying to understand Apache Solr application requirements and map them into how to best size and deploy the Solr service.
Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives
Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri
Rock the JVM
SEPTEMBER 4, 2020
Scala syntax can be confusing: discover almost all uses of underscores and why understanding their inconsistent philosophy is worthwhile
Confluent
SEPTEMBER 3, 2020
ksqlDB, the event streaming database, is becoming one of the most popular ways to work with Apache Kafka.
DareData
SEPTEMBER 2, 2020
The world of data science and information technology is a constantly evolving landscape, where dozens of new tools and methodologies are created and updated daily, and many others quickly become obsolete. Every organization has their own ecosystem of applications, but even the most advanced organizations sometimes fall behind in certain areas when compared to the bleeding edge of technological advances.
Cloudera
SEPTEMBER 2, 2020
Recently, Cloudera announced the release of Cloudera CDP Private Cloud, delivering the final component of our hybrid cloud strategy. There’s nothing comparable to it in the industry. CDP Private Cloud offers benefits of a public cloud architecture—autoscaling, isolation, agile provisioning, etc.—in an on-premise environment. Additionally, lines of business (LOBs) are able to gain access to a shared data lake that is secured and governed by the use of Cloudera Shared Data Experience (SDX).
Advertisement
With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to: Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to Write DAGs that adapt to your data at runtime and set up alerts and notifications Scale you
Teradata
SEPTEMBER 2, 2020
CEOs of banks know all about change. But the existential challenge posed by Big Tech requires a totally new set of skills. What do they need to learn to survive?
Preset
SEPTEMBER 1, 2020
Preset and Superset User Documentation is available for everyone that want to become a superset expert
Rock the JVM
AUGUST 30, 2020
Clarifying the differences between two essential repartitioning operations in Apache Spark
Cloudera
AUGUST 31, 2020
Service Management Group ( SMG ) offers an easy-to-use experience management (XM) platform that combines end-to-end customer and employee experience management software with hands-on professional services to deliver actionable insights and help brands get smarter about their customers. The XM platform, smg360 , helps customers across verticals, including restaurants, retail, and healthcare, drive changes that boost loyalty and improve business outcomes. .
Speaker: Tamara Fingerlin, Developer Advocate
In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!
Teradata
SEPTEMBER 1, 2020
With Teradata Vantage on Azure, customers have access to the same dynamic resource optimization tools that they have come to love with the added agility that Azure brings to the table.
Rockset
SEPTEMBER 3, 2020
MongoDB’s Advantages & Disadvantages MongoDB has comprehensive aggregation capabilities. You can run many analytic queries on MongoDB without exporting your data to a third-party tool. However, these aggregation queries are frequently CPU-intensive and can block or delay the execution of other queries. For example, Online Transactional Processing (OLTP) queries are usually short read operations that have direct impacts on the user experience.
Netflix Tech
SEPTEMBER 2, 2020
Edgar helps Netflix teams troubleshoot distributed systems efficiently with the help of a summarized presentation of request tracing, logs, analysis, and metadata. by Elizabeth Carretto Everyone loves Unsolved Mysteries. There’s always someone who seems like the surefire culprit. There’s a clear motive, the perfect opportunity, and an incriminating footprint left behind.
Teradata
SEPTEMBER 1, 2020
The Utah Jazz create winning customer experiences using Teradata Vantage on AWS with consumption pricing for flexible and elastic modern cloud analytics.
Speaker: Ben Epstein, Stealth Founder & CTO | Tony Karrer, Founder & CTO, Aggregage
When tasked with building a fundamentally new product line with deeper insights than previously achievable for a high-value client, Ben Epstein and his team faced a significant challenge: how to harness LLMs to produce consistent, high-accuracy outputs at scale. In this new session, Ben will share how he and his team engineered a system (based on proven software engineering approaches) that employs reproducible test variations (via temperature 0 and fixed seeds), and enables non-LLM evaluation m
Expert insights. Personalized for you.
133 
Let's personalize your content