Start Data Engineering

JUNE 19, 2020

Intro A lot of tutorials show how to write spark code with just the API and code samples, but they do not explain how to write “efficient Apache Spark” code.

Coding 130

Coding IT Data 130

Simon Späti

JUNE 14, 2020

Today we have more requirements with ever-growing tools and framework, complex cloud architectures, and with data stack that is changing rapidly. I hear claims: “Business Intelligence (BI) takes too long to integrate new data”, or “understanding how the numbers match up is very hard and needs lots of analysis”. The goal of this article is to make business intelligence easier, faster and more accessible with techniques from the sphere of data engineering.

Business Intelligence 130

Business Intelligence Data Engineering Data Engineer Technology 130

Confluent

JUNE 19, 2020

When I have a small software project that I want to share with the world, I don’t write my own version control system with a web UI. I don’t even […].

Kafka 121

Kafka Engineering Project Systems 121

Teradata

JUNE 17, 2020

Juneteenth has been declared a U.S. holiday at Teradata, as we stand with the black community and reflect on what we can do to fight racism and injustice, and embrace diversity.

119

119

Speaker: Tamara Fingerlin, Developer Advocate

Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.

Data

Data Engineering Podcast

JUNE 15, 2020

Summary Machine learning is a process driven by iteration and experimentation which requires fast and easy access to relevant features of the data being processed. In order to reduce friction in the process of developing and delivering models there has been a recent trend toward building a dedicated feature. In this episode Simba Khadder discusses his work at StreamSQL building a feature store to make creation, discovery, and monitoring of features fast and easy to manage.

Machine Learning 100

Machine Learning Google Cloud Kafka Data Engineer 100

Advancing Analytics: Data Engineering

JUNE 16, 2020

You might have seen that I’ve been pretty busy recently, digging into the new Azure Synapse Analytics preview, announced back at Microsoft Build 2020. I’ve explored the spark engine, SQL serverless/On-Demand and various other bits… but I’m still getting the same question of “Cool!…. but what actually is it?”. One of the problems here is that Azure SQL Data Warehouse was rebranded as “Azure Synapse Analytics”… but it’s not the same as the full workspace.

SQL 59

SQL Data Warehouse Engineering Data Engineering 59

Teradata

JUNE 16, 2020

Integration with AWS first-party services gives our enterprise customers as much cloud-native functionality as they want for their Vantage environments. Learn more.

AWS 71

AWS Cloud 71

Rock the JVM

JUNE 16, 2020

Explore how Akka Streams, Kafka Streams, and Spark Streaming stack up and find out which one is best for your use case

Kafka 52

Kafka 52

Rockset

JUNE 16, 2020

MongoDB.live took place last week, and Rockset had the opportunity to participate alongside members of the MongoDB community and share about our work to make MongoDB data accessible via real-time external indexing. In our session, we discussed the need for modern data-driven applications to perform real-time aggregations and joins, and how Rockset uses MongoDB change streams and Converged Indexing to deliver fast queries on data from MongoDB.

MongoDB 52

MongoDB Data Lake PostgreSQL Kafka 52

Confluent

JUNE 18, 2020

Many organisations rely on commercial or open source monitoring tools to measure the performance and stability of business-critical applications. AppDynamics, Datadog, and Prometheus are widely used commercial and open source […].

Building 68

Building Cloud Kafka 68

Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage

There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.

Systems

Teradata

JUNE 18, 2020

Integration with Azure first-party services enables Vantage users to tap into new sources of innovation across all aspects of the analytic process from start to finish.

Process 59

Process 59

dbt Developer Hub

JUNE 19, 2020

If you’ve been using dbt for over a year, your project is out-of-date. This is natural. New functionalities have been released. Warehouses change. Best practices are updated. Over the last year, I and others on the Fishtown Analytics (now dbt Labs!) team have conducted seven audits for clients who have been using dbt for a minimum of 2 months. In every single audit, we found opportunities to: Improve performance Improve maintainability Make it easier for new people to get up-to-speed on the proj

Project 40

Project BI SQL Utilities 40

Teradata

JUNE 17, 2020

Lloyds Banking Group executes analytic projects that benefit the customer journey for multiple brands within Lloyds Banking Group.

Banking 52

Banking Project 52

Teradata

JUNE 14, 2020

Learn how leveraging Machine Learning for advanced analytics enables Telcos to tackle problems from identifying network anomalies to customer churn. Read more.

Machine Learning 52

Machine Learning 52

Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives

Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri

Manufacturing