KDnuggets
NOVEMBER 28, 2019
Open source is becoming the standard for sharing and improving technology. Some of the largest organizations in the world namely: Google, Facebook and Uber are open sourcing their own technologies that they use in their workflow to the public.
Uber Engineering
NOVEMBER 26, 2019
When something goes wrong with a piece of code, engineers want to know all the relevant details of the error immediately so they can get right to work remedying the malfunction. . However, as technology has advanced, measuring system metrics and … The post Optimizing Observability with Jaeger, M3, and XYS at Uber appeared first on Uber Engineering Blog.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Confluent
NOVEMBER 20, 2019
Today marks a new release of KSQL, one so significant that we’re giving it a new name: ksqlDB. Like KSQL, ksqlDB remains freely available and community licensed, and you can […].
Data Engineering Podcast
NOVEMBER 26, 2019
Summary The team at Sentry has built a platform for anyone in the world to send software errors and events. As they scaled the volume of customers and data they began running into the limitations of their initial architecture. To address the needs of their business and continue to improve their capabilities they settled on Clickhouse as the new storage and query layer to power their business.
Speaker: Tamara Fingerlin, Developer Advocate
Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.
Netflix Tech
NOVEMBER 4, 2019
by Artem Shtatnov and Ravi Srinivas Ranganathan Almost a year ago we described our learnings from adopting GraphQL on the Netflix Marketing Tech team. We have a lot more to share since then! There are plenty of existing resources describing how to express a search query in GraphQL and paginate the results. This post looks at the other side of search: how to index data and make it searchable.
Teradata
NOVEMBER 19, 2019
How do you ensure your Customer Data Platform is enabling breakthrough customer experience business outcomes, rather than hindering them? Find out more!
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
|
Uber Engineering
NOVEMBER 19, 2019
A restaurant’s menu is arguably its most important feature. When ordering online or via the app with Uber Eats, potential customers can’t peer in through a restaurant’s windows or smell the scents wafting from their kitchens, so digital menus become … The post Introducing Menu Maker: Uber Eats’ New Menu Management Tool appeared first on Uber Engineering Blog.
Confluent
NOVEMBER 21, 2019
ksqlDB is a new kind of database purpose-built for stream processing apps, allowing users to build stream processing applications against data in Apache Kafka® and enhancing developer productivity. ksqlDB simplifies […].
Data Engineering Podcast
NOVEMBER 18, 2019
Summary With the constant evolution of technology for data management it can seem impossible to make an informed decision about whether to build a data warehouse, or a data lake, or just leave your data wherever it currently rests. What’s worse is that any time you have to migrate to a new architecture, all of your analytical code has to change too.
Netflix Tech
NOVEMBER 22, 2019
by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! Please stop by our “Living Room” for an opportunity to connect or reconnect with Netflixers. We’ve compiled our speaking events below so you know what we’ve been working on. We look forward to seeing you there!
Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage
There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.
Teradata
NOVEMBER 11, 2019
An integrated data foundation allows data science models to be more accurate, actionable and engage more customers. Find out how your model can positively impact your bottom line.
KDnuggets
NOVEMBER 28, 2019
Sebastian Thrun, the founder of Udacity, ruined my machine learning project and wedding.
U-Next
NOVEMBER 9, 2019
Introduction to Workforce Analytics Today, the need to understand what attracts skillful individuals to join an organization, stay motivated, and deliver outstanding results has become more important than ever. However, this is not a task which can be shouldered by the HR team alone; they need the right tools to deliver optimal results. Over the years, organizations around the globe have spent billions of dollars on employee performance analysis, talent recruitment, leadership training, and deve
Confluent
NOVEMBER 25, 2019
Potential advantages of hybrid cloud architectures include avoiding vendor lock-in, increasing system resilience, optimizing costs, and inducing price competition among cloud providers. Hybrid cloud architectures require the ability to securely […].
Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives
Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri
Data Engineering Podcast
NOVEMBER 11, 2019
Summary The practice of data management is one that requires technical acumen, but there are also many policy and regulatory issues that inform and influence the design of our systems. With the introduction of legal frameworks such as the EU GDPR and California’s CCPA it is necessary to consider how to implement data protectino and data privacy principles in the technical and policy controls that govern our data platforms.
InData Labs
NOVEMBER 26, 2019
Today we witness new technologies arising and new tendencies influencing the way we work and live. In order to stay competitive, it’s important to be aware of the latest trends, maintain a level of strong expertise, and offer best-in-class services. Our team is always striving to move with the times and keep all modern requirements. Запись TDA Announces InData Labs as the Winners in Global Agency Awards впервые появилась InData Labs.
Teradata
NOVEMBER 24, 2019
Every organization, in every industry, needs a Chief Data Officer. Use these guidelines to choose the one with the optimal skills and background for success.
KDnuggets
NOVEMBER 27, 2019
Also: Deep Learning for Image Classification with Less Data; How to Speed up Pandas by 4x with one line of code; 25 Useful #Python Snippets to Help in Your Day-to-Day Work; Automated Machine Learning Project Implementation Complexities.
Advertisement
With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to: Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to Write DAGs that adapt to your data at runtime and set up alerts and notifications Scale you
Dataquest
NOVEMBER 4, 2019
If you’ve ever wanted to learn Python online with streaming data, or data that changes quickly, you may be familiar with the concept of a data pipeline. Data pipelines allow you transform data from one representation to another through a series of steps. Data pipelines are a key part of data engineering, which we teach in our new Data Engineer Path.
Confluent
NOVEMBER 19, 2019
In the past 12 months, games and other forms of content made with the Unity platform were installed 33 billion times reaching 3 billion devices worldwide. Apart from our real-time […].
Data Engineering Podcast
NOVEMBER 4, 2019
Summary As data engineers the health of our pipelines is our highest priority. Unfortunately, there are countless ways that our dataflows can break or degrade that have nothing to do with the business logic or data transformations that we write and maintain. Sean Knapp founded Ascend to address the operational challenges of running a production grade and scalable Spark infrastructure, allowing data engineers to focus on the problems that power their business.
Netflix Tech
NOVEMBER 12, 2019
Page Simulation for Better Offline Metrics at Netflix by David Gevorkyan , Mehmet Yilmaz , Ajinkya More , Gaurav Agrawal , Richard Wellington , Vivek Kaushal , Prasanna Padmanabhan , Justin Basilico At Netflix, we spend a lot of effort to make it easy for our members to find content they will love. To make this happen, we personalize many aspects of our service, including which movies and TV shows we present on each member’s homepage.
Advertisement
Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.
Teradata
NOVEMBER 17, 2019
Moving part of your analytic ecosystem to the cloud requires the inspection of all the ecosystem elements to make sure they perform well over a WAN. Read more.
KDnuggets
NOVEMBER 29, 2019
Read this article on training Markov chains to generate George R. R. Martin style text.
Rockset
NOVEMBER 6, 2019
Events are messages that are sent by a system to notify operators or other systems about a change in its domain. With event-driven architectures powered by systems like Apache Kafka becoming more prominent, there are now many applications in the modern software stack that make use of events and messages to operate effectively. In this blog, we will examine the use of three different data backends for event data - Apache Druid , Elasticsearch and Rockset.
Confluent
NOVEMBER 6, 2019
Today, we are proud to make Confluent Cloud available to companies leveraging the Microsoft Azure ecosystem of services, in addition to the previous rollouts on Google Cloud Platform (GCP) and […].
Advertisement
Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?
KDnuggets
NOVEMBER 26, 2019
A guide to build a content-based movie recommender model based on NLP.
KDnuggets
NOVEMBER 22, 2019
If you are a new Data Scientists early in your professional journey, and you’re a bit confused and lost, then follow this advice to figure out how to best contribute to your company.
KDnuggets
NOVEMBER 29, 2019
Where does Java stand in the world of artificial intelligence, machine learning, and deep learning? Learn more about how to do these things in Java, and the libraries and frameworks to use.
KDnuggets
NOVEMBER 22, 2019
To demonstrate the implementation complexity differences along the AutoML highway, let's have a look at how 3 specific software projects approach the implementation of just such an AutoML "solution," namely Keras Tuner, AutoKeras, and automl-gs.
Speaker: Tamara Fingerlin, Developer Advocate
In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!
Expert insights. Personalized for you.
160 
Let's personalize your content