KDnuggets
JULY 17, 2023
The All Things Insights and marketing analytics and data science community completed an extensive survey covering what executives are thinking, how they’re spending and the issues and opportunities they face. Grab your free copy now.
The Pragmatic Engineer
JULY 20, 2023
👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. In this article, we cover one out of five topics from today’s subscriber-only deepdive on Advice on how to sell a startup. To get full issues twice a week, subscribe here.
Waitingforcode
JULY 21, 2023
Starting from Apache Spark 3.2.0 is now possible to load an initial state of the arbitrary stateful pipelines. Even though the feature is easy to implement, it hides some interesting implementation details!
Start Data Engineering
JULY 20, 2023
1. Introduction 2. Sample project 3. Best practices 3.1. Use standard patterns that progressively transform your data 3.2. Ensure data is valid before exposing it to its consumers (aka data quality checks) 3.3. Avoid data duplicates with idempotent pipelines 3.4. Write DRY code & keep I/O separate from data transformation 3.5. Know the when, how, & what (aka metadata) of pipeline runs for easier debugging 3.
Advertisement
Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.
Data Engineering Podcast
JULY 16, 2023
Summary Data has been one of the most substantial drivers of business and economic value for the past few decades. Bob Muglia has had a front-row seat to many of the major shifts driven by technology over his career. In his recent book "Datapreneurs" he reflects on the people and businesses that he has known and worked with and how they relied on data to deliver valuable services and drive meaningful change.
Seattle Data Guy
JULY 16, 2023
The ETL & ELT tool market is experiencing continuous transformation, propelled by fluctuating pricing structures and the advent of inventive alternatives. This industry remains fiercely competitive due to these changing elements and a swiftly growing user base. In the following sections, we will explore four emerging alternatives to Fivetran. Of course, that is if you… Read more The post 4 Alternatives to Fivetran: The Evolving Dynamics of the ETL & ELT Tool Market appeared first
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
KDnuggets
JULY 21, 2023
Let's explore the journeys of SAS users who harnessed the power of SAS to unlock new opportunities and achieve their career goals.
databricks
JULY 18, 2023
Today, Meta released their latest state-of-the-art large language model (LLM) Llama 2 to open source for commercial use1. This is a significant development.
Confluent
JULY 18, 2023
Gain the easiest solution for data streaming and increase data flow to your platform through native integrations with Confluent Cloud and 120+ Kafka connectors.
ThoughtSpot
JULY 18, 2023
At ThoughtSpot, we believe making data accessible to every knowledge worker requires human-centered technology—an analytics experience that bridges the “language” barrier between technology and people. AI is the perfect compliment to search because it empowers organizations to analyze, understand, and act on data. In order to achieve this vision, we knew we’d need to work with some of the best, most innovative technology companies across the modern data stack —companies that put their users fir
Speaker: Tamara Fingerlin, Developer Advocate
In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!
KDnuggets
JULY 20, 2023
Unveiling GPT-4: Deciphering its impact on data science and exploring its strengths and boundaries.
databricks
JULY 18, 2023
We are excited to announce enhanced monitoring and observability features in Databricks Workflows. This includes a new real-time insights dashboard to see all.
ArcGIS
JULY 17, 2023
This blog shares the resource to the ArcGIS Pro Learn Series about SAR satellite imagery.
Engineering at Meta
JULY 17, 2023
Meta has made it possible for people to upload high dynamic range (HDR) videos from their phone’s camera roll to Reels on Facebook and Instagram. To show standard dynamic range (SDR) UI elements and overlays legibly on top of HDR video, we render them at a brightness level comparable to the video itself. We solved various technical challenges to ensure a smooth transition to HDR video across the diverse range of old and new devices that people use to interact with our services every day.
Advertisement
Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.
KDnuggets
JULY 21, 2023
Don’t have any coding experience? Don’t worry. Check out this drag-and-drop tool that helps you to build your own customized LLM flows. And guess what, you don’t have to be a tech professional!
databricks
JULY 19, 2023
Today, we’re excited to share that we’ve completed our acquisition of MosaicML, a leading platform for creating and customizing generative AI models for you.
ArcGIS
JULY 18, 2023
The purpose is to learn what network diagram storage means and provide guidance to avoid unnecessarily increasing database sizes.
Pinterest Engineering
JULY 18, 2023
Brian Overstreet | Software Engineer, Observability; Humsheen Geo | Software Engineer, Observability Time series is a critical part of Observability at Pinterest, powering 60,000 alerts and 5,000 dashboards. A time series is an identifier with values where the values are associated with a timestamp. Given the widespread use and critical nature of time series, it’s important to give engineers the ability to adequately express what operations to perform on the time series in a readable, understand
Advertisement
Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?
KDnuggets
JULY 17, 2023
Putting the world to shame.
databricks
JULY 21, 2023
Let's say a distributor reached out wanting to understand what factors are driving the sale of carbonated beverages from customers in their convenience.
Towards Data Science
JULY 17, 2023
Deep Dive Guide for When and How to Use 8 Types of SCD Continue reading on Towards Data Science »
Cloudera
JULY 18, 2023
In the realm of big data analytics, Hive has been a trusted companion for summarizing, querying, and analyzing huge and disparate datasets. But let’s face it, navigating the world of any SQL engine is a daunting task, and Hive is no exception. As a Hive user, you will find yourself wanting to go beyond surface-level analysis, and deep dive into the intricacies of how a Hive query is executed.
Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali
As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.
KDnuggets
JULY 17, 2023
A guide to using ChatGPT for exploratory data analysis. Use ChatGPT to explore a dataset, generate visualizations, and gain insights.
databricks
JULY 21, 2023
This is part seven of a multi-part series to share key insights and tactics with Senior Executives leading data and AI transformation initiatives.
LinkedIn Engineering
JULY 19, 2023
Co-Authors: Sumedh Sakdeo , Lei Sun , Sushant Raikar , Stanislav Pak , and Abhishek Nath Introduction At LinkedIn, we build and operate an open source data lakehouse deployment to power Analytics and Machine Learning workloads. Leveraging data to drive decisions allows us to serve our members with better job insights, and connect the world’s professionals with each other.
Cloudera
JULY 21, 2023
Congratulations on your pregnancy! Finding out you are pregnant is an exciting and life-changing experience, but it can also bring some unexpected challenges – especially if you find out you’re pregnant with twins after accepting a new job offer. That’s exactly what happened to me. I was thrilled to secure a job at Cloudera, a company I greatly admired.
Speaker: Nikhil Joshi, Founder & President of Snic Solutions
Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.
KDnuggets
JULY 21, 2023
Addressing the potential risks associated with superintelligence systems.
databricks
JULY 17, 2023
For over a year now, Databricks and dbt Labs have been working together to realize the vision of simplified real-time analytics engineering, combining.
Lyft Engineering
JULY 17, 2023
Our cross-functional development process By: Aastha Bhargava , Jake Hercules , Erik Kamp , Michael Ramdatt , Nathan Van Fleet , Rex Lam , Kieran Gupta Product For years, drivers have been clear about what they wanted: native Lyft support for CarPlay and Android Auto. They’ve made the request across social media platforms, through the app, and in feedback sessions with Lyft researchers.
Workfall
JULY 18, 2023
Reading Time: 8 minutes Picture yourself in the bustling world of a leading streaming platform, where countless users rely on personalized recommendations for their next binge-watching adventure. Behind the scenes, a team of data wizards tirelessly crunches mountains of data to make those recommendations sparkle. As one of those wizards, we’ve seen the challenges we face: the struggle to transform massive datasets into meaningful insights, all while keeping queries fast and our system scal
Advertisement
Cloud Development Environments (CDEs) are changing how software teams work by moving development to the cloud. Our Cloud Development Environment Adoption Report gathers insights from 223 developers and business leaders, uncovering key trends in CDE adoption. With 66% of large organizations already using CDEs, these platforms are quickly becoming essential to modern development practices.
Expert insights. Personalized for you.
105 
Let's personalize your content