Knowledge Hut
JANUARY 29, 2024
In today's data-driven world, where information reigns supreme, businesses rely on data to guide their decisions and strategies. However, the sheer volume and complexity of raw data from various sources can often resemble a chaotic jigsaw puzzle. It is in this intricate process of assembling, cleaning, and refining data that the magic of Data Wrangling unfolds.
KDnuggets
FEBRUARY 1, 2024
Tune in to these 5 AI podcasts at the gym or on your commute to keep up to date with the world of AI.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Data Engineering Podcast
JANUARY 28, 2024
Summary Monitoring and auditing IT systems for security events requires the ability to quickly analyze massive volumes of unstructured log data. The majority of products that are available either require too much effort to structure the logs, or aren't fast enough for interactive use cases. Cliff Crosland co-founded Scanner to provide fast querying of high scale log data for security auditing.
databricks
FEBRUARY 1, 2024
As Chief Scientist (Neural Networks) at Databricks, I lead our research team toward the goal of giving everyone the ability to build and.
Advertisement
In Airflow, DAGs (your data pipelines) support nearly every use case. As these workflows grow in complexity and scale, efficiently identifying and resolving issues becomes a critical skill for every data engineer. This is a comprehensive guide with best practices and examples to debugging Airflow DAGs. You’ll learn how to: Create a standardized process for debugging to quickly diagnose errors in your DAGs Identify common issues with DAGs, tasks, and connections Distinguish between Airflow-relate
ArcGIS
JANUARY 31, 2024
Exploring the value of critique as part of the process of creating a new map of the Total Eclipse that will cross the United States on April 8th
KDnuggets
FEBRUARY 2, 2024
Learn how to use OpenAI Python API for accessing language, embedding, audio, vision, and image generation models.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
|
databricks
JANUARY 30, 2024
At Databricks, we believe that AI will change the way that enterprises interact with their data. That’s why today, we're excited to welcome t.
ArcGIS
JANUARY 30, 2024
ArcGIS Pro now includes Neighborhood Explorer: an experience that will help you understand and refine spatial relationships in your analysis.
KDnuggets
JANUARY 31, 2024
Looking to make a career in data analytics? Take the first steps today with these free courses.
Waitingforcode
JANUARY 30, 2024
Previously you could read about transformation of a user job definition into an executable stream graph. Since this explanation was relatively high-level, I decided to deep dive into the final step executing the code.
Speaker: Tamara Fingerlin, Developer Advocate
Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.
databricks
JANUARY 30, 2024
Quantization is a technique for making machine learning models smaller and faster. We quantize Llama2-70B-Chat, producing an equivalent-quality model that generates 2.2x more.
Jesse Anderson
JANUARY 29, 2024
The premier of my latest talk covering The State of Data Engineering. I go through the history of the industry to see where we’re heading. This starts with data warehousing and goes into data science. I finish off by showing how data engineering can avoid the same fate as data warehousing and data science. Sorry, we didn’t have a microphone for the questions and I forgot to repeat some of the questions.
KDnuggets
JANUARY 29, 2024
ChatGPT can be a great tool for data scientists. Here’s what I learned about where it excels and where it is less so.
Snowflake
JANUARY 29, 2024
Imagine seamlessly integrating your Google Analytics data with Snowflake, allowing you to combine it effortlessly with other key sources like CRM, ERP, social media metrics, email campaign data, and whatever data sources compose the full scope of your data estate. The good news is that it’s possible with the native Snowflake Connector for Google Analytics, now available in public preview.
Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage
There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.
databricks
JANUARY 31, 2024
Today, we're excited to announce that AI model sharing is available in both Databricks Delta Sharing and on the Databricks Marketplace. With Delta.
ArcGIS
JANUARY 31, 2024
How to get exquisite control over polygon generalization without actually generalizing
KDnuggets
FEBRUARY 1, 2024
Learn about the most common questions asked during data science interviews. This blog covers non-technical, Python, SQL, statistics, data analysis, and machine learning questions.
Snowflake
JANUARY 31, 2024
Today, we’re excited to announce the general availability of the Snowflake Native App Framework on AWS and Azure! We’ve seen incredible momentum around Snowflake Native Apps. More than 90 Snowflake Native Apps are currently available in Snowflake Marketplace. You can purchase, install and run Snowflake Native Apps—ranging from connectors to clean rooms—directly in your Snowflake account.
Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives
Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri
databricks
JANUARY 31, 2024
Welcome to the blog series covering product advancements in 2023 for Databricks SQL, the serverless data warehouse from Databricks. This is part 2.
ArcGIS
JANUARY 31, 2024
To improve a map's legend, often all that’s needed is a bit of tidying: renaming, reordering, and removing items.
KDnuggets
JANUARY 29, 2024
Want to learn about AI? You can for FREE with LinkedIn.
Snowflake
FEBRUARY 1, 2024
It’s not hyperbole to say that generative AI (gen AI) is radically transforming the advertising, media and entertainment industry. There has been widespread excitement about the potential of gen AI to open brand-new creative opportunities and unlock unprecedented efficiencies. At the same time, there has been understandable concern about issues such as inherent bias, deep fakes and the impact of gen AI on jobs.
Advertisement
With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to: Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to Write DAGs that adapt to your data at runtime and set up alerts and notifications Scale you
databricks
JANUARY 29, 2024
Databricks launches hands-on labs solution and cohort-based learning From the data + AI experts, today, we're announcing two unique ways that practitioners can.
ThoughtSpot
JANUARY 31, 2024
Every winter, members of ThoughtSpot’s research and development teams participate in a company-wide hackathon called Codex. The ideas that come out of Codex are always inspiring, but the Winter 22/23 hackathon was special—OpenAI had just released ChatGPT and the world was buzzing about generative AI. We knew then that this would be the beginning of a new era of analytics, for ThoughtSpot and the broader industry, but none of us could have predicted the rapid evolution of analytics and BI in the
KDnuggets
JANUARY 30, 2024
LLMs aren't the right tool for most business applications. Find out why — and learn which AI techniques are a better match.
Snowflake
JANUARY 30, 2024
In the buildup to this week’s World Economic Forum Annual Meeting in Davos, Switzerland, the talk of polycrisis becoming permacrisis painted a picture of impending doom. These terms have been used to describe the global condition today, citing the “ cascading and connected crises ” triggered by war and geopolitics, economic uncertainty, and environmental concerns, and their persistence.
Speaker: Tamara Fingerlin, Developer Advocate
In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!
databricks
FEBRUARY 2, 2024
This blog was written in collaboration with Anand Iyer, PhD, MBA, Chief Analytics Officer and Abhi Kumbara, Data Science Manager at Welldoc The.
ThoughtSpot
JANUARY 31, 2024
It’s an exciting time to be in the world of data and business intelligence. Recent advances in AI and machine learning are not only changing the way we interact with data, but also pushing those of us who build analytics and BI platforms to think critically about how our products can best serve our customers moving forward. Some will always love getting hands-on with data—but that’s no longer the only option.
KDnuggets
FEBRUARY 2, 2024
Navigating Complex Data Structures with Python's json_normalize.
Cloudera
FEBRUARY 2, 2024
Forrester Research recently released the Forester Wave for Streaming Platforms, Q4 2023. We are happy to share that Cloudera ranked as a strong performer, with a top three score for current offering. This score was stronger than anyone outside of one-cloud vendors Microsoft and Google, including a stronger current offering than Confluent. Cloudera is also the strongest on-prem offering and the only fully hybrid offering to achieve a strong performer score.
Speaker: Ben Epstein, Stealth Founder & CTO | Tony Karrer, Founder & CTO, Aggregage
When tasked with building a fundamentally new product line with deeper insights than previously achievable for a high-value client, Ben Epstein and his team faced a significant challenge: how to harness LLMs to produce consistent, high-accuracy outputs at scale. In this new session, Ben will share how he and his team engineered a system (based on proven software engineering approaches) that employs reproducible test variations (via temperature 0 and fixed seeds), and enables non-LLM evaluation m
Expert insights. Personalized for you.
52 
Let's personalize your content