KDnuggets
JANUARY 14, 2025
What are essential commands for data analysis in SQL? This article will answer this question by mentioning 10 SQL commands. As a bonus, well add some other SQL commands that are not that common but will certainly add flexibility to your analyses.
Data Engineering Weekly
JANUARY 15, 2025
The Critical Role of AI Data Engineers in a Data-Driven World How does a chatbot seamlessly interpret your questions? How does a self-driving car understand a chaotic street scene? The answer lies in unstructured data processing—a field that powers modern artificial intelligence (AI) systems. Unlike neatly organized rows and columns in spreadsheets, unstructured data—such as text, images, videos, and audio—requires advanced processing techniques to derive meaningful insights.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Hevo
JANUARY 17, 2025
Digital tools and technologies help organizations generate large amounts of data daily, requiring efficient governance and management. This is where the AWS data lake comes in. With the AWS data lake, organizations and businesses can store, analyze, and process structured and unstructured data of any size.
Waitingforcode
JANUARY 15, 2025
A few months ago I wrote a blog post about event skew and how dangerous it is for a stateful streaming job. Since it was a high-level explanation, I didn't cover Apache Spark Structured Streaming deeply at that moment. Now the watermark topic is back to my learning backlog and it's a good opportunity to return to the event skew topic and see the dangers it brings for Structured Streaming stateful jobs.
Speaker: Tamara Fingerlin, Developer Advocate
Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.
KDnuggets
JANUARY 13, 2025
Ive been programming with Python for over 6 years now. But if I could start over, heres how Id go about learning Python in 2025.
Cloudyard
JANUARY 15, 2025
Read Time: 2 Minute, 33 Second Snowflakes PARSE_DOCUMENT function revolutionizes how unstructured data, such as PDF files, is processed within the Snowflake ecosystem. Traditionally, this function is used within SQL to extract structured content from documents. However, Ive taken this a step further, leveraging Snowpark to extend its capabilities and build a complete data extraction process.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Snowflake
JANUARY 15, 2025
Over the last three geospatial-centric blog posts, weve covered the basics of what geospatial data is, how it works in the broader world of data and how it specifically works in Snowflake based on our native support for GEOGRAPHY , GEOMETRY and H3. Those articles are great for dipping your toe in, getting a feel for the water and maybe even wading into the shallow end of the pool.
Pinterest Engineering
JANUARY 13, 2025
Angel Vargas | Software Engineer, API Platform; Swati Kumar | Software Engineer, API Platform; Chris Bunting | Engineering Manager, APIPlatform The inside of the Pinterest lobby in Mexico City, showing a patterned ceiling, a receptionist deck with a plant on it, a light above it, and a gallery of images of pins youd find on Pinterest, behind it. To the left, a glowing Pinterest P sign hovers in front of a glasswall.
Precisely
JANUARY 13, 2025
As we approach 2025, the customer experience (CX) landscape is evolving rapidly, driven by technological innovation, heightened consumer expectations, and a growing emphasis on trust and compliance. For regulated industries such as healthcare, insurance, and financial services, the stakes are especially high. These organizations operate in complex ecosystems where delivering exceptional CX must align with stringent regulatory requirements and the need for data security.
Christophe Blefari
JANUARY 11, 2025
HNY 2025 ( credits ) Happy new year ✨ I wish you the best for 2025. There are multiple ways to start a new year, either with new projects, new ideas, new resolutions or by just keeping doing the same music. I hope you will enjoy 2025. The Data News are here to stay, the format might vary during the year, but here we are for another year. Thank you so much for your support through the years.
Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage
There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.
Snowflake
JANUARY 15, 2025
Y Combinator founder Paul Graham advises startup founders to live in the future, then build whats missing. I had the privilege of glimpsing the future through a series of interviews with investors on the bleeding edge of the AI landscape. Insights from these candid conversations laid the foundation for Startup 2025: Building a Business in the Age of AI, the AI startup report that Snowflake is publishing today.
Engineering at Meta
JANUARY 16, 2025
Do types actually make developers more productive? Or is it just more typing on the keyboard? To answer that question were revisiting Diff Authoring Time (DAT) how Meta measures how long it takes to submit changes to a codebase. DAT is just one of the ways e measure developer productivity and this latest episode of the Meta Tech Podcast takes a look at two concrete use cases for DAT, including a type-safe mocking framework in Hack.
Confessions of a Data Guy
JANUARY 17, 2025
A Deep Dive into Databricks Labs’ DQX: The Data Quality Game Changer for PySpark DataFrames Recently, a LinkedIn announcement caught my eyeand honestly, it had me on the edge of my seat. Databricks Labs has unveiled DQX, a Python-based Data Quality framework explicitly designed for PySpark DataFrames. Finally, a Dedicated Data Quality Tool for PySpark […] The post PySpark Data Quality on Databricks with DQX. appeared first on Confessions of a Data Guy.
KDnuggets
JANUARY 15, 2025
Rust is a systems programming language that offers high performance and safety. Python programmers will find Rust's syntax familiar but with more control over memory and performance.
Speaker: Andrew Skoog, Founder of MachinistX & President of Hexis Representatives
Manufacturing is evolving, and the right technology can empower—not replace—your workforce. Smart automation and AI-driven software are revolutionizing decision-making, optimizing processes, and improving efficiency. But how do you implement these tools with confidence and ensure they complement human expertise rather than override it? Join industry expert Andrew Skoog as he explores how manufacturers can leverage automation to enhance operations, streamline workflows, and make smarter, data-dri
Snowflake
JANUARY 16, 2025
Large language models (LLMs) are at the heart of generative AI transformations, driving solutions across industries from efficient customer support to simplified data analysis. Enterprises need performant, cost-effective and low-latency inference to scale their gen AI solutions. Yet, the complexity and computational demands of LLM inference present a challenge.
DareData
JANUARY 15, 2025
DareData will close 2024 with a 5% revenue growth compared to 2023. At first glance, given the rapid growth in our market, one might be tempted to classify this year as underwhelming. However, 2024 has been a transformative year for us. We started the year as a 100% consulting business. Consulting is highly dependent on people, and in small boutique firms like ours, this often means being heavily reliant on the partners.
Scott Logic
JANUARY 13, 2025
From crafting logos to leading stand-ups, my journey with vAirify wasnt just about codeit was a crash course in full-stack development, cross-organizational collaboration, and the art of turning weather data into actionable insights. As a recently promoted developer who began as a freshly trained associate, my experience was initially limited to the graduate projectan internal project designed to provide new graduates with practical development experience.
KDnuggets
JANUARY 14, 2025
Looking for extra income? Here are 7 creative ways to use large language models for passive earnings!
Advertisement
With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to: Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to Write DAGs that adapt to your data at runtime and set up alerts and notifications Scale you
Snowflake
JANUARY 16, 2025
AI is proving that its here to stay. While 2023 brought wonder, and 2024 saw widespread experimentation, 2025 will be the year that manufacturing enterprises get serious about AI's applications. But its complicated: AI proofs of concept are graduating from the sandbox to production, just as some of AIs biggest cheerleaders are turning a bit dour. How to navigate such a landscape is top of mind for me and top executives such as Snowflakes CEO, Sridhar Ramaswamy; Snowflakes Distinguished AI Engine
Striim
JANUARY 17, 2025
In todays competitive environment, enterprises need to harness data the instant its created. But data teams often face challenges when it comes to capturing, processing, and integrating high-velocity data streams from diverse sourcesmaking it difficult to keep AI applications timely and relevant. Simultaneously, generative AI (GenAI) is becoming indispensable for delivering dynamic, real-time solutions, from chatbots and personalized marketing to adaptive decision-making.
ArcGIS
JANUARY 14, 2025
Mix colors to build a theme for the new multidimensional Global Wind Atlas that's now available in ArcGIS Living Atlas.
KDnuggets
JANUARY 17, 2025
A not-to-be-missed list of elegant Python solutions to perform common programming and processing tasks in a single line of code.
Speaker: Tamara Fingerlin, Developer Advocate
In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!
Precisely
JANUARY 16, 2025
The 2025 Outlook: Data Integrity Trends and Insights report is here! What are the latest data integrity trends you need to know about? How does your data program compare to your peers? Find out in the report, published in partnership between Precisely and Drexel Universitys LeBow College of Business. This years report is filled with actionable strategic insights from over 550 leading data and analytics professionals worldwide and its going to be an essential resource as you plan your 2025 data
Data Engineering Weekly
JANUARY 12, 2025
Try Fully Managed Apache Airflow for FREE Astro is the fully-managed DataOps platform powered by Apache Airflow. With Astro, you can build, run, and observe your data pipelines in one place, ensuring your mission critical data is delivered on time. Try Astro Free → JetBrains: State of Developer Ecosystem Report 2024 JetBrains published its annual developer survey, and there is tons of insight on the developer adoption of various programming languages.
ArcGIS
JANUARY 14, 2025
Learn how to use spatial components to help with various spatial statistics workflows.
KDnuggets
JANUARY 16, 2025
Want to become an NLP pro by 2025? Check out these top free courses and learn from experts whove shaped the future of language models.
Speaker: Ben Epstein, Stealth Founder & CTO | Tony Karrer, Founder & CTO, Aggregage
When tasked with building a fundamentally new product line with deeper insights than previously achievable for a high-value client, Ben Epstein and his team faced a significant challenge: how to harness LLMs to produce consistent, high-accuracy outputs at scale. In this new session, Ben will share how he and his team engineered a system (based on proven software engineering approaches) that employs reproducible test variations (via temperature 0 and fixed seeds), and enables non-LLM evaluation m
WeCloudData
JANUARY 17, 2025
Our interactions with AI models are as important as the technology itself. Prompt engineering is the secret to maximizing the potential of AI systems like ChatGPT, Gemini, DALL-E, and other Large Language Models, whether you’re summarizing complex documents, or brainstorming creative ideas. At WeCloudData, we are committed to giving students real-world skills that make a […] The post Introduction to Prompt Engineering appeared first on WeCloudData.
databricks
JANUARY 14, 2025
Electronic products are evolving at lightning speed, driven by an insatiable demand for new consumer devices, energy, transport, robotics, connectivity, data and beyond.
ArcGIS
JANUARY 17, 2025
Use this dictionary renderer symbology to display clear and concise status information for locations of interest.
KDnuggets
JANUARY 17, 2025
A product data scientist breaks down why AI wont replace us anytime soon.
Advertisement
Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.
Expert insights. Personalized for you.
112 
Let's personalize your content