The Pragmatic Engineer
JANUARY 21, 2025
Hi, this is Gergely with a bonus issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. This article is one out of five sections from The Pulse #119. Full subscribers received this issue a week and a half ago. To get articles like this in your inbox, subscribe here.
KDnuggets
JANUARY 21, 2025
Data science is still among the best careers to choose from in terms of compensation, with data scientists earning higher than the average salary. Lets see what data professionals stand to earn in 2025.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
KDnuggets
JANUARY 14, 2025
Looking for extra income? Here are 7 creative ways to use large language models for passive earnings!
Waitingforcode
JANUARY 15, 2025
A few months ago I wrote a blog post about event skew and how dangerous it is for a stateful streaming job. Since it was a high-level explanation, I didn't cover Apache Spark Structured Streaming deeply at that moment. Now the watermark topic is back to my learning backlog and it's a good opportunity to return to the event skew topic and see the dangers it brings for Structured Streaming stateful jobs.
Advertisement
With over 30 million monthly downloads, Apache Airflow is the tool of choice for programmatically authoring, scheduling, and monitoring data pipelines. Airflow enables you to define workflows as Python code, allowing for dynamic and scalable pipelines suitable to any use case from ETL/ELT to running ML/AI operations in production. This introductory tutorial provides a crash course for writing and deploying your first Airflow pipeline.
Seattle Data Guy
JANUARY 18, 2025
PDF files are one of the most popular file formats today. Because they can preserve the visual layout of documents and are compatible with a wide range of devices and operating systems, PDFs are used for everything from business forms and educational material to creative designs. However, PDF files also present multiple challenges when it… Read more The post What Is PDFMiner And Should You Use It – How To Extract Data From PDFs appeared first on Seattle Data Guy.
ArcGIS
JANUARY 14, 2025
Learn how to use spatial components to help with various spatial statistics workflows.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Engineering at Meta
JANUARY 16, 2025
Do types actually make developers more productive? Or is it just more typing on the keyboard? To answer that question were revisiting Diff Authoring Time (DAT) how Meta measures how long it takes to submit changes to a codebase. DAT is just one of the ways e measure developer productivity and this latest episode of the Meta Tech Podcast takes a look at two concrete use cases for DAT, including a type-safe mocking framework in Hack.
Confessions of a Data Guy
JANUARY 17, 2025
A Deep Dive into Databricks Labs’ DQX: The Data Quality Game Changer for PySpark DataFrames Recently, a LinkedIn announcement caught my eyeand honestly, it had me on the edge of my seat. Databricks Labs has unveiled DQX, a Python-based Data Quality framework explicitly designed for PySpark DataFrames. Finally, a Dedicated Data Quality Tool for PySpark […] The post PySpark Data Quality on Databricks with DQX. appeared first on Confessions of a Data Guy.
ArcGIS
JANUARY 14, 2025
Mix colors to build a theme for the new multidimensional Global Wind Atlas that's now available in ArcGIS Living Atlas.
Snowflake
JANUARY 15, 2025
Over the last three geospatial-centric blog posts, weve covered the basics of what geospatial data is, how it works in the broader world of data and how it specifically works in Snowflake based on our native support for GEOGRAPHY , GEOMETRY and H3. Those articles are great for dipping your toe in, getting a feel for the water and maybe even wading into the shallow end of the pool.
Speaker: Tamara Fingerlin, Developer Advocate
In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!
KDnuggets
JANUARY 15, 2025
Rust is a systems programming language that offers high performance and safety. Python programmers will find Rust's syntax familiar but with more control over memory and performance.
Engineering at Meta
JANUARY 21, 2025
Were sharing details about Strobelight, Metas profiling orchestrator. Strobelight combines several technologies, many open source, into a single service that helps engineers at Meta improve efficiency and utilization across our fleet. Using Strobelight, weve seen significant efficiency wins, including one that has resulted in an estimated 15,000 servers worth of annual capacity savings.
databricks
JANUARY 14, 2025
Electronic products are evolving at lightning speed, driven by an insatiable demand for new consumer devices, energy, transport, robotics, connectivity, data and beyond.
ArcGIS
JANUARY 14, 2025
Master the Spatial Analyst Fill tool to eliminate sinks and extract better hydrologic information from your elevation data
Advertisement
Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.
Snowflake
JANUARY 15, 2025
Y Combinator founder Paul Graham advises startup founders to live in the future, then build whats missing. I had the privilege of glimpsing the future through a series of interviews with investors on the bleeding edge of the AI landscape. Insights from these candid conversations laid the foundation for Startup 2025: Building a Business in the Age of AI, the AI startup report that Snowflake is publishing today.
KDnuggets
JANUARY 17, 2025
A product data scientist breaks down why AI wont replace us anytime soon.
Data Engineering Weekly
JANUARY 15, 2025
The Critical Role of AI Data Engineers in a Data-Driven World How does a chatbot seamlessly interpret your questions? How does a self-driving car understand a chaotic street scene? The answer lies in unstructured data processing—a field that powers modern artificial intelligence (AI) systems. Unlike neatly organized rows and columns in spreadsheets, unstructured data—such as text, images, videos, and audio—requires advanced processing techniques to derive meaningful insights.
ThoughtSpot
JANUARY 15, 2025
With the ever-growing focus on GenAI, many legacy BI tools have failed to invest in the analyst. By focusing solely on AI experiences for business teams, theyve alienated data teams, relegating analysts to disjointed tools and data silos. When in reality, businesses still need people who can help decision-makers assess messy data to diagnose and evaluate business problems.
Advertisement
Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?
Towards Data Science
JANUARY 19, 2025
Make the right choice for YOU Continue reading on Towards Data Science
Snowflake
JANUARY 16, 2025
Large language models (LLMs) are at the heart of generative AI transformations, driving solutions across industries from efficient customer support to simplified data analysis. Enterprises need performant, cost-effective and low-latency inference to scale their gen AI solutions. Yet, the complexity and computational demands of LLM inference present a challenge.
KDnuggets
JANUARY 16, 2025
Want to become an NLP pro by 2025? Check out these top free courses and learn from experts whove shaped the future of language models.
Confluent
JANUARY 14, 2025
PodPrep AI, an AI-powered research assistant, leverages EDA and real-time streaming data using Confluent and Flink, in order to help its author with podcast preparation.
Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali
As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.
ArcGIS
JANUARY 17, 2025
Use this dictionary renderer symbology to display clear and concise status information for locations of interest.
Towards Data Science
JANUARY 16, 2025
Questions that guide architectural decisions to balance functional requirements with non-functional ones, like latency and scalability Continue reading on Towards Data Science
Snowflake
JANUARY 16, 2025
AI is proving that its here to stay. While 2023 brought wonder, and 2024 saw widespread experimentation, 2025 will be the year that manufacturing enterprises get serious about AI's applications. But its complicated: AI proofs of concept are graduating from the sandbox to production, just as some of AIs biggest cheerleaders are turning a bit dour. How to navigate such a landscape is top of mind for me and top executives such as Snowflakes CEO, Sridhar Ramaswamy; Snowflakes Distinguished AI Engine
KDnuggets
JANUARY 17, 2025
A not-to-be-missed list of elegant Python solutions to perform common programming and processing tasks in a single line of code.
Advertisement
Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.
Cloudyard
JANUARY 15, 2025
Read Time: 2 Minute, 33 Second Snowflakes PARSE_DOCUMENT function revolutionizes how unstructured data, such as PDF files, is processed within the Snowflake ecosystem. Traditionally, this function is used within SQL to extract structured content from documents. However, Ive taken this a step further, leveraging Snowpark to extend its capabilities and build a complete data extraction process.
ArcGIS
JANUARY 21, 2025
Learn how to make a map with SNODAS data in ArcGIS Pro and build a map quickly by planning your data and design early.
Towards Data Science
JANUARY 19, 2025
From Data Lakehouses to Event-Driven Architecture — Master 12 data concepts and turn them into simple projects to stay ahead in IT.
Confluent
JANUARY 14, 2025
CC 2024 Q4 adds private networking and mTLS authentication, follower fetching, Flink updates, WarpStream features to support migration and governance, and more!
Speaker: Nikhil Joshi, Founder & President of Snic Solutions
Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.
Expert insights. Personalized for you.
147 
Let's personalize your content