KDnuggets

NOVEMBER 30, 2023

Data engineers must prepare and manage the infrastructure and tools necessary for the whole data workflow in a data-driven company.

Data Engineer 149

Data Engineer Data Engineering Engineering Data Workflow 149

Analytics Vidhya

NOVEMBER 26, 2023

In this Leading with Data episode, explore the analytics landscape with Dr. Swati Jain, a seasoned leader boasting over two decades of experience. From her unforeseen foray into analytics to steering EXL Analytics’ India business, Dr. Jain imparts invaluable insights into the ever-evolving world of data science. Read on to know more about her career, […] The post Unlocking the Power of Analytics with Dr.

Data Science 214

Data Science Data Analysis Data Data Engineer 214

Jesse Anderson

NOVEMBER 30, 2023

Lately, I’ve been learning how to trade options. Although there’s data and programming involved in options trading, it isn’t as technical as data engineering or software engineering. However, it reflects the current state of learning, whether that’s data engineering or options trading. It gave me a look into learning a skill using videos. Each lesson I learned will directly apply to your learning or skill improvement.

Data Engineer 147

Data Engineer Data Engineering Algorithm Engineering 147

Data Engineering Podcast

NOVEMBER 26, 2023

Summary Building a data platform that is enjoyable and accessible for all of its end users is a substantial challenge. One of the core complexities that needs to be addressed is the fractal set of integrations that need to be managed across the individual components. In this episode Tobias Macey shares his thoughts on the challenges that he is facing as he prepares to build the next set of architectural layers for his data platform to enable a larger audience to start accessing the data being ma

Architecture 147

Architecture Data Lake High Quality Data SQL 147

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

KDnuggets

DECEMBER 1, 2023

The blog covers machine learning courses, bootcamps, books, tools, interview questions, cheat sheets, MLOps platforms, and more to master ML and secure your dream job.

Machine Learning 160

Machine Learning 160

Confessions of a Data Guy

NOVEMBER 25, 2023

Ok. Get off your high horse. You are human just like the rest of us. Just like your ancient ancestors who were throwing rocks and sticks at each other a thousand years ago … you are looking for a leg up on the competition. Isn’t that the world we live in? At the end of […] The post How to be Better Than Everyone Else appeared first on Confessions of a Data Guy.

Data 147

Data Data Engineer Data Engineering Engineering 147

Confluent

NOVEMBER 27, 2023

The top 7 free online courses, tutorials, get started guides, and examples for the easiest way to learn Apache Kafka.

Kafka 132

Kafka 132

KDnuggets

DECEMBER 1, 2023

Image by Author When you are getting started with machine learning, logistic regression is one of the first algorithms you’ll add to your toolbox.

Python 160

Python Building Machine Learning Algorithm 160

ArcGIS

NOVEMBER 30, 2023

Learn how to filter coordinate systems based on a spatial extent, GCS, or projection property.

Project 130

Project Systems 130

Waitingforcode

NOVEMBER 28, 2023

In March I wrote a blog showing how to use accumulators to know the application of each filter statement. Turns out, the solution may not be perfect as mentioned by Aravind in one of the comments. I bet you already have an idea but if not, keep reading. Everything will be clear in the end!

130

130

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Confluent

NOVEMBER 28, 2023

Learn how to write code that produces messages via librdkafka, how it will behave during error situations, and how your application should detect and respond to them.

Coding 131

Coding IT Kafka 131

KDnuggets

DECEMBER 1, 2023

Curious about optimizing AI for everyday devices? Dive into the complete overview of MIT's TinyML and Efficient Deep Learning Computing course. Explore strategies to make AI smarter on small devices. Read the full article for an in-depth look!

Deep Learning 155

Deep Learning Machine Learning 155

Seattle Data Guy

NOVEMBER 28, 2023

Data warehousing would be easy if all data were structured and formatted in the data source. Maybe we wouldn’t even need to build a data warehouse. But as anyone who has worked with data from more than one source knows, that’s rarely the case. Businesses today need to pull data from a plethora of sources,… Read more The post Finding The Right ETL/ELT Solution – What Is Estuary And Should You Use It?

IT 130

IT Data Warehouse Building Data 130

ArcGIS

NOVEMBER 28, 2023

New in ArcGIS Pro 3.

Government 124

Government 124

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

Tweag

NOVEMBER 27, 2023

Sponsored by Antithesis (distributed systems reliability testing experts), I’ve developed a new library to filter local files in Nix which I’d like to introduce! This post requires some familiarity with Nix and its language. So if you don’t know what Nix is yet, take a look first, it’s pretty neat. In this post we’re going to look at what source filtering is, why it’s useful, why a new library was needed for it, and the basics of the new library.

Building 122

Building Systems IT Project 122

KDnuggets

NOVEMBER 29, 2023

From ANI to AGI and Beyond: Deciphering AI's Evolutionary Path.

155

155

Seattle Data Guy

NOVEMBER 27, 2023

If you’re a data engineer, then you’ve likely at least heard of Airflow. Apache Airflow is one of the most popular open-source workflow orchestration solutions that gets used for data pipelines. This is what spurred me to write the article “Should You Use Airflow” because there are plenty of people who don’t enjoy Airflow or… Read more The post Common Pitfalls in Deploying Airflow for Data Teams appeared first on Seattle Data Guy.

Data Pipeline 130

Data Pipeline Data Engineer Data Engineering Data 130

ArcGIS

NOVEMBER 27, 2023

A suite of ArcGIS Solutions to support common workflows in the stormwater industry.

Utilities 122

Utilities Government Architecture Engineering 122

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

databricks

NOVEMBER 30, 2023

We are excited to introduce five new integrations in Databricks Partner Connect—a one-stop portal enabling you to use partner solutions with your Databricks D.

119

119

KDnuggets

NOVEMBER 28, 2023

Want to support the behavior of built-in functions and method calls in your Python classes? Magic methods in Python let you do just that! So let’s uncover the method behind the magic.

Python 154

Python 154

Cloudera

DECEMBER 1, 2023

Recent Government Initiatives on Public Sector AI Solutions In recent years, governments across the globe have recognized the transformative potential of artificial intelligence (AI) and have embarked on initiatives to harness this technology to drive innovation and serve their citizens more effectively. These government-led efforts have had a profound impact on the development and adoption of AI solutions in the public sector, paving the way for a future where data-driven decision-making and au

Building 115

Building Government Transportation Data Governance 115

ArcGIS

DECEMBER 1, 2023

Learn how to resolve error 00374: Unique numeric IDs are not assigned.

118

118

Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

databricks

NOVEMBER 29, 2023

Background: Modernizing Data Delivery Today's enterprise data estates are vastly different from 10 years ago. Industries have transitioned their analytics from monolithic data.

Healthcare 118

Healthcare Government Data 118

KDnuggets

NOVEMBER 29, 2023

Probability is one of the foundational elements of computer science. Some bootcamps will skim over the topic, however, it is integral to your computer science knowledge.

Computer Science 152

Computer Science IT Data Science Data 152

Snowflake

NOVEMBER 30, 2023

At TCS , we help companies shift their enterprise data warehouse (EDW) platforms to the cloud as well as offering IT services. We’re extremely familiar with just how tricky a cloud migration can be, especially when it involves moving historical business data. Choosing a migration approach involves balancing cloud strategy, architecture needs and business priorities.

Data Warehouse 105

Data Warehouse Banking Data Cloud 105

Netflix Tech

NOVEMBER 29, 2023

by Aditya Mavlankar , Zhi Li , Lukáš Krasula and Christos Bampis High dynamic range ( HDR ) video brings a wider range of luminance and a wider gamut of colors, paving the way for a stunning viewing experience. Separately, our invention of Dynamically Optimized ( DO ) encoding helps achieve optimized bitrate-quality tradeoffs depending on the complexity of the content.

Metadata 105

Metadata Electronics Algorithm Software Engineer 105

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

databricks

NOVEMBER 28, 2023

We’re thrilled to share that Databricks has won the AWS ISV Partner of the Year award for North America. This award recognizes top I.

AWS 111

AWS 111

KDnuggets

NOVEMBER 30, 2023

The blog discusses five platforms designed for data scientists with specialized capabilities in managing large datasets, models, workflows, and collaboration beyond what GitHub offers.

Data Science 151

Data Science Project Datasets Data 151

Lyft Engineering

NOVEMBER 29, 2023

Written by Ritesh Varyani and Jeana Choi at Lyft. Introduction At Lyft, we have used systems like Apache ClickHouse and Apache Druid for near real-time and sub-second analytics. Sub-second query systems allow for near real-time data explorations and low latency, high throughput queries, which are particularly well-suited for handling time-series data.

Kafka 105

Kafka Data Ingestion Datasets Architecture 105

Precisely

NOVEMBER 30, 2023

Today’s customers expect businesses to engage with them on their own terms. They want companies to anticipate their needs, personalize offerings to their individual preferences, and present them with multiple ways to interact with the brand. That requires a deep, contextual understanding of each customer’s behaviors, intentions, and wishes. It often means predicting what the customer wants, even before they go looking for it.

Electronics 103

Electronics Management Technology Government 103

Advertisement

Cloud Development Environments (CDEs) are changing how software teams work by moving development to the cloud. Our Cloud Development Environment Adoption Report gathers insights from 223 developers and business leaders, uncovering key trends in CDE adoption. With 66% of large organizations already using CDEs, these platforms are quickly becoming essential to modern development practices.

Cloud