Precisely

NOVEMBER 20, 2023

AI transforms how we interact with technology, make decisions, and solve complex problems. It has been at the heart of many innovations over the past two years, powering everything from the chatbots that enhance our customer experiences to the predictive analytics engines that help us make financial decisions. What defines a successful AI initiative, and how can your organization ensure that your investments and hard work deliver maximum value for your organization?

Raw Data 121

Raw Data Insurance Data Portfolio 121

Start Data Engineering

NOVEMBER 14, 2023

1. Introduction 2. What is an Open Table Format (OTF) 3. Why use an Open Table Format (OTF) 3.0. Setup 3.1. Evolve data and partition schema without reprocessing 3.2. See previous point-in-time table state, aka time travel 3.3. Git like branches & tags for your tables 3.4. Handle multiple reads & writes concurrently 4. Conclusion 5. Further reading 6.

Data 322

Data 322

The Pragmatic Engineer

NOVEMBER 21, 2023

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. In this article, we cover thee out of nine topics from today’s subscriber-only issue: The Past and Future of Modern Backend Practices.

Engineering 301

Engineering Bytes Cloud Computing AWS 301

Analytics Vidhya

NOVEMBER 8, 2023

Introduction Imagine yourself in command of a sizable cargo ship sailing through hazardous waters. It is your responsibility to deliver precious cargo to its destination safely. Determine success by the precision of your charts, the equipment’s dependability, and your crew’s expertise. A single mistake, glitch, or slip-up could endanger the trip. In the data-driven world […] The post Monitoring Data Quality for Your Big Data Pipelines Made Easy appeared first on Analytics Vidhya.

Big Data 246

Big Data Data Pipeline Data IT 246

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

KDnuggets

NOVEMBER 23, 2023

Are you excited to learn about and build machine learning models? Start learning today with these free machine learning courses.

Machine Learning 157

Machine Learning Building 157

Jesse Anderson

NOVEMBER 30, 2023

Lately, I’ve been learning how to trade options. Although there’s data and programming involved in options trading, it isn’t as technical as data engineering or software engineering. However, it reflects the current state of learning, whether that’s data engineering or options trading. It gave me a look into learning a skill using videos. Each lesson I learned will directly apply to your learning or skill improvement.

Data Engineering 147

Data Engineering Data Engineer Algorithm Engineering 147

Confessions of a Data Guy

NOVEMBER 25, 2023

Ok. Get off your high horse. You are human just like the rest of us. Just like your ancient ancestors who were throwing rocks and sticks at each other a thousand years ago … you are looking for a leg up on the competition. Isn’t that the world we live in? At the end of […] The post How to be Better Than Everyone Else appeared first on Confessions of a Data Guy.

Data 147

Data Data Engineering Data Engineer Engineering 147

The Pragmatic Engineer

NOVEMBER 9, 2023

The below topic was sent out to full subscribers of The Pragmatic Engineer , three weeks ago, in The Pulse #66. I have received several messages from people asking if they can pay to “unlock” this information for others, given how vital it is for software engineers. It is vital, and so I’m sharing this with all readers, without a paywall.

Software Engineering 226

Software Engineering Software Engineer Engineering Coding 226

Analytics Vidhya

NOVEMBER 26, 2023

In this Leading with Data episode, explore the analytics landscape with Dr. Swati Jain, a seasoned leader boasting over two decades of experience. From her unforeseen foray into analytics to steering EXL Analytics’ India business, Dr. Jain imparts invaluable insights into the ever-evolving world of data science. Read on to know more about her career, […] The post Unlocking the Power of Analytics with Dr.

Data Science 214

Data Science Data Analysis Data Data Engineering 214

KDnuggets

NOVEMBER 24, 2023

Data scientists love it! See how ChatGPT creates jaw-dropping data viz with just a few words - it's almost unfair how easy it is.

Data 153

Data IT Data Science 153

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Jesse Anderson

NOVEMBER 14, 2023

A Guest Post by Ole Olesen-Bagneux In this blog post I would like to describe a new data team, that I call ‘the data discovery team’. It’s a team that connects naturally into the constellation of the three data teams Operations team Data engineering team Data Science team as described in Jesse Anderson’s book Data Teams (2020) Before I explain what the data discovery team should do, it is necessary to add a bit of context on the concept of data discovery itself.

Metadata 147

Metadata Data Science Big Data Data 147

Data Engineering Podcast

NOVEMBER 19, 2023

Summary The dbt project has become overwhelmingly popular across analytics and data engineering teams. While it is easy to adopt, there are many potential pitfalls. Dustin Dorsey and Cameron Cyr co-authored a practical guide to building your dbt project. In this episode they share their hard-won wisdom about how to build and scale your dbt projects.

Project 147

Project Data Lake High Quality Data SQL 147

databricks

NOVEMBER 15, 2023

The observation that "software is eating the world" has shaped the modern tech industry. Today, software is ubiquitous in our lives, from the.

Data 144

Data 144

ArcGIS

NOVEMBER 7, 2023

From oriented imagery to engaging thematic map series, there is something for everyone in this release of ArcGIS Pro 3.2.

143

143

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

Zalando Engineering

NOVEMBER 8, 2023

Introduction This blog post describes a recent contribution from Zalando to the Postgres JDBC driver to address a long-standing issue with the driver’s integration with Postgres’ logical replication that resulted in runaway Write-Ahead Log (WAL) growth. We will describe the issue, how it affected us at Zalando, and detail the fix made upstream in the JDBC driver that fixes the issue for Debezium and all other clients of the Postgres JDBC driver.

PostgreSQL 138

PostgreSQL Java Database Bytes 138

KDnuggets

NOVEMBER 29, 2023

Probability is one of the foundational elements of computer science. Some bootcamps will skim over the topic, however, it is integral to your computer science knowledge.

Computer Science 152

Computer Science IT Data Science Data 152

Confluent

NOVEMBER 27, 2023

The top 7 free online courses, tutorials, get started guides, and examples for the easiest way to learn Apache Kafka.

Kafka 132

Kafka 132

Data Engineering Podcast

NOVEMBER 12, 2023

Summary Software development involves an interesting balance of creativity and repetition of patterns. Generative AI has accelerated the ability of developer tools to provide useful suggestions that speed up the work of engineers. Tabnine is one of the main platforms offering an AI powered assistant for software engineers. In this episode Eran Yahav shares the journey that he has taken in building this product and the ways that it enhances the ability of humans to get their work done, and when t

Software Engineering 147

Software Engineering Software Engineer Engineering Data Lake 147

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

databricks

NOVEMBER 6, 2023

We are excited to announce that we have completed our acquisition of Arcion, a leading provider for real-time data replication technologies. Arcion’s capabilities w.

Data 137

Data Technology 137

ArcGIS

NOVEMBER 21, 2023

Contextual keyboard shortcuts open up a whole new world of customization possibilities in ArcGIS Pro 3.2.

133

133

Waitingforcode

NOVEMBER 28, 2023

In March I wrote a blog showing how to use accumulators to know the application of each filter statement. Turns out, the solution may not be perfect as mentioned by Aravind in one of the comments. I bet you already have an idea but if not, keep reading. Everything will be clear in the end!

130

130

KDnuggets

NOVEMBER 6, 2023

Thinking about getting into the data analytical world but do not know where to start? Have a look at these 5 FREE university courses on data analytics.

Data Analytics 152

Data Analytics Data Data Science 152

Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

Confluent

NOVEMBER 28, 2023

Learn how to write code that produces messages via librdkafka, how it will behave during error situations, and how your application should detect and respond to them.

Coding 131

Coding IT Kafka 131

Data Engineering Podcast

NOVEMBER 5, 2023

Summary Databases are the core of most applications, but they are often treated as inscrutable black boxes. When an application is slow, there is a good probability that the database needs some attention. In this episode Lukas Fittl shares some hard-won wisdom about the causes and solution of many performance bottlenecks and the work that he is doing to shine some light on PostgreSQL to make it easier to understand how to keep it running smoothly.

PostgreSQL 147

PostgreSQL Data Lake High Quality Data SQL 147

databricks

NOVEMBER 21, 2023

We recently announced our AI-generated documentation feature, which uses large language models (LLMs) to automatically generate documentation for tables and columns in Unity.

Data Science 136

Data Science Engineering Data 136

Seattle Data Guy

NOVEMBER 28, 2023

Data warehousing would be easy if all data were structured and formatted in the data source. Maybe we wouldn’t even need to build a data warehouse. But as anyone who has worked with data from more than one source knows, that’s rarely the case. Businesses today need to pull data from a plethora of sources,… Read more The post Finding The Right ETL/ELT Solution – What Is Estuary And Should You Use It?

IT 130

IT Data Warehouse Building Data 130

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

Engineering at Meta

NOVEMBER 8, 2023

New optional features in WhatsApp have helped make calling on WhatsApp more secure. “Silence Unknown Callers” is a new setting on WhatsApp that not only quiets annoying calls but also blocks sophisticated cyber attacks. “Protect IP Address in Calls” is a new setting on WhatsApp that helps hide your location from other parties on the call. Privacy and security are at the core of WhatsApp.

Metadata 130

Metadata Process Designing Data Process 130

KDnuggets

NOVEMBER 29, 2023

From ANI to AGI and Beyond: Deciphering AI's Evolutionary Path.

151

151

Waitingforcode

NOVEMBER 21, 2023

Even though you may be thinking now about Data+AI Summit 2024, I still owe you my retrospective for the 2023 edition. Let's start with the first part covering stream processing talks!

Data 130

Data Process 130

ArcGIS

NOVEMBER 14, 2023

Here's everything new in ArcGIS Pro 3.2 from the Geodatabase Team. Schema Reports, 64-bit OIDs, Big Integer fields, new date fields, etc.

Data Management 128

Data Management Management Data 128

Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage

When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.

Systems