article thumbnail

Top 11 GenAI Powered Data Engineering Tools to Follow in 2025

Analytics Vidhya

What will data engineering look like in 2025? How will generative AI shape the tools and processes Data Engineers rely on today? As the field evolves, Data Engineers are stepping into a future where innovation and efficiency take center stage.

article thumbnail

Mentoring software engineers or engineering leaders

The Pragmatic Engineer

I get asked every now and then if I offer 1:1 mentoring for either software engineers or engineering managers or leaders. I collected much of the advice I have to offer for software engineers in The Software Engineer's Guidebook. While I used to do this in the past, I don't offer this any more.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

The Roots of Today's Modern Backend Engineering Practices

The Pragmatic Engineer

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers.   To succeed as a software engineer, you needed to be a jack-of-all-trades.

article thumbnail

Is Critical Thinking the Most Important Skill for Software Engineers?

The Pragmatic Engineer

When I think back on the software engineers I looked up to, they all shared this trait where they never took anything at face value. After the meeting, I asked other engineers in private who were nodding along if they knew what idempotency was. Some senior+ engineers loved using jargon, and used it all the time.

article thumbnail

Apache Airflow® 101 Essential Tips for Beginners

Due to its widespread adoption, Airflow knowledge is paramount to success in the field of data engineering. It is a versatile tool used in companies across the world from agile startups to tech giants to flagship enterprises across all industries.

article thumbnail

The “10x engineer:" 50 years ago and now

The Pragmatic Engineer

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. The “10x engineer” has always been a source of debate and contention.

article thumbnail

Data Engineering Projects

Start Data Engineering

Introduction Whether you are new to data engineering or have been in the data field for a few years, one of the most challenging parts of learning new frameworks is setting them up! Projects from least to most complex 3.2. Batch pipelines 3.3. Stream pipelines 3.4. Event-driven pipelines 3.5. LLM RAG pipelines 4. Conclusion 1.

article thumbnail

Agent Tooling: Connecting AI to Your Tools, Systems & Data

Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage

As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward. There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data.

article thumbnail

Generative AI Deep Dive: Advancing from Proof of Concept to Production

Speaker: Maher Hanafi, VP of Engineering at Betterworks & Tony Karrer, CTO at Aggregage

💡 This new webinar featuring Maher Hanafi, VP of Engineering at Betterworks, will explore a practical framework to transform Generative AI prototypes into impactful products! There's no question that it is challenging to figure out where to focus and how to advance when it’s a new field that is evolving everyday.

article thumbnail

Improving the Accuracy of Generative AI Systems: A Structured Approach

Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage

Using this case study, he'll also take us through his systematic approach of iterative cycles of human feedback, engineering, and measuring performance. . 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.

article thumbnail

LLMs in Production: Tooling, Process, and Team Structure

Speaker: Dr. Greg Loughnane and Chris Alexiuk

Greg Loughnane and Chris Alexiuk in this exciting webinar to learn all about: How to design and implement production-ready systems with guardrails, active monitoring of key evaluation metrics beyond latency and token count, managing prompts, and understanding the process for continuous improvement Best practices for setting up the proper mix of open- (..)

article thumbnail

Monetizing Analytics Features: Why Data Visualizations Will Never Be Enough

Think your customers will pay more for data visualizations in your application? Five years ago they may have. But today, dashboards and visualizations have become table stakes. Discover which features will differentiate your application and maximize the ROI of your embedded analytics. Brought to you by Logi Analytics.

article thumbnail

How to Optimize the Developer Experience for Monumental Impact

Speaker: Anne Steiner and David Laribee

As an innovative concept, Developer Experience (DX) has gained significant attention in the tech industry, and emphasizes engineers’ efficiency and satisfaction during the product development process. Attendance of this webinar will earn one PDH toward your NPDP certification for the Product Development and Management Association.

article thumbnail

The Ultimate Guide to Apache Airflow DAGS

With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer. This eBook provides a comprehensive overview of DAG writing features with plenty of example code.

article thumbnail

How to Achieve High-Accuracy Results When Using LLMs

Speaker: Ben Epstein, Stealth Founder & CTO | Tony Karrer, Founder & CTO, Aggregage

In this new session, Ben will share how he and his team engineered a system (based on proven software engineering approaches) that employs reproducible test variations (via temperature 0 and fixed seeds), and enables non-LLM evaluation metrics for at-scale production guardrails.