Data Engineering Podcast

NOVEMBER 19, 2023

Summary The dbt project has become overwhelmingly popular across analytics and data engineering teams. Dustin Dorsey and Cameron Cyr co-authored a practical guide to building your dbt project. In this episode they share their hard-won wisdom about how to build and scale your dbt projects. What was your path to adoption of dbt?

Project 147

Project Data Lake High Quality Data SQL 147

Seattle Data Guy

JULY 2, 2024

Data teams are expected to juggle a combination of ad-hoc requests, big bet projects, migrations, etc. All while keeping up with the latest changes in technology.

Data Management 130

Data Management Management Data Project 130

KDnuggets

NOVEMBER 14, 2024

Manage Python projects, run scripts and tools, handle dependencies, and install packages—all with the uv tool.

Python 128

Python Management Project 128

KDnuggets

JANUARY 22, 2025

Are you looking for an easier way to manage files across different storage systems? fsspec is a Python library that simplifies file handling by providing a unified interface for file management.

Python 93

Python Management Project Systems 93

Data Engineering Podcast

MAY 12, 2024

Once it is running, the next challenge is figuring out how to address release management for all of the different component parts. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is supported by Code Comments, an original podcast from Red Hat.

Management 147

Management Data Lake High Quality Data Machine Learning 147

KDnuggets

DECEMBER 4, 2024

Learn to build, run, and manage data engineering pipelines both locally and in the cloud using popular tools.

Data Engineering 150

Data Engineering Data Engineer Engineering Project 150

Data Engineering Podcast

MARCH 31, 2024

To bring observability to dbt projects the team at Elementary embedded themselves into the workflow. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. Can you start by outlining what elements of observability are most relevant for dbt projects?

Project 130

Project Data Lake High Quality Data Data Workflow 130

Start Data Engineering

SEPTEMBER 23, 2024

Steps to decide on a data project to build 2.1. Potential referral/hiring manager research 2.2.3. Introduction 2. Objective 2.2. Research 2.2.1. Job description 2.2.2. Company research 2.3. Data 2.3.1. Dataset Search 2.3.2. Generate fake data 2.4. Outcome 2.4.1. Visualization 2.5. Presentation 3. Conclusion 4. Read these 1.

Portfolio 130

Portfolio Project Datasets Data 130

KDnuggets

SEPTEMBER 17, 2024

In this post, you'll learn what BigQuery is, understand its capabilities, and set up a project in Google Cloud which we will later use to practice using BigQuery for loading, querying, and analyzing data.

Project 117

Project Google Cloud Cloud Management 117

KDnuggets

AUGUST 5, 2024

Get noticed by recruiters and hiring managers by creating and documenting the following AI projects.

Portfolio 117

Portfolio Project Recruitment Management 117

The Pragmatic Engineer

OCTOBER 10, 2024

Wordpress is the most popular content management system (CMS),  estimated  to power around 43% of all websites; a staggering number! Automattic generates most of its revenue by offering managed Wordpress hosting. WP Engine also sells managed Wordpress services, making it a direct competitor to Automattic.

Consulting 316

Consulting AWS Engineering Software Engineer 316

Data Engineering Podcast

JUNE 11, 2023

Datafold has invested a lot of time into integrating with the workflow of dbt projects to add early verification that the changes you are making are correct. What are some of the error conditions/failure modes that data-diff can help identify in a dbt project? What are the parallels to that in data projects?

Project 130

Project Building Data Lake Machine Learning 130

The Pragmatic Engineer

SEPTEMBER 19, 2024

Without the backing of management, a large-scale rewrite is likely to fail. This project helped onboard me to the software, its structure, its build, and our issue tracking and version control workflows. My first project was supporting i18n (internationalization) in the app. Big rewrites need heavyweight support.

Recruitment 260

Recruitment Java Coding Project 260

Knowledge Hut

MAY 21, 2024

When it comes to IT projects, the first step is always creating a project charter. This document outlines the project's goals and how everyone involved will work together to achieve them. The importance of project charter cannot be overlooked. What is Project Charter ? Why do you Need a Project Charter?

Project 98

Project IT Certification Consulting 98

KDnuggets

OCTOBER 17, 2024

This article will discuss how to set up and streamline data science projects using Monday.com for project management among teams.

Data Science 115

Data Science Project Data Management 115

Snowflake

NOVEMBER 2, 2023

In the ever-evolving world of data management, Snowflake is at the forefront of empowering our customers to make informed decisions about data while ensuring cost efficiency and control. Admins know that managing and optimizing platform costs can be a complex and time-consuming task.

Management 118

Management Retail SQL AWS 118

Knowledge Hut

MAY 6, 2024

The scope keeps changing as the project makes progress on the agreed timeline, so, the estimation becomes a challenging task for the product owner. Agile software estimation is used for realistic planning to implementing the project requirements as per commitments. Master the art of project management and achieve new heights.

Project 98

Project Programming Certification Database 98

The Pragmatic Engineer

OCTOBER 18, 2024

Corporate conflict recap Automattic is the creator of open source WordPress content management system (CMS), and WordPress powers an incredible 43% of webpages and 65% of CMSes. But leveraging a supposedly neutral platform (the WordPress plugin manager) should not be the way to win in business – at least not in open source.

Government 335

Government Engineering Project AWS 335

Tweag

NOVEMBER 15, 2023

Modern development projects are synthesizer-like these days. Organist is a tool meant to do just that: Provide you with an ergonomic way of managing the complexity of your development environment. Be tailored to your project. We combine a plethora of tools, each one coming with its own set of knobs.

Management 120

Management Python Programming Language Project 120

Engineering at Meta

MAY 22, 2024

In recent years, Meta’s data management systems have evolved into a composable architecture that creates interoperability, promotes reusability, and improves engineering efficiency. A few years ago we embarked on a journey to address these shortcomings by rethinking how our data management systems were designed.

Data Management 104

Data Management Management Data SQL 104

Knowledge Hut

MAY 9, 2024

The Lean Six Sigma Green Belt certification is an important step in becoming a master of the lean six sigma technique and leading improvement projects for a company. Before we jump to the six sigma green belt project examples and ideas, let us learn some of the basic aspects of Six Sigma Green Belt. It should not need help.

Project 98

Project Manufacturing Certification Consulting 98

Engineering at Meta

FEBRUARY 20, 2024

This new convergence helps Meta and the larger community build data management systems that are unified, more efficient, and composable. Meta’s Data Infrastructure teams have been rethinking how data management systems are designed. An introduction to Velox Velox is the first project in our composable data management system program.

Data Management 113

Data Management Bytes Management Datasets 113

Knowledge Hut

SEPTEMBER 24, 2024

What is management ? What is the difference between governing body and management? What is the relevance of meetings in management? Does the management layer need to conduct so many meetings? I am sure most of us have attended or conducted meetings as a part of management governance.

Government 98

Government Management Project Certification 98

The Pragmatic Engineer

SEPTEMBER 22, 2023

The npm or pnpm package managers do this additional check and so are slower, though arguably more reliable. It begins with a clean state, and can ship something that works for, say, 90% of existing Node projects, and break the remaining 10%. The project would simply need volunteer contributors, or sponsors who pay contributions.

Programming Language 281

Programming Language Project Coding Engineering 281

KDnuggets

NOVEMBER 30, 2023

The blog discusses five platforms designed for data scientists with specialized capabilities in managing large datasets, models, workflows, and collaboration beyond what GitHub offers.

Data Science 142

Data Science Project Datasets Data 142

The Pragmatic Engineer

MARCH 12, 2024

In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. This is how Brooks put it: “Programming managers have long recognized wide productivity variations between good programmers and poor ones. (.) Progress does not. Two secretaries.

Engineering 299

Engineering Programming Language Hospitality Programming 299

Snowflake

APRIL 4, 2024

As companies collect more data, from disparate sources and in disparate formats, building and managing transformations has become exponentially more complex and time-consuming. This expands the number of users who can contribute to data projects and enhances collaboration. The Coalesce platform is uniquely built for Snowflake.

Cloud 121

Cloud Management Data Pipeline SQL 121

Analytics Vidhya

MARCH 7, 2023

Introduction Apache Cassandra is a NoSQL database management system that is open-source and distributed. Facebook created Cassandra, which ultimately became an Apache Software Foundation project. It is well-known for its rapid write […] The post Top 6 Cassandra Interview Questions appeared first on Analytics Vidhya.

NoSQL 251

NoSQL Database Project Systems 251

Analytics Vidhya

FEBRUARY 6, 2023

Introduction While working with multiple projects, there are chances of issues with versions of packages in python; for example, a project needs a new version of a package, and another requires a different version. Sometimes the python version itself changes from project to project.

Python 218

Python Project Management Data Engineering 218

Snowflake

FEBRUARY 13, 2025

. "Serverless computing" has enabled customers to use cloud capabilities without provisioning, deploying and managing either hardware or software resources. Snowflake has embraced serverless since our founding in 2012, with customers providing their code to load, manage and query data and us taking care of the rest.

Management 73

Management Government Cloud Unstructured Data 73

The Pragmatic Engineer

NOVEMBER 21, 2023

In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. After Zynga, he rejoined Amazon, and was the General Manager (GM) for Compute services at AWS, and later chief of staff, and advisor to AWS executives like Charlie Bell and Andy Jassy (Amazon’s current CEO.)

Engineering 325

Engineering Bytes Cloud Computing AWS 325