Analytics Vidhya

FEBRUARY 22, 2023

Introduction Suppose you are appearing in an interview for the manager or senior role.

Management 235

Management Google Cloud Cloud Cloud Computing 235

Data Engineering Podcast

FEBRUARY 25, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. What are some of the other tools/technologies that can benefit from some or all of the pieces of the FDAP stack?

Database 162

Database Technology Data Lake High Quality Data 162

Precisely

JANUARY 9, 2025

Disclaimer: Throughout this post, I discuss a variety of complex technologies but avoid trying to explain how these technologies work. The goal of this post is to understand how data integrity best practices have been embraced time and time again, no matter the technology underpinning. Then came Big Data and Hadoop!

Data Integration 118

Data Integration Hadoop Data Warehouse Data Lake 118

Precisely

FEBRUARY 6, 2025

The traditional ways of operations management are over modernization and holistic approaches are now essential. For IT operations (ITOps) teams, 2025 means reassessing technology stacks, processes, and people. Success in tackling modernization of IT operations management starts with assessing where your team is. Whats next?

IT 63

IT Management Datasets Systems 63

Data Engineering Podcast

MAY 12, 2024

Once it is running, the next challenge is figuring out how to address release management for all of the different component parts. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is supported by Code Comments, an original podcast from Red Hat.

Management 147

Management Data Lake High Quality Data Machine Learning 147

Seattle Data Guy

JULY 2, 2024

All while keeping up with the latest changes in technology. In the past few years I have gotten to work with dozens of teams and see how various directors and managers deal… Read more The post 9 Habits Of Effective Data Managers – Running A Data Team appeared first on Seattle Data Guy.

Data Management 130

Data Management Management Data Project 130

Data Engineering Weekly

DECEMBER 28, 2024

Investment in an Agent Management System (AMS) is crucial, as it offers a framework for scaling, monitoring, and refining AI agents. AI engineers, in particular, will find their skills in high demand as they navigate managing and optimizing agents to ensure reliability within enterprise systems.

Unstructured Data 112

Unstructured Data Metadata Data Government 112

databricks

SEPTEMBER 30, 2024

This blog explores how new technologies such as Databricks Data Intelligence Platform can pave the way for more effective and efficient multi-omics data management.

Data Management 118

Data Management Management Data Technology 118

RandomTrees

DECEMBER 4, 2024

As one of the most important sectors of the global economy, the food and beverage (F&B) industry works in highly volatile conditions and ensures its success by reducing waste and managing inventories. Managing production and consumption, meeting deadlines, cutting waste, and being environmentally friendly are always a challenge.

Food 52

Food Management Manufacturing Machine Learning 52

The Pragmatic Engineer

NOVEMBER 21, 2023

In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. After Zynga, he rejoined Amazon, and was the General Manager (GM) for Compute services at AWS, and later chief of staff, and advisor to AWS executives like Charlie Bell and Andy Jassy (Amazon’s current CEO.)

Engineering 345

Engineering Bytes Cloud Computing AWS 345

Snowflake

JANUARY 8, 2025

Though AI is (still) the hottest technology topic, its not the overriding issue for enterprise security in 2025. AI is both a contributor to the problem more data to secure, more attack surface and a potential boon, providing tools to manage amounts of data that humans cant grasp on their own.

Data Lake 105

Data Lake Data Security Machine Learning Technology 105

Engineering at Meta

JANUARY 22, 2025

Data lineage is an instrumental part of Metas Privacy Aware Infrastructure (PAI) initiative, a suite of technologies that efficiently protect user privacy. Thus we have developed a powerful flow discovery tool as part of our PAI tool suite, Policy Zone Manager (PZM), based on data lineage.

Data Warehouse 115

Data Warehouse SQL Programming Language Data 115

Monte Carlo

NOVEMBER 26, 2024

Operational use cases were rising to the surface, technology was reducing barriers to entry, and general artificial intelligence was obviously right around the corner. So, if it’s not generating revenue, AI needs to be cutting costs—and in that regard, this budding technology has certainly found some footing. Well, sort of.

Data Engineering 105

Data Engineering Data Engineer Engineering Unstructured Data 105

Snowflake

APRIL 4, 2024

As companies collect more data, from disparate sources and in disparate formats, building and managing transformations has become exponentially more complex and time-consuming. Our goal at Snowflake is to provide developers, data engineers, and other users with optimal choice in the tools they use to prepare and manage data.

Cloud 131

Cloud Management Data Pipeline SQL 131

Analytics Vidhya

MARCH 5, 2023

A distributed file system runs on commodity hardware and manages massive data collections. It is a fully managed cloud-based environment for analyzing and processing enormous volumes of data. Introduction Microsoft Azure HDInsight(or Microsoft HDFS) is a cloud-based Hadoop Distributed File System version.

Hadoop 254

Hadoop Data Collection Cloud Technology 254

Snowflake

MARCH 31, 2025

Despite the best efforts of many ML teams, most models still never make it to production due to disparate tooling, which often leads to fragmented data and ML pipelines and complex infrastructure management. Customers include CHG Healthcare, Keysight Technologies and Avios. With over $5.5

Healthcare 93

Healthcare Medical Government Food 93

The Pragmatic Engineer

SEPTEMBER 22, 2023

The npm or pnpm package managers do this additional check and so are slower, though arguably more reliable. For most commercial open source companies, the approach tends to be to offer a managed, dedicated service of the open source software, or to provide additional features on top of it.

Programming Language 316

Programming Language Project Coding Engineering 316

Snowflake

DECEMBER 4, 2024

In the enterprise technology space, both the greatest certainties and the most significant potential surprises come from one area: the rapidly advancing field of artificial intelligence. But businesses will continue to hesitate to put in front of customers a technology that may display bias or provide inaccurate responses.

Unstructured Data 95

Unstructured Data Data Lake Deep Learning Structured Data 95

Precisely

JANUARY 30, 2025

As we reflect on the data management outcomes our customers achieved in 2024 and the goals they have for their 2025 business initiatives, one thing is clear: modern data governance remains at the heart of successful data strategies. Align people, processes, and technology Successful data governance requires a holistic approach.

Data Governance 102

Data Governance Government Metadata Data 102

The Pragmatic Engineer

SEPTEMBER 19, 2024

Without the backing of management, a large-scale rewrite is likely to fail. In 2004, I was hired by ISO-NE, a non-profit that manages the electric grid in New England. They hired a manager who had done this kind of project before, and set a target date of nearly two years. Big rewrites need heavyweight support.

Recruitment 297

Recruitment Java Coding Project 297

Snowflake

JANUARY 21, 2025

Data and technology (yes, AI) can now deeply impact the relevance of advertising creative, but that data needs to be secured and democratized across all levels and all departments within the agency landscape. How do their agencies manage new privacy regulations both globally and in the U.S.?

Media 96

Media Unstructured Data Data Retail 96

Data Engineering Podcast

APRIL 23, 2023

In this episode DeVaris Brown discusses the types of applications that are possible when teams don't have to manage the complex infrastructure necessary to support continuous data flows. Can you describe what Meroxa is and the story behind it? How have the focus and goals of the platform and company evolved over the past 2 years?

Data Lake 277

Data Lake Kafka Machine Learning Data Warehouse 277

Snowflake

APRIL 23, 2025

Free comprehensive teaching resources and a no-hassle setup: Teaching AI/ML, data science, apps and data cloud technologies shouldnt be bogged down by logistical challenges. Access to cutting-edge data technology: Snowflake is widely used by more than 10,000 enterprises across industries such as healthcare, financial services and retail.

Education 94

Education Certification Portfolio Retail 94

Data Engineering Podcast

FEBRUARY 18, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. To start, can you share your definition of what constitutes a "Data Lakehouse"? What do you have planned for the future of Trino/Starburst?

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Precisely

NOVEMBER 18, 2024

A long-term approach to your data strategy is key to success as business environments and technologies continue to evolve. The rapid pace of technological change has made data-driven initiatives more crucial than ever within modern business strategies. However, they require a strong data foundation to be effective.

Data Analytics 71

Data Analytics Data Governance Data Integration Government 71

Analytics Vidhya

FEBRUARY 21, 2023

Introduction Nowadays, the corporate environment changes according to technology. Organizations are converting them to cloud-based technologies for the convenience of data collecting, reporting, and analysis.

Data Warehouse 211

Data Warehouse IT Data Collection Data 211

Snowflake

APRIL 2, 2025

Without adopting Iceberg tables, data teams are forced to spend significant time and resources managing migrations and governance before being able to capture the opportunities new technologies and solutions offer. As technology continues to evolve, Snowflake continues to prioritize its customers by supporting open source initiatives.

Data Lake 96

Data Lake Cloud Storage Metadata Data Warehouse 96

Snowflake

NOVEMBER 6, 2024

But as technology speeds forward, organizations of all sizes are realizing that generative AI isn’t just aspirational: It’s accessible and applicable now. You can begin to recognize that this technology, built with Cortex AI, can even start driving care, including suggesting and creating orders.

Hospitality 95

Hospitality Medical Government Software Engineer 95

The Pragmatic Engineer

JUNE 1, 2023

Juraj created a systems design explainer on how he built this project, and the technologies used: The systems design diagram for the Rides application The app uses: Node.js Both as an engineer, and especially as an engineering manager, don’t forget there’s a real cost to setting up and then maintaining infrastructure.

Education 364

Education Project PostgreSQL Software Engineer 364

Snowflake

NOVEMBER 26, 2024

In essence, that was the story of WHOOP , the Boston-based wearable technology company aimed at enhancing human performance and endorsed by superstar athletes such as LeBron James and Cristiano Ronaldo. Data becomes distributed across multiple platforms; different teams end up using different tools.

Data Warehouse 104

Data Warehouse Cloud PostgreSQL Hadoop 104