Knowledge Hut

JANUARY 22, 2024

Entity set definitions usually include a name and a description of the entities in the set. Entity sets are used to model real-world objects, such as customers, products, or orders. They can also be used to model concepts, such as profit margins or customer satisfaction levels.

MongoDB 98

MongoDB Database Data Mining Relational Database 98

Knowledge Hut

FEBRUARY 1, 2024

Are you looking for the perfect movie recommendation based on your tastes? Machine learning (ML) offers just that - an advanced technology to help users find the exact films they want.

Systems 98

Systems Entertainment Algorithm Datasets 98

Knowledge Hut

JANUARY 2, 2024

The preferred monitoring solution for Kubernetes and Docker is gradually evolving towards Prometheus monitoring. This manual demonstrates how to use Prometheus for Kubernetes monitoring.

Architecture 98

Architecture Metadata Utilities Data Collection 98

Netflix Tech

DECEMBER 17, 2024

Metric definitions are often scattered across various databases, documentation sites, and code repositories, making it difficult for analysts and data scientists to find reliable information quickly. DJ acts as a central store where metric definitions can live and evolve. Enter DataJunction (DJ).

Engineering 84

Engineering Entertainment Amazon Web Services Utilities 84

The Pragmatic Engineer

AUGUST 1, 2023

Amazon’s SDE3 definition is closer to what is considered a “team lead” or “tech lead” role at many companies, and so I view this SDE3 role as being at the upper end of a senior engineer definition. ” A summary of this level’s typical expectations: A summary of Amazon’s SDE3 level.

Software Engineering 246

Software Engineering Software Engineer Engineering Designing 246

Christophe Blefari

MARCH 1, 2023

In a nutshell the dbt journey starts with sources definition on which you will define models that will transform these sources to something else you'll need in your downstream usage of the data. You can read dbt's official definitions. You can read my What is a metrics store to help you understand.

Data Warehouse 278

Data Warehouse SQL Metadata Raw Data 278

The Pragmatic Engineer

JUNE 22, 2023

The press release: “Squarespace announced today it has entered into a definitive asset purchase agreement with Google, whereby Squarespace will acquire the assets associated with the Google Domains business, which will be winding down following a transition period. ” So what’s being sold, exactly?

Google Cloud 304

Google Cloud Media Cloud Engineering 304

dbt Developer Hub

APRIL 20, 2025

With the dbt MCP server, LLMs can understand and query these metrics directly, ensuring that AI-generated analyses are consistent with your organization's definitions. For human stakeholders : Request metrics using natural language.

Structured Data 98

Structured Data SQL BI Project 98

KDnuggets

JUNE 13, 2022

Gain a beginner's perspective on artificial neural networks and deep learning with this set of 14 straight-to-the-point related key concept definitions.

Deep Learning 160

Deep Learning Machine Learning 160

Cloudera

NOVEMBER 13, 2024

In today’s heterogeneous data ecosystems, integrating and analyzing data from multiple sources presents several obstacles: data often exists in various formats, with inconsistencies in definitions, structures, and quality standards.

Metadata 90

Metadata Management Data Governance Government 90

Knowledge Hut

NOVEMBER 25, 2024

Even if you are not planning to take the PSPO test (for whatever reason), you should nonetheless follow the processes outlined in the PSPO study guide and the PSPO Scrum Certification Guide before continuing.

Certification 97

Certification Business Analyst Consulting Education 97

The Pragmatic Engineer

AUGUST 10, 2023

Ayhan visualized this data and observed a definite fall in all metrics: page views, visits, questions asked, votes. Q&A activity is definitely down: the company is aware of this metric taking a dive, and said they’re actively working to address it.

Retail 207

Retail Utilities Software Engineer Software Engineering 207

Precisely

JANUARY 9, 2025

But those end users werent always clear on which data they should use for which reports, as the data definitions were often unclear or conflicting. And then a wide variety of business intelligence (BI) tools popped up to provide last mile visibility with much easier end user access to insights housed in these DWs and data marts.

Data Integration 119

Data Integration Hadoop Data Warehouse Data Lake 119

Snowflake

APRIL 2, 2025

Manage secure multi-engine access with Snowflakes Open Catalog , a fully managed service for Polaris that preserves the option to self-manage by maintaining role-based access controls (RBAC), namespaces and definitions intact, regardless of where the catalog is hosted nearly eliminating migration complexity.

Data Lake 97

Data Lake Metadata Cloud Storage Data Warehouse 97

The Pragmatic Engineer

APRIL 19, 2023

ChatGPT can be an unexpectedly useful to break down jargon terms: just don't forget to verify its definition, as ChatGPT can make things up. If someone is telling you jargon terms, ask them to explain simply, and challenge them if they cannot do so. Otherwise, understand the jargon in simple terms, yourself.

Software Engineering 343

Software Engineering Software Engineer Engineering Media 343

Data Engineering Podcast

FEBRUARY 18, 2024

To start, can you share your definition of what constitutes a "Data Lakehouse"? To start, can you share your definition of what constitutes a "Data Lakehouse"? What are the technical/architectural/UX challenges that have hindered the progression of lakehouses?

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

The Pragmatic Engineer

MARCH 23, 2023

By this definition, the peak was February-November 2021: The “peak job market” for the number of companies posting positions on Hacker News. So when was the peak in the number of “roles vacant” posts? “Peak” refers to 50% more companies posting than in Feb 2020.

Software Engineering 279

Software Engineering Software Engineer Engineering Recruitment 279

Towards Data Science

JANUARY 30, 2025

Key findings across multipleruns: Furthest-from-centroid consistently outperformed othermethods There definitely is a sweet spot between compute time and and model accuracy if you want to find it for your use case. Image byauthor. Median Run time across Data Pruning methods. Image byauthor. More work needs to be donehere.

Database-centric 106

Database-centric Datasets Data Architecture 106

DataKitchen

FEBRUARY 20, 2025

Enhanced Testing & Profiling Copy & Move Tests with Ease The Test Definitions page now supports seamless test migration between test suites. Custom Branding Replace the DataKitchen logo in the navigation menu with your companys branding for a personalized experience.

Datasets 73

Datasets Metadata Data Government 73

The Pragmatic Engineer

SEPTEMBER 26, 2023

CDE vendors: a definite trend With more than 20 players in this space, let’s begin with a timeline of when the products launched. It’s also possible to posit that the “spark” which ignited this confluence of factors was the Covid-19 pandemic of 2020-21, and the accompanying rise of remote work.

Cloud 316

Cloud Software Engineer Software Engineering Cloud Computing 316

databricks

FEBRUARY 6, 2025

FOX Sports has a long history of driving the evolution of broadcast technology, from its high-definition coverage to experiments with virtual reality. Eventually.

Technology 120

Technology IT 120

Waitingforcode

JANUARY 30, 2024

Previously you could read about transformation of a user job definition into an executable stream graph. Since this explanation was relatively high-level, I decided to deep dive into the final step executing the code.

Coding 130

Coding 130

Waitingforcode

FEBRUARY 3, 2023

We all have our habits and as programmers, libraries and frameworks are definitely a part of the group. In this blog post I'll share with you a list of Java and Scala classes I use almost every time in data engineering projects. The part for Python will follow next week!

Scala 130

Scala Java Data Engineering Data Engineer 130

Waitingforcode

FEBRUARY 3, 2023

Watch out the definition of your predicate because from time to time, even though the pushdown predicate is supported by the data source, the predicate can still be executed by the Apache Spark job! Pushdowns in Apache Spark are great to delegate some operations to the data sources. However, there is one important gotcha.

IT 130

IT Process Data 130

Snowflake

NOVEMBER 20, 2023

simplifies access to model training data with Snowflake Native Apps and Snowpark Container Services H2O.ai , a leading machine learning company, offers its state-of-the-art ML engine, including H2O Driverless AI AutoML engine and H2O LLM Studio for fine-tuning and making custom GPTs, as a Snowflake Native App.

Utilities 125

Utilities Machine Learning Coding AWS 125

Seattle Data Guy

MARCH 14, 2023

But there are definite ways to avoid having your costs grow at an unsustainable rate. Photo by Lukas As you increase your analytical processes and abilities, you’ll unavoidably increase costs. … Read more The post How To Scale Your Data Team’s Impact Without Scaling Costs appeared first on Seattle Data Guy.

Data Science 130

Data Science Data Engineering Data Engineer Data 130

Simon Späti

JANUARY 3, 2023

Here’s the definition from our Data Glossary: The Modern Data Stack (MDS) is a heap of open-source tools to achieve end-to-end analytics from ingestion to [transformation]([link] We’ll cover integration, transformation, orchestration, analytics, and ML tools as a starter guide to the latest open data stack.

Data 130

Data IT 130

Netflix Tech

JANUARY 6, 2025

While this is a critical business need and we definitely should solve it, its essential to evaluate how it stacks up against other priorities across different areas of the organization. Defining TitleHealth Navigating such an ambiguous space required a shared understanding to foster clarity and collaboration.

Metadata 87

Metadata Algorithm Systems Building 87

Netflix Tech

DECEMBER 3, 2022

Workflow Definitions Below you can see a typical file structure of a sample workflow package written in SparkSQL. ??? In every sample workflow package there are three workflow definition files that work together to provide flexible functionality. See an example high water mark job from the main workflow definition. -

Data Pipeline 144

Data Pipeline Scala Metadata Food 144

The Pragmatic Engineer

FEBRUARY 23, 2023

Stock prices do have correlation with the broader economy, and should the stock market recover, this would definitely be good news for Big Tech, and it would also likely encourage VCs investing their current funds with more confidence.   For my view , I am not sure I fully agree with Chris that a recover hinges on just the stock prices.

Software Engineer 276

Software Engineer Software Engineering Recruitment Portfolio 276

Precisely

JANUARY 30, 2025

He emphasized that organizations need to make it easy for employees to do the right thing by providing tools like business glossaries and clear data definitions.

Data Governance 103

Data Governance Government Metadata Data 103

The Pragmatic Engineer

AUGUST 3, 2023

Now after 7 years, Google has announced it will retire Firebase Dynamic Links, but with no definite successor lined up. A timeline of more than 12 months is generous, and by the time Google announces the definite sunset timeline in Q3 of this year, they will likely have given around 18 months’ notice.

Metadata 189

Metadata Engineering Building Technology 189