Snowflake

SEPTEMBER 24, 2024

Today, Snowflake is officially launching the AI Data Cloud for Travel and Hospitality. Snowflake’s newest AI Data Cloud offers a unified and secure platform that streamlines AI and ML development to support the growth of travel and hospitality businesses, empowering organizations to harness their data’s full potential.

Hospitality 88

Hospitality Cloud Database-centric Data 88

Data Engineering Weekly

JUNE 2, 2024

Data Engineering Weekly is sponsored by Astronomer—Enterprise-Grade Apache Airflow. Deliver data on time with the speed and scale your application demands. Data sovereignty will play a vital role as countries formulate regulations. How should one think about a data strategy if you’re a startup?

Data Engineer 103

Data Engineer Data Engineering Engineering Pipeline-centric 103

Grouparoo

MAY 13, 2021

In this post, we’ll look at the historical reasons for the 191 character limit as a default in most relational databases. The first question you might ask is why limit the length of the strings you can store in a database at all? Indexes can be made to perform better when assumptions can be made about the type of data they store.

Database 52

Database Bytes MySQL Database-centric 52

Striim

DECEMBER 12, 2023

Exciting news is on the horizon as Striim proudly announces its Technology Partnership with YugabyteDB, a collaboration set to reshape the landscape of data management. As we embark on this thrilling journey, we share a vision of empowering organizations with the tools they need to thrive in a data-driven world.

Data Management 59

Data Management Database-centric Management PostgreSQL 59

Data Engineering Podcast

AUGUST 13, 2022

Summary Data engineers have typically left the process of data labeling to data scientists or other roles because of its nature as a manual and process heavy undertaking, focusing instead on building automation and repeatable systems. Data stacks are becoming more and more complex. Sifflet also offers a 2-week free trial.

Machine Learning 100

Machine Learning Pipeline-centric Database-centric MongoDB 100

Christophe Blefari

APRIL 8, 2023

Data News entering in town ( credits ) Hey you, if I wasn't late in my newsletter writing it wouldn't be me. But here is your usual Data News. Data modeling Dear readers, I have to confess something. I did not care about data modeling for years. Data modeling should not be a required data engineer skill.

Pipeline-centric 130

Pipeline-centric Database-centric Algorithm Data 130

Christophe Blefari

APRIL 8, 2023

Data News entering in town ( credits ) Hey you, if I wasn't late in my newsletter writing it wouldn't be me. But here is your usual Data News. Data modeling Dear readers, I have to confess something. I did not care about data modeling for years. Data modeling should not be a required data engineer skill.

Pipeline-centric 130

Pipeline-centric Database-centric Algorithm Data 130

RandomTrees

SEPTEMBER 27, 2024

A cloud-based software as a service (SaaS) called Microsoft Fabric combines several data and analytics technologies that businesses require. Data Factory, Data Activator, Power BI, Synapse Real-Time Analytics, Synapse Data Engineering, Synapse Data Science, and Synapse Data Warehouse are some of them.

Database-centric 52

Database-centric Pipeline-centric IT BI 52

Knowledge Hut

APRIL 23, 2024

Big data and data mining are neighboring fields of study that analyze data and obtain actionable insights from expansive information sources. Big data encompasses a lot of unstructured and structured data originating from diverse sources such as social media and online transactions.

Data Mining 52

Data Mining Big Data Database-centric Unstructured Data 52

DareData

MARCH 5, 2024

At DareData Engineering, we believe in a human-centric approach, where AI agents work together with humans to achieve faster and more efficient results. At its core, RAG harnesses the power of large language models and vector databases to augment pre-trained models (such as GPT 3.5 ).

Database-centric 52

Database-centric Telecommunication SQL Unstructured Data 52

Data Engineering Weekly

MARCH 3, 2024

RudderStack is the Warehouse Native CDP, built to help data teams deliver value across the entire data activation lifecycle, from collection to unification and activation. Editor’s Note: Chennai, India Meetup - March-08 Update We are thankful to Ideas2IT to host our first Data Hero’s meetup.

Data Engineer 67

Data Engineer Data Engineering Engineering Pipeline-centric 67

U-Next

AUGUST 17, 2022

Introduction to 2022 Data Engineer Roles and Responsibilities. Companies and enterprises, large and small, are built on data. Data Engineer roles and responsibilities include aiding in the collection of issues and the delivery of remedies addressing customer demand and product accessibility.

Data Engineer 52

Data Engineer Data Engineering Pipeline-centric Database-centric 52

Ripple Engineering

JANUARY 29, 2024

Why do we need a Data Platform? At Ripple, different business units require a platform to quickly and easily turn their data into insights and compelling customer experiences.   A lack of a centralized system makes building a single source of high-quality data difficult. Why is it difficult to build a centralized platform?

Database-centric 52

Database-centric Pipeline-centric NoSQL High Quality Data 52

Precisely

APRIL 20, 2023

To accomplish that, they must stay on top of data and market insights daily. The National Association of REALTORS ® clearly understands this challenge, which is why it built RPR (Realtors Property Resource), the nation’s largest parcel-centric database, exclusively for REALTORS ®. While RPR can now offer high accuracy for U.S.

Database-centric 52

Database-centric Database Data Datasets 52

Monte Carlo

MAY 30, 2024

Generative AI has the potential to transform your business and your data engineering team, but only when it’s done right. So how can your data team actually drive value with your LLM or GenAI initiative? Data retrieval: Based on the query, the RAG system searches the database to find relevant data.

Pipeline-centric 52

Pipeline-centric Database-centric Datasets Data Pipeline 52

Data Engineering Weekly

JULY 2, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork, and SQL grunt work out of building complete customer profiles, so you can quickly ship actionable, enriched data to every downstream team. So, let's shape the future of Data Engineering together. See how it works today.

Data Engineer 52

Data Engineer Data Engineering Engineering Pipeline-centric 52

Precisely

JULY 18, 2024

Key Takeaways Data Fabric is a modern data architecture that facilitates seamless data access, sharing, and management across an organization. Data management recommendations and data products emerge dynamically from the fabric through automation, activation, and AI/ML analysis of metadata. What is Data Fabric ?

Data Management 52

Data Management Management Metadata Database-centric 52

Ascend.io

SEPTEMBER 14, 2023

The rise of data-intensive operations has positioned data engineering at the core of today’s organizations. As the demand to efficiently collect, process, and store data increases, data engineers have started to rely on Python to meet this escalating demand. Why Python for Data Engineering?

Data Engineer 52

Data Engineer Data Engineering Python Engineering 52

Knowledge Hut

JUNE 20, 2023

A novice data scientist prepared to start a rewarding journey may need clarification on the differences between a data scientist and a machine learning engineer. Many people are learning data science for the first time and need help comprehending the two job positions. Facial reorganization, social media optimization, etc.

Machine Learning 52

Machine Learning Data Engineer Data Engineering Engineering 52

DataKitchen

NOVEMBER 5, 2024

The Race For Data Quality In A Medallion Architecture The Medallion architecture pattern is gaining traction among data teams. It is a layered approach to managing and transforming data. By systematically moving data through these layers, the Medallion architecture enhances the data structure in a data lakehouse environment.

Architecture 79

Architecture Raw Data Pipeline-centric Data Ingestion 79

Knowledge Hut

MARCH 29, 2024

Showcases a data-driven approach to continuous improvement and process optimization. Data Scientist Template Link: Data Scientist (Download here) Why This Cover Letter Works: Highlights technical proficiency in relevant data analysis tools and programming languages. Showcases Python, Scala, and data modeling skills.

Database-centric 52

Database-centric Programming Language BI Finance 52

Data Engineering Podcast

APRIL 7, 2019

Summary How much time do you spend maintaining your data pipeline? This was a fascinating conversation with someone who has spent his entire career working on simplifying complex data problems. Managing and auditing access to your servers and databases is a problem that grows in difficulty alongside the growth of your teams.

Data Pipeline 100

Data Pipeline Pipeline-centric Database-centric AWS 100

Knowledge Hut

MARCH 15, 2024

Making decisions in the database space requires deciding between RDBMS (Relational Database Management System) and NoSQL, each of which has unique features. RDBMS uses SQL to organize data into structured tables, whereas NoSQL is more flexible and can handle a wider range of data types because of its dynamic schemas.

NoSQL 52

NoSQL Database-centric Relational Database PostgreSQL 52

ThoughtSpot

OCTOBER 18, 2024

This requirement prompted us to explore Build Analytics—harnessing data from our build processes to gain actionable insights. It aims to explain how we transformed our development practices with a data-centric approach and offers recommendations to help your teams address similar challenges in your software development lifecycle.

Building 52

Building Process Pipeline-centric Database-centric 52

Cloudera

JANUARY 24, 2022

Many organizations initiate data projects because they want to increase revenue, but a select few tackle projects that truly transform society. . The bank’s data project aimed to give Filipinos better access to credit and financial opportunities. As you can imagine, searching a database that large is no easy task.

Hospitality 64

Hospitality Database-centric Banking Medical 64

Towards Data Science

JULY 13, 2023

The example we’ll walk you through will mirror a typical LLM application workflow you’d run to populate a vector database with some text knowledge. Specifically, we’ll cover pulling data from the web, creating text embeddings (vectors) and pushing them to a vector store. The application will receive a small data input (e.g.,

Building 69

Building Database-centric Database Coding 69

Cloudera

OCTOBER 17, 2023

The adoption of Flink mirrors growth in streaming data volumes and maturity of the streaming market. Cloudera’s perspective: Cloudera saw the increasing volumes of data our customers were moving via streams early on. Also, why solve for connectivity and data distribution again if it’s already solved for? What about data lock-in?

Kafka 78

Kafka Database-centric Pipeline-centric Database 78

Knowledge Hut

JANUARY 30, 2024

In today's world, where data rules the roost, data extraction is the key to unlocking its hidden treasures. As someone deeply immersed in the world of data science, I know that raw data is the lifeblood of innovation, decision-making, and business progress. What is data extraction?

ETL Tools 52

ETL Tools Database-centric Data Mining Raw Data 52

Knowledge Hut

NOVEMBER 2, 2023

For those aspiring to build a career within the Azure ecosystem, navigating the choices between Azure Data Engineers and Azure DevOps Engineers can be quite challenging. Azure Data Engineers and Azure DevOps Engineers are two critical components of the Azure ecosystem for different but interconnected reasons.

Data Engineer 52

Data Engineer Data Engineering Engineering Pipeline-centric 52

dbt Developer Hub

NOVEMBER 13, 2023

Environmental segmentation has always been an important part of the analytics engineering workflow: When developing new models you can process a smaller subset of your data by using target.name or an environment variable. We need to exchange a job-oriented strategy for a more mature and scalable environment-centric view of the world.

Cloud 52

Cloud Database-centric Metadata Data Warehouse 52

Dataquest

JANUARY 25, 2017

From helping cars drive themselves to helping Facebook tag you in photos , data science has attracted a lot of buzz recently. Data scientists have become extremely sought after , and for good reason — a skilled data scientist can add incredible value to a business. But what about data engineers?

Data Engineer 52

Data Engineer Data Engineering Pipeline-centric Database-centric 52

Meltano

JANUARY 25, 2023

A curated list of the top 9 must read blogs on data. The data world is in turmoil and lots of exciting things happen every day, week and year. At Meltano we’re ourselves avid users of data, data engineers, data PMs, and data enthusiasts through and through. Happy reading!

Pipeline-centric 52

Pipeline-centric Database-centric SQL Data Warehouse 52

Rockset

AUGUST 2, 2023

With 90% of trade being transported via sea , this data is crucial to keeping the global supply chain on track but can be difficult to disentangle and take action on. In 2022, Windward embarked on several changes to its application prompting a reconsideration of its underlying data stack.

Database-centric 52

Database-centric PostgreSQL Transportation Insurance 52

Data Science Blog: Data Engineering

SEPTEMBER 9, 2020

Data is centric in testing of several applications because data is critical to organizations. Businesses are becoming more data-driven, and hence it is imperative that as Automation Test developers, the value of the test-data is understood and completely harnessed during Test Automation development.

Data Management 52

Data Management Database-centric Management PostgreSQL 52

Maxime Beauchemin

JANUARY 20, 2017

By the time I left in 2013, I was a data engineer. We were data engineers! Data Engineering? Data science as a discipline was going through its adolescence of self-affirming and defining itself. At the same time, data engineering was the slightly younger sibling, but it was going through something similar.

Data Engineer 130

Data Engineer Data Engineering Engineering ETL Tools 130

Ascend.io

JUNE 8, 2023

Because of its transformative potential, data has graduated from being merely a by-product of business operations to a critical asset in its own right. Is it possible to treat data not just as a necessary operational output, but as a product that holds immense strategic value? Without them, data products can’t exist.

Pipeline-centric 52

Pipeline-centric Database-centric Data Ingestion Data Pipeline 52

Knowledge Hut

NOVEMBER 17, 2023

The demand for data-related professions, including data engineering, has indeed been on the rise due to the increasing importance of data-driven decision-making in various industries. Becoming an Azure Data Engineer in this data-centric landscape is a promising career choice. Who is an Azure Data Engineer?

Data Engineer 52

Data Engineer Data Engineering Engineering Scala 52