Data Engineering Weekly

MARCH 3, 2024

Editor’s Note: Chennai, India Meetup - March-08 Update We are thankful to Ideas2IT to host our first Data Hero’s meetup. There will be food, networking, and real-world talks around data engineering.

Data Engineer 67

Data Engineer Data Engineering Pipeline-centric Engineering 67

ProjectPro

MAY 26, 2015

The webinar discusses about the working of beacon technology (Beaconstac) and the production beacon analytics system Morpheus at MobStac that leverages Hadoop for analysing huge amounts of unstructured data generated from beacons (IoT).Beacons

Hadoop 52

Hadoop Unstructured Data Big Data Retail 52

Precisely

MARCH 27, 2023

Transforming Go-to-Market After years of acquiring and integrating smaller companies, a $37 billion multinational manufacturer of confectionery, pet food, and other food products was struggling with complex and largely disparate processes, systems, and data models that needed to be normalized.

Food 52

Food Consulting Pharmaceutical Data Lake 52

Precisely

FEBRUARY 9, 2024

Increase in data trust: Data that doesn’t meet rigorous quality metrics, or that isn’t governed with a robust framework, puts AI systems at risk of generating inaccurate predictions, recommendations, and output. These more complete datasets will both reduce bias and increase accuracy.

Data Integration 69

Data Integration Government Data Metadata 69

ProjectPro

JUNE 6, 2025

AI technology in agriculture is completely changing food production, making farming more innovative, efficient, and sustainable. In this blog, we’ll explore 7 AI projects in agriculture that are helping to increase yields, reduce waste, and build a more resilient smart farming system. Show visual stress maps using Leaflet.js

Project 40

Project Building Google Cloud Datasets 40

ProjectPro

JUNE 26, 2015

Despite the challenges of adopting big data, there are several companies exploiting big data analysis in an innovative way to exhibit the versatility and usefulness of big data. ”- says Liv Buli, data journalist at The Next Big Sound. “Watson amplifies human creativity.

Big Data 40

Big Data Food Unstructured Data Data Analytics 40

Monte Carlo

FEBRUARY 21, 2023

A data fabric isn’t a standalone technology—it’s a data management architecture that leverages an integrated data layer atop underlying data in order to empower business leaders with real-time analytics and data-driven insights. To integrate and unify that distributed data, Domino’s implemented a data fabric.

Data Architecture 52

Data Architecture Architecture Metadata Unstructured Data 52

Monte Carlo

FEBRUARY 21, 2023

A data fabric isn’t a standalone technology—it’s a data management architecture that leverages an integrated data layer atop underlying data in order to empower business leaders with real-time analytics and data-driven insights. To integrate and unify that distributed data, Domino’s implemented a data fabric.

Data Architecture 52

Data Architecture Architecture Metadata Unstructured Data 52

ProjectPro

JUNE 6, 2025

You can pick any of these cloud computing project ideas to develop and improve your skills in the field of cloud computing along with other big data technologies. You can pick any of these cloud computing project ideas to develop and improve your skills in the field of cloud computing along with other big data technologies.

Cloud Computing 40

Cloud Computing Cloud Project Google Cloud 40

Knowledge Hut

NOVEMBER 20, 2023

This is not true for automated systems at the core. So, if you give a command that hasn’t been configured, the system won’t work. Etymologically, AI refers to the intelligence of a computer-controlled system that performs tasks commonly associated with humans. Learning AI systems learn from the data fed into them.

Manufacturing 52

Manufacturing Pharmaceutical Finance Healthcare 52

Knowledge Hut

MAY 31, 2023

However, managing data can be a challenging task, especially when dealing with large amounts of information. This is where database management systems come in handy. A database management system (DBMS) is a software system that helps organize, store and manage information efficiently.

Database 52

Database Coding MongoDB Project 52

Monte Carlo

JANUARY 23, 2024

Gen AI can whip up serviceable code in moments — making it much faster to build and test data pipelines. Today’s LLMs can already process enormous amounts of unstructured data, automating much of the monotonous work of data science. But what does that mean for the roles of data engineers and data scientists going forward?

Database-centric 52

Database-centric Pipeline-centric Metadata Unstructured Data 52

Knowledge Hut

MAY 2, 2024

Spark is being used in more than 1000 organizations who have built huge clusters for batch processing, stream processing, building warehouses, building data analytics engine and also predictive analytics platforms using many of the above features of Spark. Let’s look at some of the use cases in a few of these organizations.

Scala 52

Scala Hospitality Machine Learning Healthcare 52

Knowledge Hut

OCTOBER 26, 2023

You can develop voice-enabled applications that can do different tasks like booking a hotel room, ordering food, or playing music. Content Recommendation System You can build a content recommendation system using Amazon SageMaker, a machine learning service offered by AWS. Source Code: GitHub 4. Source Code: GitHub 5.

AWS 52

AWS Coding Project Cloud Computing 52

ProjectPro

MAY 23, 2015

Use market basket analysis to classify shopping trips Walmart Data Analyst Interview Questions Walmart Hadoop Interview Questions Walmart Data Scientist Interview Question American multinational retail giant Walmart collects 2.5 petabytes of unstructured data from 1 million customers every hour.

Big Data 52

Big Data Data Analysis Hadoop Retail 52

AltexSoft

DECEMBER 15, 2021

Let’s explore the stages where current AutoML systems already show or at least promise the best results. Data preprocessing. The process of cooking the right food for your algorithm falls into two key steps. Besides tabular data, the system performs text and image processing.

Machine Learning 59

Machine Learning Deep Learning Algorithm Telecommunication 59

Knowledge Hut

APRIL 23, 2024

Drug Discovery and Development: By analyzing enormous volumes of biological, chemical, and clinical data, big data helps to speed up the drug discovery process, resulting in the identification of prospective therapeutic targets and more effective clinical trials.

Big Data 52

Big Data Recruitment Transportation Retail 52

ProjectPro

APRIL 27, 2015

Big Data startups are banking upon analytics to disrupt the market in the foresight based approach. Big data startups are leveraging recommendation systems by making sense of big data-predicting user intentions and rendering services and products people are looking for before they even know that they need them.

Big Data 40

Big Data Retail Hadoop Unstructured Data 40

ProjectPro

OCTOBER 18, 2021

Recommender System Projects Have you ever seen movies or web series on online streaming platforms? They have a well-researched collection of data such as ratings, reviews, timestamps, price, category information, customer likes, and dislikes. It contains all the attributes you need to build your stock price prediction system.

Machine Learning 52

Machine Learning Project Datasets Algorithm 52

ProjectPro

JANUARY 28, 2022

Systems are already in place in most major banks where the authorities are alerted when unusually high spending or credit activity occurs on someone’s account. Given that data can back the decision and sufficiently reliable data is available, anomaly detection can be potentially life-saving.

Machine Learning 52

Machine Learning Python Algorithm Datasets 52

ProjectPro

JUNE 6, 2025

It offers a vast collection of datasets covering various topics, including natural disasters, food security, health, and displacement. They maintain a vast repository of healthcare data and data on several health-related topics, including data on diseases, health systems, and health outcomes.

Datasets 40

Datasets Data Science Project Google Cloud 40

AltexSoft

MAY 27, 2021

The documents often come in semi-structured and unstructured data formats, which makes them difficult to process quickly and accurately. Let’s see how all three work together and what enables the “intelligent” part of the system. The technology has proved to be effective in working with data presented in a structured format.

Technology 64

Technology Process Insurance Electronics 64