Start Data Engineering
OCTOBER 5, 2021
1. Introduction 2. What is a staging area 3. The advantages of having a staging area 5. Conclusion 6. Further reading 1. Introduction Working with data pipelines, you might have noticed a staging area in most data pipelines. If you work in the data space and have questions like Why is there a staging area? Can’t we just load data into the destination tables?
Confluent
OCTOBER 6, 2021
Today, an organization’s strategic objective is to deliver innovations for a connected life and to improve the quality of life worldwide. With connected devices comes data, and with data comes […].
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Data Engineering Podcast
OCTOBER 8, 2021
Summary The key to making data valuable to business users is the ability to calculate meaningful metrics and explore them along useful dimensions. Business intelligence tools have provided this capability for years, but they don’t offer a means of exposing those metrics to other systems. Metriql is an open source project that provides a headless BI system where you can define your metrics and share them with all of your other processes.
Cloudera
OCTOBER 7, 2021
October sees the launch of Partner Appreciation Month and during the next few weeks we will be sharing success stories, updates and interviews with our valued partners across the world. . We’re on a mission to make data and analytics easy and accessible, for everyone, and the hybrid data cloud is how we’ll get there. Today’s world is a hybrid world—there’s hybrid data, hybrid infrastructure, hybrid work—and leading businesses are embracing these changes, unafraid to transform their processes and
Advertisement
Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?
Start Data Engineering
OCTOBER 3, 2021
1. Introduction 2. Business requirements: dashboards and analytics 3. What is a data warehouse 4. OLTP vs OLAP based data warehouses 5. Conclusion 6. Further reading 7. References 1. Introduction If you are a student, analyst, engineer, or anyone in the data space, it’s important to understand what a data warehouse is. If you are wondering What is a data warehouse?
Teradata
OCTOBER 4, 2021
An interdisciplinary team from Volkswagen, AWS and Teradata have created an intelligent solution that enables greater transparency and efficiency in car body construction. Find out more.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Cloudera
OCTOBER 5, 2021
Cloudera Data Platform (CDP) supports access controls on tables and columns, as well as on files and directories via Apache Ranger since its first release. It is common to have different workloads using the same data – some require authorizations at the table level (Apache Hive queries) and others at the underlying files (Apache Spark jobs). Unfortunately, in such instances you would have to create and maintain separate Ranger policies for both Hive and HDFS, that correspond to each othe
Netflix Tech
OCTOBER 7, 2021
Martin Tingley with Wenjing Zheng , Simon Ejdemyr , Stephanie Lane , and Colin McFarland This is the third post in a multi-part series on how Netflix uses A/B tests to inform decisions and continuously innovate on our products. Need to catch up? Have a look at Part 1 (Decision Making at Netflix) and Part 2 (What is an A/B Test?). Subsequent posts will go into more details on experimentation across Netflix, how Netflix has invested in infrastructure to support and scale experimentation, and the i
DataKitchen
OCTOBER 7, 2021
The post How Predictive and Prescriptive Analytics Improve the Call Center Experience first appeared on DataKitchen.
Data Engineering Podcast
OCTOBER 2, 2021
Summary Aerospike is a database engine that is designed to provide millisecond response times for queries across terabytes or petabytes. In this episode Chief Strategy Officer, Lenley Hensarling, explains how the ability to process these large volumes of information in real-time allows businesses to unlock entirely new capabilities. He also discusses the technical implementation that allows for such extreme performance and how the data model contributes to the scalability of the system.
Speaker: Jay Allardyce, Deepak Vittal, and Terrence Sheflin
As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.
Cloudera
OCTOBER 8, 2021
Introduction. Apache Impala is a massively parallel in-memory SQL engine supported by Cloudera designed for Analytics and ad hoc queries against data stored in Apache Hive, Apache HBase and Apache Kudu tables. Supporting powerful queries and high levels of concurrency Impala can use significant amounts of cluster resources. In multi-tenant environments this can inadvertently impact adjacent services such as YARN, HBase, and even HDFS.
Netflix Tech
OCTOBER 7, 2021
By Minal Mishra Quality of a client application is of paramount importance to global digital products, as it is the primary way customers interact with a brand. At Netflix, we have significant investments in ensuring new versions of our applications are well tested. However, Netflix is available for streaming on thousands of types of devices and it is powered by hundreds of micro-services which are deployed independently, making it extremely challenging to comprehensively test internally.
DataKitchen
OCTOBER 8, 2021
The post DataOps Lowers The Cost Of Asking Analytic Questions first appeared on DataKitchen.
Pipeline Data Engineering
OCTOBER 8, 2021
In this interview series we’ll share some of the stories that Daniel and I get to watch unfold at Pipeline Academy. Check out what our graduates have to say about the course, how they’ve tackled its challenges and what they are doing now with their new data engineering superpowers. Peter: Michele, it's great to see you again. Thanks for taking the time to have a chat with me.
Speaker: Nikhil Joshi, Founder & President of Snic Solutions
Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.
Cloudera
OCTOBER 5, 2021
If your organization is using multi-tenant big data clusters (and everyone should be), do you know the usage and cost efficiency of resources in the cluster by tenants? A chargeback or showback model allows IT to determine costs and resource usage by the actual analytic users in the multi-tenant cluster, instead of attributing those to the platform (“overhead’) or IT department.
ProjectPro
OCTOBER 8, 2021
Retail is one of the first industries that started leveraging the power of machine learning and artificial intelligence. There are machine learning projects for almost every retail use case - right from inventory management to customer satisfaction. Machine learning projects in retail directly convert into profits and increase an organization’s market share with better customer acquisition and satisfaction.
DataKitchen
OCTOBER 6, 2021
The post Why Enterprise AI Needs Human Intervention first appeared on DataKitchen.
Monte Carlo
OCTOBER 7, 2021
I first “joined” Monte Carlo exactly a year ago, as a data science intern. I met Lior , our co-founder, on Zoom in August of 2020. I had cast a volley of solicitous emails into my network — “(sort of) Stanford C.S. student looking to be (sort of) hired and avoid school for a while” — and one opportunity had come back from Oren and Glenn , former colleagues and now mentors of mine at GGV.
Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage
When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.
AltexSoft
OCTOBER 8, 2021
To drive deeper business insights and greater revenues, organizations — whether they are big or small — need quality data. But more often than not data is scattered across a myriad of disparate platforms, databases, and file systems. What’s more, that data comes in different forms and its volumes keep growing rapidly every day — hence the name of Big Data.
ProjectPro
OCTOBER 8, 2021
Microsoft Power BI is the most used business intelligence tool according to peer ratings on Gartner. Companies like Adobe, Heathrow, PharmD have shown their trust in Power BI and continue to do so year after year. For 14 consecutive years, Power BI has bagged the first position in Magic Quadrant. As Power BI customers keep increasing, companies look for Power BI developers and analysts who can drive their business analytics and intelligence tasks.
DataKitchen
OCTOBER 5, 2021
A DataOps Engineer owns the assembly line that’s used to build a data and analytic product. Data operations (or data production) is a series of pipeline procedures that take raw data, progress through a series of processing and transformation steps, and output finished products in the form of dashboards, predictions, data warehouses or whatever the business requires.
Grouparoo
OCTOBER 6, 2021
Last month, we decided that we should all read a book and talk about it as a company. It was a fun experience and I think we made a good choice by picking 97 Things Every Data Engineer Should Know. This was the first book I have read in this series and I liked the format. It is made up of 97 small vignettes that are 2-3 pages each. This provided a nice overview of the breadth of topics that are relevant to data engineering including data warehouses/lakes, pipelines, metadata, security, complianc
Speaker: Donna Laquidara-Carr, PhD, LEED AP, Industry Insights Research Director at Dodge Construction Network
In today’s construction market, owners, construction managers, and contractors must navigate increasing challenges, from cost management to project delays. Fortunately, digital tools now offer valuable insights to help mitigate these risks. However, the sheer volume of tools and the complexity of leveraging their data effectively can be daunting. That’s where data-driven construction comes in.
A Cloud Guru: Data Engineering
OCTOBER 6, 2021
In this post, we’ll discuss the difference between ETL vs ELT and when you might choose ETL or ELT. We’ll also include a flowchart to help walk you through the ETL vs ELT decision-making process. The difference between ETL vs ELT What’s the difference between ETL and ELT? The short answer is it’s all about […] The post ETL vs ELT flowchart: When to use each appeared first on A Cloud Guru.
ProjectPro
OCTOBER 6, 2021
With 67 zones, 140 edge locations, over 90 services, and 940163 organizations using GCP across 200 countries - GCP is slowly garnering the attention of cloud users in the market. Flexera’s State of Cloud report highlighted that 41% of the survey respondents showed the most interest in using Google Cloud Platform for their future cloud computing projects.
Big Data Tools
OCTOBER 5, 2021
In most countries, students start learning in September. As data engineers, let’s follow their lead and learn something new, too! I’m Pasha Finkelshteyn , and I’ll be your guide through this month’s news. I’ll offer my impressions of developments and highlight ideas from the wider community. If you think I missed something worthwhile, ping me on Twitter and suggest a topic, link, or anything else.
Datakin
OCTOBER 5, 2021
Last week, Matt Turck and John Wu published the latest annual report on the state of data, the 2021 Machine Learning, AI and Data (MAD) Landscape. If you haven’t read it yet, we recommend it as a comprehensive snapshot of the intricate world of AI, machine learning, and data science & engineering. Our team enjoyed reading it. We represent several of the pixels on this chart (hey, cool!
Speaker: Evelyn Chou
Choosing the right business intelligence (BI) platform can feel like navigating a maze of features, promises, and technical jargon. With so many options available, how can you ensure you’re making the right decision for your organization’s unique needs? 🤔 This webinar brings together expert insights to break down the complexities of BI solution vetting.
Zalando Engineering
OCTOBER 4, 2021
The problem When building Machine Learning (ML) applications - such as recommender systems - there is often a need to provide a "feature store" which can enrich the request to the system with additional ML features. For example: whether a user had looked at an article before is often very informative about whether the user will click or buy that article this time.
ProjectPro
OCTOBER 6, 2021
This blog is a step-by-step guide for a beginner in NLP. If you are someone who wants to know what is the best way to learn NLP from scratch, then please go through our blog till the end. We assure you will build the confidence and gear up yourself to make a career transition into data science as an NLP Engineer. We will first begin with what are the essential subjects one must be aware of, to prepare them for diving into the world of Natural Language Processing.
Big Data Tools
OCTOBER 5, 2021
In most countries, students start learning in September. As data engineers, let’s follow their lead and learn something new, too! I’m Pasha Finkelshteyn , and I’ll be your guide through this month’s news. I’ll offer my impressions of developments and highlight ideas from the wider community. If you think I missed something worthwhile, ping me on Twitter and suggest a topic, link, or anything else.
Teradata
OCTOBER 6, 2021
This how-to guide will help you connect Teradata Vantage with the AWS Data Exchange service. Read more for step-by-step instructions.
Speaker: Aindra Misra, Senior Manager, Product Management (Data, ML, and Cloud Infrastructure) at BILL
Join us for an insightful webinar that explores the critical intersection of data privacy and AI governance. In today’s rapidly evolving tech landscape, building robust governance frameworks is essential to fostering innovation while staying compliant with regulations. Our expert speaker, Aindra Misra, will guide you through best practices for ensuring data protection while leveraging AI capabilities.
Expert insights. Personalized for you.
130 
Let's personalize your content