Confluent

SEPTEMBER 16, 2021

The full inventory of three online Kafka Summits in 2021 is now complete. Kafka Summit Americas wrapped just yesterday. Being a part of the event team and the Program Committee, […].

Kafka 145

Kafka Programming 145

Start Data Engineering

SEPTEMBER 16, 2021

1. Introduction 2. What is scaling & why do we need it? 3. Types of scaling 4. Choose your scaling strategy 5. Conclusion 6. Further reading 7. References 1. Introduction Choosing tools/frameworks to scale your data pipelines can be confusing. If you have struggled with Data pipelines that randomly crash Finding guides on how to scale your data pipelines from the ground up Then this post is for you.

Data Pipeline 130

Data Pipeline Data IT 130

Cloudera

SEPTEMBER 15, 2021

Leveraging the Internet of Things (IoT) allows you to improve processes and take your business in new directions. But it requires you to live on the edge. That’s where you find the ability to empower IoT devices to respond to events in real time by capturing and analyzing the relevant data. Edge computing relies on squeezing the power and functionality of a data center into a micro site as close to data sources as possible to enable real-time tasks.

Medical 125

Medical Retail Datasets Algorithm 125

Data Engineering Podcast

SEPTEMBER 12, 2021

Summary The Cassandra database is one of the first open source options for globally scalable storage systems. Since its introduction in 2008 it has been powering systems at every scale. The community recently released a new major version that marks a milestone in its maturity and stability as a project and database. In this episode Ben Bromhead, CTO of Instaclustr, shares the challenges that the community has worked through, the work that went into the release, and how the stability and testing

Database 100

Database Metadata Kafka Data Storage 100

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

Confluent

SEPTEMBER 15, 2021

GraphQL and Apache Kafka® are sometimes troubled with misconceptions. One of the reasons for this is that people are often familiar with one but not the other. GraphQL is mostly […].

Kafka 122

Kafka 122

Teradata

SEPTEMBER 12, 2021

Groupon is modernizing with Vantage on AWS to better match its data & analytics with demands of its global business. The Cloud allows Groupon to better leverage infrastructure dollars, support more technology projects and capture opportunity.

AWS 98

AWS Cloud Technology Project 98

Netflix Tech

SEPTEMBER 15, 2021

By Ricky Gardiner , Alex Borysov Background In our previous post , we discussed how we utilize FieldMask as a solution when designing our APIs so that consumers can request the data they need when fetched via gRPC. In this blog post we will continue to cover how Netflix Studio Engineering uses FieldMask for mutation operations such as update and remove.

Designing 97

Designing Utilities Building Engineering 97

Confluent

SEPTEMBER 14, 2021

Data governance initiatives aim to manage the availability, integrity, and security of data used across an organization. With the explosion in volume, variety, and velocity of data powering the modern […].

Government 119

Government Data Governance Management Data 119

Cloudera

SEPTEMBER 14, 2021

In June, we announced the beginning of a new chapter for Cloudera, with a mission to make data and analytics easy and accessible, for everyone. With transformation comes change, and today I’m thrilled to announce the promotion of Sudhir “Suds” Menon, Ram Venkatesh and Paul Codding, three leaders driving our mission forward. The foundation of our mission is a move to a hybrid data cloud platform , an evolution of our Cloudera Data Platform , a hybrid and multi-cloud solution purpose-built with th

Cloud 86

Cloud AWS Cloud Computing Government 86

DataKitchen

SEPTEMBER 17, 2021

Below is our final post (5 of 5) on combining data mesh with DataOps to foster innovation while addressing the challenges of a data mesh decentralized architecture. We see a DataOps process hub like the DataKitchen Platform playing a central supporting role in successfully implementing a data mesh. DataOps excels at the type of workflow automation that can coordinate interdependent domains, manage order-of-operations issues and handle inter-domain communication.

Architecture 94

Architecture Data Architecture Raw Data Government 94

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Netflix Tech

SEPTEMBER 13, 2021

Written by Jose Fernandez , Arthur Gonigberg , Julia Knecht , and Patrick Thomas In 2017, Netflix Studios was hitting an inflection point from a period of merely rapid growth to the sort of explosive growth that throws “how do we scale?” into every conversation. The vision was to create a “Studio in the Cloud”, with applications supporting every part of the business from pitch to play.

Portfolio 91

Portfolio Cloud Software Engineer Software Engineering 91

Datakin

SEPTEMBER 14, 2021

Blog Building a Remote-First Culture Written by Amanda Bulger on Sep 15, 2021 This morning I was planning an offsite for our team – our first one since Datakin was founded during the pandemic – and I had a realization: I haven’t met most of these people in person yet! We’ve been working together for months and months, solving interesting problems and planning social events, but we have been restricted to knowing each other through a tiny box on a screen.

Building 64

Building Recruitment Management IT 64

ProjectPro

SEPTEMBER 16, 2021

Looking to land a job as a data analyst or a data scientist, SQL is a must-have skill on your resume. Everyone uses SQL to query data and perform analysis, from the biggest names in tech like Amazon, Netflix, and Google to fast-growing seed-stage startups in data. Before the world was taken over by the buzz of data science and analytics, data management still existed.

SQL 52

SQL MySQL MongoDB Data 52

Rockset

SEPTEMBER 14, 2021

We’re introducing a new Rockset Integration for Apache Kafka that offers native support for Confluent Cloud and Apache Kafka, making it simpler and faster to ingest streaming data for real-time analytics. This new integration comes on the heels of several new product features that make Rockset more affordable and accessible for real-time analytics including SQL-based rollups and transformations.

Kafka 52

Kafka SQL MongoDB Computer Science 52

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

dbt Developer Hub

SEPTEMBER 14, 2021

Hello there, Do you remember? The 21st day of September? ? Course you do it was two days ago. Well that's a win in your bucket and the day's barely begun! So let's get a win for someone else -- like Jeremy Cohen, the dbt Core product manager. I'm sure you know that half of the updates in this email are pushed automatically when we upgrade everyone to the latest version of dbt Cloud ?

Metadata 52

Metadata Data Pipeline Python Cloud 52

Teradata

SEPTEMBER 14, 2021

Learn what Mitsui Sumitomo Insurance, one of Japan’s leading insurance and finance groups, has achieved through leveraging the power of data analytics.

Insurance 52

Insurance Finance Data Analytics Data 52

ProjectPro

SEPTEMBER 16, 2021

NoSQL databases are the new-age solutions to distributed unstructured data storage and processing. The speed, scalability, and fail-over safety offered by NoSQL databases are needed in the current times in the wake of Big Data Analytics and Data Science technologies. The edge that NoSql provides over their SQL counterparts is high scalability and faster read/write performances, highly appreciated features in Distributed Systems.

NoSQL 52

NoSQL Database Hadoop Big Data 52

Preset

SEPTEMBER 13, 2021

Here we go step by step to build an open-source ingestion layer with Airbyte.

Building 52

Building 52

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

Datakin

SEPTEMBER 13, 2021

Blog Datakin in 104 seconds Written by Ross Turk on Sep 13, 2021 Hi! I’m Ross from Datakin I’d like to show you a new approach to keeping your pipelines running smoothly. Datakin observes your jobs as they run, collecting metadata that helps you understand how data flows through your ecosystem. We believe that lineage provides the context required to keep troubleshooting and resolution times low.

Metadata 52

Metadata Datasets Data Pipeline Systems 52

Zalando Engineering

SEPTEMBER 12, 2021

2016 - First attempt at rolling out SRE Welcome to the first installment of our three part series following Zalando’s SRE journey. Be sure to come back for the other two, with the next one being published in a week. Site Reliability Engineering (SRE) is a recent discipline in the Software Engineering field that is growing in popularity, with many companies turning to this new way of working to solve their operational issues, or to support its growing scale.

Software Engineer 52

Software Engineer Software Engineering Architecture Cloud 52

ProjectPro

SEPTEMBER 16, 2021

87% of Data Science Projects never make it to production - VentureBeat According to an analytics firm, Cognilytica, the MLOps market is anticipated to be worth $4 billion by end of 2025. Jobs over the next decade will be built on top of Data Science, but for production. Data Science has flourished over the decade on the promise that organizations will leverage analytics for profitable business decision-making.

Project 52

Project Amazon Web Services Machine Learning Data Science 52

Teradata

SEPTEMBER 12, 2021

The new Vantage BYOM feature allows data scientists and data engineers to finally operationalize all their predictive models. Find out more.

Data Science 52

Data Science Data Engineer Data Engineering Data 52

Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

RudderStack

SEPTEMBER 16, 2021

Learn how Touchless used a data-first design approach and leveraged RudderStack to help Wavedirect 4X Leads with dynamically generated SEO landing pages.

Designing 40

Designing Data 40

Cloudera

SEPTEMBER 13, 2021

About Cruise Control. There are two big gaps in the Apache Kafka project when we think of operating a cluster. The first is monitoring the cluster efficiently and the second is managing failures and changes in the cluster. There are no solutions for these inside the Kafka project but there are many good 3rd party tools for both problems. Cruise Control is one of the earliest open source tools to provide a solution for the failure management problem but lately for the monitoring problem as well.

Kafka 76

Kafka Bytes Utilities Management 76

ProjectPro

SEPTEMBER 14, 2021

Cloud technology is widely used, with 94% of enterprises already using one or multiple cloud services. In a couple of years, the public cloud market will reach $623.3 billion. There are abundant options available in the cloud technology market, with AWS and Openstack as the two trendy choices. Table of Contents AWS vs. OpenStack - A Head to Head Comparison OpenStack vs.

AWS 52

AWS Amazon Web Services Cloud Computing Big Data 52

ProjectPro

SEPTEMBER 14, 2021

As a data science aspirant, you would have probably come across the following phrase more than once: “A data scientist is a person who is better at statistics than any programmer and better at programming than any statistician.” Before data science became a well-known career path, companies would hire statisticians to process their data and develop insights based on trends observed.

Data Science 52

Data Science Datasets Machine Learning Data 52

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

ProjectPro

SEPTEMBER 11, 2021

This blog contains a list of business analyst interview questions and answers. You will find it helpful if you are a hiring manager who is looking for business analyst questions to ask during an interview and also if you are a job seeker who is interested in business analyst jobs. Table of Contents Role of a Business Analyst: Skills and Opportunities 50 Business Analyst Interviews Questions and Answers Junior Business Analyst Interview Questions/Entry-level Business Analyst Interview Questions T

Business Analyst 52

Business Analyst Database-centric MySQL SQL 52

RudderStack

SEPTEMBER 14, 2021

This guide covers how to send data from your website to customer.io in less than five minutes.

Data 40

Data 40