Sat.Nov 28, 2020 - Fri.Dec 04, 2020

article thumbnail

A Data Scientist in Engineering Wonderland

Team Data Science

As a data scientist, I always felt a missing link between my developed models and putting them in the production process. Yes, I can create a pipeline, write a model, get results, and interpret the results, but if I cannot scale it, these all will sit on my Jupiter notebooks. This thought led me to my data engineering adventure. I am confident that learning data engineering will make me a better data scientist.

article thumbnail

Streaming Data Integration Without The Code at Equalum

Data Engineering Podcast

Summary The first stage of every good pipeline is to perform data integration. With the increasing pace of change and the need for up to date analytics the need to integrate that data in near real time is growing. With the improvements and increased variety of options for streaming data engines and improved tools for change data capture it is possible for data teams to make that goal a reality.

article thumbnail

Project Metamorphosis Month 8: Complete Apache Kafka in Confluent Cloud

Confluent

This is the eighth and final month of Project Metamorphosis: an initiative that brings the best characteristics of modern cloud-native data systems to the Apache Kafka® ecosystem, served from Confluent […].

Kafka 99
article thumbnail

2020 Data Impact Award Winner Spotlight: Rush University Medical Center

Cloudera

After a tumultuous year, the final award category at the Data Impact Awards was a much needed pick me up for everyone in attendance. Showcasing some of the most inspiring and uplifting use cases of Cloudera’s technology, The Data for Good category recognizes organizations that are tackling the challenging issues affecting society and the planet — and we all know there are plenty of them in 2020!

Medical 77
article thumbnail

Apache Airflow® Best Practices for ETL and ELT Pipelines

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

article thumbnail

Teradata at AWS re:Invent

Teradata

Teradata is participating in AWS re:Invent 2020, demonstrating our cloud-first stance as a Gold sponsor. Find out more.

AWS 59
article thumbnail

Immutable Linked Lists in Scala With Call-By-Name and Lazy Values

Rock the JVM

Discover how to harness lazy values and call-by-name techniques to craft a fully immutable doubly-linked list in Scala

Scala 52

More Trending

article thumbnail

How to configure clients to connect to Apache Kafka Clusters securely – Part 1: Kerberos

Cloudera

This is the first installment in a short series of blog posts about security in Apache Kafka. In this article we will explain how to configure clients to authenticate with clusters using different authentication mechanisms. Secured Apache Kafka clusters can be configured to enforce authentication using different methods, including the following: SSL – TLS client authentication.

Kafka 70
article thumbnail

Risk-Based Wealth Management: What the Insurance Industry Gets Wrong

Teradata

Product-centric processes degrade customer experience. Insurers must insulate consumers from internal & regulatory-driven controls by placing them in the center of the customer experience.

article thumbnail

Open Source Highlight: Klio

Data Council

Klio is a framework for easy large-scale processing and ML research on binary files, such as audio files -- its original use case. As a matter of fact, it was developed for audio intelligence at Spotify, which open-sourced it earlier this year at the 2020 International Society for Music Information Retrieval Conference.

Process 52
article thumbnail

Real-Time Serverless Ingestion, Streaming, and Analytics using AWS and Confluent Cloud

Confluent

Due to the distributed architecture of Apache Kafka®, the operational burden of managing it can quickly become a limiting factor on adoption and developer agility. For this reason, it is […].

AWS 86
article thumbnail

Apache Airflow®: The Ultimate Guide to DAG Writing

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

article thumbnail

2020 Data Impact Award Winner Spotlight: Telkomsel

Cloudera

2020 is a year that’s been defined by transformation. The way we work, how businesses operate, and even serve customers have all transformed in order to cope with the challenges that have been thrown our way. Amongst the chaos, some organizations have excelled. The Industry Transformation category at our Data Impact Awards celebrates these organizations— the ones that have looked digital transformation in the eye and said “bring it on!

article thumbnail

Data and Strategic Alignment in the Bank of the Future

Teradata

Strategic alignment is a fundamental building block for the bank of the future. It must rest on integrated data & financial data analysis that inform each stage on the enterprise value chain.

Banking 52
article thumbnail

2020 Retrospective (and What's Coming in 2021)

Rock the JVM

In this article, I'll recap 2020's highlights, share key insights and achievements, and unveil exciting plans for the future of Rock the JVM

52
article thumbnail

Getting Started with Spring Cloud Data Flow and Confluent Cloud

Confluent

Data is the currency of competitive advantage in today’s digital age. All organizations struggle with their data due to the sheer variety of data types and ways that it can […].

Cloud 59
article thumbnail

Optimizing The Modern Developer Experience with Coder

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

article thumbnail

#ClouderaLife: Unplugged

Cloudera

It’s a trick as old as time… or at least as old as technology. We all know that step one to solving for any tech issue is to turn it off and then turn it back on again. But would it solve for issues in advance of them happening? And could this work not only for technology but for the people behind the technology? Our leadership team decided to explore that theory.

article thumbnail

How to Tackle Data Skew

Teradata

Learn how to use use Teradata's Global Space Accounting to counter our biggest villain: data skew.

Data 52
article thumbnail

5 things you should know about Real-Time Analytics

A Cloud Guru: Data Engineering

Running analytics on real-time data is a challenge many data engineers are facing today. But not all analytics can be done in real time! Many are dependent on the volume of the data and the processing requirements. Even logic conditions are becoming a bottleneck. For example, think about join operations on huge tables with more […] The post 5 things you should know about Real-Time Analytics appeared first on A Cloud Guru.

article thumbnail

A Visual Tour of the Global COVID-19 Vaccine Efforts

Preset

In response to the COVID-19 pandemic, hundreds of countries, organizations, universities, and companies came together to fund many vaccine candidates.

Data 40
article thumbnail

15 Modern Use Cases for Enterprise Business Intelligence

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

article thumbnail

Cloudera Operational Database Infrastructure Planning Considerations

Cloudera

In this blog post, let us take a look at how you can plan your infrastructure planning that you may have to do when deploying an operational database cluster on a CDP Private Cloud Base deployment. Note that you may have to do some planning assumptions when designing your initial infrastructure, and it must be flexible enough to scale up or down based on your future needs. .

article thumbnail

Intertoys

Teradata

Toy retailer uses Vantage on Azure, the modern cloud data analytics platform, as the building blocks for agility and cost-savings.

Retail 52
article thumbnail

Coffee with Cloudera: Cindy Maike, VP of Industry Solutions

Cloudera

Meet Cindy Maike, VP of Industry Solutions at Cloudera. Cindy has led the Industry Solutions team for over 3 years, with 6 years with Cloudera, and has been at the forefront of developing targeted vertical solutions for our customers and partners. Cindy is an exceptional female leader and we hope this blog gives you insight into the great work Cindy is doing with the Industry Solutions team!

article thumbnail

Making Privacy an Essential Business Process

Cloudera

Canada is poised to become a world-leader in privacy regulation and with new regulation comes record-breaking fines for those who can’t keep up. . In November, Canada introduced the Digital Charter Implementation Act. If passed, companies could face fines of up to five percent of global revenue or $25 million CAD — whichever is greater — for violating Canadians’ privacy.

Process 71
article thumbnail

Prepare Now: 2025s Must-Know Trends For Product And Data Leaders

Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.