Data Engineering Podcast

MAY 27, 2018

Summary Building an ETL pipeline is a common need across businesses and industries. It’s easy to get one started but difficult to manage as new requirements are added and greater scalability becomes necessary. Rather than duplicating the efforts of other engineers it might be best to use a hosted service to handle the plumbing so that you can focus on the parts that actually matter for your business.

Data Pipeline 100

Data Pipeline MongoDB Google Cloud Scala 100

Uber Engineering

MAY 16, 2018

At Uber, we combine real-time systems monitoring with intelligent alerting mechanisms to ensure the availability and reliability of our apps. In our push to empower our engineers to author more accurate alerts, Uber’s Observability Applications team sought to introduce alert … The post Engineering a Job-based Forecasting Workflow for Observability Anomaly Detection appeared first on Uber Engineering Blog.

Engineering 81

Engineering Systems Architecture 81

Zalando Engineering

MAY 30, 2018

Comparing Redux, MobX & setState in React by Kaiser Anwar Shad and revised by Eugen Kiss Introduction React is a declarative, efficient, and flexible JavaScript library for building user interfaces. Compared to other frontend libraries and frameworks, React’s core concept is simple but powerful: ‘React makes it painless to design simple views and renders by using virtual DOM’.

Management 52

Management Software Engineering Software Engineer Database 52

InData Labs

MAY 7, 2018

Digital Transformation Definition Digital transformation has been a big topic for a few years now, and it has many definitions. From a business perspective, digital transformation is about leveraging digital technologies to improve processes, competencies, and business models. It is also about changing the culture of the company because it requires letting go of old.

Process 52

Process Technology IT Data Science 52

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

Cloud Academy

MAY 2, 2018

A 2017 IDC White Paper “recommend[s] that organizations that want to get the most out of cloud should train a wide range of stakeholders on cloud fundamentals and provide deep training to key technical teams ” (emphasis ours). Regular readers of the Cloud Academy blog know we’ve been talking about this for a long time. Future-proofing your organization requires technical excellence, collective experience, business context, and shared understanding.

Google Cloud 52

Google Cloud Machine Learning Big Data AWS 52

Cloudera

MAY 2, 2018

Global Technology Partner of the Year. A Cloudera contingent, including Business Development, Marketing, Sales, and Software Engineering just got back from Qlik Qonnections in Orlando, Florida where the vibe to #DisruptEverything was strong! Qonnections is the premier Business Intelligence (BI) conference that brings data lovers together to share insights and discover data in new ways.

BI 45

BI Business Intelligence Software Engineering Software Engineer 45

ProjectPro

MAY 1, 2018

News on Hadoop - April 2018 Big Data and Cambridge Analytica: 5 Big Picture Truths.Datamation.com, April 2, 2018. Cambridge Analytica is in the news headlines as people realize the fact that how Trump campaign and its donors used FB ads and created a sociopolitical shakeup.Let’s understand what big truths does the #deletefacebook movement reveal and what implications it has - i) This data leak is not due to any breach but is a quite common and nothing unusual.

Hadoop 40

Hadoop Banking Healthcare Big Data 40

Zalando Engineering

MAY 2, 2018

Our cooperation with CODE University At Zalando, we strive to create an environment in which all our engineers, product, and design specialists feel they can inspire each other, make their ideas a reality, and contribute to providing the best possible platform for Zalando’s customers to have the ultimate customer experience. Part of this is making sure we understand what the future generation of product managers, interaction designers, and software engineers are thinking and what ideas and innov

Designing 52

Designing Engineering Software Engineering Software Engineer 52

Data Engineering Podcast

MAY 13, 2018

Summary The Open Data Science Conference brings together a variety of data professionals each year in Boston. This week’s episode consists of a pair of brief interviews conducted on-site at the conference. First up you’ll hear from Andy Eschbacher of Carto. He dscribes some of the complexities inherent to working with geospatial data, how they are handling it, and some of the interesting use cases that they enable for their customers.

Data Science 100

Data Science Algorithm Database Data Engineering 100

Data Engineering Podcast

MAY 6, 2018

Summary The Open Data Science Conference brings together a variety of data professionals each year in Boston. This week’s episode consists of a pair of brief interviews conducted on-site at the conference. First up you’ll hear from Alan Anders, the CTO of Applecart about their challenges with getting Spark to scale for constructing an entity graph from multiple data sources.

Data Science 100

Data Science Machine Learning Data Engineering Data Engineer 100

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Cloudera

MAY 15, 2018

Over the past decade, we have observed open source powered big data and analytics platforms evolve from large data storage containers to massively scalable advanced modeling platforms that seamlessly operate on-premises and in a multi-cloud environment. Cloudera Enterprise is the market leader in this space and working closely with the wider open source community to integrate the latest innovations in machine learning and artificial intelligence (AI).

Finance 42

Finance Machine Learning Big Data Banking 42

Cloudera

MAY 16, 2018

The Keystone to a Holistic Trade Surveillance Strategy. Criminals don’t refer to a playbook of best practices to execute a crime. They are creative in their thinking and collaborative in their efforts (including with parties who may not know they are complicit) to obtain their objectives and avoid getting caught. Trade surveillance in the financial services industry must respond to their creativity and the complexities of a global trading network, the myriad of digital and physical communication

Data Lake 42

Data Lake Electronics Media Unstructured Data 42

Cloudera

MAY 22, 2018

Cloudera Data Science Workbench (CDSW) makes secure, collaborative data science at scale a reality for the enterprise and accelerates the delivery of new data products. With CDSW, organizations can research and experiment faster, deploy models easily and with confidence, as well as rely on the wider Cloudera platform to reduce the risks and costs of data science projects.

Data Science 42

Data Science Machine Learning Big Data Metadata 42

Cloudera

MAY 10, 2018

Cloudera 2017 Data Impact Award Winners. We are excited to kick off the 2018 Data Impact Awards ! Nominations are now open and will remain open until July 20th. Since 2012, the Data Impact Awards have showcased how organizations are using Cloudera and the power of data to transform themselves and achieve dramatic results. Click to tweet : Nominations are now open for the sixth annual Cloudera Data Impact Awards!

Healthcare 41

Healthcare Machine Learning Media Data Science 41

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

Cloudera

MAY 22, 2018

Why Enterprises Need to Unify ML, Analytics, and Cloud. Times are changing, and the traditional models of analytics and data management don’t serve the needs of the modern enterprise, so the way to address these topics is changing too. While organizations are moving more workloads to the cloud, many mission-critical workloads remain on-prem. End users, data stewards, governance groups, and security groups alike can easily get overwhelmed with multiple access points, inconsistent user interfaces,

Building 40

Building Data Warehouse Government Machine Learning 40

Cloudera

MAY 22, 2018

It was exactly one year ago at Strata London that we introduced the world to Cloudera Altus Data Engineering. The premise was simple: make it quicker and easier for customers to drive data to their machine learning and analytics services by leveraging cloud resources, while at the same time, eliminating the pain associated with managing datacenter or cloud infrastructure.

AWS 40

AWS Data Warehouse Data Lake Data Engineering 40

Zalando Engineering

MAY 16, 2018

Sharing successful large scale agile experiences Zalando has been known for radical approaches to agility since 2015. In order to keep growing and staying successful we took the next step in 2017 forming around 30 business units. Each business unit is formed around one particular business problem, topic or product with end2end responsibility. All disciplines needed are inside this business unit from commercial roles to tech teams.

Data Science 40

Data Science Engineering Coding Process 40

Zalando Engineering

MAY 14, 2018

How to establish and evolve your data science community In Zalando, we have many guilds: self-organized groups of people who share interests. The topics, scope, size, and ways to organize the guilds varies. We have technical guilds like the web or API guilds , local and artistic guilds like the knitting guild in Helsinki, and some guilds that support the growth of people in certain job families, like the Data Science Guild.

Data Science 40

Data Science Data Portfolio Designing 40

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

Zalando Engineering

MAY 8, 2018

Moving from a more traditional internal IT setup to a product-driven culture I love building enterprise systems, because you get to work with your customers/users every day and literally see their lives change as you release new features. In my case, at Zalando, these are systems for fashion buying, supply chain management, inventory management and procure-to-pay processes (e.g. paying our suppliers for merchandise we bought from them).

Systems 40

Systems Management Recruitment Retail 40

Zalando Engineering

MAY 7, 2018

Real-time joins in event-driven microservices As discussed in my previous blog post , Kafka is one of the key components of our event-driven microservice architecture in Zalando’s Smart Product Platform. We use it for sequencing events and building an aggregated view of data hierarchies. This post expands on what I previously wrote about the one-to-many data model and introduces more complex many-to-many relationships.

Kafka 40

Kafka Scala Java Media 40

Zalando Engineering

MAY 1, 2018

Celebrating Zalando’s first international tech hub Three years ago, Zalando decided to start looking beyond Germany’s borders to tap into Europe’s pool of tech talent. Diverse and brilliant minds from other European cities and beyond contributed to cementing Zalando’s place as Europe’s most fashionable tech company. So, back in 2015, Zalando’s first move was across the Irish Sea, and now the team is very excited to celebrate its third anniversary!

Software Engineering 40

Software Engineering Software Engineer Datasets Big Data 40