Data Engineering Podcast

JANUARY 28, 2019

Summary Controlling access to a database is a solved problem… right? It can be straightforward for small teams and a small number of storage engines, but once either or both of those start to scale then things quickly become complex and difficult to manage. After years of running across the same issues in numerous companies and even more projects Justin McCarthy built strongDM to solve database access management for everyone.

Accessibility 100

Accessibility Accessible Database Management 100

Netflix Tech

JANUARY 31, 2019

by Richard Cool Netflix has over 139M members streaming on more than half a billion devices spanning over 1,700 different types of devices from hundreds of brands. This diverse device ecosystem results in a high dimensionality feature space, often with sparse data, and can make identifying device performance issues challenging. Identifying ways to scale solutions in this space is vital as the ecosystem continues to grow both in volume and diversity.

Electronics 88

Electronics Entertainment Machine Learning Engineering 88

Cloudera

JANUARY 30, 2019

It’s official – Cloudera and Hortonworks have merged , and today I’m excited to announce the availability of Cloudera Data Science Workbench (CDSW) for Hortonworks Data Platform (HDP). Trusted by large data science teams across hundreds of enterprises —. Western Union and IQVIA to name just a couple — CDSW is now also ready to help Hortonworks customers accelerate the delivery of new data products through secure, collaborative data science at scale.

Data Science 75

Data Science Machine Learning Scala Government 75

Uber Engineering

JANUARY 16, 2019

Lau Skorstengaard is a Ph.D. student at Aarhus University who pursued a 2018 internship with Uber Engineering’s Aarhus, Denmark office. In this article, Lau discusses his path to Uber and the technical challenges faced while building his internship project as … The post Aarhus Engineering Internship: Building Aggregation Support for YQL, Uber’s Graph Query Language for Grail appeared first on Uber Engineering Blog.

Engineering 64

Engineering Building Project Programming Language 64

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

Teradata

JANUARY 7, 2019

David Socha compares Karl Benz and Sigmund Freud, two people that fundamentally and indisputably influenced how we live today.

75

75

Pandora Engineering

JANUARY 14, 2019

Photo credit: Stewart Sutton | DigitalVision via Getty Images You’re in a maze of twisty little passages, all alike The state of the art in online computer gaming circa 1976 was a game called Adventure , which consisted of typing short instructions into a computer terminal and getting terse responses, which you had to interpret to complete a vaguely-defined mission.

Software Engineer 52

Software Engineer Software Engineering Utilities Programming 52

Netflix Tech

JANUARY 31, 2019

By Gang Su & Ian Yohai From living rooms in Bogota, to morning commutes in Tokyo, to beaches in Los Angeles and dorms in Berlin, Netflix strives to bring joy to over 139 million members around the globe and connect people with stories they’ll love. Every bit of the customer experience is imbued with innovation, right from the very first encounter with Netflix during the signup process?

Healthcare 84

Healthcare Process Designing Accessibility 84

Cloudera

JANUARY 3, 2019

A new year is always an opportunity for change. This year, we’re making a big one. On January 3, we closed the merger of Cloudera and Hortonworks — the two leading companies in the big data space — creating a single new company that is the leader in our category. We are well positioned to deliver even more innovation and success than we have independently over the last decade.

Hadoop 74

Hadoop Machine Learning Big Data Data Warehouse 74

InData Labs

JANUARY 2, 2019

These days, office employees need an AI hero. Can you imagine the number of hours wasted on handling a paper-based workflow? Isn’t it time to save employees from piles of paper? No one is saying it will be easy to eliminate paper documents promptly. For instance, in the legal sphere where the cost of a. Запись How OCR Can Help Employees Fight Through Most Mundane Tasks впервые появилась InData Labs.

IT 52

IT Data Engineering Data Engineer Engineering 52

Teradata

JANUARY 1, 2019

Regulations like GDPR are an opportunity for many organizations, Reiner Kappenberger explains how data privacy can be good for your business.

Data 63

Data 63

Speaker: Jay Allardyce, Deepak Vittal, and Terrence Sheflin

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

Pandora Engineering

JANUARY 7, 2019

How We Updated Pandora on iOS 12 Launch Day Photo Credit: Stavros Constantinou The Story That Shook the Press The Pandora app was amongst the very few enterprise apps that successfully released an update for iOS 12 on Apple’s day one September 21 launch date, supporting the exciting new Siri Shortcuts feature. Here are some notable quotes: Engadget, “Music app Pandora is taking advantage of Shortcuts at iOS 12’s launch.

Cloud Computing 52

Cloud Computing Architecture Cloud Building 52

Data Engineering Podcast

JANUARY 13, 2019

Summary The past year has been an active one for the timeseries market. New products have been launched, more businesses have moved to streaming analytics, and the team at Timescale has been keeping busy. In this episode the TimescaleDB CEO Ajay Kulkarni and CTO Michael Freedman stop by to talk about their 1.0 release, how the use cases for timeseries data have proliferated, and how they are continuing to simplify the task of processing your time oriented events.

Database 100

Database PostgreSQL SQL MongoDB 100

Zalando Engineering

JANUARY 30, 2019

Shared language and visualizing to deliver great products *Football is an environment with changing variables that players and coaches need to react to. Teams attempt to move the ball down the field by running or passing in a set number of plays. *If you’ve ever watched a football game you will see coaches holding a subset of plays from the coach’s playbook they think may work for the game they are playing.

Designing 40

Designing Building IT Data 40

Cloudera

JANUARY 22, 2019

Today’s data landscape is characterized by exponentially increasing volumes of data, comprising a variety of structured, unstructured, and semi-structured data types originating from an expanding number of disparate data sources located on-premises, in the cloud, and at the edge. In conjunction with the evolving data ecosystem are demands by business for reliable, trustworthy, up-to-date data to enable real-time actionable insights.

Big Data 44

Big Data NoSQL Hadoop Data Lake 44

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

Rockset

JANUARY 23, 2019

Redash is a powerful open source query and visualization tool that helps you make sense of your data. It connects to variety of data sources and also includes a native connector for Rockset. In this post we will demonstrate how to use Redash to build live dashboards on Rockset data sets. Configure If you've never used Redash before, you need to set it up first.

SQL 40

SQL Building IT Data 40

Teradata

JANUARY 21, 2019

Applying deep reinforcement learning to real world problems has the potential to revolutionize how businesses tackle many of their core business challenges.

Data 45

Data 45

Data Engineering Podcast

JANUARY 6, 2019

Summary The Hadoop platform is purpose built for processing large, slow moving data in long-running batch jobs. As the ecosystem around it has grown, so has the need for fast data analytics on fast moving data. To fill this need the Kudu project was created with a column oriented table format that was tuned for high volumes of writes and rapid query execution across those tables.

Data Analytics 100

Data Analytics Hadoop Kafka Media 100

Zalando Engineering

JANUARY 28, 2019

Nakadi is Zalando’s open source event streaming platform. It is based on Apache Kafka. It started as a simple HTTP proxy, providing a REST interface to publish and consume JSON messages. It quickly evolved, with the addition of schema validation and evolution, self-service authorization, a subscription API for easy consumption, deep integration with Zalando’s infrastructure, a SQL-over-streams engine, and much more.

Scala 40

Scala Java Kafka SQL 40

Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage

When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.

Systems

Rockset

JANUARY 23, 2019

Have you ever wanted to run SQL queries on Amazon DynamoDB tables without impacting your production workloads? Wouldn't it be great to do so without needing to set up an ETL job and then having to manually monitor that job? In this blog, I will discuss how Rockset integrates with DynamoDB and continuously updates a collection automatically as new objects are added to a DynamoDB table.

SQL 40

SQL NoSQL AWS Datasets 40

Zalando Engineering

JANUARY 23, 2019

You’ve probably never had the same day twice at your current job. At Zalando it’s no different. Here, it not only depends on the product you're currently working on but also on your peers. Actually, what's expected from a frontend engineer can vary according to a company philosophy or your own previous experience: usually a frontend engineer can be seen as a Swiss army knife when in reality at Zalando, for example, we see them as masters of trades.

Engineering 40

Engineering Coding AWS Technology 40

Rockset

JANUARY 21, 2019

An incredible amount of business data is floating around in Excel spreadsheets - so data scientists often need to analyze data across multiple worksheets or even multiple spreadsheets using SQL. Additionally, this data may need to be joined with other data sets that are in JSON, CSV or Parquet formats. Microsoft Excel currently has some basic SQL support in place: Use SQL for connecting to an external database like Access or SQL Server, parsing field or table contents and importing the data.

SQL 40

SQL Data Lake AWS Data Science 40

Rockset

JANUARY 10, 2019

This post outlines how to use SQL for querying and joining raw data sets like nested JSON and CSV - for enabling fast, interactive data science. Data scientists and analysts deal with complex data. Much of what they analyze could be third-party data, over which there is little control. In order to make use of this data, significant effort is spent in data engineering.

Data Science 40

Data Science SQL Relational Database Raw Data 40

Speaker: Donna Laquidara-Carr, PhD, LEED AP, Industry Insights Research Director at Dodge Construction Network

In today’s construction market, owners, construction managers, and contractors must navigate increasing challenges, from cost management to project delays. Fortunately, digital tools now offer valuable insights to help mitigate these risks. However, the sheer volume of tools and the complexity of leveraging their data effectively can be daunting. That’s where data-driven construction comes in.

Project

Zalando Engineering

JANUARY 9, 2019

How the Zalando Personalization Unit improved with a diagnostic In our coaching work, doing diagnostics can already create huge improvements without a lot of action on our part. Working at scale, Zalando has around 150 tech teams, this helps create an impact on the whole organisation. In this blog post, I will share the story of a diagnostic done in a unit of seven machine learning and data scientist teams (ML/DS) in Berlin, Helsinki and Dublin.

Machine Learning 40

Machine Learning Systems Engineering Management 40

Rockset

JANUARY 8, 2019

Rockset makes it easy to develop serverless microservices, data APIs, and data-driven applications. This video demo shows an example of what's possible with Rockset. For this exercise, we will build a serverless microservice to discover the stock symbols with the most mentions on Twitter. Ingest Our Twitter stream comes from Amazon Kinesis and is continuously ingested into Rockset.

AWS 40

AWS Building SQL Java 40

Zalando Engineering

JANUARY 6, 2019

Project Highlights Patroni - one of the most well-known open source projects of Zalando is now deployed as the Postgres Failover Manager on GitLab.com. Patroni was created a few years back when we needed an automatic failover to manage hundreds of in-house clusters. The project was a fork of Compose Governor , Patroni quickly overtook the original version and became one of the most widely used template for PostgreSQL High Availability these days.

Machine Learning 40

Machine Learning PostgreSQL Python Database 40

Teradata

JANUARY 29, 2019

Ben MacKenzie reflects on some of the unique challenges to building models with relational data.

Building 53

Building Data 53

Speaker: Aindra Misra, Senior Manager, Product Management (Data, ML, and Cloud Infrastructure) at BILL

Join us for an insightful webinar that explores the critical intersection of data privacy and AI governance. In today’s rapidly evolving tech landscape, building robust governance frameworks is essential to fostering innovation while staying compliant with regulations. Our expert speaker, Aindra Misra, will guide you through best practices for ensuring data protection while leveraging AI capabilities.

Government

Teradata

JANUARY 21, 2019

Applying deep reinforcement learning to real world problems has the potential to revolutionize how businesses tackle many of their core business challenges.

Data 40

Data 40

Teradata

JANUARY 14, 2019

The definition of insanity is doing the same thing over and over and expecting different results.

Big Data 45

Big Data Data 45

Teradata

JANUARY 10, 2019

Reinforcement learning is the machine learning approach that is behind some of the most talked about advances in AI.

Machine Learning 40

Machine Learning 40

Teradata

JANUARY 10, 2019

Reinforcement learning is the machine learning approach that is behind some of the most talked about advances in AI.

Machine Learning 40

Machine Learning 40

Advertisement

Entity Resolution Sometimes referred to as data matching or fuzzy matching, entity resolution, is critical for data quality, analytics, graph visualization and AI. Learn what entity resolution is, why it matters, how it works and its benefits. Advanced entity resolution using AI is crucial because it efficiently and easily solves many of today’s data quality and analytics problems.

IT