Data Engineering Podcast

JUNE 3, 2018

Summary Using a multi-model database in your applications can greatly reduce the amount of infrastructure and complexity required. ArangoDB is a storage engine that supports documents, dey/value, and graph data formats, as well as being fast and scalable. In this episode Jan Steeman and Jan Stücke explain where Arango fits in the crowded database market, how it works under the hood, and how you can start working with it today.

Data Storage 100

Data Storage PostgreSQL Database Architecture 100

Dataquest

JUNE 8, 2018

The data science life cycle is generally comprised of the following components: data retrieval data cleaning data exploration and visualization statistical or predictive modeling While these components are helpful for understanding the different phases, they don’t help us think about our programming workflow. Often, the entire data science life cycle ends up as an arbitrary mess of notebook cells in either a Jupyter Notebook or a single messy script.

Data Science 52

Data Science Programming Data Data Pipeline 52

ProjectPro

JUNE 4, 2018

News on Hadoop - May 2018 Data-Driven HR: How Big Data And Analytics Are Transforming Recruitment.Forbes.com, May 4, 2018. With platforms like LinkedIn and Glassdoor giving every employer access to valuable big data, the world of recruitment transforming to intelligent recruitment.HR teams that make use of big data in future are likely to be successful in recruiting the right talent in the coming years.

Hadoop 52

Hadoop Recruitment Banking Big Data 52

Cloudera

JUNE 4, 2018

Authors: Mai N. Nguyen, Accenture & Mitch Gomulinski, Cloudera. Imagine storing the DNA of the entire population of the US – and then cloning them, twice. That’s the equivalent of 1 petabyte ( ComputerWeekly ) – the amount of unstructured data available within our large pharmaceutical client’s business. Then imagine the insights that are locked in that massive amount of data.

Pharmaceutical 46

Pharmaceutical Unstructured Data Electronics Metadata 46

Advertisement

In Airflow, DAGs (your data pipelines) support nearly every use case. As these workflows grow in complexity and scale, efficiently identifying and resolving issues becomes a critical skill for every data engineer. This is a comprehensive guide with best practices and examples to debugging Airflow DAGs. You’ll learn how to: Create a standardized process for debugging to quickly diagnose errors in your DAGs Identify common issues with DAGs, tasks, and connections Distinguish between Airflow-relate

Data Pipeline