Data Engineering Podcast
MARCH 15, 2021
Summary The team at DoorDash has a complex set of optimization challenges to deal with using data that they collect from a multi-sided marketplace. In order to handle the volume and variety of information that they use to run and improve the business the data team has to build a platform that analysts and data scientists can use in a self-service manner.
Confluent
MARCH 19, 2021
LiveStreams is a YouTube show about Confluent, real-time data streaming, and related technologies that help you maximize data in motion on any cloud. Every episode of LiveStreams will teach you […].
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Cloudera
MARCH 18, 2021
The Fundamental Review of the Trading Book (FRTB), introduced by the Basel Committee on Banking Supervision (BCBS), will transform how banks measure risk. FRTB is designed to address some fundamental weaknesses that did not get addressed in the post-2008 financial crisis regulatory reforms. In order to help make banks more resilient to drastic market changes, it will impose capital requirements that are more closely aligned with the market’s actual risk factors.
Teradata
MARCH 18, 2021
Without a strong foundation, any house will crumble. The same holds true for CFO Analytics – without a core foundational model, a strong analytics platform will be difficult to build.
Advertisement
Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?
Retail Insight
MARCH 18, 2021
As a former distance runner, I have seen first-hand how investment in elite sport is accelerating athletic performance. Just as the world has developed, our physical capabilities have too. New lifestyles, technologies, science , and world-class facilities all help to enabl e athletes to go ‘Faster – Higher – Stronger’, as the Olympics motto states.
Confluent
MARCH 17, 2021
In order for a technology like Confluent Cloud to make it easy to set data in motion, many different software systems are required to interact with each other using API […].
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Monte Carlo
MARCH 18, 2021
As founders of companies that build solutions designed to help teams deliver on the promise of data , we knew we wanted to build great products that are easy to deploy and manage for our customers. We also knew that since we would be integrating with our customers’ data stacks, we would need to offer the highest level of security and compliance. The question was: how are we going to build them?
ProjectPro
MARCH 17, 2021
“Machine Learning” and “Deep Learning” – are two of the most often confused and conflated terms that are used interchangeably in the AI world. However, there is one undeniable fact that both machine learning and deep learning are undergoing skyrocketing growth. According to Forbes , the global machine learning market will be worth $30.6 billion by 2024 and the deep learning market size is expected to reach $10.2 billion by 2025, expanding at a CAGR of 42.8% and 52.1
Confluent
MARCH 18, 2021
Building cross-platform solutions enables organizations to leverage technology driven by real-time data and enabled with both highly available services and low-latency databases hosted on Microsoft Azure. Azure Cosmos DB is […].
Grouparoo
MARCH 16, 2021
Don’t use underscores in your HTTP Headers. at least according to AWS and Nginx! curl --HEADER "AUTH_TOKEN: abc" example.com # is bad curl --HEADER "AUTH-TOKEN: abc" example.com # is OK Grouparoo is a self-hosted application, and we are always helping folks run and deploy our service in new ways. Recently, we’ve been working an example application for Amazon Web Service’s (AWS) Elastic Beanstalk service.
Speaker: Jay Allardyce, Deepak Vittal, and Terrence Sheflin
As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.
Teradata
MARCH 16, 2021
This guide describes the procedure to stream data into Teradata Vantage on AWS with AWS Glue Streaming ETL jobs and Amazon Kinesis, and to visualize the data with Amazon QuickSight.
Zalando Engineering
MARCH 15, 2021
Introduction The CSS Flexible Box Layout specification (AKA flexbox) is a useful abstraction for describing layouts in a platform agnostic way. For this reason, it is widely used on the web and even on mobile. Readers familiar with ConstraintLayout can think of flexbox as conceptually similar to the Flow virtual layout it supports. This type of layout is ideal for grids or other groups of views with varying sizes.
Rock the JVM
MARCH 18, 2021
This article provides a gentle introduction to Monoids and Semigroups, exploring functional abstractions and the workings of the Cats library
Grouparoo
MARCH 15, 2021
Last week when working on Grouparoo's OneSignal integration , I wrote a quick CLI script to create users in their system that we could use for testing. This script would take in a single argument, which would be the external_user_id on OneSignal: $./create_user < external_user_id > Parsing command-line arguments To allow parsing command-line arguments, Node provides process.argv.
Speaker: Nikhil Joshi, Founder & President of Snic Solutions
Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.
Teradata
MARCH 14, 2021
Often departments and project teams strive to solve their analytic needs independently. At first blush, it seems quicker that way. But inadvertently these efforts create technical debit.
Monte Carlo
MARCH 19, 2021
For Resident, a collection of premium direct-to-consumer brands, Data Observability has become a part of the Data Engineering team’s tech stack, helping them prevent broken data pipelines and achieve highly reliable data — at scale. “I only use 3 tabs at work: Gmail, BigQuery, and Monte Carlo,” says Resident’s Head of Data Engineering, Daniel Rimon.
Rock the JVM
MARCH 13, 2021
Learn how to tackle a Google-style algorithm interview question in Scala using pure functional programming
Grouparoo
MARCH 15, 2021
One thing we can observe from a table of CRM rate limits is a inverse correlation between "Enterprise-ness" systems and their normalized limits. Systems like Pardot, Marketo, Eloqua, and Salesforce have daily limits as opposed to per minute or second like the others. This encourages the use of their batching APIs. Batching in this context means performing API operations related to multiple people with one API request.
Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage
When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.
Teradata
MARCH 14, 2021
Healthcare provider uses Teradata Vantage for Patient 360 and Connected Identity efforts, improving the health of their 7M community members.
ProjectPro
MARCH 19, 2021
Natural language processing is a subfield of artificial intelligence that aims at making machines understand natural languages just like humans do. The power of NLP techniques dates back to the 1950’s Turing Test ( also known as the Imitation Game ), a test to determine if a machine can be considered intelligent.“ A computer would deserve to be called intelligent if it could deceive a human into believing that it was human.”- Alan Turing The Turing test serves as a milestone fo
Zalando Engineering
MARCH 17, 2021
In our first post about How we use GraphQL at Zalando , we briefly shared about performance optimizations using GraphQL-JIT. GraphQL-JIT allowed us to scale our implementation without performance degradations. In this post, we share another optimization we use - Lookaheads. Same Model; Different Views In our GraphQL service, we do not have resolvers for every single field in the schema.
Grouparoo
MARCH 14, 2021
TLDR: This command will fully get rid of the master branch after you've switched to using main. git remote prune origin What I was seeing was git checkout master still referenced the existence of the old branch I had used. I needed to get rid of that old one so that it would be an error. The switch We have switched to using the main branch instead of the master branch on all our Github projects.
Speaker: Donna Laquidara-Carr, PhD, LEED AP, Industry Insights Research Director at Dodge Construction Network
In today’s construction market, owners, construction managers, and contractors must navigate increasing challenges, from cost management to project delays. Fortunately, digital tools now offer valuable insights to help mitigate these risks. However, the sheer volume of tools and the complexity of leveraging their data effectively can be daunting. That’s where data-driven construction comes in.
Monte Carlo
MARCH 16, 2021
Data pipelines can break for a million different reasons, but how can we have awesome data monitoring in place to ensure data quality issues are identified and addressed in real time—at scale? Sometimes, all it takes is a bit of SQL, some precision and recall, and a holistic approach to data observability. In this article , we walk through how you can create your own data monitors from scratch and leverage basic principles of machine learning to apply them at scale across your data pipelines.
Rockset
MARCH 15, 2021
When working with a real-time analytics system you need your database to meet very specific requirements. This includes making the data available for query as soon as it is ingested, creating proper indexes on the data so that the query latency is very low, and much more. Before it can be ingested, there’s usually a data pipeline for transforming incoming data.
Preset
MARCH 15, 2021
In this tutorial, we'll showcase how to ingest data on Wikipedia edits into Apache Druid. Then, we'll use a time-series chart to visualize the data in Superset.
Cloudera
MARCH 16, 2021
In celebration of International Women’s Day, on Wednesday, March 10 women from across Cloudera in America and EMEA came together with an external audience to debate and discuss the state of women. Our transatlantic panel featuring Bianka Racz, Software Engineer; Christine Sherry, Director Critical Incident Response; Dominique Hollins, Director of Culture and Inclusion and Tahmeena Tariq Senior Corporate Counsel, came together to discuss how COVID has changed our expectations of employers, the wo
Speaker: Evelyn Chou
Choosing the right business intelligence (BI) platform can feel like navigating a maze of features, promises, and technical jargon. With so many options available, how can you ensure you’re making the right decision for your organization’s unique needs? 🤔 This webinar brings together expert insights to break down the complexities of BI solution vetting.
Monte Carlo
MARCH 18, 2021
You’re on Snowflake and Looker? Great. But for most companies, having a cloud data stack is just the tip of the iceberg when it comes to operationalizing their data and analytics at scale. We share five non-obvious roadblocks businesses face when becoming data driven and what some of the industry’s leading data engineering and analytics teams are doing to overcome them.
Expert insights. Personalized for you.
100 
Let's personalize your content