Data Engineering Podcast
AUGUST 13, 2023
Summary Data pipelines are the core of every data product, ML model, and business intelligence dashboard. If you're not careful you will end up spending all of your time on maintenance and fire-fighting. The folks at Rivery distilled the seven principles of modern data pipelines that will help you stay out of trouble and be productive with your data.
Waitingforcode
AUGUST 17, 2023
Lessons learned why it's always worth checking the code implementation to avoid surprises later. Even for vendor-supported solutions.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
KDnuggets
AUGUST 16, 2023
LangChain simplifies building AI assistants with large language models, providing an intuitive API, memory capabilities, access to external tools, the ability to chain LLM actions, and prompt templating. Check out our newest cheat sheet to get up and running now.
Engineering at Meta
AUGUST 15, 2023
Instagram has introduced Immortal Objects – PEP-683 – to Python. Now, objects can bypass reference count checks and live throughout the entire execution of the runtime, unlocking exciting avenues for true parallelism. At Meta, we use Python (Django) for our frontend server within Instagram. To handle parallelism, we rely on a multi-process architecture along with asyncio for per-process concurrency.
Advertisement
Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.
Knowledge Hut
AUGUST 17, 2023
Staying current with rapidly advancing technology holds significant importance. Obtaining the latest certifications can enhance your professional standing by providing you with sought-after skills, thereby increasing your attractiveness to potential employers. It's noteworthy that AXELOS is a renowned authority in awarding certifications across a diverse spectrum of IT domains.
databricks
AUGUST 14, 2023
One of the key challenges that organizations face when adopting the open data lakehouse is selecting the optimal format for their data. Among.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
KDnuggets
AUGUST 14, 2023
Vector databases enable fast similarity search and scale across data points. For LLM apps, vector indexes can simplify architecture over full vector databases by attaching vectors to existing storage. Choosing indexes vs databases depends on specialized needs, existing infrastructure, and broader enterprise requirements.
Lyft Engineering
AUGUST 14, 2023
Where’s My Data — A Unique Encounter with Flink Streaming’s Kinesis Connector For years now, Lyft has not only been a proponent of but also a contributor to Apache Flink. Lyft’s pipelines have evolved drastically over the years , yet, time and time again, we run into unique cases that stretch Flink to its breaking points — this is one of those times.
databricks
AUGUST 16, 2023
Thousands of Databricks customers use Databricks Workflows every day to orchestrate business critical workloads on the Databricks Lakehouse Platform. As is often the.
ArcGIS
AUGUST 15, 2023
Summer internship experience with the Raster Analysis team at Esri- experience the world of GIS with Rakibul Ahasan.
Advertisement
Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?
KDnuggets
AUGUST 18, 2023
"You may also like" is a simple phrase that implies a new era in the way businesses interact and connect with their customers, and graph databases can easily help to build recommendation engines.
Cloudera
AUGUST 17, 2023
Machine Learning (ML) is at the heart of the boom in AI Applications, revolutionizing various domains. From powering intelligent Large Language Model (LLM) based chatbots like ChatGPT and Bard , to enabling text-to-AI image generators like Stable Diffusion , ML continues to drive innovation. Its transformative impact advances multiple fields from genetics to medicine to finance.
databricks
AUGUST 13, 2023
Today we are announcing the general availability of Delta Live Tables (DLT) on Google Cloud. DLT pipelines empower data engineers to build reliable.
Netflix Tech
AUGUST 16, 2023
By Arthur Gonigberg , Argha C Plaintext Past When Zuul was designed and developed , there was an inherent assumption that connections were effectively free, given we weren’t using mutual TLS (mTLS). It’s built on top of Netty , using event loops for non-blocking execution of requests, one loop per core. To reduce contention among event loops, we created connection pools for each, keeping them completely independent.
Speaker: Jay Allardyce, Deepak Vittal, and Terrence Sheflin
As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.
KDnuggets
AUGUST 16, 2023
How to Use ChatGPT to Convert Text into a PowerPoint Presentation • Best Python Tools for Building Generative AI Applications Cheat Sheet • Data Scientists Need to Specialize to Survive the Tech Winter • Python Vector Databases and Vector Indexes: Architecting LLM Apps • How To Speed Up SQL Queries Using Indexes [Python Edition]
Cloudera
AUGUST 16, 2023
Remember when we announced our redesigned partner program Cloudera Partner Network (CPN) last year? Our goal was to create a more competency-based approach and more comprehensive tools and support to help partners guide their customers adopting modern data strategies based on the Cloudera hybrid data platform. In addition, CPN helps our partners go to market faster, and provides industry-leading incentives and promotions aligned with partner business and sales models.
databricks
AUGUST 16, 2023
The Personalization Paradigm: Balancing Business Self-Service and Data Governance Personalization transforms businesses, shaping and reshaping the way brands connect with their audiences. Its.
Netflix Tech
AUGUST 17, 2023
By: Hamid Shahid , Laura Johnson , Tiffany Low Synopsis At Netflix, we have created millions of artwork to represent our titles. Each artwork tells a story about the title it represents. From our testing on promotional assets , we know which of these assets have performed well and which ones haven’t. Through this, our teams have developed an intuition of what visual and thematic artwork characteristics work well for what genres of titles.
Speaker: Nikhil Joshi, Founder & President of Snic Solutions
Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.
KDnuggets
AUGUST 15, 2023
Forget ChatGPT, This New AI Assistant Is Leagues Ahead and Will Change the Way You Work Forever • Best Python Tools for Building Generative AI Applications Cheat Sheet • 3 Ways to Access GPT-4 for Free • 5 Python Packages For Geospatial Data Analysis • Time Series Analysis: ARIMA Models in Python
Workfall
AUGUST 14, 2023
Reading Time: 7 minutes In today’s data-driven world, efficient data pipelines have become the backbone of successful organizations. These pipelines ensure that data flows smoothly from various sources to its intended destinations, enabling businesses to make informed decisions and gain valuable insights. Two powerful tools that have emerged to simplify the management of data pipelines are DBT (Data Build Tool) and Airflow.
Towards Data Science
AUGUST 17, 2023
Unit testing dbt models has always been one of the most critical missing pieces of the dbt ecosystem.
Ascend.io
AUGUST 18, 2023
Jupyter Notebooks have fundamentally revolutionized how data scientists approach their tasks. They offer an unparalleled environment for experimentation and visualization. Yet, there’s an interest in putting notebooks directly into production environments. While it’s great to take the ideas from notebooks and use them in real-world settings, trying to put the entire notebook directly into production as a code artifact can cause problems.
Advertisement
Cloud Development Environments (CDEs) are changing how software teams work by moving development to the cloud. Our Cloud Development Environment Adoption Report gathers insights from 223 developers and business leaders, uncovering key trends in CDE adoption. With 66% of large organizations already using CDEs, these platforms are quickly becoming essential to modern development practices.
KDnuggets
AUGUST 15, 2023
Devart is excited to extend a special offer to its valued customers on its 26th birthday. From August 15th to August 31st, 2023, you can dive into a world of seamless data connectivity with an incredible 20% discount on their top-notch Data Connectivity tools.
Knowledge Hut
AUGUST 18, 2023
In my early career, I knew that getting certified in AWS would be essential for success. Now that I have obtained multiple AWS certifications, I can vouch for their value to professionals & companies alike. With cloud computing becoming the new norm in today's marketplace, AWS certifications are nothing short of essential. From AWS Certified Solutions Architect to AWS Certified DevOps Engineer, there are many different paths to choose from as per your career goals & skill set.
Hevo
AUGUST 18, 2023
Are you struggling to connect PostgreSQL on Amazon RDS to Snowflake? Do you find it challenging to integrate the two platforms and leverage the powerful analytics features of Snowflake? You’re not alone. To analyze the vast amounts of data stored in PostgreSQL databases on Amazon RDS, integration with Snowflake becomes a viable solution.
Monte Carlo
AUGUST 17, 2023
TOCA Football is a technology enabled experience that helps players elevate their soccer game and is an official training partner of Major League Soccer. Their proprietary training system collects data on every session at a ball-by-ball level giving players the insights they need to take their training, and their game, to the next level. TOCA’s business also needs insights from high-quality data to perform at its peak.
Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage
When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.
KDnuggets
AUGUST 14, 2023
A speedy way to convert a long text to a short PowerPoint Presentation using only ChatGPT.
Knowledge Hut
AUGUST 18, 2023
As technology gets more advanced, cloud-native apps become more complicated. We need experts who can handle and set up these apps well. When I started as a new developer, I wasn't sure about learning Kubernetes. But working with different cloud apps showed me how powerful Kubernetes can be in managing and setting up my work. If you're in a similar situation, going for a Kubernetes certification is a great idea.
Hevo
AUGUST 18, 2023
There are several reasons why data replication from PostgreSQL on Amazon RDS to SQL Server may become necessary. These reasons include changes in business processes, increased data volumes, and enhanced performance requirements. By replicating data from PostgreSQL on Amazon RDS to SQL Server, you can also enhance data quality, improve accessibility, and ensure data security.
Monte Carlo
AUGUST 17, 2023
Data pipelines are a set of processes that enable the flow of data from one system to another, and one such process you can use is ETL ( e xtract, t ransform, l oad). The way to think of the relationship between the two is that ETL is a data pipeline, but not all data pipelines are ETL. Besides ETL, there are quite a few ways to process data that fall under the umbrella of data pipelines, including real time processing , data sharing , ELT , and Zero ETL.
Speaker: Donna Laquidara-Carr, PhD, LEED AP, Industry Insights Research Director at Dodge Construction Network
In today’s construction market, owners, construction managers, and contractors must navigate increasing challenges, from cost management to project delays. Fortunately, digital tools now offer valuable insights to help mitigate these risks. However, the sheer volume of tools and the complexity of leveraging their data effectively can be daunting. That’s where data-driven construction comes in.
Expert insights. Personalized for you.
130 
Let's personalize your content