Waitingforcode

AUGUST 17, 2023

Lessons learned why it's always worth checking the code implementation to avoid surprises later. Even for vendor-supported solutions.

Coding 130

Coding IT 130

Data Engineering Podcast

AUGUST 13, 2023

Summary Data pipelines are the core of every data product, ML model, and business intelligence dashboard. If you're not careful you will end up spending all of your time on maintenance and fire-fighting. The folks at Rivery distilled the seven principles of modern data pipelines that will help you stay out of trouble and be productive with your data.

Data Pipeline 130

Data Pipeline BI SQL Machine Learning 130

KDnuggets

AUGUST 14, 2023

A speedy way to convert a long text to a short PowerPoint Presentation using only ChatGPT.

108

108

ArcGIS

AUGUST 15, 2023

Summer internship experience with the Raster Analysis team at Esri- experience the world of GIS with Rakibul Ahasan.

98

98

Advertisement

Apache Airflow® is the open-source standard to manage workflows as code. It is a versatile tool used in companies across the world from agile startups to tech giants to flagship enterprises across all industries. Due to its widespread adoption, Airflow knowledge is paramount to success in the field of data engineering.

Datasets

databricks

AUGUST 14, 2023

One of the key challenges that organizations face when adopting the open data lakehouse is selecting the optimal format for their data. Among.

Data 98

Data Engineering 98

KDnuggets

AUGUST 16, 2023

LangChain simplifies building AI assistants with large language models, providing an intuitive API, memory capabilities, access to external tools, the ability to chain LLM actions, and prompt templating. Check out our newest cheat sheet to get up and running now.

Accessible 107

Accessible Accessibility Building Machine Learning 107

Engineering at Meta

AUGUST 15, 2023

Instagram has introduced Immortal Objects – PEP-683 – to Python. Now, objects can bypass reference count checks and live throughout the entire execution of the runtime, unlocking exciting avenues for true parallelism. At Meta, we use Python (Django) for our frontend server within Instagram. To handle parallelism, we rely on a multi-process architecture along with asyncio for per-process concurrency.

Python 98

Python Metadata Architecture Process 98

databricks

AUGUST 13, 2023

Today we are announcing the general availability of Delta Live Tables (DLT) on Google Cloud. DLT pipelines empower data engineers to build reliable.

Google Cloud 98

Google Cloud Cloud Data Engineering Data Engineer 98

Knowledge Hut

AUGUST 17, 2023

Staying current with rapidly advancing technology holds significant importance. Obtaining the latest certifications can enhance your professional standing by providing you with sought-after skills, thereby increasing your attractiveness to potential employers. It's noteworthy that AXELOS is a renowned authority in awarding certifications across a diverse spectrum of IT domains.

Certification 90

Certification Portfolio Consulting Business Analyst 90

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

KDnuggets

AUGUST 18, 2023

The improvement you need to get the best result from GPT.

Python 106

Python 106

DoorDash Engineering

AUGUST 15, 2023

As DoorDash experienced rapid growth over the last few years, we began to see the limits of our traditional methods of monitoring. Metrics, logs, and traces provide vital information about our service ecosystem. But these signals almost entirely rely on application-level instrumentation, which can leave gaps or conflicting semantics across different systems.

Bytes 84

Bytes PostgreSQL Coding Database 84

databricks

AUGUST 16, 2023

Thousands of Databricks customers use Databricks Workflows every day to orchestrate business critical workloads on the Databricks Lakehouse Platform. As is often the.

98

98

Lyft Engineering

AUGUST 14, 2023

Where’s My Data — A Unique Encounter with Flink Streaming’s Kinesis Connector For years now, Lyft has not only been a proponent of but also a contributor to Apache Flink. Lyft’s pipelines have evolved drastically over the years , yet, time and time again, we run into unique cases that stretch Flink to its breaking points — this is one of those times.

Data 80

Data Engineering Process Management 80

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

KDnuggets

AUGUST 17, 2023

Integrating Open Source LLMs and LangChain for Free Generative Question Answering (No API Key required).

103

103

Cloudera

AUGUST 17, 2023

Machine Learning (ML) is at the heart of the boom in AI Applications, revolutionizing various domains. From powering intelligent Large Language Model (LLM) based chatbots like ChatGPT and Bard , to enabling text-to-AI image generators like Stable Diffusion , ML continues to drive innovation. Its transformative impact advances multiple fields from genetics to medicine to finance.

Manufacturing 72

Manufacturing Finance Python Machine Learning 72

databricks

AUGUST 16, 2023

The Personalization Paradigm: Balancing Business Self-Service and Data Governance Personalization transforms businesses, shaping and reshaping the way brands connect with their audiences. Its.

Data Governance 96

Data Governance Government Data IT 96

Netflix Tech

AUGUST 16, 2023

By Arthur Gonigberg , Argha C Plaintext Past When Zuul was designed and developed , there was an inherent assumption that connections were effectively free, given we weren’t using mutual TLS (mTLS). It’s built on top of Netty , using event loops for non-blocking execution of requests, one loop per core. To reduce contention among event loops, we created connection pools for each, keeping them completely independent.

Algorithm 69

Algorithm Utilities AWS Cloud Computing 69

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

KDnuggets

AUGUST 14, 2023

Vector databases enable fast similarity search and scale across data points. For LLM apps, vector indexes can simplify architecture over full vector databases by attaching vectors to existing storage. Choosing indexes vs databases depends on specialized needs, existing infrastructure, and broader enterprise requirements.

Database 102

Database Python Architecture Data 102

Cloudera

AUGUST 16, 2023

Remember when we announced our redesigned partner program Cloudera Partner Network (CPN) last year? Our goal was to create a more competency-based approach and more comprehensive tools and support to help partners guide their customers adopting modern data strategies based on the Cloudera hybrid data platform. In addition, CPN helps our partners go to market faster, and provides industry-leading incentives and promotions aligned with partner business and sales models.

Telecommunication 65

Telecommunication Insurance Data Architect Manufacturing 65

Workfall

AUGUST 14, 2023

Reading Time: 7 minutes In today’s data-driven world, efficient data pipelines have become the backbone of successful organizations. These pipelines ensure that data flows smoothly from various sources to its intended destinations, enabling businesses to make informed decisions and gain valuable insights. Two powerful tools that have emerged to simplify the management of data pipelines are DBT (Data Build Tool) and Airflow.

Data Pipeline 62

Data Pipeline Data Raw Data Database 62

Netflix Tech

AUGUST 17, 2023

By: Hamid Shahid , Laura Johnson , Tiffany Low Synopsis At Netflix, we have created millions of artwork to represent our titles. Each artwork tells a story about the title it represents. From our testing on promotional assets , we know which of these assets have performed well and which ones haven’t. Through this, our teams have developed an intuition of what visual and thematic artwork characteristics work well for what genres of titles.

Media 69

Media Algorithm Architecture Building 69

Advertisement

With over 30 million monthly downloads, Apache Airflow is the tool of choice for programmatically authoring, scheduling, and monitoring data pipelines. Airflow enables you to define workflows as Python code, allowing for dynamic and scalable pipelines suitable to any use case from ETL/ELT to running ML/AI operations in production. This introductory tutorial provides a crash course for writing and deploying your first Airflow pipeline.

Cloud

KDnuggets

AUGUST 15, 2023

In this free ebook, readers will learn how to employ data cleaning and preprocessing for data science using the Python ecosystem.

Data Science 102

Data Science Python Data 102

Ascend.io

AUGUST 18, 2023

Jupyter Notebooks have fundamentally revolutionized how data scientists approach their tasks. They offer an unparalleled environment for experimentation and visualization. Yet, there’s an interest in putting notebooks directly into production environments. While it’s great to take the ideas from notebooks and use them in real-world settings, trying to put the entire notebook directly into production as a code artifact can cause problems.

Data Pipeline 52

Data Pipeline Programming Language Data Science Coding 52

Knowledge Hut

AUGUST 18, 2023

In my early career, I knew that getting certified in AWS would be essential for success. Now that I have obtained multiple AWS certifications, I can vouch for their value to professionals & companies alike. With cloud computing becoming the new norm in today's marketplace, AWS certifications are nothing short of essential. From AWS Certified Solutions Architect to AWS Certified DevOps Engineer, there are many different paths to choose from as per your career goals & skill set.

AWS 52

AWS Certification Amazon Web Services Cloud 52

Confluent

AUGUST 18, 2023

Learn how Australia’s Alex Bank leveraged real-time streaming to create a data-driven, customer-focused banking experience.

Banking 52

Banking Data 52

Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

KDnuggets

AUGUST 18, 2023

Do we still need data science, or are tools such as ChatGPT taking over the majority of tasks?

Data Science 101

Data Science Data 101

Hevo

AUGUST 18, 2023

Are you struggling to connect PostgreSQL on Amazon RDS to Snowflake? Do you find it challenging to integrate the two platforms and leverage the powerful analytics features of Snowflake? You’re not alone. To analyze the vast amounts of data stored in PostgreSQL databases on Amazon RDS, integration with Snowflake becomes a viable solution.

PostgreSQL 52

PostgreSQL Database Data IT 52

Knowledge Hut

AUGUST 18, 2023

As technology gets more advanced, cloud-native apps become more complicated. We need experts who can handle and set up these apps well. When I started as a new developer, I wasn't sure about learning Kubernetes. But working with different cloud apps showed me how powerful Kubernetes can be in managing and setting up my work. If you're in a similar situation, going for a Kubernetes certification is a great idea.

Certification 52

Certification AWS Amazon Web Services Cloud 52