KDnuggets

SEPTEMBER 12, 2024

Develop unique yet standing-out data science projects to improve your data portfolio.

Data Science 149

Data Science Project Portfolio Data 149

Analytics Vidhya

SEPTEMBER 12, 2024

Introduction Imagine yourself as a data professional tasked with creating an efficient data pipeline to streamline processes and generate real-time information. Sounds challenging, right? That’s where Mage AI comes in to ensure that the lenders operating online gain a competitive edge. Picture this: thus, unlike many other extensions that require deep setup and constant coding, […] The post Setup Mage AI with Postgres to Build and Manage Your Data Pipeline appeared first on Analytics Vidhy

Data Pipeline 144

Data Pipeline Building Management Data 144

Confluent

SEPTEMBER 9, 2024

Confluent has acquired WarpStream, an innovative Kafka-compatible streaming solution. Read the full statement by Jay Kreps, co-founder and CEO of Confluent.

Cloud 142

Cloud Kafka 142

databricks

SEPTEMBER 10, 2024

We are excited to share the latest new features and performance improvements that make Databricks SQL simpler, faster and lower cost than ever.

SQL 128

SQL 128

Advertisement

Apache Airflow® is the open-source standard to manage workflows as code. It is a versatile tool used in companies across the world from agile startups to tech giants to flagship enterprises across all industries. Due to its widespread adoption, Airflow knowledge is paramount to success in the field of data engineering.

Datasets

KDnuggets

SEPTEMBER 12, 2024

The GitHub repository includes up-to-date learning resources, research papers, guides, popular tools, tutorials, projects, and datasets.

Datasets 147

Datasets Project 147

Netflix Tech

SEPTEMBER 10, 2024

By Jose Fernandez , Sebastien Dabdoub , Jason Koch , Artem Tkachuk The Compute and Performance Engineering teams at Netflix regularly investigate performance issues in our multi-tenant environment. The first step is determining whether the problem originates from the application or the underlying infrastructure. One issue that often complicates this process is the "noisy neighbor" problem.

Utilities 103

Utilities Coding Programming Java 103

databricks

SEPTEMBER 11, 2024

We are excited to announce that Databricks was named one of the 2024 Fortune Best Workplaces in Technology™. This award reflects our.

Technology 124

Technology 124

KDnuggets

SEPTEMBER 11, 2024

Kickstart your data analyst career with all these free courses.

Data Analytics 147

Data Analytics Data Data Science 147

Confessions of a Data Guy

SEPTEMBER 13, 2024

Did you know there are only 3 types of Data Engineers? It’s true. I hope you are the right one. The post The 3 Types of Data Engineers. appeared first on Confessions of a Data Guy.

Data Engineering 100

Data Engineering Data Engineer Engineering Data 100

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Striim

SEPTEMBER 11, 2024

Data pipelines are the backbone of your business’s data architecture. Implementing a robust and scalable pipeline ensures you can effectively manage, analyze, and organize your growing data. Most importantly, these pipelines enable your team to transform data into actionable insights, demonstrating tangible business value. According to an IBM study, businesses expect that fast data will enable them to “make better informed decisions using insights from analytics (44%), improved data quality and

Data Pipeline 98

Data Pipeline Designing Data Lake Data Warehouse 98

databricks

SEPTEMBER 11, 2024

Personal Access Tokens (PATs) are a convenient way to access services like Azure Databricks or Azure DevOps without logging in with your password.

Accessible 114

Accessible Accessibility Data Science Engineering 114

KDnuggets

SEPTEMBER 9, 2024

Exploring the not-so-famous data science libraries that can be useful in your data workflow.

Data Science 145

Data Science Python Data Workflow Data 145

Snowflake

SEPTEMBER 12, 2024

The Snowflake World Tour is making 23 stops around the globe, so you can learn about the latest innovations in the AI Data Cloud in a city near you. This tour will cover Snowflake’s latest advancements that can help you accelerate AI and application development in your organization while advancing the data foundation that makes it all possible. This includes new capabilities related to Snowflake Cortex, streaming, Iceberg open table formats and more.

Data Lake 94

Data Lake Government Cloud Machine Learning 94

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

Netflix Tech

SEPTEMBER 10, 2024

By Karthik Yagna , Baskar Odayarkoil , and Alex Ellis Pushy is Netflix’s WebSocket server that maintains persistent WebSocket connections with devices running the Netflix application. This allows data to be sent to the device from backend services on demand, without the need for continually polling requests from the device. Over the last few years, Pushy has seen tremendous growth, evolving from its role as a best-effort message delivery service to be an integral part of the Netflix ecosystem.

Metadata 92

Metadata Coding Transportation Systems 92

databricks

SEPTEMBER 9, 2024

Imagine giving your business an intelligent bot to talk to customers. Chatbots are commonly used to talk to customers and provide them with.

108

108

KDnuggets

SEPTEMBER 10, 2024

Learn about machine learning APIs for datasets, models, web applications, free GPUs, and text, audio, and image generation.

Machine Learning 134

Machine Learning Datasets 134

Snowflake

SEPTEMBER 13, 2024

Snowflake has always been committed to helping customers protect their accounts and data. To further our commitment to protect against cybersecurity threats and to champion the advancement of industry standards for security, Snowflake recently signed the Cybersecurity and Infrastructure Security Agency (CISA) Secure By Design Pledge. In line with CISA’s Secure By Design principles, we recently announced a number of security enhancements in the platform — most notably the general availability of

Consulting 87

Consulting Designing IT Data 87

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

Monte Carlo

SEPTEMBER 12, 2024

You know what they say about rules: they’re meant to be broken. Or, when it comes to data quality, it’s more like they’re bound to be broken. Data breaks, that much is certain. The challenge is knowing when, where, and why it happens. For most data analysts, combating that means writing data rules – lots of data rules – to ensure your data products are accurate and reliable.

Data 80

Data Machine Learning Data Validation Algorithm 80

databricks

SEPTEMBER 9, 2024

Personalization and scale have historically been mutually exclusive. For all the talk of one-to-one marketing and hyper-personalization , the reality has been that.

Building 105

Building Retail 105

KDnuggets

SEPTEMBER 13, 2024

Learn how to use the OpenAI o1-preview & o1-mini for decision-making, coding, and building an end-to-end machine learning project from scratch.

Machine Learning 125

Machine Learning Coding Project Building 125

Towards Data Science

SEPTEMBER 11, 2024

One answer and many best practices for how larger organizations can operationalizing data quality programs for modern data platforms An answer to “who does what” for enterprise data quality. Image courtesy of the author. I’ve spoken with dozens of enterprise data professionals at the world’s largest corporations, and one of the most common data quality questions is, “who does what?

Government 75

Government Data Engineering Data Engineer Data 75

Advertisement

With over 30 million monthly downloads, Apache Airflow is the tool of choice for programmatically authoring, scheduling, and monitoring data pipelines. Airflow enables you to define workflows as Python code, allowing for dynamic and scalable pipelines suitable to any use case from ETL/ELT to running ML/AI operations in production. This introductory tutorial provides a crash course for writing and deploying your first Airflow pipeline.

Cloud

Confluent

SEPTEMBER 10, 2024

Engineers can put generative AI to work to improve the quality of their data, allowing them to build more accurate and trustworthy AI-powered applications.

Data 72

Data Engineering Building 72

databricks

SEPTEMBER 10, 2024

We are excited to announce the addition of three new integrations in Databricks Partner Connect—a centralized hub that allows you to integrate partner.

98

98

KDnuggets

SEPTEMBER 12, 2024

In order for Lakehouse to become a unified data layer for both analytics and AI, it needs to be extended with new capabilities

IT 122

IT Data 122

Tweag

SEPTEMBER 11, 2024

We’ve all been there: wasting a couple of days on a silly bug. Good news for you: formal methods have never been easier to leverage. In this post, I will discuss the contributions I made during my internship to Liquid Haskell (LH), a tool that makes proving that your Haskell code is correct a piece of cake. LH lets you write contracts for your functions inside your Haskell code.

Coding 70

Coding IT Programming Accessible 70

Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

Confluent

SEPTEMBER 11, 2024

To make application testing for topics with schemas easier, you can now produce messages that are serialized with schemas using the Confluent Cloud Console UI.

Cloud 69

Cloud 69

databricks

SEPTEMBER 9, 2024

We are excited to introduce several powerful new capabilities to Mosaic AI Gateway, designed to help our customers accelerate their AI initiatives with.

Government 84

Government Designing Machine Learning 84

KDnuggets

SEPTEMBER 11, 2024

Access a pre-built Python environment with free GPUs, persistent storage, and large RAM. These Cloud IDEs include AI code assistants and numerous plugins for a fast and efficient development experience.

Cloud 122

Cloud Data Science Python Coding 122