Waitingforcode

SEPTEMBER 6, 2023

In our social media and marketing-driven era, it's quite hard to get things right. For me there is one common misconception brought by the Modern Data Stack idea that everything should be now ELT. In fact no, it shouldn't but only can.

Media 228

Media IT Data 228

Data Engineering Podcast

SEPTEMBER 3, 2023

Summary Cloud data warehouses and the introduction of the ELT paradigm has led to the creation of multiple options for flexible data integration, with a roughly equal distribution of commercial and open source options. The challenge is that most of those options are complex to operate and exist in their own silo. The dlt project was created to eliminate overhead and bring data integration into your full control as a library component of your overall data system.

Data Integration 162

Data Integration BI SQL Python 162

KDnuggets

SEPTEMBER 3, 2023

This tutorial covers Python's foundational data structures - lists, tuples, dictionaries, and sets. Learn their characteristics, use cases, and practical examples, all in 5 steps.

Python 149

Python Data 149

Engineering at Meta

SEPTEMBER 7, 2023

Earlier this year, a small team of engineers at Meta started working on an idea for a new app. It would have all the features people expect from a text-based conversations app, but with one very key, distinctive goal – being an app that would allow people to share their content across multiple platforms. We wanted to build a decentralized (or federated) app that would enable people to post content that is viewable by anyone on other social apps, and vice versa.

Media 143

Media Engineering Coding Python 143

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

Seattle Data Guy

SEPTEMBER 2, 2023

SSIS, short for SQL Server Integration Service, is an essential data migration tool for modern businesses. As a key part of Microsoft’s SQL database software, It allows you to easily complete many complex tasks, including data extraction, merging data, loading and transformation, aggregating data, and more. It’s a comprehensive solution to your data management needs.

IT 130

IT Aggregated Data SQL Database 130

Snowflake

SEPTEMBER 7, 2023

Snowpark is the set of libraries and runtimes that enables data engineers, data scientists and developers to build data engineering pipelines, ML workflows, and data applications in Python, Java, and Scala. Functions or procedures written by users in these languages are executed inside of Snowpark’s secure sandbox environment , which runs on the warehouse.

Amazon Web Services 122

Amazon Web Services AWS Government Python 122

ArcGIS

SEPTEMBER 5, 2023

Access a vast collection of satellite imagery with new EarthCache add-In for ArcGIS Pro.

Accessible 121

Accessible Accessibility 121

Knowledge Hut

SEPTEMBER 6, 2023

As a seasoned software developer with almost a decade of experience in the tech industry, I vividly remember the excitement of taking my first web development course. Back then, I was just starting my journey as a front-end web developer, and that course was a stepping-stone that transformed my career. Today, I am thrilled to share my insights on some of the top software development courses available, hoping to empower aspiring developers like you to find the perfect path to success.

Computer Science 119

Computer Science Software Engineer Software Engineering Certification 119

Confluent

SEPTEMBER 7, 2023

Kafka-on-Windows tutorials are everywhere, but most run Kafka directly on Windows.

Kafka 116

Kafka 116

KDnuggets

SEPTEMBER 8, 2023

Understand the relevance of databases in data science. Also learn the fundamentals of relational databases, NoSQL database categories, and more.

Database 146

Database Data Science NoSQL Relational Database 146

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Engineering at Meta

SEPTEMBER 7, 2023

We’re introducing Arcadia, Meta’s unified system that simulates the compute, memory, and network performance of AI training clusters. Extracting maximum performance from an AI cluster and increasing overall efficiency warrants a multi-input system that accounts for various hardware and software parameters across compute, storage, and network collectively.

Systems 112

Systems Architecture Designing Utilities 112

databricks

SEPTEMBER 7, 2023

Check out our Solution Accelerator for RFM Segmentation for more details and to download the notebooks. For retail brands, effective customer engagement depends.

Retail 111

Retail 111

ThoughtSpot

SEPTEMBER 6, 2023

In a survey by the Harvard Business Review , 87% of respondents stated their organizations would be more successful if frontline workers were empowered to make important decisions in the moment. And 86% of respondents stated that they needed better technology to enable those in-the-moment decisions. Those coveted insights live at the end of a process lovingly known as the data pipeline.

Data Workflow 105

Data Workflow Raw Data Data Lake Business Intelligence 105

KDnuggets

SEPTEMBER 7, 2023

We have a bad case of generative AI fatigue.

146

146

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

Engineering at Meta

SEPTEMBER 7, 2023

Meta presents Chakra execution traces , an open graph-based representation of AI/ML workload execution, laying the foundation for benchmarking and network performance optimization. Chakra execution traces represent key operations, such as compute, memory, and communication, data and control dependencies, timing, and resource constraints. In collaboration with MLCommons , we are seeking industry-wide adoption for benchmarking.

Metadata 109

Metadata Utilities Systems Designing 109

databricks

SEPTEMBER 4, 2023

We are thrilled to announce great enhancements to onboard more workloads to Unity Catalog clusters in shared access mode, Databricks' highly efficient, secure.

Accessible 105

Accessible Accessibility Engineering 105

Pinterest Engineering

SEPTEMBER 5, 2023

Pong Eksombatchai | Principal Engineer; Karthik Anantha Padmanabhan | Manager II, Engineering Image from [link] Pinterest’s mission is to bring everyone the inspiration to create a life they love. We rely on an extensive suite of AI powered products to connect over 460M users to hundreds of billions of Pins, resulting in hundreds of millions of ML inferences per second, hundreds of thousands of ML training jobs per month by just a couple of hundreds of ML engineers.

Engineering 104

Engineering Deep Learning Architecture Project 104

KDnuggets

SEPTEMBER 5, 2023

This step-by-step tutorial is for beginners to guide them through the process of data cleaning and preprocessing using the powerful Pandas library.

Data 146

Data Process Data Science 146

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

Confluent

SEPTEMBER 5, 2023

Want to run Kafka on AWS? Our full tutorial provides expert recommendations on how to deploy, monitor, and manage Kafka clusters on AWS.

Kafka 98

Kafka AWS Designing Management 98

databricks

SEPTEMBER 5, 2023

In May this year, we opened our latest development center in Bengaluru, India. We've been busy building out our R&D teams in India.

Building 105

Building 105

Knowledge Hut

SEPTEMBER 6, 2023

Scrum Alliance training is crucial when it comes to proving competency in project management practices. A good Scrum Alliance certification can imminently help you to excel in your career. It is a versatile Agile Project Management framework suitable for any industry. Scrum Alliance certifications not only help to improve an organization's productivity but are also widely responsible for improving product qualities, risk mitigation, and robust team dynamics.

Certification 97

Certification Education Recruitment Project 97

KDnuggets

SEPTEMBER 2, 2023

Want to learn Python? Get started today by learning Python's syntax, supported data types, and control structures.

Python 147

Python Data 147

Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

Snowflake

SEPTEMBER 6, 2023

In today’s hyper-connected world of retail and consumer goods, understanding the customer journey is more critical than ever. As digital disruption and evolving customer expectations continue to shape the future of these sectors, organizations are striving to achieve ‘Shopper 360,’ a comprehensive and integrated view of their shoppers that is the retail equivalent of ‘Customer 360.

Retail 94

Retail Machine Learning Data Science Data Analytics 94

Cloudera

SEPTEMBER 6, 2023

At Cloudera, we’re known for making innovative technological solutions that drive change and impact the world. Our mission is to make data and analytics easy and accessible to everyone. And that doesn’t end with our customer base. We also aim to provide equitable access to career opportunities within data and analytics to the workforce of tomorrow.

Programming 88

Programming IT Accessible Accessibility 88

Knowledge Hut

SEPTEMBER 6, 2023

Nowadays, I often hear people saying they aspire to become data scientists or they want to work with data, but they don’t know the path to do so. I myself have faced this problem and data science certifications come as a rescue for this problem. As we all know Data Science is the most demanding job in the IT industry, today tons of data is created by just a single click, hence it is extremely important that this data is properly tailored and utilized to make viable business decisions.

Data Science 97

Data Science Certification Business Analyst Machine Learning 97

KDnuggets

SEPTEMBER 5, 2023

From cleaning data to wowing recruiters - this blog shares 5 killer data science projects to launch your data science career and get hired!

Data Science 146

Data Science Portfolio Project Recruitment 146

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

Snowflake

SEPTEMBER 7, 2023

Higher education institutions have a lot of plates to spin, and the University of Birmingham is no exception. Following a tough pandemic, the need to digitally transform had never been more pressing. The university needed to modernize its data capabilities to better serve staff, students and researchers—and it used the Snowflake Data Cloud to do it.

Education 93

Education Professional Services Business Intelligence Cloud 93

Cloudera

SEPTEMBER 7, 2023

With the emergence of new creative AI algorithms like large language models (LLM) fromOpenAI’s ChatGPT, Google’s Bard, Meta’s LLaMa, and Bloomberg’s BloombergGPT—awareness, interest and adoption of AI use cases across industries is at an all time high. But in highly regulated industries where these technologies may be prohibited, the focus is less on off the shelf generative AI, and more on the relationship between their data and how AI can transform their business.

Insurance 88

Insurance Finance Portfolio Government 88

Knowledge Hut

SEPTEMBER 6, 2023

Projects undergo a multitude of challenges when they begin or start. What commences as a simple activity may undergo a series of alterations - due to unknown or unforeseen constraints. To face and overcome such adversities, the project manager needs to rely on ways or techniques of playing a balancing act. For constraints related to the project schedule, the two schedule compression techniques of fast tracking and crashing come in very handy in critical situations.

Project 95

Project Certification Management IT 95

KDnuggets

SEPTEMBER 6, 2023

Happy 30th Anniversary KDnuggets!

Python 145

Python Data 145

Advertisement

Cloud Development Environments (CDEs) are changing how software teams work by moving development to the cloud. Our Cloud Development Environment Adoption Report gathers insights from 223 developers and business leaders, uncovering key trends in CDE adoption. With 66% of large organizations already using CDEs, these platforms are quickly becoming essential to modern development practices.

Cloud