The Pragmatic Engineer

MAY 16, 2023

There are a growing number of AI coding tools that are alternatives to Copilot. A list of other popular, promising options.

Coding 326

Coding Computer Science Machine Learning Technology 326

Analytics Vidhya

MAY 17, 2023

How can we sift through many variables to identify the most influential factors for accurate predictions in machine learning? Recursive Feature Elimination offers a compelling solution, and RFE iteratively removes less important features, creating a subset that maximizes predictive accuracy. By leveraging a machine learning algorithm and an importance-ranking metric, RFE evaluates each feature’s impact […] The post Recursive Feature Elimination: Working, Advantages & Examples ap

Machine Learning 174

Machine Learning Algorithm Data Analysis Big Data 174

Data Engineering Podcast

MAY 14, 2023

Summary All of the advancements in our technology is based around the principles of abstraction. These are valuable until they break down, which is an inevitable occurrence. In this episode the host Tobias Macey shares his reflections on recent experiences where the abstractions leaked and some observances on how to deal with that situation in a data platform architecture.

Data Lake 147

Data Lake Machine Learning Data Warehouse AWS 147

Christophe Blefari

MAY 19, 2023

TWO YEARS — HAPPY BIRTHDAY 👋 Here is a special edition for me. Exactly 2 years ago, I sent out my first email newsletter. At the time, only 3 people received it. I already told the story in Robin's podcast , here is a written version. In 2021, I was doing Twitch lives twice a week, every Wednesday I was doing a data news round-up.

Data 130

Data Data Engineer Data Engineering Hadoop 130

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

Waitingforcode

MAY 17, 2023

Finally, the time has come to start the analysis of the new features in Apache Spark. The first of them that grabbed my attention was the Async progress tracking from Structured Streaming.

130

130

Tweag

MAY 16, 2023

Today, I am very excited to announce the 1.0 release of Nickel. A bit more than one year ago, we released the very first public version Nickel (0.1). Throughout various write-ups and public talks ( 1 , 2 , 3 ), we’ve been telling the story of our dissatisfaction with the state of configuration management. The need for a New Deal Configuration is everywhere.

MySQL 134

MySQL Metadata Data Validation Coding 134

Christophe Blefari

MAY 18, 2023

( credits ) Data Council Austin is a yearly conference that features a great panel of speakers giving talks about the future of the data field. As I often do I've overlooked the 70 presentations and here a medley of what I've liked. Data Council 2023 YouTube playlist My personal selection If you had only 3 videos to watch it should be the 3 following: Malloy an experimental language — This is my favourite talk.

Data 130

Data BI Consulting Data Science 130

KDnuggets

MAY 16, 2023

AutoGPT has taken the world by storm and has even surpassed ChatGPT itself. So, get ready to dive into the exciting world of Auto-GPT.

Process 140

Process 140

databricks

MAY 18, 2023

Today, we are thrilled to announce that serverless compute for Databricks SQL is Generally Available on AWS and Azure! Databricks SQL (DB SQL).

SQL 126

SQL AWS 126

Data Engineering Weekly

MAY 16, 2023

In the first part of this series, we talked about design patterns for data creation and the pros & cons of each system from the data contract perspective. In the second part, we will focus on architectural patterns to implement data quality from a data contract perspective. Why is Data Quality Expensive? I posted this LinkedIn post that sparked some exciting conversation.

Engineering 110

Engineering Kafka Data Pipeline Data Warehouse 110

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Towards Data Science

MAY 19, 2023

Data Entropy — More Data, More Problems? How to navigate and embrace complexity in a modern data organisation. Source: [link] “It’s like the more money we come across, the more problems we see” Notorious B.I.G Webster’s dictionary defines Entropy in thermodynamics as a measure of the unavailable energy in a closed thermodynamic system that is also usually considered to be a measure of the system’s disorder.

Pipeline-centric 104

Pipeline-centric Data Software Engineer Software Engineering 104

KDnuggets

MAY 18, 2023

This article discusses the key components that contribute to the successful scaling of data science projects. It covers how to collect data using APIs, how to store data in the cloud, how to clean and process data, how to visualize data, and how to harness the power of data visualization through interactive dashboards.

Cloud Computing 108

Cloud Computing Data Science Cloud Project 108

databricks

MAY 16, 2023

The Databricks Lakehouse Platform provides a unified set of tools for building, deploying, sharing, and maintaining enterprise-grade data solutions at scale. Databricks integrates.

Data Solutions 100

Data Solutions Data Building 100

Netflix Tech

MAY 19, 2023

By Chris Wolfe , Joey Schorr , and Victor Roldán Betancort Introduction The authorization team at Netflix recently sponsored work to add Attribute Based Access Control (ABAC) support to AuthZed’s open source Google Zanzibar inspired authorization system, SpiceDB. Netflix required attribute support in SpiceDB to support core Netflix application identity constructs.

PostgreSQL 101

PostgreSQL Systems Accessible Accessibility 101

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

Hevo

MAY 16, 2023

As the volume of data that businesses collect today increases, the need for tools that can help manage this data also increases. One of the most significant requirements of businesses for managing data is a tool that can seamlessly replicate the high volume of data that has been collected.

Data 97

Data Management Data Warehouse Data Integration 97

KDnuggets

MAY 15, 2023

The road to simpler Data Analysis for data scientists and analysts, powered by OpenAI.

Python 152

Python Data Analysis Data 152

databricks

MAY 15, 2023

Apache Spark Structured Streaming is the leading open source stream processing platform. It is also the core technology that powers streaming on the.

Technology 105

Technology Process IT Engineering 105

ArcGIS

MAY 17, 2023

Learn how to produce a monthly elevation dataset for the Greenland Ice Sheet using Trajectory Dataset

Datasets 123

Datasets Data 123

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

Cloudera

MAY 17, 2023

During Women’s History Month, Cloudera hosted a fantastic fireside chat featuring Irma Laxamana, Chief Legal Officer for Cloudera, and Cloudera’s CHRO, Amy Nelson. The discussion was wide-ranging from reflecting on career lessons learned, to advice on navigating the workplace. Below are the highlights of the chat. About Irma Laxamana Irma is the Chief Legal Officer at Cloudera leading a global team of lawyers and legal professionals supporting all areas of the business.

Building 87

Building Technology IT Systems 87

KDnuggets

MAY 16, 2023

In today's highly competitive job market, practitioners need every advantage they can get to stand out from the crowd and accelerate in their roles as a high-performing employee. With that in mind, here are 5 reasons why you should earn a SAS certification, and stand out to employers.

Certification 106

Certification 106

Pinterest Engineering

MAY 17, 2023

Isabel Tallam | Sw Eng, Real Time Analytics; Charles Wu | Sw Eng, Real Time Analytics; Kapil Bajaj | Eng Manager, Real Time Analytics Detecting anomalous events has been becoming increasingly important in recent years at Pinterest. Anomalous events, broadly defined, are rare occurrences that deviate from normal or expected behavior. Because these types of events can be found almost anywhere, opportunities and applications for anomaly detection are vast.

Algorithm 85

Algorithm Engineering Machine Learning Project 85

ArcGIS

MAY 18, 2023

Part 1 - explains why personal geodatabases are not supported within ArcGIS Pro and begins the quest to migrate data to a mobile geodatabase.

Data 98

Data Data Management Management 98

Speaker: Jay Allardyce, Deepak Vittal, and Terrence Sheflin

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

Snowflake

MAY 18, 2023

Welcome to Snowflake’s Startup Spotlight, where we learn about innovative companies building businesses on Snowflake. In this edition, we’ll hear from Pipedream Co-Founder Dylan Sather about what it takes to build integrations right and how an engaged community becomes a powerful resource. Tell us about yourself. I’m Dylan Sather, co-founder and Software Engineer at Pipedream.

Finance 82

Finance Building SQL Python 82

KDnuggets

MAY 15, 2023

Mojo Lang: The New Programming Language • Stop Doing this on ChatGPT and Get Ahead of the 99% of its Users • 3 Ways to Access GPT-4 for Free • 8 Open-Source Alternative to ChatGPT and Bard • Exploratory Data Analysis Techniques for Unstructured Data

Programming Language 101

Programming Language Programming Unstructured Data Data Analysis 101

databricks

MAY 17, 2023

Today, we are excited to announce the general availability of the Variable Explorer for Python in the Databricks Notebook. The Variable Explorer allows.

Python 89

Python 89

ArcGIS

MAY 18, 2023

This second blog in a series explains how ArcGIS Pro can be used to create an OLE DB connection to a.mdb,accdb, and a MySQL database.

MySQL 98

MySQL Database Data Accessible 98

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Workfall

MAY 16, 2023

Reading Time: 5 minutes When Rust compiles code, you get an executable if you created the application using the --bin command. In this blog, we shall look at how we can create a Dockerfile to create an image with this executable. We shall then deploy this image on EC2 using GitHub Actions which will be set on our repository [link] which also has the source code for our web application.

AWS 81

AWS Cloud Computing Cloud Coding 81

KDnuggets

MAY 15, 2023

Transitioning your career to DataOps could be just the change you need - not only will it provide the possibility to expand your technical skills, but also a rewarding salary with many job openings.

IT 91

IT Data Engineering Data Engineer Engineering 91

databricks

MAY 15, 2023

We recently announced our partnership with Databricks to bring multi-cloud data clean room collaboration capabilities to every Lakehouse. Our integration with Databricks combines.

Cloud 81

Cloud Data Entertainment Media 81

Snowflake

MAY 15, 2023

Multi-touch attribution (MTA) is a data-driven approach to measuring the impact of various marketing channels and touchpoints on a consumer’s journey toward making a purchase or completing a desired action. Unfortunately, marketers struggle with gaining such a view because most solutions make it difficult, if not impossible, to centralize data and deliver data-driven insights in real-time.

Data Collection 78

Data Collection Utilities Cloud AWS 78

Advertisement

Cloud Development Environments (CDEs) are changing how software teams work by moving development to the cloud. Our Cloud Development Environment Adoption Report gathers insights from 223 developers and business leaders, uncovering key trends in CDE adoption. With 66% of large organizations already using CDEs, these platforms are quickly becoming essential to modern development practices.

Cloud