Analytics Vidhya
SEPTEMBER 20, 2023
Data engineering plays a pivotal role in the vast data ecosystem by collecting, transforming, and delivering data essential for analytics, reporting, and machine learning. Aspiring data engineers often seek real-world projects to gain hands-on experience and showcase their expertise. This article presents the top 20 data engineering project ideas with their source code.
The Pragmatic Engineer
SEPTEMBER 22, 2023
👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. We cover one out of four topics in yesterday’s subscriber-only The Pulse issue. To get full newsletters twice a week, subscribe here. Two weeks ago, a JavaScript runtime and toolkit called Bun was released and took the Node.js world by storm. Bun was mostly built by Jared Sumner , a former Stripe engineer, and recipient of the Thiel Fellowship (a grant of $100,000 for young people to drop out of s
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Marc Lamberti
SEPTEMBER 22, 2023
Wondering how to share data between tasks? What are XCOMs in Apache Airflow? Well, you are at the right place. In this tutorial, you will learn about XComs in Airflow. What they are, how they work, how you can define them, how to get them, and more. If you checked my course “Apache Airflow: The Hands-On Guide”, Aiflow XCom should not sound unfamiliar.
Data Engineering Podcast
SEPTEMBER 17, 2023
Summary A significant amount of time in data engineering is dedicated to building connections and semantic meaning around pieces of information. Linked data technologies provide a means of tightly coupling metadata with raw information. In this episode Brian Platz explains how JSON-LD can be used as a shared representation of linked data for building semantic data products.
Advertisement
Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.
Analytics Vidhya
SEPTEMBER 20, 2023
Data engineering plays a pivotal role in the vast data ecosystem by collecting, transforming, and delivering data essential for analytics, reporting, and machine learning. Aspiring data engineers often seek real-world projects to gain hands-on experience and showcase their expertise. This article presents the top 20 data engineering project ideas with their source code.
KDnuggets
SEPTEMBER 16, 2023
This tutorial offers a comprehensive hands-on walkthrough of machine learning with Scikit-learn. Readers will learn key concepts and techniques including data preprocessing, model training and evaluation, hyperparameter tuning, and compiling ensemble models for enhanced performance.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Waitingforcode
SEPTEMBER 20, 2023
It's time for another part of "What's new on the cloud for data engineers" Let's see what happened in the last 4 months.
Rock the JVM
SEPTEMBER 17, 2023
By Lucas Nouguier Hey everyone, Daniel here. Lucas’ story is shared by lots of beginner Scala developers, which is why I wanted to post it here on the blog. I’ve watched thousands of developers learn Scala from scratch, and, like Lucas, they love it! If you want to learn Scala well and fast, take a look at my Scala Essentials course at Rock the JVM.
KDnuggets
SEPTEMBER 20, 2023
KDnuggets' latest cheat sheet covers 10 curated hands-on projects to boost data science workflows with ChatGPT across ML, NLP, and full stack dev, including links to full project details.
Marc Lamberti
SEPTEMBER 21, 2023
Looking to create your first Airflow DAG? Wondering how to process data in Airflow? What are the steps to code your data pipelines? You’ve come to the right place! At the end of this short tutorial, you will have your first Airflow DAG! You might think starting with Apache Airflow is hard, but it is not. The truth is Airflow has so many features that it can be overwhelming.
Advertisement
Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?
ArcGIS
SEPTEMBER 19, 2023
In collaboration with NOAA, we used the Presence-Only Prediction (Maxent) tool to predict snow crab habitat under changing climate conditions.
databricks
SEPTEMBER 20, 2023
This blog post is in collaboration with Greg Rokita, AVP of Technology at Edmunds. Long envisioned as a key milestone in computing, we've.
KDnuggets
SEPTEMBER 20, 2023
On October 11th, 2023 the Feature Store Summit will bring together leading ML companies, such as Uber, WeChat and more, for in-depth discussions about data and AI.
Snowflake
SEPTEMBER 19, 2023
Why AI has everyone’s attention, what it means for different data roles, and how Alteryx and Snowflake are bringing AI to data use cases There’s a llama on the loose! Well, more specifically, LLaMA (Large Language Model Meta AI), along with other large language models (LLMs) that have suddenly become more open and accessible for everyday applications.
Speaker: Jay Allardyce, Deepak Vittal, and Terrence Sheflin
As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.
ArcGIS
SEPTEMBER 22, 2023
This Climate Week renews focus on nature. Learn more about how ArcGIS supports nature-related assessments to run sustainable organizations.
databricks
SEPTEMBER 21, 2023
Introduction In this blog post, we'll explore a set of advanced SQL functions available within Apache Spark that leverage the HyperLogLog algorithm, enabling.
KDnuggets
SEPTEMBER 20, 2023
This tutorial provides hands-on experience with the key concepts and implementation of K-Means clustering, a popular unsupervised learning algorithm, for customer segmentation and targeted advertising applications.
Workfall
SEPTEMBER 18, 2023
Reading Time: 9 minutes Imagine your data as pieces of a complex puzzle scattered across different platforms and formats. Making sense of this scattered information often feels like solving a gigantic puzzle blindfolded. This is where the power of data integration comes into play. If you’ve ever wished for a simplified way to seamlessly connect these puzzle pieces, then you’re in for a treat.
Speaker: Nikhil Joshi, Founder & President of Snic Solutions
Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.
DoorDash Engineering
SEPTEMBER 19, 2023
As a tech company, it’s our products and platform – and the engineers that build them – that power what DoorDash is able to offer our Consumers, Dashers, and Merchants every day. We thrive on tackling challenging technical problems and creating opportunities for customers, and take a lot of pride in what we do (and, as regular readers know, periodically share details about our work on this blog).
databricks
SEPTEMBER 18, 2023
Costa Rica is known for several things, both culturally and ecologically. Among those are biodiversity, coffee, Pura Vida, and most recently a rapidly.
KDnuggets
SEPTEMBER 20, 2023
Python in Excel: This Will Change Data Science Forever • KDnuggets Survey: Benchmark With Your Peers On Data Science Spend & Trends 2023 H2 • 5 Best AI Tools For Maximizing Productivity • And much more!
LinkedIn Engineering
SEPTEMBER 20, 2023
Since learning frontend and backend skills, Rishika’s passion for engineering has expanded beyond her team at LinkedIn to grow into her own digital community. As she develops as an engineer, giving back has become the most rewarding part of her role. From intern to engineer—life at LinkedIn My career with LinkedIn began with a college internship, where I got to dive into all things engineering.
Advertisement
Cloud Development Environments (CDEs) are changing how software teams work by moving development to the cloud. Our Cloud Development Environment Adoption Report gathers insights from 223 developers and business leaders, uncovering key trends in CDE adoption. With 66% of large organizations already using CDEs, these platforms are quickly becoming essential to modern development practices.
Snowflake
SEPTEMBER 21, 2023
The marketing technology landscape has exploded in the last decade. With over 11,000 available solutions , an increase of 7,258% over the last 12 years, marketing organizations have never had more tool options to choose from. In this post, we’ll take a look at how leading vendors in the 2023 Modern Marketing Data Stack are differentiating their products in a crowded market. 360-degree customer view broken into 120 data silos As of 2019, the average enterprise used 120 marketing applications.
databricks
SEPTEMBER 19, 2023
“Supply chains compete, not companies” — Martin Christopher No two supply chains are identical - the unique combination of products, industries, and geographic locat.
KDnuggets
SEPTEMBER 18, 2023
If you're looking for a hands-on experience with a detailed yet beginner-friendly tutorial on implementing Linear Regression using Scikit-learn, you're in for an engaging journey.
DoorDash Engineering
SEPTEMBER 19, 2023
As an Engineering org, we are tremendously proud of our accomplishments throughout the history of DoorDash, particularly in recent years as we’ve grown and scaled in service of our customers. This success has been heavily influenced by the strength and leadership of our Eng management team; great companies are built by great people who have great managers.
Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage
When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.
Snowflake
SEPTEMBER 20, 2023
ADP provides products, services and experiences that simplify work for more than 1 million clients in 140 countries. Large and small organizations across virtually every industry rely on ADP’s cloud-based human capital management (HCM) solutions to streamline HR, payroll, time, tax and benefits administration. Self-service HCM analytics help ADP’s clients understand workforce trends and benchmark their metrics against aggregated, anonymized data from over 30 million employee records.
databricks
SEPTEMBER 18, 2023
We are thrilled to introduce the support of a new SQL feature in Apache Spark and Databricks: Lateral Column Alias (LCA). This feature.
KDnuggets
SEPTEMBER 18, 2023
You can now run Python code in Excel to analyze data, build machine learning models, and create visualizations.
Cloudera
SEPTEMBER 19, 2023
Perhaps your C-suite is already a bit crowded. The typical hierarchy will include a CEO, COO, CFO, CTO, CMO, CIO, and a few more. Adding another position may not be terribly appealing, but there is one C-suite role every company should consider—chief data and analytics officer (CDO or CDAO). The CDO is the point person for your data strategy: the leader who oversees how data is collected, managed, and put to use to improve the organization; the person who ensures that wherever there are opportun
Speaker: Donna Laquidara-Carr, PhD, LEED AP, Industry Insights Research Director at Dodge Construction Network
In today’s construction market, owners, construction managers, and contractors must navigate increasing challenges, from cost management to project delays. Fortunately, digital tools now offer valuable insights to help mitigate these risks. However, the sheer volume of tools and the complexity of leveraging their data effectively can be daunting. That’s where data-driven construction comes in.
Expert insights. Personalized for you.
268 
Let's personalize your content