Towards Data Science

AUGUST 22, 2023

Advice from a startup founder in the data space on how to find a startup that works for you Photo by Leeloo Thefirst from Pexels.com So you want to join a startup huh? I’m not talking about a fancy Series E startup that’s about to go IPO funded by a16z. I’m talking about a real startup, from seed to series B — where every day can feel like you’re either about to soar or crash and burn — and there’s little in between.

Data Engineer 87

Data Engineer Data Engineering Engineering Data Pipeline 87

Seattle Data Guy

AUGUST 9, 2023

Learning about how to data models from basic star schemas on the internet is like learning data science using the IRIS data set. It works great as a toy example. But it doesn’t match real life at all. Data modeling in real life requires you fully understand the data sources and your business use cases.… Read more The post Why Is Data Modeling So Challenging – How To Data Model For Analytics appeared first on Seattle Data Guy.

Data Science 246

Data Science Data Consulting IT 246

The Pragmatic Engineer

AUGUST 1, 2023

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. To get full issues twice a week, subscribe here. The past month, we’ve done deepdives in the newsletter on what a senior software engineer is at Big Tech , and at scaleups.

Software Engineer 245

Software Engineer Software Engineering Engineering Designing 245

Analytics Vidhya

AUGUST 30, 2023

Introduction In the bustling arena of database management systems, two heavyweight contenders emerge, each carrying its arsenal of features and capabilities. In one corner, we have the suave and sophisticated Microsoft SQL Server (MSSQL), donned in the elegance of enterprise-level prowess. And in the other corner the scrappy and open-source MySQL, armed with its community-driven […] The post MSSQL vs MySQL: Comparing Powerhouses of Databases appeared first on Analytics Vidhya.

MySQL 228

MySQL Database SQL Systems 228

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

KDnuggets

AUGUST 30, 2023

7 Projects Built with Generative AI • Beyond Numpy and Pandas: Unlocking the Potential of Lesser-Known Python Libraries • 5 Ways You Can Use ChatGPT’s Code Interpreter For Data Science • GPT-4: 8 Models in One; The Secret is Out

Python 147

Python Project Data Science Coding 147

databricks

AUGUST 30, 2023

It’s no secret that modern organizations are doubling down on their investments in data - investments that uncover deep customer insights that provide a.

Data 131

Data 131

Data Engineering Podcast

AUGUST 27, 2023

Summary Data persistence is one of the most challenging aspects of computer systems. In the era of the cloud most developers rely on hosted services to manage their databases, but what if you are a cloud service? In this episode Vignesh Ravichandran explains how his team at Cloudflare provides PostgreSQL as a service to their developers for low latency and high uptime services at global scale.

Database 130

Database Building PostgreSQL BI 130

The Pragmatic Engineer

AUGUST 22, 2023

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. In this article, we cover one out of for topics from the past newsletter issue Game Development Basics. To get the full issues, twice a week, subscribe here.

Software Engineer 207

Software Engineer Software Engineering Consulting Entertainment 207

Analytics Vidhya

AUGUST 22, 2023

Introduction In today’s data-driven world, seamless data integration plays a crucial role in driving business decisions and innovation. Two prominent methodologies have emerged to facilitate this process: Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT). In this article, we will discuss ELT vs ETL, comparing their characteristics, benefits, and suitability for various use cases. […] The post ELT vs ETL: Unveiling the Differences and Similarities appeared first on Ana

Data Integration 185

Data Integration Process Datasets Data 185

KDnuggets

AUGUST 31, 2023

These are my predictions based on my personal experiences, recent research and reports from leading platforms.

Data Management 146

Data Management Management Data Data Engineer 146

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Confessions of a Data Guy

AUGUST 24, 2023

I still remember that day. A day that shall live on in infamy in my mind. Well over a decade ago, in the days when SQL Server roamed the land devouring souls on the Altar of Stored Procedures. There was only one tool available at the time. SQL. That’s it. There was one problem that […] The post The Case of the Mysterious Recursive CTE appeared first on Confessions of a Data Guy.

SQL 130

SQL Data IT Data Engineer 130

Waitingforcode

AUGUST 22, 2023

One of the great features of modern table file formats is the ability to handle write conflicts. It wouldn't be possible without commits that are the topic of this new blog post.

IT 130

IT 130

Data Engineering Podcast

AUGUST 20, 2023

Summary Generative AI has unlocked a massive opportunity for content creation. There is also an unfulfilled need for experts to be able to share their knowledge and build communities. Illumidesk was built to take advantage of this intersection. In this episode Greg Werner explains how they are using generative AI as an assistive tool for creating educational material, as well as building a data driven experience for learners.

Education 130

Education BI SQL Database 130

The Pragmatic Engineer

AUGUST 10, 2023

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. In this article, we cover one out of five topics from today’s subscriber-only The Pulse issue. To get full issues twice a week, subscribe here.

Engineering 196

Engineering Recruitment Retail Software Engineer 196

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

Seattle Data Guy

AUGUST 2, 2023

A few months ago, I uploaded a video where I discussed data warehouses, data lakes, and transactional databases. However, the world of data management is evolving rapidly, especially with the resurgence of AI and machine learning. There are numerous other methods that technical teams are utilizing to handle their data effectively. In this presentation, I… Read more The post Data Warehouses Vs Operational Data Stores Vs Data Lakes – How To Store Your Data For Analytics appeared first

Data Lake 130

Data Lake Data Warehouse Data Machine Learning 130

KDnuggets

AUGUST 31, 2023

And why you should learn how to use them to filter Python sequences more elegantly.

Python 144

Python 144

Confessions of a Data Guy

AUGUST 11, 2023

Do you think I’m just trying to get you to click? Maybe. Maybe not. After working in and around Data Teams for well over a decade, with both the smartest people to touch the keyboard, and the others, it’s become quite clear to me what the number one skill that identifies a Senior level Engineering […] The post Senior Engineer – The Number One Skill appeared first on Confessions of a Data Guy.

Engineering 130

Engineering Data IT Data Engineer 130

Waitingforcode

AUGUST 17, 2023

Lessons learned why it's always worth checking the code implementation to avoid surprises later. Even for vendor-supported solutions.

Coding 130

Coding IT 130

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

Data Engineering Podcast

AUGUST 13, 2023

Summary Data pipelines are the core of every data product, ML model, and business intelligence dashboard. If you're not careful you will end up spending all of your time on maintenance and fire-fighting. The folks at Rivery distilled the seven principles of modern data pipelines that will help you stay out of trouble and be productive with your data.

Data Pipeline 130

Data Pipeline BI SQL Business Intelligence 130

The Pragmatic Engineer

AUGUST 10, 2023

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. In this article, we cover one out of five topics from today’s subscriber-only The Pulse issue. To get full issues twice a week, subscribe here.

Retail 176

Retail Utilities Software Engineer Software Engineering 176

Snowflake

AUGUST 30, 2023

In the past few days, the scope and trajectory of Instacart’s use of Snowflake has been misrepresented by some on social media. Snowflake has partnered closely with Instacart to scale up to meet the company’s massive demand growth, and then to optimize for efficiency. Optimizations are undertaken on a workload-by-workload basis, and have been extremely successful.

Media 118

Media Retail Machine Learning Data Pipeline 118

KDnuggets

AUGUST 30, 2023

This post discusses the importance of media mix modeling and how it can be used to maximize the business impact of advertising. It also discusses the impact of seasonality on media advertising and how media mix modeling can be used to minimize the impact of seasonality on business outcomes.

Media 144

Media IT Data Science Data 144

Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

Confessions of a Data Guy

AUGUST 4, 2023

The post Introduction to Delta Lake appeared first on Confessions of a Data Guy.

Data 130

Data Data Engineer Data Engineering Engineering 130

Waitingforcode

AUGUST 10, 2023

There are probably not that many people working today on the flat files with Structured Streaming than 5 years ago thanks to the table file formats. However, if you are in this group and are still generating CSVs or JSONs with the streaming sink, brace yourself, the memory problems are coming if you don't take action!

130

130

Data Engineering Podcast

AUGUST 6, 2023

Summary As businesses increasingly invest in technology and talent focused on data engineering and analytics, they want to know whether they are benefiting. So how do you calculate the return on investment for data? In this episode Barr Moses and Anna Filippova explore that question and provide useful exercises to start answering that in your company.

Machine Learning 130

Machine Learning SQL Data Engineer Data Engineering 130

The Pragmatic Engineer

AUGUST 3, 2023

👋 Hi, this is Gergely with a free issue of the Pragmatic Engineer Newsletter. We cover one out of five topics in today’s subscriber-only The Pulse issue. If you’re not yet a full subscriber, you missed this week’s deepdive: The 2023 tech market, as seen by hiring managers. To get full newsletters twice a week, subscribe here.

Metadata 163

Metadata Engineering Building Technology 163

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

databricks

AUGUST 30, 2023

With the rapid advancement of neural network-based techniques and Large Language Model (LLM) research, businesses are increasingly interested in AI applications for value.

Engineering 105

Engineering 105

KDnuggets

AUGUST 31, 2023

The Burtch Works 2023 Data Science & AI Professionals salary report is here, and includes insightful data such as hiring and marketplace trends, compensation changes over time, and salary data. Get your copy here.

Data Science 157

Data Science Data 157

Confessions of a Data Guy

AUGUST 4, 2023

The post Introduction to AWS Lambda (deployment) appeared first on Confessions of a Data Guy.

AWS 130

AWS Data Data Engineer Data Engineering 130

Waitingforcode

AUGUST 2, 2023

When you wrote your first arbitrary stateful processing pipelines, the state expiration is maybe the first tricky point you had to deal with. Why is that? After all, it's just about setting the timeout, doesn't it? Most of the time, yes, but there is an exception.

Process 130

Process IT 130

Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage

When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.

Systems