Seattle Data Guy

MAY 20, 2023

Starting new data engineering projects can be challenging. Data engineers can get stuck on finding the right data for their data engineering project or picking the right tools. And many of my Youtube followers agree as they confirmed in a recent poll that starting a new data engineering project was difficult. Here were the key… Read more The post 7 Data Engineering Projects To Put On Your Resume appeared first on Seattle Data Guy.

Data Engineer 147

Data Engineer Data Engineering Project Engineering 147

The Pragmatic Engineer

MAY 25, 2023

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and high-growth startups through the lens of engineering managers and senior engineers. In this issue, we cover one out of six topics from today’s subscriber-only The Scoop issue. To get full articles twice a week, subscribe here.

Software Engineer 316

Software Engineer Software Engineering AWS Engineering 316

Analytics Vidhya

MAY 22, 2023

Introduction Join us in this interview as Sumeet shares his background, journey as a former Data Scientist to a software engineer, and learn the captivating aspects of his current job. He provides insights into the future of data science and software engineering and offers valuable advice for career transitioners. Let’s dive into our conversation with […] The post Conversation with Sumeet, Software Engineer at Natwest Group appeared first on Analytics Vidhya.

Software Engineer 46

Software Engineer Software Engineering Engineering Data Science 46

Data Engineering Podcast

MAY 21, 2023

Summary Batch vs. streaming is a long running debate in the world of data integration and transformation. Proponents of the streaming paradigm argue that stream processing engines can easily handle batched workloads, but the reverse isn't true. The batch world has been the default for years because of the complexities of running a reliable streaming system at scale.

Data Lake 162

Data Lake Machine Learning Kafka Data Warehouse 162

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

KDnuggets

MAY 26, 2023

ChatGPT might have some loophole to provide unreliable facts.

160

160

Snowflake

MAY 24, 2023

Comments

130

130

Simon Späti

MAY 26, 2023

Welcome to the third and final installment of our series “Data Modeling: The Unsung Hero of Data Engineering.” If you’ve journeyed with us from Part 1, where we dove into the importance and history of data modeling, or joined us in Part 2 to explore various approaches and techniques, I’m delighted you’ve stuck around. In this third part, we’ll delve into data architecture patterns and their influence on data modeling.

Architecture 130

Architecture Data Engineer Data Engineering Engineering 130

KDnuggets

MAY 24, 2023

When it's all said and done, and AI has been universally recognized as our rightful overlords, the idea of data science as a standalone field will have been but a blip on our collective radar.

Data Science 156

Data Science Data IT 156

Waitingforcode

MAY 24, 2023

Apache Spark is infamous for its correctness issue for chained stateful operations. Fortunately things get improved in each release. The most recent one, the 3.4.0, also got some important changes on that field!

IT 130

IT 130

ArcGIS

MAY 22, 2023

Esri's updated statement regarding Log4j vulnerabilities (Log4Shell) and ArcGIS products

116

116

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Simon Späti

MAY 26, 2023

Welcome to the third and final installment of our series “Data Modeling: The Unsung Hero of Data Engineering.” If you’ve journeyed with us from Part 1, where we dove into the importance and history of data modeling, or joined us in Part 2 to explore various approaches and techniques, I’m delighted you’ve stuck around. In this third part, we’ll delve into data architecture patterns and their influence on data modeling.

Architecture 130

Architecture Data Engineer Data Engineering Engineering 130

KDnuggets

MAY 22, 2023

What Generative AI is currently capable of and the current challenges it needs to overcome to explore the next wave of generative AI models?

IT 149

IT 149

Tweag

MAY 24, 2023

Tweagers have an engineering mantra — Functional. Typed. Immutable. — that begets composable software which can be reasoned about and avails itself to static analysis. These are all “good things” for building robust software, which inevitably lead us to using languages such as Haskell, OCaml and Rust. However, it would be remiss of us to snub languages that don’t enforce the same disciplines, but are nonetheless popular choices in industry.

Python 110

Python Programming Language Programming Coding 110

databricks

MAY 24, 2023

Special thanks to EY's Mario Schlener, Wissem Bouraoui and Tarek Elguebaly for their support throughout this journey and their contributions to this blog.

Management 105

Management 105

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

ArcGIS

MAY 23, 2023

Discover the power of ArcGIS Bathymetry in our upcoming webinar on June 20th. Learn how this advanced tool can empower your organization.

105

105

KDnuggets

MAY 25, 2023

The blog focuses on GPT models, providing an in-depth understanding and analysis. It explains the three main components of GPT models: generative, pre-trained, and transformers.

IT 139

IT Process 139

Monte Carlo

MAY 26, 2023

What is data freshness and why is it important? Data freshness, sometimes referred to as data timeliness, is the frequency in which data is updated for consumption. It is an important dimension of data quality and a pillar of data observability because recently refreshed data is more accurate, and thus more valuable. Since it is impractical and expensive to have all data refreshed on a near real-time basis, data engineers ingest and process most analytical data in batches with pipelines designed

Data Pipeline 98

Data Pipeline Data Data Warehouse Machine Learning 98

databricks

MAY 23, 2023

Want to build your own LLM-enabled bot? Download our end-to-end solution accelerator here. Business leaders are universally excited for the potential of large.

Building 105

Building Machine Learning 105

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

ArcGIS

MAY 24, 2023

Learn more about a suite of sample data management tools to enable, add, remove or disable media hyperlinks to feature classes in geodatabases.

Management 98

Management Media Data Management Data 98

KDnuggets

MAY 23, 2023

With all the buzz surrounding the ChatGPT. Are you eager to make the most out of it? Here is the FREE video course that offers a comprehensive education about OpenAI API through detailed explanations and hands-on projects.

Project 139

Project Coding Education IT 139

Precisely

MAY 23, 2023

Author’s note: this article about marketing trends has been adapted from an article originally published in The CMO. What are your goals in 2023, and which marketing trends can help you achieve them? In my role as Chief Marketing Officer (CMO) here at Precisely, an important part of what I do is to keep a finger on the pulse of the latest marketing innovations and strategize with my team around how we may be able to capitalize on industry trends to produce even bigger and better results.

Data Integration 98

Data Integration Technology Process IT 98

databricks

MAY 26, 2023

The Asian Employee Network (AEN) launched two years ago, during Lunar New Year 2021. AEN was created with the objective of building a.

Building 98

Building 98

Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

ArcGIS

MAY 24, 2023

Learn how to port schematic diagrams created with ArcGIS Schematics to network diagrams from utility or trace networks using ArcGIS Pro

Utilities 98

Utilities Telecommunication Data Management Management 98

KDnuggets

MAY 26, 2023

An LLM Trained on 1T Tokens of Text and Code by MosaicML Foundation Series.

Coding 116

Coding Process 116

Pinterest Engineering

MAY 25, 2023

Representation online matters: practical end-to-end diversification in search and recommender systems Bhawna Juneja | Senior Machine Learning Engineer; Pedro Silva | Senior Machine Learning Engineer; Shloka Desai | Machine Learning Engineer II; Ashudeep Singh | Machine Learning Engineer II; Nadia Fawaz | (former) Inclusive AI Tech Lead Introduction Pinterest is a platform designed to bring everyone the inspiration to create a life they love.

Utilities 97

Utilities Food Machine Learning Algorithm 97

databricks

MAY 25, 2023

A key piece of your data and AI transformation strategy will involve the decision around which components of the data ecosystem are built.

Data Analytics 98

Data Analytics Building Data 98

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

ArcGIS

MAY 24, 2023

Learn how tasks in ArcGIS Pro help standardize and share workflows across your organization.

97

97

KDnuggets

MAY 25, 2023

Simple tips on doing less and achieving more from VSCode.

Python 116

Python 116

Confluent

MAY 23, 2023

Learn how Confluent Cloud helped Michelin streamline Apache Kafka® operations, reduce costs, and go to market 8-9 months faster.

Kafka 96

Kafka Cloud 96

databricks

MAY 23, 2023

We are excited to announce Azure Databricks support for Azure confidential computing (ACC) in preview! With this announcement, customers can run their Azure.

98

98

Advertisement

Cloud Development Environments (CDEs) are changing how software teams work by moving development to the cloud. Our Cloud Development Environment Adoption Report gathers insights from 223 developers and business leaders, uncovering key trends in CDE adoption. With 66% of large organizations already using CDEs, these platforms are quickly becoming essential to modern development practices.

Cloud