LinkedIn Engineering

NOVEMBER 20, 2023

Co-Authors: Shweta Patira , Ankan Saha , Yilin Li, and Manas Somaiya Earlier this year, we launched Collaborative Articles with the vision of making LinkedIn the one-stop destination for all work-related questions. Among our 1 billion members, there are seasoned experts who have encountered every conceivable workplace problem. If we could present their thoughts on LinkedIn, then mentors, experts, and coaches would be right at our members' fingertips.

Engineering 82

Engineering Systems Building Coding 82

The Pragmatic Engineer

NOVEMBER 21, 2023

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. In this article, we cover thee out of nine topics from today’s subscriber-only issue: The Past and Future of Modern Backend Practices.

Engineering 302

Engineering Bytes Cloud Computing AWS 302

KDnuggets

NOVEMBER 24, 2023

Data scientists love it! See how ChatGPT creates jaw-dropping data viz with just a few words - it's almost unfair how easy it is.

Data 158

Data IT Data Science 158

Data Engineering Podcast

NOVEMBER 19, 2023

Summary The dbt project has become overwhelmingly popular across analytics and data engineering teams. While it is easy to adopt, there are many potential pitfalls. Dustin Dorsey and Cameron Cyr co-authored a practical guide to building your dbt project. In this episode they share their hard-won wisdom about how to build and scale your dbt projects.

Project 147

Project Data Lake High Quality Data SQL 147

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

databricks

NOVEMBER 21, 2023

We recently announced our AI-generated documentation feature, which uses large language models (LLMs) to automatically generate documentation for tables and columns in Unity.

Data Science 145

Data Science Engineering Data 145

ArcGIS

NOVEMBER 21, 2023

Contextual keyboard shortcuts open up a whole new world of customization possibilities in ArcGIS Pro 3.2.

134

134

Waitingforcode

NOVEMBER 21, 2023

Even though you may be thinking now about Data+AI Summit 2024, I still owe you my retrospective for the 2023 edition. Let's start with the first part covering stream processing talks!

Data 130

Data Process 130

Tweag

NOVEMBER 22, 2023

This article aims to introduce and explore the practice of splitting debug symbols away from C/C++ build artifacts to save space and time when building large codebases. Note that we want to retain access to the debug symbols if and when they are needed at a later date, hence we don’t want to merely remove (aka strip ) the debug symbols. 1 This exploration is largely inspired and based on what I have learned in various places around the web, most notably: Improving C++ Builds with Split DWARF, by

Bytes 122

Bytes Coding Building Project 122

databricks

NOVEMBER 19, 2023

A key element in orchestrating multi-stage data and AI processes and pipelines is control flow management. This is why we continue to invest.

Process 122

Process Management Data 122

KDnuggets

NOVEMBER 22, 2023

Large Language Models (LLMs) have now become an integral part of various applications. This article provides an extensive list of resources for anyone interested to dive into the world of LLMs.

155

155

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Precisely

NOVEMBER 20, 2023

AI transforms how we interact with technology, make decisions, and solve complex problems. It has been at the heart of many innovations over the past two years, powering everything from the chatbots that enhance our customer experiences to the predictive analytics engines that help us make financial decisions. What defines a successful AI initiative, and how can your organization ensure that your investments and hard work deliver maximum value for your organization?

Raw Data 121

Raw Data Insurance Data Portfolio 121

ArcGIS

NOVEMBER 20, 2023

The November 2023 release of ArcGIS Image Analyst for ArcGIS Pro brings exciting enhancements to deep learning, image science and SAR.

Deep Learning 115

Deep Learning 115

Engineering at Meta

NOVEMBER 21, 2023

Python plays a big part at Meta. It powers Instagram’s backend and plays an important role in our configuration systems, as well as much of our AI work. Meta even made contributions to Python 3.12 , the latest version of Python. On this episode of the Meta Tech Podcast , Meta engineer Pascal Hartig ( @passy ) is joined by Amethyst Reese, a production engineer at Meta, to discuss all things Python at Meta.

Python 112

Python Engineering Systems IT 112

KDnuggets

NOVEMBER 22, 2023

This article is about the four key soft skills every data scientist needs, and how to work on them.

Data 152

Data 152

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

databricks

NOVEMBER 22, 2023

The costs of fraud are staggering. In 2022, just one type of fraud, card-not-present fraud, resulted in almost $6bn in losses in the.

111

111

Pinterest Engineering

NOVEMBER 22, 2023

Improving Efficiency Of Goku Time Series Database at Pinterest (Part — 1) Monil Mukesh Sanghavi, Kapil Bajaj, Ming-May Hu, Xiao Li and Zhenxiao Luo Introduction At Pinterest, one of the pillars of the observability stack provides internal engineering teams (our users) the opportunity to monitor their services using metrics data and set up alerting on it.

Database 109

Database Bytes Kafka Architecture 109

Scott Logic

NOVEMBER 23, 2023

If you want to try your hand at fine-tuning an LLM (Large Language Model): one of the first things you’re going to need to know is “will it fit on my GPU”. The pre-eminent guide to estimating (VRAM) memory requirements is Transformer Math 101. It bears mentioning, though, that its heuristics are written in the context of frameworks such as GPT-NeoX and Megatron-DeepSpeed.

Bytes 102

Bytes Education IT Utilities 102

KDnuggets

NOVEMBER 21, 2023

Are you a programmer looking to get better at Python? Learn some of Python’s features that’ll help you write more elegant and Pythonic code.

Python 151

Python Coding 151

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

databricks

NOVEMBER 21, 2023

Upskilling through role-based pathways to accelerate your data + AI career Databricks has spent years crafting and iterating technical trainings for learners across.

Data 104

Data 104

Snowflake

NOVEMBER 20, 2023

The enterprise app market has been growing faster than ever before, due to the recent spike in demand for AI / ML workloads. These new types of apps operate over large sets of data, have increasingly higher compute demands, require strict data privacy protections, provide very sophisticated web experiences, and need to be secure at all stages of their life cycles.

Utilities 100

Utilities Machine Learning Coding AWS 100

Knowledge Hut

NOVEMBER 23, 2023

Product Owners and Scrum Masters are two integral roles in Scrum, an Agile project management methodology. While both roles are essential for successful Scrum teams, their responsibilities are distinct. When talking about Product Owner vs Scrum Master responsibilities, it is necessary to note that the Product Owner looks after the vision and content of the product.

Certification 98

Certification Project Process Management 98

KDnuggets

NOVEMBER 20, 2023

Generative AI is an exciting and fast-moving area of research and application. Check out these 5 courses to get up to speed and stay ahead of the curve.

151

151

Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

Netflix Tech

NOVEMBER 20, 2023

by Jun He , Yingyi Zhang , and Pawan Dixit Incremental processing is an approach to process new or changed data in workflows. The key advantage is that it only incrementally processes data that are newly added or updated to a dataset, instead of re-processing the complete dataset. This not only reduces the cost of compute resources but also reduces the execution time in a significant manner.

Process 88

Process Data Pipeline Datasets SQL 88

Zalando Engineering

NOVEMBER 19, 2023

What this article is about What kind of changes we had to make to the codebase How we did the actual upgrade What challenges we faced How we did the data transfer How the data was kept in sync What this article is not A step-by-step guide on how to upgrade Elasticsearch (read on to find out why). Who we are We are a team from the Search & Browse department, the department in Zalando that is responsible for all things search (read: relevance, personalisation, sorting, filters, full text searc

Scala 82

Scala Java Coding IT 82

Knowledge Hut

NOVEMBER 20, 2023

The world is gradually having a paradigm shift from desktop computers to mobile devices- it doesn’t mean that the desktop experience is not good. According to a recent mobile survey , many mobile users have confirmed that it is more appealing, unique, user-friendly and functional. Here are some guides that will help you in learning how to optimise website for mobile users: Choose your site layout You need to ensure your website layout is well streamlined.

Media 98

Media Designing IT 98

KDnuggets

NOVEMBER 24, 2023

In this post we explored the potential challenges and pitfalls of remote work in data science.

Data Science 148

Data Science Data 148

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

Confluent

NOVEMBER 20, 2023

MQTT and Confluent work together to stream and process IoT device data in real time, powering fleet management systems. Learn how here.

Management 80

Management Systems Process Data 80

ArcGIS

NOVEMBER 21, 2023

ArcGIS Maritime 3.2 introduces new tools to streamline chart production and create depth contours, and support for PostgreSQL.

PostgreSQL 78

PostgreSQL 78

Knowledge Hut

NOVEMBER 19, 2023

Over the past few years, there has been a paradigm shift in the world of computing, with cloud computing being on the forefront. It is a computing model based on the internet and it provides on-demand data and shared computer processing to computers and devices. With cloud computing, a pool of computing resources can be shared and accessed. This allows the transfer of information in an effortless manner.

Certification 98

Certification Cloud Computing Amazon Web Services AWS 98

KDnuggets

NOVEMBER 21, 2023

Master algorithms, including deep learning like LSTMs, GRUs, RNNs, and Generative AI & LLMs such as ChatGPT, with Packt's 50 Algorithms Every Programmer Should Know.

Algorithm 148

Algorithm Computer Science Machine Learning Deep Learning 148

Advertisement

Cloud Development Environments (CDEs) are changing how software teams work by moving development to the cloud. Our Cloud Development Environment Adoption Report gathers insights from 223 developers and business leaders, uncovering key trends in CDE adoption. With 66% of large organizations already using CDEs, these platforms are quickly becoming essential to modern development practices.

Cloud