The Pragmatic Engineer

JANUARY 26, 2023

Comments

326

326

KDnuggets

JANUARY 26, 2023

Impress your friends and loved ones by perfecting your ChatGPT prompt engineering game with this incredibly useful resource.

Engineering 160

Engineering Process 160

Engineering at Meta

JANUARY 27, 2023

Managing network solutions amidst a growing scale inherently brings challenges around performance, deployment, and operational complexities. At Meta, we’ve found that these challenges broadly fall into three themes: 1.) Data center networking: Over the past decade, on the physical front, we have seen a rise in vendor-specific hardware that comes with heterogeneous feature and architecture sets (e.g., non-blocking architecture).

Engineering 143

Engineering Software Engineer Software Engineering Transportation 143

databricks

JANUARY 26, 2023

This is a collaborative post from Databricks and wisecube.ai. We thank Vishnu Vettrivel, Founder, and Alex Thomas, Principal Data Scientist, for their contributions.

Data Lake 137

Data Lake Building Data 137

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

Christophe Blefari

JANUARY 27, 2023

My view from the train window ( credits ) Dear Data News readers it's a joy every week to write this newsletter, we are slowly approaching the second birthday of this newsletter. In order to celebrate this together I'd love to receive your stories about data —can be short or long, anonymous or not. This is an open box, just write me with what you have on the mind and I'll bundle an edition with it.

Data 130

Data Algorithm Data Pipeline SQL 130

KDnuggets

JANUARY 24, 2023

Effective solutions exist when you don't have enough data for your models. While there is no perfect approach, five proven ways will get your model to production.

Machine Learning 160

Machine Learning Data 160

Data Engineering Podcast

JANUARY 22, 2023

Summary The most interesting and challenging bugs always happen in production, but recreating them is a constant challenge due to differences in the data that you are working with. Building your own scripts to replicate data from production is time consuming and error-prone. Tonic is a platform designed to solve the problem of having reliable, production-like data available for developing and testing your software, analytics, and machine learning projects.

PostgreSQL 130

PostgreSQL MongoDB MySQL Machine Learning 130

Netflix Tech

JANUARY 25, 2023

Scalable Annotation Service — Marken by Varun Sekhri , Meenakshi Jindal Introduction At Netflix, we have hundreds of micro services each with its own data models or entities. For example, we have a service that stores a movie entity’s metadata or a service that stores metadata about images. All of these services at a later point want to annotate their objects or entities.

Algorithm 117

Algorithm Media Metadata Data Ingestion 117

KDnuggets

JANUARY 23, 2023

Get your hands on these gems to learn Python, data analytics, machine learning, and deep learning.

Data Science 160

Data Science Deep Learning Machine Learning Python 160

Engineering at Meta

JANUARY 26, 2023

The technical journey discusses the motivations, challenges, and technical solutions employed for warehouse schematization, especially a change to the wire serialization format employed in Meta’s data platform for data interchange related to Warehouse Analytics Logging. Here, we discuss the engineering, scaling, and nontechnical challenges of modernizing Meta’s exabyte-scale data platform by migrating to the new Tulip format.

Bytes 111

Bytes Data Engineering Coding 111

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Precisely

JANUARY 27, 2023

In today’s hypercompetitive business environment, companies must deliver a standout experience for their target audience. Companies that excel at customer experience (CX) are better at building brand loyalty, increasing total customer lifetime value, and turning occasional customers into brand evangelists. This compelling drive for outstanding CX coincides with an intensive shift toward digitization, personalization, and omnichannel alignment.

High Quality Data 105

High Quality Data Data Governance Government Banking 105

ArcGIS

JANUARY 24, 2023

This blog will answer questions such as “What is SAR?”, “What can SAR be used for?”, and “How is SAR beneficial?”.

Education 100

Education 100

KDnuggets

JANUARY 23, 2023

Here are 6 stages of a novel Data Science Project; From Data Collection to Model in Production, backed by research and examples.

Data Collection 158

Data Collection Data Science Project Data 158

databricks

JANUARY 27, 2023

“This blog is authored by Hai Nguyen, Senior Data Scientist at Gousto” Gousto is the UK's best value recipe box, serving up more rec.

Data 98

Data 98

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

LinkedIn Engineering

JANUARY 25, 2023

Co-Authors: Xianyun Mao , Stan Xu , Rachit Kumar , Vikas R , Xia Hong , and�� Divyakumar Menghani �� As a LinkedIn member, you can subscribe to LinkedIn Premium on a monthly or annual basis. For our customers, we offer the same option for our Talent Solutions and/or Sales Navigator products. For each, LinkedIn offers subscription renewal payments. These subscription renewal payments used to go through a rule-based routing engine to selected payment gateways, which often resulted in a less-than-o

Banking 97

Banking Machine Learning Datasets Engineering 97

ArcGIS

JANUARY 27, 2023

ArcGIS Online users and administrators - learn best practices for managing ArcGIS Online credits and get answers to frequently asked questions.

Management 98

Management 98

KDnuggets

JANUARY 24, 2023

Learn about machine learning libraries for building and deploying machine learning models.

Machine Learning 143

Machine Learning Building 143

databricks

JANUARY 26, 2023

For your data-centered workloads, Databricks offers the best-in-class development experience and gives you the tools you need to adhere to code development best.

Coding 98

Coding Data 98

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

Pinterest Engineering

JANUARY 24, 2023

Armen Tashjian | Security Engineer, Corporate Security Intro Pinterest has enforced the use of managed and compliant devices in our Okta authentication flow, using a passwordless implementation, so that access to our tools always requires a healthy Pinterest device. Following the phishing-based attacks against our peers in the tech industry, Pinterest decided to take a two pronged approach to defend against similar attacks.

Certification 95

Certification Accessibility Accessible Engineering 95

ArcGIS

JANUARY 26, 2023

Another five hacks in an endless stream of one-minute how-to videos.

98

98

KDnuggets

JANUARY 26, 2023

Learn how to easily build, iterate and deploy a state-of-the-art deep learning model to predict customer ratings with a declarative approach to machine learning.

Deep Learning 133

Deep Learning Coding Machine Learning Building 133

databricks

JANUARY 26, 2023

This is part two of a three-part series in Best Practices and Guidance for Cloud Engineers to deploy Databricks on AWS. You can.

AWS 98

AWS Cloud Engineering 98

Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

Cloudera

JANUARY 24, 2023

United Bank Limited (UBL), a Pakistani banking and financial services leader, serves over 11 million customers nationwide and operates 1,338 branches and 1,445 ATMs, along with its branchless banking proposition (combination ATM and online banking). In 2022, UBL was awarded Best Bank for Digital Solutions by Asiamoney and Market Leader of Digital Banking in Pakistan by Euromoney, a testament to its track record as the best in digital banking.

Banking 86

Banking Data Analytics Pipeline-centric IT 86

Snowflake

JANUARY 25, 2023

Data, data, data. It does seem we are not only surrounded by talk about data, but by the actual data itself. We are collecting data from every nook and cranny of the universe (literally!). IoT devices in every industry; geolocation information on our phones, watches, cars, and every other mobile device; every website or app we access—all are collecting data.

Metadata 85

Metadata Data Pipeline Government Data 85

KDnuggets

JANUARY 27, 2023

Best oversampling techniques for the imbalanced data.

Machine Learning 131

Machine Learning Data 131

databricks

JANUARY 24, 2023

We are excited to announce a new AutoML capability to quickly and easily use Feature Store data to improve model outcomes. AutoML users.

Data 98

Data 98

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

ArcGIS

JANUARY 24, 2023

Find out how including web maps in your development workflows using the ArcGIS Maps SDKs for Native Apps can increase your productivity!

84

84

Uber Engineering

JANUARY 26, 2023

We recently containerized Hadoop NameNodes and upgraded hardware, improving NameNode RPC queue time from ~200 to ~20ms – A 10x improvement! With this radical change, Uber’s Hadoop customers are happier and admins rest more at night.

Hadoop 82

Hadoop Data 82

KDnuggets

JANUARY 26, 2023

Take your Data Science journey to the next level by joining these Slack communities in 2023.

Data Science 131

Data Science Data 131

databricks

JANUARY 23, 2023

This is a collaborative post from Databricks and Census. We thank Parker Rogers, Data Community Advocate, at Census for his contributions. In this.

Data 98

Data 98

Advertisement

Cloud Development Environments (CDEs) are changing how software teams work by moving development to the cloud. Our Cloud Development Environment Adoption Report gathers insights from 223 developers and business leaders, uncovering key trends in CDE adoption. With 66% of large organizations already using CDEs, these platforms are quickly becoming essential to modern development practices.

Cloud