Tue.Mar 12, 2024

article thumbnail

The “10x engineer:" 50 years ago and now

The Pragmatic Engineer

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. In this article, we cover one out of five topics from today’s subscriber-only article What Changed in 50 Years of Computing.

article thumbnail

5 Free University Courses to Learn Computer Science

KDnuggets

Want to switch to a tech career? Make it happen with these free computer science courses.

article thumbnail

Building Meta’s GenAI Infrastructure

Engineering at Meta

Marking a major investment in Meta’s AI future, we are announcing two 24k GPU clusters. We are sharing details on the hardware, network, storage, design, performance, and software that help us extract high throughput and reliability for various AI workloads. We use this cluster design for Llama 3 training. We are strongly committed to open compute and open source.

Building 145
article thumbnail

7 GPTs to Help Improve Your Data Science Workflow

KDnuggets

Don’t sleep on these GPTs from the GPT Store.

article thumbnail

Apache Airflow® Best Practices for ETL and ELT Pipelines

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

article thumbnail

Processing time trigger, to be or not to be?

Waitingforcode

That's the question. The lack of the processing time trigger means more a reactive micro-batch triggering but it cannot be considered as the single true best practice. Let's see why.

Process 130
article thumbnail

Keeping track of engineering-wide goals and migrations

Yelp Engineering

What is Engineering Effectiveness Metrics (EE Metrics)? EE Metrics was envisioned as a hub that helps teams manage their technical debt. EE Metrics provides every team with a detailed web page that contains information about technical debt that needs to be addressed. It also serves as a platform to highlight top engineering initiatives at the organization level.

More Trending

article thumbnail

Improving ETAs with Multi-Task Models, Deep Learning, and Probabilistic Forecasts

DoorDash Engineering

The DoorDash ETA team is committed to providing an accurate and reliable estimated time of arrival (ETA) as a cornerstone DoorDash consumer experience. We want to ensure that every customer can trust our ETAs, ensuring a high-quality experience in which their food arrives on time every time. With more than 2 billion orders annually, our dynamic engineering challenge is to improve and maintain accuracy at scale while managing a variety of conditions within diverse delivery and merchant scenarios.

article thumbnail

Real-Time Data Streaming for Smart Warehouses

Confluent

Leverage Confluent Data Streaming Platform to bring real time to your smart warehouse, powering greater IoT automation, efficiency and cost savings.

Data 84
article thumbnail

Lyft’s Reinforcement Learning Platform

Lyft Engineering

Summary At Lyft we have built a platform for developing, training and serving Reinforcement Learning models for typical internet industry applications with a focus on Contextual Bandits. Those models have been critical for decision making problems that other techniques such as supervised learning or optimization models struggled with. In this article we describe how we extended our existing machine learning ecosystem to support Reinforcement Learning models, develop models using Off-Policy Evalu

article thumbnail

Les tendances en matière de location intelligence pour 2024

Precisely

Alors que les organisations s’efforcent de tirer le meilleur parti de leurs données et de leurs analyses avancées, les premières découvrent la valeur unique de l’emplacement physique. Pratiquement tous les points de données disponibles sont associés à un élément de localisation. Les organisations ont généralement une présence physique qui prend la forme de bureaux, de points de vente, d’entrepôts ou de sites de production.

article thumbnail

Apache Airflow®: The Ultimate Guide to DAG Writing

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

article thumbnail

Transforming the Road: Generative AI in Intelligent Driver Care

RandomTrees

Introduction: In the ever-evolving landscape of automotive technology, the integration of artificial intelligence has reached new heights. One of the most promising applications is the use of Generative Artificial Intelligence (Generative AI) in the realm of Intelligent Driver Care. In this blog post, we’ll explore how Generative AI is revolutionizing the way we approach driver assistance, safety, and overall well-being on the road.

article thumbnail

Automating Relevance Tuning for Event Search

Eventbrite Engineering

Authored by: Zelal Gungordu and Delaine Wendling. 1 Background Relevance tuning is the process of making incremental changes to a search algorithm to improve the ranking of search results to better meet the information needs and preferences of users. Ideally, each such change would improve the quality of results, which can be measured using engagement … Continue reading "Automating Relevance Tuning for Event Search" The post Automating Relevance Tuning for Event Search appeared first on En

article thumbnail

B2B Data Enrichment for Beginners

Precisely

Your competitors use data to target your customers, discover unmet needs, and explore new markets. If you’re not using data to your advantage, you risk falling behind the rest of the pack. In a 2023 survey conducted by Gartner , customer service and support leaders cited customer data and analytics as a top priority for achieving their organizational goals.

article thumbnail

9 Ways to Improve Your Dataplex Auto Data Quality Scans

Monte Carlo

Google Cloud’s Dataplex is a data fabric tool that enables organizations to discover, manage, monitor, and govern their data across all of their data systems, including their data lakes, data warehouses, data lakehouses, and data marts. With Dataplex, teams get lineage and visibility into their data management no matter where it’s housed, centralizing the security, governance, search and discovery across potentially distributed systems.

article thumbnail

Optimizing The Modern Developer Experience with Coder

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

article thumbnail

Mainframes vs Midrange Servers: What’s the Difference, Anyway?

Precisely

Mainframe and midrange servers are probably among the least understood systems among today’s IT professionals. in IT, due in part to the fact that many people just lump them into a single category, unclear about the distinctions between them. Unless you work with mainframes or midrange servers specifically, you probably think of them as old-school systems with very different technical specifications, administration, and software.

article thumbnail

Now Featuring: Orchestration Lineage

Monte Carlo

Data pipelines get complex quickly. A pipeline supporting a single critical use case may be ingesting data from dozens of sources and refining it via multiple layers of transformation across hundreds of tables. Just like with mathematical equations, the order of operations matter. If there is a delay or failure in one part of this intricate web of tasks, then a cascade of data issues is around the corner.

BI 52
article thumbnail

What is Long Short Term Memory (LSTM) - Complete Guide

Knowledge Hut

What is Long Short-Term Memory (LSTM)? This question has been asked lately with the release of several new smartphones and devices that include LSTM technology. While LSTM hasn't quite hit the mainstream yet, it's something you should be familiar with. In this post, we'll take a detailed look at what LSTM is and how it works. We'll also explore some of the benefits of using LSTM and discuss possible applications for this innovative technology.

article thumbnail

Key Success Metrics, Benefits, and Results for Data Observability Using DataKitchen Software

DataKitchen

Key Success Metrics, Benefits, and Results for Data Observability Using DataKitchen Software Lowering Serious Production Errors Key Benefit Errors in production can come from many sources – poor data, problems in the production process, being late, or infrastructure problems. Reducing the errors your customers find and those they do not are key success metrics of Data Observability Using DataKitchen DataOps Observability and DataOps TestGen.

article thumbnail

15 Modern Use Cases for Enterprise Business Intelligence

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

article thumbnail

SQL vs SQLite: Key Differences and Similarities

Knowledge Hut

SQL databases are one of the most widely used types of database systems available. SQL is a structured query language that these databases enable users to utilize for data management, retrieval, and storage. A number of SQL databases are available. However SQLite is one of the most widely used. Often seen in online browsers, embedded systems, and mobile applications, SQLite is a small, file-based SQL database.

SQL 52
article thumbnail

Mapbox Snowflake Native App Opens Geospatial Analytics to New Audiences 

Snowflake

Geospatial data can give a business a competitive edge — especially when it’s combined with the company’s own data resources. Considering a new store location? You’ll want to analyze not just where your nearest competitors and potential customers are, but also retail footfall numbers, historical traffic patterns, distance from distribution centers, environmental factors, potential delivery times to customers and more.

article thumbnail

Transition From Software Engineer to Product Manager: Step-by-Step Guide

Knowledge Hut

As more and more companies need software, some interesting changes are happening in the tech world. A recent report from LinkedIn says that 25% more software engineers become product managers in the last three years. But why is this happening? Is it because people want to lead in making products, or do they want more say in how a company works? This blog post aims to assist software engineers considering a transition to product management by providing straightforward explanations.

article thumbnail

Democratizing Finance With the Customer Experience Representative Pathways Program

Robinhood

Robinhood was founded on a simple idea: that our financial markets should be accessible to all. With customers at the heart of our decisions, Robinhood is lowering barriers and providing greater access to financial information and investing. Together, we are building products and services that help create a financial system everyone can participate in. … We sat down for a virtual chat with Robinhoodies Cody Borsellino and Tyler Vatcoskay to discuss their work on the Customer Experience Represent

Finance 72
article thumbnail

The Cloud Development Environment Adoption Report

Cloud Development Environments (CDEs) are changing how software teams work by moving development to the cloud. Our Cloud Development Environment Adoption Report gathers insights from 223 developers and business leaders, uncovering key trends in CDE adoption. With 66% of large organizations already using CDEs, these platforms are quickly becoming essential to modern development practices.

article thumbnail

Azure vs Azure DevOps: Key Differences and Similarities

Knowledge Hut

In today’s rapidly еvolving digital landscapе, cloud computing, and еfficiеnt softwarе dеvеlopmеnt practices arе important for businеssеs to stay compеtitivе. Microsoft, a global lеadеr in the technology industry, provides two kеy solutions in thеsе domains: Azurе and Azurе DеvOps. Although both products are part of the Microsoft family, they have different functions and address various facеts of softwarе dеvеlopmеnt.

article thumbnail

Artificial Intelligence Salary in India [For Beginners & Experienced] in 2024

Knowledge Hut

Artificial intelligence has become tremendously popular in recent years, taking the internet by storm. The size of the Indian artificial intelligence market is projected to reach US$5.47 billion (Source: Statista) by the end of this year, which has substantially increased demand for AI jobs. This makes me believe that a career in AI is one to seek vigorously, as the market is expanding by the day!

article thumbnail

Machine Learning Expert Salary in India [Fresher to Experienced]

Knowledge Hut

In today’s fast-paced technological world, the need for qualified machine learning specialists in India is growing. Machine learning experts are becoming increasingly important as artificial intelligence transforms more and more sectors. In this article, I will walk you through the complex world of machine learning expert salaries in India and examine pay scales, industry standards, and earning potential determining variables.

article thumbnail

AWS Solutions Architect Salary in India [2024 Updated]

Knowledge Hut

I’m an AWS Solutions Architect in India. I started my journey not long ago, exploring the world of AWS on a crisp morning. Little did I know it would reshape my career and open doors to endless possibilities in cloud computing. Now, let’s talk about the question on many minds – the AWS solution architect jobs salary in India. As I became engrossed in designing robust cloud solutions, my curiosity about the earning potential naturally followed.

AWS 52
article thumbnail

Prepare Now: 2025s Must-Know Trends For Product And Data Leaders

Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

article thumbnail

Top Skills for Production Managers in 2024

Knowledge Hut

Production management is an expansive process in my experience. I say this because managing, monitoring, and improving the operational efficiency of different manufacturing processes has demanded the best of my efforts and expertise. This is why I believe this field demands diverse skills, including technical skills like financial understanding, knowledge of industry trends and guidelines, agile management, and interpersonal skills like communication, problem-solving, leadership, etc.

article thumbnail

Product Manager Salary in India in 2024 [Fresher to Experienced]

Knowledge Hut

With technology and innovation driving today's fast-changing corporate environment, the role of a product manager is crucial. Understanding the many aspects that influence a product manager's salary in India is vital for both experienced professionals and aspiring professionals. From what I've learned, grasping how a particular industry works involves examining how job titles influence salaries, exploring the impact of experience, and understanding how business knowledge is related t

article thumbnail

How to Become a Hacker for the Government in 2024?

Knowledge Hut

An interesting and fulfilling job choice can be working as a government hacker. Hackers can explore a variety of opportunities. Capable people are being randomly selected by the government. These experts are undoubtedly necessary for the authority to maintain the security of its networks. Though challenging at times, government hacking presents an exciting blend of technological know-how, security knowledge, and investigative work.

article thumbnail

Benefits of PMP Certification for Professionals and Employers

Knowledge Hut

Project managers are in high demand, and companies are willing to pay top dollar for qualified individuals. In today's fast-paced business world, having a Project Management Professional (PMP) certification can give you a significant advantage over other job candidates. As per the PMI statistics, a certified PMP is entitled to a pay increase of 20%.

article thumbnail

Introducing CDEs to Your Enterprise

Explore how enterprises can enhance developer productivity and onboarding by adopting self-hosted Cloud Development Environments (CDEs). This whitepaper highlights the simplicity and flexibility of cloud-based development over traditional setups, demonstrating how large teams can leverage economies of scale to boost efficiency and developer satisfaction.