Sat.Jun 10, 2023 - Fri.Jun 16, 2023

article thumbnail

The Journey of a Senior Data Scientist and Machine Learning Engineer at Spice Money

Analytics Vidhya

Introduction Meet Tajinder, a seasoned Senior Data Scientist and ML Engineer who has excelled in the rapidly evolving field of data science. Tajinder’s passion for unraveling hidden patterns in complex datasets has driven impactful outcomes, transforming raw data into actionable intelligence. In this article, we explore Tajinder’s inspiring success story.

article thumbnail

Inside Agoda’s Private Cloud - Exclusive

The Pragmatic Engineer

👋 Hi, this is Gergely with the monthly, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover challenges at Big Tech and startups through the lens of engineering managers and senior engineers. If you’re not a subscriber, you missed the issue on Shopify’s leveling split and a few others. Subscribe to get two full issues every week.

Cloud 201
article thumbnail

A Comprehensive Guide to Convolutional Neural Networks

KDnuggets

Artificial Intelligence has been witnessing monumental growth in bridging the gap between the capabilities of humans and machines. Researchers and enthusiasts alike, work on numerous aspects of the field to make amazing things happen. One of many such areas is the domain of Computer Vision.

article thumbnail

Migrating Netflix to GraphQL Safely

Netflix Tech

By Jennifer Shin , Tejas Shikhare , Will Emmanuel In 2022, a major change was made to Netflix’s iOS and Android applications. We migrated Netflix’s mobile apps to GraphQL with zero downtime, which involved a total overhaul from the client to the API layer. Until recently, an internal API framework, Falcor , powered our mobile apps. They are now backed by Federated GraphQL , a distributed approach to APIs where domain teams can independently manage and own specific sections of the API.

Utilities 143
article thumbnail

Apache Airflow® Best Practices for ETL and ELT Pipelines

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

article thumbnail

The Journey of a Senior Data Scientist and Machine Learning Engineer in Fintech Domain

Analytics Vidhya

Introduction Meet Tajinder, a seasoned Senior Data Scientist and ML Engineer who has excelled in the rapidly evolving field of data science. Tajinder’s passion for unraveling hidden patterns in complex datasets has driven impactful outcomes, transforming raw data into actionable intelligence. In this article, we explore Tajinder’s inspiring success story.

article thumbnail

An explosion in software engineers using AI coding tools?

The Pragmatic Engineer

GitHub surveyed 500 developers in the US for a sense of how they use AI coding tools. I examine the results and add context on how the survey was conducted.

More Trending

article thumbnail

Data News — Week 23.24

Christophe Blefari

The newsletter, a metaphor ( credits ) Hello, after the good weather comes the storm. I'm now under the Berlin rain with 20° When I write in these conditions I feel like a tortured author writing a depressing novel while actually today I'll speak about the AI Act, Python, SQL and data platforms. Casual day at the office finally. Some personal news, next Monday and Tuesday I'll be at Berlin Buzzwords, if you're ping me, it would be a pleasure to meet and hang together.

article thumbnail

What's new in Apache Spark 3.4.0 - Spark Connect

Waitingforcode

Spark Connect is probably the most expected feature in Apache Spark 3.4.0. It was announced in the Data+AI Summit 2022 keynotes and has a lot of coverage in social media right now. I'll try to add my small contribution to this by showing some implementation details.

Media 130
article thumbnail

How to become a valuable data engineer

Start Data Engineering

1. Introduction 2. Skills 2.1. Business Impact 2.1.1. Know your business 2.1.2. Money & Time 2.2. Technical skills 3. Build impactful projects 4. Conclusion 5. Further reading 1. Introduction So you are a new data engineer (or looking for a DE job) and want to better yourself as a data engineer. However, when you look at job postings or company tech stack, you are overwhelmed by the sheer amount of tools you have to learn!

article thumbnail

Your Ultimate Guide to Chat GPT and Other Abbreviations

KDnuggets

Everyone seems to have gone crazy about ChatGPT, which has become a cultural phenomenon. If you’re not on the ChatGPT train yet, this article might help you better understand the context and excitement around this innovation.

160
160
article thumbnail

Apache Airflow®: The Ultimate Guide to DAG Writing

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

article thumbnail

Build Better Tests For Your dbt Projects With Datafold And data-diff

Data Engineering Podcast

Summary Data engineering is all about building workflows, pipelines, systems, and interfaces to provide stable and reliable data. Your data can be stable and wrong, but then it isn't reliable. Confidence in your data is achieved through constant validation and testing. Datafold has invested a lot of time into integrating with the workflow of dbt projects to add early verification that the changes you are making are correct.

Project 130
article thumbnail

Unlock the Power of Real-time Data Processing with Databricks and Google Cloud

databricks

We are excited to announce the official launch of the Google Pub/Sub connector for the Databricks Lakehouse Platform. This new connector adds to.

article thumbnail

Migrating Critical Traffic At Scale with No Downtime?—?Part 2

Netflix Tech

Migrating Critical Traffic At Scale with No Downtime — Part 2 Shyam Gala , Javier Fernandez-Ivern , Anup Rokkam Pratap , Devang Shah Picture yourself enthralled by the latest episode of your beloved Netflix series, delighting in an uninterrupted, high-definition streaming experience. Behind these perfect moments of entertainment is a complex mechanism, with numerous gears and cogs working in harmony.

Systems 114
article thumbnail

5 Free Julia Books For Data Science

KDnuggets

Discover the full potential of the Julia programming language for data analysis and modeling with a comprehensive guide that covers everything from its syntax to advanced techniques.

article thumbnail

Optimizing The Modern Developer Experience with Coder

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

article thumbnail

Pivot your Perspective: Embracing the Full Power of Microsoft's Power Platform

FreshBI

Switching from Tableau to PowerBI is just the first step. At our company, we don't just transition you to a new data visualization tool. We connect you to an entire ecosystem of powerful, user-friendly solutions: the Microsoft Power Platform. Here's how we do it and why it's a game changer. The Starting Point: A Power-Packed Punch The Power Platform is Microsoft's suite of business analytics tools, which includes PowerBI , Power Apps , Power Automate , and Power Virtual Agents.

Coding 105
article thumbnail

Lakehouse Orchestration with Databricks Workflows

databricks

Organizations across industries are adopting the lakehouse architecture and using a unified platform for all their data, analytics and AI workloads. When moving.

article thumbnail

Understanding level of detail in Business Analyst’s color-coded maps

ArcGIS

The new color-coded map functionality introduces enhanced map visualization, interactive panels and level of detail control.

Coding 104
article thumbnail

How to MLOps like a Boss: A Guide to Machine Learning without Tears

KDnuggets

If you have ever emailed a.pickle file to engineers for deployment, this is for YOU!

article thumbnail

15 Modern Use Cases for Enterprise Business Intelligence

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

article thumbnail

Open-Sourcing AvroTensorDataset: A Performant TensorFlow Dataset For Processing Avro Data

LinkedIn Engineering

Co-authors: Jonathan Hung , Pei-Lun Liao , Lijuan Zhang , Abin Shahab , Keqiu Hu TensorFlow is one of the most popular frameworks we use to train machine learning (ML) models at LinkedIn. It allows us to develop various ML models across our platform that power relevance and matching in the news feed, advertisements, recruiting solutions, and more. To ensure the best member experience, we want our models to be accurate and up-to-date, which requires training the models as fast as possible.

Datasets 102
article thumbnail

Introducing the Well-Architected Data Lakehouse from Databricks

databricks

To provide customers with a framework for planning and implementing their data lakehouse, we are pleased to announce that we have recently published.

Data 105
article thumbnail

Repurposing Deep Learning Models using Transfer Learning in ArcGIS

ArcGIS

Re-train a deep learning model using transfer learning in ArcGIS. Start with an Esri pre-trained model, then create more training samples.

article thumbnail

How to Optimize SQL Queries for Faster Data Retrieval

KDnuggets

Today, we’ll talk about why SQL query optimization is important and which techniques can be used to optimize it.

SQL 154
article thumbnail

Prepare Now: 2025s Must-Know Trends For Product And Data Leaders

Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

article thumbnail

Deep Multi-task Learning and Real-time Personalization for Closeup Recommendations

Pinterest Engineering

Haomiao Li | Software Engineer, Closeup Ranking & Blending; Travis Ebesu | Software Engineer, Closeup Ranking & Blending; Fan Jiang | Software Engineer, Closeup Candidates; Jay Adams | Software Engineer, Pinner Growth & Signals; Olafur Gudmundsson | Software Engineer, Pinner Discovery; Yan Sun | Engineering Manager, Closeup Ranking & Blending; Huizhong Duan | Engineering Manager, Closeup Relevance Introduction At Pinterest, Closeup recommendations (aka Related Pins) is typically

article thumbnail

What’s New with Databricks Notebooks

databricks

Databricks Notebooks offers developers a managed authoring experience where data and AI teams can efficiently collaborate on projects together. The team here is.

Project 105
article thumbnail

Blend mode helper

ArcGIS

A color card reference layer, for exploring and understanding the visual effects of blend modes in any map.

article thumbnail

OpenAI’s Approach to AI Safety

KDnuggets

What will happen with safety approaches in AI systems after OpenAI’s CEO Sam Altman testified about the concerns around new technology?

article thumbnail

How to Drive Cost Savings, Efficiency Gains, and Sustainability Wins with MES

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

article thumbnail

One Big Cluster Stuck: Platform Health

Cloudera

Platform Health Clearly environmental health and high performance are dependent on the proper implementation, tuning, and use of CDP, hardware, and microservices. Ideally you have Visibility and Transparency into existing high priority problems in your environment. The links below will carry you to regions within the Cloudera Community where you will find best practices to properly implement and tune hardware and services.

article thumbnail

Guest Post: Using Lamini to train your own LLM on your Databricks data

databricks

This is a guest post from our startup partner, Lamini. Play with this LLM pictured above, trained on Lamini documentation. Live now! You.

Data 103
article thumbnail

Unleashing the Power of CDC With Snowflake

Workfall

Reading Time: 9 minutes Picture a vibrant landscape of bustling tech hubs, where businesses thrive and decisions are made in the blink of an eye. In this dynamic realm of data engineering, a monumental challenge takes centre stage: efficiently managing the ever-changing tides of real-time data. Data, the lifeblood of organisations, holds the key to unlocking untapped potential and propelling businesses forward.

article thumbnail

The Effects of ChatGPT in Schools and Why It’s Getting Banned

KDnuggets

Many schools are banning ChatGPT for plagiarism, accuracy and privacy concerns. However, the chatbot could help students and teachers with the right application.

144
144
article thumbnail

The Cloud Development Environment Adoption Report

Cloud Development Environments (CDEs) are changing how software teams work by moving development to the cloud. Our Cloud Development Environment Adoption Report gathers insights from 223 developers and business leaders, uncovering key trends in CDE adoption. With 66% of large organizations already using CDEs, these platforms are quickly becoming essential to modern development practices.