Sat.Jul 29, 2023 - Fri.Aug 04, 2023

article thumbnail

What is a Senior Software Engineer at Wise and Amazon?

The Pragmatic Engineer

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. To get full issues twice a week, subscribe here. The past month, we’ve done deepdives in the newsletter on what a senior software engineer is at Big Tech , and at scaleups.

article thumbnail

Data Warehouses Vs Operational Data Stores Vs Data Lakes – How To Store Your Data For Analytics

Seattle Data Guy

A few months ago, I uploaded a video where I discussed data warehouses, data lakes, and transactional databases. However, the world of data management is evolving rapidly, especially with the resurgence of AI and machine learning. There are numerous other methods that technical teams are utilizing to handle their data effectively. In this presentation, I… Read more The post Data Warehouses Vs Operational Data Stores Vs Data Lakes – How To Store Your Data For Analytics appeared first

Data Lake 130
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

The first state in Apache Spark Structured Streaming arbitrary stateful processing

Waitingforcode

When you wrote your first arbitrary stateful processing pipelines, the state expiration is maybe the first tricky point you had to deal with. Why is that? After all, it's just about setting the timeout, doesn't it? Most of the time, yes, but there is an exception.

Process 130
article thumbnail

Strategies For A Successful Data Platform Migration

Data Engineering Podcast

Summary All software systems are in a constant state of evolution. This makes it impossible to select a truly future-proof technology stack for your data platform, making an eventual migration inevitable. In this episode Gleb Mezhanskiy and Rob Goretsky share their experiences leading various data platform migrations, and the hard-won lessons that they learned so that you don't have to.

article thumbnail

15 Modern Use Cases for Enterprise Business Intelligence

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

article thumbnail

Google Shutting down Firebase Dynamic Links

The Pragmatic Engineer

👋 Hi, this is Gergely with a free issue of the Pragmatic Engineer Newsletter. We cover one out of five topics in today’s subscriber-only The Pulse issue. If you’re not yet a full subscriber, you missed this week’s deepdive: The 2023 tech market, as seen by hiring managers. To get full newsletters twice a week, subscribe here.

Metadata 160
article thumbnail

Introduction to AWS Lambda (deployment)

Confessions of a Data Guy

The post Introduction to AWS Lambda (deployment) appeared first on Confessions of a Data Guy.

AWS 130

More Trending

article thumbnail

Sunrise: Zalando's developer platform based on Backstage

Zalando Engineering

Introduction Since 2021, Zalando invested in building up a developer portal called Sunrise, aimed to become the starting point for Builders at Zalando. The portal is based on Spotify's Backstage platform with additional extensions built internally. Sunrise enables everyone at Zalando to view and discover information about teams, applications, APIs, events, CI/CD pipelines, Infrastructure accounts and costs, and much more.

article thumbnail

Forging a Data Strategy for Success in Uncertain Times

Precisely

The results are in! The 2023 Data Integrity Trends and Insights Report , published in partnership between Precisely and Drexel University’s LeBow College of Business, delivers groundbreaking insights into the importance of trusted data. For the report, more than 450 data and analytics professionals worldwide were surveyed about the state of their data programs.

article thumbnail

Introduction to Delta Lake

Confessions of a Data Guy

The post Introduction to Delta Lake appeared first on Confessions of a Data Guy.

Data 130
article thumbnail

A step-by-step guide to build an Effective Data Quality Strategy from scratch

Towards Data Science

A Step-by-Step Guide to Building an Effective Data Quality Strategy from Scratch How to build an interpretable data quality framework based on user expectations Photo by Rémi Müller on Unsplash As data engineers, we are (or should be) responsible for the quality of the data we provide. This is nothing new, but every time I join a data project I ask myself the same questions: When should I start working on data quality?

article thumbnail

Prepare Now: 2025s Must-Know Trends For Product And Data Leaders

Speaker: Jay Allardyce, Deepak Vittal, and Terrence Sheflin

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

article thumbnail

Modern Overview of the MIT CDOIQ Symposium

The Modern Data Company

Modern Announces Partnership with Data Mesh Pioneers, ThoughtWorks In July, we collaborated with ThoughtWorks at the annual CDOIQ Conference in Cambridge, MA to discuss real-world Data Products implementation and best practices for Data Mesh. The data community, especially CDOs, emphasized the importance of raising awareness and gaining clarity about data products.

article thumbnail

Create the engineering career you love at Pinterest

Pinterest Engineering

An interview with Behnam Rezaei | Pinterest VP, Engineering At Pinterest, we’re on a mission to bring everyone the inspiration to create a life they love. For our employees, this extends further to creating the life and career they love. The Pinterest Engineering Blog team sat down with Behnam Rezaei to get an inside scoop into the Monetization Engineering team, what makes Pinterest different and why now is a great time to join our team.

article thumbnail

Introducing Databricks Assistant, a context-aware AI assistant

databricks

Today, we are excited to announce the public preview of Databricks Assistant, a context-aware AI assistant, available natively in Databricks Notebooks, SQL editor.

SQL 97
article thumbnail

Leveraging The Powers of Functional Code?—?Part 2

Booking.com Engineering

Leveraging The Powers of Functional Code — Part 2 The Fully Functional Haskell Solution Part one can be found here: [link] The Solution: Regarding the Haskell code — don’t worry if you don’t understand everything. I am going to explain the main points of it by drawing a parallel to the Java implementation. If you are curious about FP, I cannot recommend this book enough, and the online version is free: [link] It is a pleasant read with lots of humor (just the illustrations by themselves make me

Coding 91
article thumbnail

How to Drive Cost Savings, Efficiency Gains, and Sustainability Wins with MES

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

article thumbnail

7 Steps to Mastering Data Cleaning and Preprocessing Techniques

KDnuggets

Are you trying to solve your first data science project? This tutorial will help you to guide you step by step to prepare your dataset before applying the machine learning model.

article thumbnail

How DoorDash Migrated from StatsD to Prometheus

DoorDash Engineering

Accurate and reliable observability is essential when supporting a large distributed service, but this is only possible if your tools are equally scalable. Unfortunately, this was a challenge at DoorDash because of peak traffic failures while using our legacy metrics infrastructure based on StatsD. Just when we most needed observability data, the system would leave us in the lurch.

AWS 83
article thumbnail

Announcing Databricks Belgrade Development Center

databricks

We are thrilled to announce the opening of Databricks’ latest development center in Belgrade, Serbia. This addition joins our existing R&D centers in A.

98
article thumbnail

Exploring the ArcGIS Utility Network Trace Framework

ArcGIS

A guided discussion on the capabilities of the tracing framework of the Utility Network and how it can be used to answer questions.

article thumbnail

Improving the Accuracy of Generative AI Systems: A Structured Approach

Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage

When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.

article thumbnail

Keras 3.0: Everything You Need To Know

KDnuggets

Unlock the power of AI collaboration with Keras 3.0! Seamlessly switch between TensorFlow, JAX, and PyTorch, revolutionizing your deep learning projects. Read now and stay ahead in the world of AI.

article thumbnail

Robinhood Reports Second Quarter 2023 Results

Robinhood

Robinhood Markets, Inc. (Nasdaq: HOOD) today reported financial results for the quarter ended June 30, 2023. Read our Q2 earnings press release here. Access more information at investors.robinhood.com. The post Robinhood Reports Second Quarter 2023 Results appeared first on Robinhood Newsroom.

article thumbnail

Smooth Sailing Ahead

databricks

The Databricks Container Infra team builds cloud-agnostic infrastructure and tooling for building, storing and distributing container images. Recently, the team worked on scaling.

Cloud 90
article thumbnail

How to Get the User’s Location Using Mapbox?

Workfall

Reading Time: 9 minutes Obtaining a user’s location is a critical requirement for many modern web applications, such as location-based services, personalized content delivery, and targeted marketing. However, without proper guidance and understanding of HTML and JavaScript geolocation techniques, developers often face challenges in implementing this feature effectively.

Coding 69
article thumbnail

The Ultimate Guide To Data-Driven Construction: Optimize Projects, Reduce Risks, & Boost Innovation

Speaker: Donna Laquidara-Carr, PhD, LEED AP, Industry Insights Research Director at Dodge Construction Network

In today’s construction market, owners, construction managers, and contractors must navigate increasing challenges, from cost management to project delays. Fortunately, digital tools now offer valuable insights to help mitigate these risks. However, the sheer volume of tools and the complexity of leveraging their data effectively can be daunting. That’s where data-driven construction comes in.

article thumbnail

KDnuggets News, August 2: ChatGPT Code Interpreter: Fast Data Science • Can’t Keep Up? Catch up on This Week in AI

KDnuggets

ChatGPT Code Interpreter: Do Data Science in Minutes • This Week in AI • Introduction to Statistical Learning, Python Edition: Free Book • 8 Programming Languages For Data Science to Learn in 2023 • Mastering GPUs: A Beginner's Guide to GPU-Accelerated DataFrames in Python

article thumbnail

Announcing Our LinkedIn-Cornell 2023 Grant Recipients

LinkedIn Engineering

​LinkedIn and Cornell Ann S. Bowers College of Computing and Information Science (Bowers CIS) embarked on a partnership , bringing together our collective research power to make technological advances that will further our goal to connect professionals with opportunities at scale. Through this partnership, we support Ph.D. students and faculty members on their research in areas in Computer Science, AI, Information Science including Diversity and Equity.

article thumbnail

Databricks and Technology Partners: Personalized Medicine with a Tailored Approach

databricks

In the ever-evolving realm of healthcare, two powerful trends have emerged: The rise of personalized medicine and the increasing emphasis on patient involvement.

article thumbnail

#ClouderaLife Employee Spotlight: Shallan Miller

Cloudera

In this Employee Spotlight, we sat down with Shallan Miller and talked about all things event marketing, her path back to this role after the Covid-19 pandemic and her perspective on the importance of cultivating resilience, communication skills, and keeping an open mind. Meet Shallan Miller Shallan is a Senior Marketing Events Manager at Cloudera.

article thumbnail

Business Intelligence 101: How To Make The Best Solution Decision For Your Organization

Speaker: Evelyn Chou

Choosing the right business intelligence (BI) platform can feel like navigating a maze of features, promises, and technical jargon. With so many options available, how can you ensure you’re making the right decision for your organization’s unique needs? 🤔 This webinar brings together expert insights to break down the complexities of BI solution vetting.

article thumbnail

Mastering Regular Expressions with Python

KDnuggets

This article dives deep into the world of regular expressions with Python, providing a comprehensive guide for anyone looking to master this complex yet powerful tool, with detailed explanations and code examples.

Python 81
article thumbnail

Career Stories: Building community and connections

LinkedIn Engineering

As Keerthana explored her passion for iOS engineering, she also found mentorship and new working styles. But the community she’s discovered as she began connecting and nurturing the next generation of female engineers has truly been the most rewarding part of her LinkedIn journey. Launching a career in iOS engineering My career started with an unexpected but exciting role as an iOS product solution engineer at an e-commerce company in Bangalore.

article thumbnail

Protecting Your Compute Resources From Bitcoin Miners With a Data Lakehouse

databricks

As cryptocurrencies, particularly Bitcoin, have grown in popularity, so has the phenomenon of Bitcoin mining. While normal mining operations are critical for blockchain.

Data 84
article thumbnail

How to import contingent values into a feature class

ArcGIS

This workflow shows how to use the Import and Export Contingent Values tools to quickly generate contingent values from existing data.

Data 75
article thumbnail

Driving Responsible Innovation: How to Navigate AI Governance & Data Privacy

Speaker: Aindra Misra, Senior Manager, Product Management (Data, ML, and Cloud Infrastructure) at BILL

Join us for an insightful webinar that explores the critical intersection of data privacy and AI governance. In today’s rapidly evolving tech landscape, building robust governance frameworks is essential to fostering innovation while staying compliant with regulations. Our expert speaker, Aindra Misra, will guide you through best practices for ensuring data protection while leveraging AI capabilities.