Engineering at Meta

DECEMBER 6, 2023

Our aim is to ensure that everyone’s personal messages on Messenger can only be accessed by the sender and the intended recipients, and that everyone can be sure the messages they receive are from an authentic sender. Third-party scrutiny E2EE implies confidentiality even if the provider wants to access the contents of a communication.

Building 145

Building Designing Consulting Accessibility 145

Data Engineering Podcast

JANUARY 28, 2024

For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics.

Data Lake 147

Data Lake Building High Quality Data AWS 147

Data Engineering Podcast

MAY 5, 2024

In this episode he explains his approach to building AI in a more human-like fashion and the emphasis on learning rather than statistical prediction. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines.

Building 147

Building Data Lake High Quality Data Machine Learning 147

Data Engineering Podcast

APRIL 16, 2023

As the capabilities of these systems has improved and become more accessible, the target of what self-serve means changes. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. Who are the target users?

Business Intelligence 147

Business Intelligence Building Data Lake BI 147

Snowflake

NOVEMBER 15, 2023

BUILD 2023 is where AI gets real. Join our two-day virtual global conference and learn how to build with the app dev innovations you heard about at Snowflake Summit and Snowday. As always, BUILD is created by builders, for builders, so you’ll find plenty to spark your interest and boost your skill set.

Building 116

Building Accessibility Accessible IT 116

databricks

DECEMBER 12, 2024

As enterprises build agent systems to deliver high quality AI apps, we continue to deliver optimizations to deliver best overall cost-efficiency for our.

Accessibility 104

Accessibility Accessible Systems Building 104

Start Data Engineering

JUNE 30, 2023

Building a self-serve data platform 3.1. Accessing data 3.3. Introduction Most companies want to build a self-serve data platform. Components of a self-serve platform 3. Creating dataset(s) 3.1.1. Gather requirements 3.1.2. Get data foundations right 3.2. Identify and remove dependencies 4. Conclusion 5. Further reading 6.

Building 130

Building Datasets Data Accessibility 130

Data Engineering Podcast

JULY 23, 2023

Summary Real-time data processing has steadily been gaining adoption due to advances in the accessibility of the technologies involved. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team.

Building 130

Building Machine Learning SQL Python 130

Data Engineering Podcast

FEBRUARY 18, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. For someone who is interested in building a data lakehouse with Trino and Iceberg, how does that influence their selection of other platform elements?

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

The Pragmatic Engineer

APRIL 21, 2024

A refresher on OpenAI, and on Evan Evan: how did you join OpenAI, and end up heading the Applied engineering group – which also builds ChatGPT? I do  not  have a PhD in Machine Learning, and was excited by the idea of building APIs and engineering teams. With this, it’s over to Evan. My questions are in italic.

Engineering 351

Engineering Software Engineer Software Engineering Programming 351

Christophe Blefari

OCTOBER 19, 2023

On the other side the finance data team wants to build a revenue model on top of the core.orders model. The contract is super important because as soon as you expose a model, you have to potential downstream consumers that are building stuff on your models, you can't delete a column or change a type without notifying. See the doc.

Project 264

Project Finance SQL Government 264

Snowflake

APRIL 17, 2024

In today’s data-driven world, developer productivity is essential for organizations to build effective and reliable products, accelerate time to value, and fuel ongoing innovation. We’re excited to share more innovations soon, making data even more accessible for all.

Data Pipeline 113

Data Pipeline Python Data Engineer Data Engineering 113

The Pragmatic Engineer

NOVEMBER 21, 2023

  He then worked at the casual games company Zynga, building their in-game advertising platform.   We didn’t build our applications in neat containers, but in bulky monoliths which commingled business, database, backend, and frontend logic. Our deployments were initially manual. What was the other driver of adoption?

Engineering 302

Engineering Bytes Cloud Computing AWS 302

The Pragmatic Engineer

MARCH 19, 2024

Today, full subscribers got access to a comprehensive Senior-and-above tech compensation research. Devin has dominated tech news for the past few days, with major media outlets running sensational headlines, including the usually reserved Bloomberg’s, Gold-Medalist Coders Build an AI That Can Do Their Job for Them.

Software Engineer 276

Software Engineer Software Engineering Programming Language Media 276

The Pragmatic Engineer

SEPTEMBER 26, 2023

This means more repositories are needed, which are fast enough to build and work with, but which increase fragmentation. Executing a build is much slower while on a call. Plus, a CPU and memory-intensive build can impact the quality of the video call, and make the local environment much less responsive.  Larger codebases.

Cloud 258

Cloud Software Engineer Software Engineering Cloud Computing 258

Snowflake

DECEMBER 18, 2023

Data clean rooms have emerged as the technology to meet this need, enabling interoperability where multiple parties can collaborate on and analyze sensitive data in a governed way without exposing direct access to the underlying data and business logic. Snowflake’s acquisition of Samooha is subject to customary closing conditions.

Cloud 120

Cloud Building Entertainment Government 120

The Pragmatic Engineer

MARCH 12, 2024

” Brooks agrees with this observation, and suggests a radical solution: have as few senior programmers as possible, and build a team around each one – a bit like how a hospital surgeon leads a whole team. A most interesting addition! The toolsmith. The tester. They come up with test cases and data.

Engineering 274

Engineering Programming Language Hospitality Programming 274

Data Engineering Podcast

DECEMBER 24, 2023

RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles.

Kafka 245

Kafka Data Lake High Quality Data SQL 245

The Pragmatic Engineer

OCTOBER 17, 2024

A €150K ($165K) grant, three people, and 10 months to build it. The startup was able to start operations thanks to getting access to an EU grant called NGI Search grant. We envision building something comparable to AWS Fargate , or Google Cloud Run. In this article, we cover: Funding and team size. Tech stack.

Cloud 278

Cloud AWS Metadata Cloud Computing 278

The Pragmatic Engineer

MAY 11, 2023

Prometheus is part of the Cloud Native Foundation, membership of which indicates that it’s safe to build on top of Prometheus, as it’s actively maintained and will continue to be. However, you would need to hire and staff a dedicated engineering team to build and run that infra. But why is this? What happens next?

AWS 318

AWS Software Engineer Software Engineering Google Cloud 318

Snowflake

OCTOBER 7, 2024

When scaling data science and ML workloads, organizations frequently encounter challenges in building large, robust production ML pipelines. Standard Snowflake RBAC is used to control access to the Feature Store and the objects within. Today we are announcing the general availability of the Snowflake Feature Store.

Management 82

Management Building Datasets Government 82

Snowflake

JUNE 5, 2024

Enterprises can utilize gen AI to extract more value from their data and build conversational interfaces for customer and employee applications. It provides access to industry-leading large language models (LLMs), enabling users to easily build and deploy AI-powered applications.

Coding 93

Coding Building Management Government 93

Data Engineering Podcast

FEBRUARY 11, 2024

In this episode Andrew Jefferson explains the complexities of building a robust system for data sharing, the techno-social considerations, and how the Bobsled platform that he is building aims to simplify the process. Dagster offers a new approach to building and running data platforms and data pipelines. tabular, image, etc.)

Data Lake 147

Data Lake High Quality Data Government Data Pipeline 147

Data Engineering Podcast

APRIL 7, 2024

Different roles and tasks in the business need their own ways to access and analyze the data in the organization. In order to enable this use case, while maintaining a single point of access, the semantic layer has evolved as a technological solution to the problem. Go to dataengineeringpodcast.com/dagster today to get started.

Data Lake 162

Data Lake High Quality Data BI Data Workflow 162

The Pragmatic Engineer

OCTOBER 10, 2024

Wordpress.org – which has a complex, intertwined setup with Automattic, and was also cofounded by Matt Mullenweg –  bans  WP Engine from accessing its plugin repository and updates infrastructure. 25 Sep: Block. This penalizes WP Engine customers. 27 Sep: Unblock.  Wordpress.org 

Consulting 306

Consulting AWS Engineering Software Engineer 306

The Pragmatic Engineer

OCTOBER 10, 2023

He’s solved interesting engineering challenges along the way, too – like building observability for Amazon’s EC2 offering, and being one of the first engineers on Uber’s observability platform. The focus seemed to shift to: invent something new → build a service for it → ship it.

Software Engineer 268

Software Engineer Software Engineering Architecture Media 268

Data Engineering Podcast

APRIL 21, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. Go to dataengineeringpodcast.com/dagster today to get started. Your first 30 days are free! Data lakes are notoriously complex.

Data Lake 182

Data Lake High Quality Data Data Pipeline Machine Learning 182

Data Engineering Podcast

NOVEMBER 26, 2023

Summary Building a data platform that is enjoyable and accessible for all of its end users is a substantial challenge. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team.

Architecture 147

Architecture Data Lake High Quality Data SQL 147

KDnuggets

JUNE 23, 2022

It will give you the power to automate tasks, build pipelines, access file systems, and enhance development operations. Essential Linux commands to improve the data science workflow.

Data Science 159

Data Science Data Accessibility Accessible 159

Data Engineering Podcast

FEBRUARY 4, 2024

For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics. Your first 30 days are free! Starburst : ![Starburst

SQL 173

SQL Data Lake High Quality Data Data Pipeline 173

KDnuggets

OCTOBER 15, 2024

Explore free platforms for learning, building portfolios, accessing code editors, engaging with communities, and hosting projects.

Data Science 141

Data Science Portfolio Coding Accessibility 141

Data Engineering Podcast

JANUARY 21, 2024

In this episode Tasso Argyros, CEO of ActionIQ, gives a summary of the major epochs in database technologies and how he is applying the capabilities of cloud data warehouses to the challenge of building more comprehensive experiences for end-users through a modern customer data platform (CDP).

Data Lake 147

Data Lake High Quality Data NoSQL Data Warehouse 147

Data Engineering Podcast

JUNE 23, 2024

For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst is an end-to-end data lakehouse platform built on Trino, the query engine Apache Iceberg was designed for, with complete support for all table formats including Apache Iceberg, Hive, and Delta Lake. Want to see Starburst in action?

Data Lake 162

Data Lake High Quality Data Hadoop Machine Learning 162