ArcGIS

MAY 13, 2024

Review best practices for designing and testing for accessibility maps and apps throughout the ArcGIS system during the development process.

Accessible 111

Accessible Accessibility Designing Systems 111

ArcGIS

MAY 16, 2024

A modern take on the London tube map with updated accessible colours, a re-classification of lines by type, and line symbols scaled by frequency

Designing 135

Designing Accessible Accessibility 135

Engineering at Meta

AUGUST 22, 2024

They cover the co-design process behind building the second generation of Meta’s first-ever custom silicon for AI workloads, including the PyTorch software ecosystem, and the model architectures for Meta’s key applications. The post Inside the hardware and co-design of MTIA appeared first on Engineering at Meta.

Designing 89

Designing Software Engineer Software Engineering Utilities 89

Striim

SEPTEMBER 11, 2024

A data pipeline is a systematic sequence of components designed to automate the extraction, organization, transfer, transformation, and processing of data from one or more sources to a designated destination. Understanding the essential components of data pipelines is crucial for designing efficient and effective data architectures.

Data Pipeline 98

Data Pipeline Designing Data Lake Data Warehouse 98

ArcGIS

FEBRUARY 8, 2024

Best practices for using layers and different display modes in Infographic templates in ArcGIS Business Analyst and Community Analyst

Business Analyst 116

Business Analyst Designing 116

Start Data Engineering

MAY 11, 2022

Design considerations 8. Introduction 2. Objective 3. Prerequisite 4.2 AWS Infrastructure costs 4.3 Data lake structure 5. Code walkthrough 5.1 Loading user purchase data into the data warehouse 5.2 Loading classified movie review data into the data warehouse 5.3 Generating user behavior metric 5.4. Checking results 6. Tear down infra 7.

Data Engineering 246

Data Engineering Data Engineer Project Data Lake 246

The Pragmatic Engineer

JUNE 1, 2023

Juraj created a systems design explainer on how he built this project, and the technologies used: The systems design diagram for the Rides application The app uses: Node.js Phase 3: a basic UX (February) Migrate to React Design a car, from a vector image Move a car with animations. Including adding unit tests.

Education 363

Education Project PostgreSQL Software Engineer 363

Analytics Vidhya

FEBRUARY 7, 2023

Introduction Data engineering is the field of study that deals with the design, construction, deployment, and maintenance of data processing systems. This includes designing and implementing […] The post Most Essential 2023 Interview Questions on Data Engineering appeared first on Analytics Vidhya.

Data Engineering 240

Data Engineering Data Engineer Engineering Data 240

Analytics Vidhya

FEBRUARY 6, 2023

Introduction A design pattern is simply a repeatable solution for problems that keep on reoccurring. Especially while working with databases, it is often considered a good practice to follow a design pattern. The pattern is not an actual code but a template that can be used to solve problems in different situations.

Accessible 234

Accessible Accessibility Python Database 234

Data Engineering Podcast

MAY 18, 2024

For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst is an end-to-end data lakehouse platform built on Trino, the query engine Apache Iceberg was designed for, with complete support for all table formats including Apache Iceberg, Hive, and Delta Lake. Want to see Starburst in action?

Building 278

Building Data Lake High Quality Data Business Intelligence 278

The Pragmatic Engineer

AUGUST 1, 2023

We put a lot of emphasis on communication and prioritization and the ability to unblock yourself or your team – this comes on top of the programming and design skills. It’s down to them to create well-designed, extensible, performant and secure solutions. Lead a strategic team effort, starting at the design stage.

Software Engineer 245

Software Engineer Software Engineering Engineering Designing 245

Analytics Vidhya

OCTOBER 6, 2023

Enter Microsoft Fabric, a cutting-edge solution designed to revolutionize how we interact with technology. In today’s rapidly evolving digital landscape, seamless data, applications, and device integration are more pressing than ever.

Designing 262

Designing Technology Data Lake Business Intelligence 262

Data Engineering Podcast

JUNE 25, 2023

SQLMesh was designed as a unifying tool that is simple to work with but powerful enough for large-scale transformations and complex projects. How have the design and goals evolved since you first started working on it? What are the lessons that you have learned from dbt which have informed the design and functionality of SQLMesh?

Data Engineering 225

Data Engineering Data Engineer Python Engineering 225

Start Data Engineering

MAY 9, 2020

Change data capture is a software design pattern used to capture changes to data and take corresponding action based on that change. The change to data is usually one of read, update or delete. The corresponding action usually is supposed to occur in another system in response to the change that was made in the source system.

Kafka 246

Kafka Data Designing Systems 246

Start Data Engineering

MAY 28, 2024

Distributed systems are scalable, resilient to failures, & designed for high availability 4.5. Cost calculation: DuckDB + Ephemeral VMs = dirt cheap data processing 4.3. Processing data less than 100GB? Use DuckDB 4.4. KISS: DuckDB + Python = easy to debug and quick to develop 4.

Data Pipeline 219

Data Pipeline Python Building Data 219

Analytics Vidhya

MARCH 10, 2023

Snowflake’s design is […] The post Top 6 Snowflake Interview Questions appeared first on Analytics Vidhya. It is intended to be a fully managed, multi-cloud solution that does not need clients to handle hardware or software. Instead, it provides high-performance analytics, flexibility, and cost-effective scaling.

Cloud 240

Cloud Designing Management Process 240

Analytics Vidhya

AUGUST 4, 2024

These powerful tools are designed to manage and query intricate data relationships effortlessly. Traditional databases, while still valuable, often falter when it comes to handling highly connected data. Enter the unsung heroes of the data world: graph databases.

Database 213

Database Data Engineering Data Engineer Engineering 213

The Pragmatic Engineer

NOVEMBER 21, 2023

Joshua also writes an excellent Substack newsletter about how to design products which customers love, how to operate live services at scale, grow and optimize your technology orgs, and the history of the tech industry. ARPANET was designed to maintain communications during outages. Subscribe here.

Engineering 302

Engineering Bytes Cloud Computing AWS 302

Analytics Vidhya

JANUARY 31, 2023

It is designed to be more flexible and generic than the original Hadoop MapReduce system, making it an attractive choice for companies looking to implement Hadoop. Introduction YARN stands for Yet Another Resource Negotiator. It is a powerful resource management system for a horizontal server environment.

Hadoop 229

Hadoop Designing Systems Process 229

The Pragmatic Engineer

NOVEMBER 26, 2024

A next-generation code editor designed for collaboration with AI. Those mentioned were IntelliJ, PyCharm, PHPStorm, and RubyMind. Windsurf Editor. An “agentic IDE” released by Codeium just two weeks ago, which is attracting users fast. Neovim with Copilot.

Software Engineer 290

Software Engineer Software Engineering Engineering Coding 290

Analytics Vidhya

FEBRUARY 27, 2023

Delta Lake is an open-source warehouse layer designed to run on top of data lakes analogous to […] The post A Comprehensive Guide on Delta Lake appeared first on Analytics Vidhya. Delta Lake allows businesses to access and break new data down in real time.

Data Lake 215

Data Lake Business Intelligence Designing Accessible 215

Analytics Vidhya

MARCH 24, 2023

That’s why we at Analytics Vidhya host a series of informative and interactive webinars designed to help you enhance your skills and expand your knowledge of data tech […] The post Don’t Miss Out: Last Few and Exciting DataHour of March appeared first on Analytics Vidhya.

Data Science 215

Data Science Designing Data Machine Learning 215

Seattle Data Guy

MAY 8, 2023

While it can work initially, these systems aren’t designed to handle complex queries. If you’re relying on your OLTP system to provide analytics, you might be in for a surprise. … Read more The post OLTP Vs OLAP – What Is The Difference appeared first on Seattle Data Guy.

MongoDB 208

MongoDB SQL Database Designing 208

Jesse Anderson

OCTOBER 5, 2023

In another post , I introduced a concept from Justin Coffey about how much systems can be changed from their original design. Can Kafka’s codebase and design change this much without other issues popping up? Clever designs can only cover up so many problems. In Kafka’s case, many of these changes were never considered.

Kafka 195

Kafka Finance Cloud Designing 195

Robinhood

OCTOBER 10, 2024

Throughout the design process, we channeled the bold and disruptive spirit that defines Robinhood to create something completely different that elevates our brand and challenges the industry as a whole. The updated design features: An iconic, simplified logo symbol and supportive wordmark.

Banking 123

Banking Retail Designing Finance 123

Analytics Vidhya

FEBRUARY 15, 2023

Source: dataedo.com It is designed to handle big data and is ideal for […] The post Best Practices For Loading and Querying Large Datasets in GCP BigQuery appeared first on Analytics Vidhya. Its importance lies in its ability to handle big data and provide insights that can inform business decisions.

Datasets 201

Datasets Big Data Designing Data Analysis 201

Analytics Vidhya

FEBRUARY 19, 2023

A well-designed data pipeline can help organizations extract valuable insights from their data, automate tedious manual processes, and ensure the accuracy of data processing. Introduction Data pipelines play a critical role in the processing and management of data in modern organizations.

Data Pipeline 197

Data Pipeline Data Designing Data Process 197

Data Engineering Podcast

FEBRUARY 4, 2024

Summary Stream processing systems have long been built with a code-first design, adding SQL as a layer on top of the existing framework. How have the design and goals/scope changed since you first started working on it? What are the core design philosophies that you rely on to prioritize the ongoing development of the project?

SQL 173

SQL Data Lake High Quality Data Data Pipeline 173

The Pragmatic Engineer

AUGUST 22, 2023

Typical roles On a typical games project, programmers work alongside designers, artists, animators, writers, sound designers and other disciplines. I often explain this working relationship as that artists make it pretty , while designers and programmers make it work.

Software Engineer 207

Software Engineer Software Engineering Consulting Entertainment 207

The Pragmatic Engineer

APRIL 21, 2024

The engineering, product, and design organization that makes and scales these products is called "Applied," and was founded in 2020 when GPT-3 was released. ChatGPT and the API each expose several classes of model: GPT-3, GPT-3.5, It’s broadly chartered with safely bringing OpenAI's research to the world.

Engineering 351

Engineering Software Engineer Software Engineering Programming 351