The Pragmatic Engineer

JUNE 1, 2023

Juraj created a systems design explainer on how he built this project, and the technologies used: The systems design diagram for the Rides application The app uses: Node.js I also wanted to get better at setting up a full-stack project from scratch and understand why certain technologies are used. path-finding) from scratch.

Education 363

Education Project PostgreSQL Software Engineer 363

Analytics Vidhya

OCTOBER 6, 2023

Enter Microsoft Fabric, a cutting-edge solution designed to revolutionize how we interact with technology. In today’s rapidly evolving digital landscape, seamless data, applications, and device integration are more pressing than ever.

Designing 262

Designing Technology Data Lake Business Intelligence 262

Christophe Blefari

MARCH 15, 2024

AI News 🤖 Mira Murati answers the Wall Street Journal about OpenAI Sora — OpenAI CTO has been asked a few questions about the underlying technology in Sora. The technology under this, is, Cityvision. With yato you give a folder with SQL queries and it guesses the DAG and runs the queries in the right order.

Metadata 272

Metadata Datasets Data Data Warehouse 272

Analytics Vidhya

FEBRUARY 23, 2023

Introduction Data analysts with the technological know-how to tackle challenging problems are data scientists. They collect, analyze, interpret data, and handle statistics, mathematics, and computer science. They are accountable for providing insights that go beyond statistical analyses.

Retail 251

Retail Computer Science Data Technology 251

Analytics Vidhya

MARCH 5, 2023

HDInsight works seamlessly with the Hadoop ecosystem, which includes technologies like MapReduce, Hive, […] The post Top 6 Microsoft HDFS Interview Questions appeared first on Analytics Vidhya. It is a fully managed cloud-based environment for analyzing and processing enormous volumes of data.

Hadoop 246

Hadoop Data Collection Cloud Technology 246

Analytics Vidhya

FEBRUARY 21, 2023

Introduction Nowadays, the corporate environment changes according to technology. Organizations are converting them to cloud-based technologies for the convenience of data collecting, reporting, and analysis.

Data Warehouse 206

Data Warehouse IT Data Collection Data 206

Data Engineering Podcast

FEBRUARY 18, 2024

Contact Info LinkedIn dain on GitHub Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Contact Info LinkedIn dain on GitHub Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today?

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Analytics Vidhya

APRIL 17, 2023

Introduction Well, hold onto your seats because the DataHour sessions are here to revolutionize how you learn about data-driven technologies. If you’re tired of boring, dry sessions that put you to sleep faster than a lullaby, you’re in for a treat.

Data Science 224

Data Science Data Technology Deep Learning 224

Christophe Blefari

MARCH 1, 2023

Also dbt only does a pass-through to your underlying data compute technology, there is not any kind of processing within dbt. There are multiple technological reasons for this, but technology is rarely the real reason. Actually dbt can be seen as an orchestrator with no scheduling capabilities.

Data Warehouse 278

Data Warehouse SQL Metadata Raw Data 278

Data Engineering Podcast

DECEMBER 24, 2023

Summary Kafka has become a ubiquitous technology, offering a simple method for coordinating events and data across different systems. Contact Info LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Operating it at scale, however, is notoriously challenging.

Kafka 245

Kafka Data Lake High Quality Data SQL 245

Snowflake

MAY 22, 2024

TruEra’s technology helps evaluate the quality of inputs, outputs, and intermediate results of LLM apps. TruEra’s technology also provides detailed, actionable insights to improve ML model performance and accuracy by revealing anomalies in model metrics and providing a specific root cause analysis for rapid debugging.

Cloud 127

Cloud Data Governance Technology Machine Learning 127

Analytics Vidhya

MARCH 14, 2023

Introduction In today’s world, technology has increased tremendously, and many people are using the internet. This results in the generation of so much data daily. This generated data is stored in the database and will maintain it. SQL is a structured query language used to read and write these databases.

SQL 202

SQL Database Technology Data 202

Data Engineering Podcast

APRIL 21, 2024

Summary Generative AI has rapidly transformed everything in the technology sector. Contact Info LinkedIn Blog Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? When Andrew Lee started work on Shortwave he was focused on making email more productive.

Data Lake 182

Data Lake High Quality Data Data Pipeline Machine Learning 182

Data Engineering Podcast

SEPTEMBER 10, 2023

Summary Data systems are inherently complex and often require integration of multiple technologies. In this episode Nick Schrock, creator of Dagster, shares his perspective on the state of data orchestration technology and its application to help inform its implementation in your environment.

BI 208

BI SQL Data Machine Learning 208

Jesse Anderson

OCTOBER 5, 2023

Data quality problems are solved by more than just technology. Data products need to be exposed with the right technologies for the various use cases. The technologies speak Kafka’s binary protocol, removing the need for any client-side changes. While Flink is a great technology, it isn’t for every use case.

Kafka 195

Kafka Finance Cloud Designing 195

Data Engineering Podcast

JUNE 2, 2024

Nicola Askham found her way into data governance by accident, and stayed because of the benefit that she was able to provide by serving as a bridge between the technology and business. What impact has the evolution of data technologies had on the implementation of governance practices? Are there any trends that concern you?

Data Governance 147

Data Governance Government Data Lake High Quality Data 147

The Pragmatic Engineer

APRIL 21, 2024

Sometimes the best explanations of how a technology solution works come from the software engineers who built it. See a longer version of this article here: Scaling ChatGPT: Five Real-World Engineering Challenges. To explain how ChatGPT (and other large language models) operate, I turned to the ChatGPT engineering team. "How

Engineering 351

Engineering Software Engineer Software Engineering Programming 351

The Pragmatic Engineer

SEPTEMBER 22, 2023

Technological innovation rarely happens in a vacuum; it builds on previous technologies. The license of a project is key to understanding how ‘risky’ it is to use. Bun uses an extremely permissive MIT license that minimizes any risk.

Programming Language 255

Programming Language Project Coding Engineering 255

Analytics Vidhya

MAY 24, 2023

The complexity of information storage technologies increases exponentially with the growth of data. As modern companies rely on data, establishing dependable, effective solutions for maintaining that data is a top task for each organization.

Data Storage 179

Data Storage IT Cloud Computing Cloud 179

Precisely

OCTOBER 2, 2024

Recognizing and supporting women in technology is a top priority at Precisely. Whether it’s hosting virtual events for women to connect, or encouraging mentoring opportunities, the Precisely Women in Technology (PWIT) program goes above and beyond to ensure that women in the organization have a great network to lean on.

Technology 105

Technology Utilities Management Programming 105

Data Engineering Podcast

MARCH 24, 2024

Contact Info Twitter LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Contact Info Twitter LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today?

Data Lake 162

Data Lake High Quality Data Hadoop Data Pipeline 162

Data Engineering Podcast

JUNE 9, 2024

In Code Comments, host Jamie Parker, Red Hatter and experienced engineer, shares the journey of technologists from across the industry and their hard-won lessons in implementing new technologies. Code Comments Podcast Logo]([link] Putting new technology to use is an exciting prospect. Closing Announcements Thank you for listening!

Process 147

Process Data Lake High Quality Data Machine Learning 147

Data Engineering Podcast

JUNE 16, 2024

In this episode Kevin Liu shares some of the interesting features that they have built by combining those technologies, as well as the challenges that they face in supporting the myriad workloads that are thrown at this layer of their data platform. What are the other systems that feed into and rely on the Trino/Iceberg service?

Data Lake 147

Data Lake High Quality Data Metadata Machine Learning 147

KDnuggets

FEBRUARY 13, 2024

Have a look at O’Reilly’s FREE technology trends report. Want to prepare your tech career for 2024 and onwards?

Technology 150

Technology 150

KDnuggets

OCTOBER 15, 2024

This guide emphasizes the growing significance of GenAI but also highlights the crucial role that data scientists play in harnessing this technology to solve real-world problems.

Data 150

Data Technology 150

Data Engineering Podcast

JUNE 23, 2024

What are the benefits of layering on top of existing technologies rather than building a fully custom solution? Contact Info LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? What are the elements of Fabric that were engineered specifically for the service?

Data Lake 162

Data Lake High Quality Data Hadoop Machine Learning 162

KDnuggets

APRIL 4, 2024

These GitHub repositories provide valuable resources for mastering computer science, including comprehensive roadmaps, free books and courses, tutorials, and hands-on coding exercises to help you gain the skills and knowledge necessary to thrive in the ever-evolving field of technology.

Computer Science 150

Computer Science Coding Technology Programming 150

Data Engineering Podcast

APRIL 7, 2024

In order to enable this use case, while maintaining a single point of access, the semantic layer has evolved as a technological solution to the problem. Contact Info LinkedIn keydunov on GitHub Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today?

Data Lake 162

Data Lake High Quality Data BI Data Workflow 162

Data Engineering Podcast

MAY 12, 2024

In Code Comments, host Jamie Parker, Red Hatter and experienced engineer, shares the journey of technologists from across the industry and their hard-won lessons in implementing new technologies. Code Comments Podcast Logo]([link] Putting new technology to use is an exciting prospect.

Management 147

Management Data Lake High Quality Data Machine Learning 147

Data Engineering Podcast

MAY 21, 2023

In this episode David Yaffe and Johnny Graettinger share the story behind the business and technology and how you can start using it today to build a real-time data lake without all of the headache. Stream processing technologies have been around for around a decade. Stream processing technologies have been around for around a decade.

Data Lake 162

Data Lake Machine Learning Kafka Data Warehouse 162