The Pragmatic Engineer

APRIL 21, 2024

Sometimes the best explanations of how a technology solution works come from the software engineers who built it. ChatGPT works impressively well with human language, and has access to more information than any one human could handle. See a longer version of this article here: Scaling ChatGPT: Five Real-World Engineering Challenges.

Engineering 351

Engineering Software Engineering Software Engineer Programming 351

Data Engineering Podcast

APRIL 7, 2024

Different roles and tasks in the business need their own ways to access and analyze the data in the organization. In order to enable this use case, while maintaining a single point of access, the semantic layer has evolved as a technological solution to the problem. What do you have planned for the future of Cube?

Data Lake 162

Data Lake High Quality Data BI Data Workflow 162

The Pragmatic Engineer

MARCH 19, 2024

Today, full subscribers got access to a comprehensive Senior-and-above tech compensation research. Source: Cognition So far, all we have is video demos, and accounts of those with access to this tool. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers.

Software Engineering 275

Software Engineering Software Engineer Programming Language Media 275

Data Engineering Podcast

DECEMBER 24, 2023

Summary Kafka has become a ubiquitous technology, offering a simple method for coordinating events and data across different systems. Contact Info LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Operating it at scale, however, is notoriously challenging.

Kafka 245

Kafka Data Lake High Quality Data SQL 245

Data Engineering Podcast

APRIL 21, 2024

Summary Generative AI has rapidly transformed everything in the technology sector. Contact Info LinkedIn Blog Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? When Andrew Lee started work on Shortwave he was focused on making email more productive.

Data Lake 182

Data Lake High Quality Data Data Pipeline Machine Learning 182

The Pragmatic Engineer

SEPTEMBER 26, 2023

Since then, all Cloud providers have expanded their availability, and the price of Cloud compute has dropped due to competition and technological progress. With full-remote work, the risk is higher that someone other than the employee accesses the codebase. Full subscribers can access a list with links here.

Cloud 258

Cloud Software Engineering Software Engineer Cloud Computing 258

Data Engineering Podcast

FEBRUARY 11, 2024

What are the requirements around governance and auditability of data access that need to be addressed when sharing data? What are the typical boundaries along which data access requires special consideration for how the sharing is managed? How does that change as a function of the type of data? tabular, image, etc.)

Data Lake 147

Data Lake High Quality Data Government Data Pipeline 147

Data Engineering Podcast

JUNE 23, 2024

What are the benefits of layering on top of existing technologies rather than building a fully custom solution? Contact Info LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? What are the elements of Fabric that were engineered specifically for the service?

Data Lake 162

Data Lake High Quality Data Hadoop Machine Learning 162

Data Engineering Podcast

JUNE 16, 2024

In this episode Kevin Liu shares some of the interesting features that they have built by combining those technologies, as well as the challenges that they face in supporting the myriad workloads that are thrown at this layer of their data platform. What are the other systems that feed into and rely on the Trino/Iceberg service?

Data Lake 147

Data Lake High Quality Data Metadata Machine Learning 147

Data Engineering Podcast

MAY 26, 2024

In Code Comments, host Jamie Parker, Red Hatter and experienced engineer, shares the journey of technologists from across the industry and their hard-won lessons in implementing new technologies. What are the characteristics or features of data technologies and the overall ecosystem that can reduce the burden of data migration in the future?

Systems 130

Systems Data Lake High Quality Data Google Cloud 130

Data Engineering Podcast

MARCH 24, 2024

Contact Info Twitter LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Contact Info Twitter LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today?

Data Lake 162

Data Lake High Quality Data Hadoop Data Pipeline 162

Data Engineering Podcast

JUNE 2, 2024

Nicola Askham found her way into data governance by accident, and stayed because of the benefit that she was able to provide by serving as a bridge between the technology and business. What impact has the evolution of data technologies had on the implementation of governance practices? Are there any trends that concern you?

Data Governance 147

Data Governance Government Data Lake High Quality Data 147

Data Engineering Podcast

JANUARY 21, 2024

In this episode Tasso Argyros, CEO of ActionIQ, gives a summary of the major epochs in database technologies and how he is applying the capabilities of cloud data warehouses to the challenge of building more comprehensive experiences for end-users through a modern customer data platform (CDP). Closing Announcements Thank you for listening!

Data Lake 147

Data Lake High Quality Data NoSQL Data Warehouse 147

Data Engineering Podcast

JUNE 9, 2024

In Code Comments, host Jamie Parker, Red Hatter and experienced engineer, shares the journey of technologists from across the industry and their hard-won lessons in implementing new technologies. Code Comments Podcast Logo]([link] Putting new technology to use is an exciting prospect. Closing Announcements Thank you for listening!

Process 147

Process Data Lake High Quality Data Machine Learning 147

Snowflake

JANUARY 15, 2024

“Informatica’s Snowflake Native App is really about meeting our customers where they are,” says Rik Tamm-Daniels, Global VP of Ecosystems and Technology at Informatica. Now, they can access the full set of capabilities of Informatica’s Intelligent Data Management Cloud (IDMC) platform through a single drag-and-drop interface.

Data Integration 104

Data Integration BI Cloud Data 104

Snowflake

JUNE 5, 2024

At the same time, organizations must ensure the right people have access to the right content, while also protecting sensitive and/or Personally Identifiable Information (PII) and fulfilling a growing list of regulatory requirements.

Government 104

Government Accessible Accessibility Cloud 104

Data Engineering Podcast

FEBRUARY 4, 2024

Contact Info yingjunwu on GitHub Personal Website LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? What do you have planned for the future of RisingWave? Closing Announcements Thank you for listening! Don't forget to check out our other shows.

SQL 173

SQL Data Lake High Quality Data Data Pipeline 173

Christophe Blefari

MARCH 2, 2024

In the end you finish with a composable solution mixing up 2 or 3 technologies to extract and load you data in your central storage, ready for transformations. On the other side of the movement there is a new open-source reverse-ETL technology called Multiwoven/multiwoven. But the field has taken a new direction.

Data 162

Data Python Database BI 162

Snowflake

JUNE 27, 2024

By bringing workloads closer to the data, Snowflake Native Apps integrated with Snowpark Container Services makes it easier for RAI’s customers to adopt its technology. Snowflake Marketplace access Aref says RelationalAI saw new customer interest and demand grow exponentially, even when their product was available only in private preview.

Data Governance 110

Data Governance Algorithm Government Cloud 110

Data Engineering Podcast

MAY 12, 2024

In Code Comments, host Jamie Parker, Red Hatter and experienced engineer, shares the journey of technologists from across the industry and their hard-won lessons in implementing new technologies. Code Comments Podcast Logo]([link] Putting new technology to use is an exciting prospect. Want to see Starburst in action?

Management 147

Management Data Lake High Quality Data Machine Learning 147

Rockset

JUNE 21, 2024

We are thrilled to join the OpenAI team and bring our technology and expertise to building safe and beneficial AGI. With this acquisition, what we’ve developed over the years will help make AI accessible to all in a safe and beneficial way. I’m excited to share that OpenAI has completed the acquisition of Rockset.

Database 145

Database Cloud Accessible Accessibility 145

Data Engineering Podcast

FEBRUARY 5, 2023

Summary This podcast started almost exactly six years ago, and the technology landscape was much different than it is now. Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? In that time there have been a number of generational shifts in how data engineering is done.

Data Engineering 147

Data Engineering Data Engineer Engineering PostgreSQL 147

Analytics Vidhya

FEBRUARY 2, 2023

Introduction SQL injection is an attack in which a malicious user can insert arbitrary SQL code into a web application’s query, allowing them to gain unauthorized access to a database. We can use this to steal sensitive information or make unauthorized changes to the data stored in the database.

Database 204

Database SQL Coding Accessible 204

Snowflake

FEBRUARY 5, 2024

And it’s no wonder — this new technology has the potential to revolutionize the industry by augmenting the value of employee work, driving organizational efficiencies, providing personalized customer experiences, and uncovering new insights from vast amounts of data. Generative AI tops every list of major financial services trends for 2024.

Unstructured Data 105

Unstructured Data Banking Government Media 105

Data Engineering Podcast

MARCH 3, 2024

Summary Artificial intelligence technologies promise to revolutionize business and produce new sources of value. Contact Info LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? What do you have planned for the future of your work at VAST Data?

Programming 162

Programming Data Lake High Quality Data Data Pipeline 162

Knowledge Hut

MARCH 28, 2024

In addition, the demand for UI designers and UX designers has grown since AR and VR technologies have made it easier to develop immersive and captivating user experiences. Companies require designers that can adjust to changing technologies and design trends in order to remain competitive.

Designing 105

Designing Accessible Accessibility Technology 105

Data Engineering Podcast

JUNE 30, 2024

For a team who is onboarding onto Synq, what are the steps required to get it integrated into their technology stack and workflows? Contact Info LinkedIn Substack Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? When is Synq the wrong choice?

Pipeline-centric 147

Pipeline-centric Engineering Data Lake High Quality Data 147

Data Engineering Podcast

JANUARY 7, 2024

Summary Data processing technologies have dramatically improved in their sophistication and raw throughput. Contact Info Website LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? What do you have planned for the future of your academic research?

Data Process 147

Data Process Process Data Lake High Quality Data 147

Snowflake

AUGUST 5, 2024

In this edition, we talk to Brent Lane, Co-founder and CEO of BigGeo, about the world of geospatial data and learn how BigGeo is turning 15 years of research into advanced technology that knocks down traditional barriers to using rich, complex location-based data throughout an organization. What inspires you as a founder?

Business Intelligence 112

Business Intelligence Technology Accessible Accessibility 112

Cloudera

APRIL 18, 2024

With the arrival of the latest AI-powered technologies like large language models (LLMs) and generative AI (GenAI), there’s a vast amount of opportunities for innovation, growth, and improved business outcomes right around the corner. All of that technology, though, depends on data to be successful.

Metadata 113

Metadata Government Data Governance Data 113

Snowflake

DECEMBER 18, 2023

Data clean rooms have emerged as the technology to meet this need, enabling interoperability where multiple parties can collaborate on and analyze sensitive data in a governed way without exposing direct access to the underlying data and business logic.

Cloud 119

Cloud Building Entertainment Government 119

Snowflake

JANUARY 9, 2024

Data cloud integration: This comprehensive solution begins with the Snowflake Data Cloud as a persistent data layer, which makes data more accessible for organizations to get started with the platform. This allows data teams to share and access governed data while preserving data privacy.

Data Ingestion 110

Data Ingestion Cloud Architecture Accessible 110

Data Engineering Podcast

MARCH 18, 2023

Summary As with all aspects of technology, security is a critical element of data applications, and the different controls can be at cross purposes with productivity. In your experience, what is a typical number of data locations that an organization is trying to manage access/permissions within?

Data Security 130

Data Security Machine Learning Data Science Metadata 130

Engineering at Meta

OCTOBER 22, 2024

Contact names are stored encrypted within WhatsApp, and we’ve built this with additional, robust protections by using IPLS to deter access to contacts to anyone except the user. IPLS incorporates new privacy technology that protects your contact lists in a privacy-preserving fashion.

Consulting 104

Consulting Accessible Accessibility Technology 104

Snowflake

JUNE 21, 2024

This is why there’s accelerating Iceberg adoption, and why Snowflake along with many other technology vendors and open source projects are supporting Iceberg ahead of other table formats. Zero Ingest with Zero Silos : Iceberg data already managed in a data lake can be accessed directly by Snowflake via an Iceberg catalog integration.

Data Lake 117

Data Lake BI Business Intelligence Metadata 117

The Pragmatic Engineer

SEPTEMBER 19, 2024

These scripts mixed database access, HTML generation, and logic in unexpected ways. Recruit expertise in a new technology and front-line managers that have done it before.  If it isn’t working, then there might be problems delivering electricity. Sometimes a script would generate another script.

Recruitment 233

Recruitment Java Coding Project 233