The Pragmatic Engineer
OCTOBER 17, 2024
Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. In this article, we cover one section from this week’s from last week’s The Pulse issue. To get full issues twice a week, subscribe here.
Start Data Engineering
OCTOBER 14, 2024
1. Introduction 2. Code & Data 3. Using nested data types effectively 3.1. Use STRUCT for one-to-one & hierarchical relationships 3.2. Use ARRAY[STRUCT] for one-to-many relationships 3.3. Using nested data types in data processing 3.3.1. STRUCT enables more straightforward data schema and data access 3.3.2. Nested data types can be sorted 3.3.3.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
The Pragmatic Engineer
OCTOBER 10, 2024
Automattic, creator of Wordpress, is being sued by one of the largest WordPress hosting providers. The conflict fits into a trend of billion-dollar companies struggling to effectively monetize open source, and are changing tactics to limit their competition and increase their revenue. This article was originally published a week ago, on 3 October 2024, in The Pragmatic Engineer.
Confessions of a Data Guy
OCTOBER 14, 2024
I figured a few of us might need the WordPress drama explained like we are 5. So, here you go. WordPress is the GOAT of internet website builders WordPress was founded by Matt Mullenweg With much of the internet running on WordPress … hosting WordPress is of course … lucrative and a big business. The […] The post What is the WordPress drama about?
Speaker: Jay Allardyce
As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.
Robinhood
OCTOBER 10, 2024
When Robinhood was founded, we set out to build a platform that gives everyone access to the financial markets. Over the last decade, we’ve disrupted and changed the industry for the better, becoming the first U.S. retail broker to offer commission-free trading, and saving investors billions in the process. In recent years, we’ve expanded our offering, ushering in a number of new cutting-edge products and services that help everyone – regardless of income – trade, invest, and earn.
KDnuggets
OCTOBER 10, 2024
NotebookLM makes it easy to create study guides from YouTube videos by using AI to summarize and organize key points. Just upload the video link, and the tool helps you turn the content into a structured guide.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Confessions of a Data Guy
OCTOBER 12, 2024
Is there anything worse than the PR process (Pull Request) at most companies? Probably not. It’s the dreaded 600-pound gorilla in the room that no one wants to talk about. Everyone hates it, everyone has to do it. But, it doesn’t have to be like that. There are a few tried and true ways to […] The post How to make the PEFECT Pull Request (PR) appeared first on Confessions of a Data Guy.
databricks
OCTOBER 10, 2024
Today, we are excited to announce the general availability of Databricks Assistant Autocomplete on all cloud platforms. Assistant Autocomplete provides personalized AI-powered code.
KDnuggets
OCTOBER 16, 2024
Where can you find projects dealing with advanced ML topics? GitHub is a perfect source with its many repositories. I’ve selected ten to talk about in this article.
Engineering at Meta
OCTOBER 15, 2024
At Open Compute Project Summit (OCP) 2024, we’re sharing details about our next-generation network fabric for our AI training clusters. We’ve expanded our network hardware portfolio and are contributing two new disaggregated network fabrics and a new NIC to OCP. We look forward to continued collaboration with OCP to open designs for racks, servers, storage boxes, and motherboards to benefit companies of all sizes across the industry.
Speaker: Nikhil Joshi, Founder & President of Snic Solutions
In an era where efficiency is king, are you leveraging the right tools to transform your manufacturing processes? A Manufacturing Execution System (MES) is critical for enhancing operational efficiency, reducing waste, and optimizing energy usage—key factors for improving your bottom line and lowering your carbon footprint. Join Nikhil Joshi, a manufacturing technology expert with 18+ years of hands-on experience, in this new webinar as he uncovers the secrets of MES and how to best utilize thes
Cloudera
OCTOBER 10, 2024
The open data lakehouse is quickly becoming the standard architecture for unified multifunction analytics on large volumes of data. It combines the flexibility and scalability of data lake storage with the data analytics, data governance, and data management functionality of the data warehouse. Open table formats are a key component of this architecture, as they provide many of the capabilities of traditional data warehousing directly on data lake storage, and Apache Iceberg is quickly becoming
Precisely
OCTOBER 16, 2024
At Trust ’24, we had the opportunity to sit down with Allan Christian, CTO of Precisely Engage, to discuss how AI is transforming customer communications and what the future holds for this technology. In this Q&A session, Allan shares insights into the AI-driven technologies Engage is offering clients today, the feedback they’re receiving from customers about AI-driven innovations, and the emerging technologies that will further enhance their products in the future.
databricks
OCTOBER 17, 2024
We are excited to announce the Public Preview of Databricks serverless budget policies. Administrators can use budget policies to ensure that the correct.
KDnuggets
OCTOBER 11, 2024
Dive into this emerging and powerful LLM-based AI tool for enhancing your business, creative, or daily processes through well-managed conversations.
Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage
When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.
Snowflake
OCTOBER 14, 2024
If innovation is the ultimate goal in business and technology today, then consider generative AI (gen AI) the vehicle taking us there — and a strong data strategy, the fuel. Despite all its promise of productivity gains and new discoveries, gen AI alone can't do it all. The technology needs a "very ready" data foundation to feed on, something the vast majority of businesses today (78%) do not possess, according to a new report by MIT Technology Review Insights , in partnership with Snowf
Confluent
OCTOBER 14, 2024
Migrate from ZooKeeper to KRaft using Confluent for Kubernetes quickly and with ease. Automate the process and migrate in minutes.
Precisely
OCTOBER 17, 2024
This year, our annual Data Integrity Summit, Trust ’24, was better than ever – and a big part of what made the event so exciting was our first-ever Data Integrity Awards ! We were thrilled to shine the spotlight on customers driving AI innovation, business results, and societal impact. As you plan for 2025, get inspired with their success stories today.
databricks
OCTOBER 14, 2024
Over the last few years, we've seen tremendous growth and adoption of Databricks SQL , our intelligent data warehouse purpose-built on the Data.
Speaker: Donna Laquidara-Carr, PhD, LEED AP, Industry Insights Research Director at Dodge Construction Network
In today’s construction market, owners, construction managers, and contractors must navigate increasing challenges, from cost management to project delays. Fortunately, digital tools now offer valuable insights to help mitigate these risks. However, the sheer volume of tools and the complexity of leveraging their data effectively can be daunting. That’s where data-driven construction comes in.
KDnuggets
OCTOBER 15, 2024
This guide emphasizes the growing significance of GenAI but also highlights the crucial role that data scientists play in harnessing this technology to solve real-world problems.
Snowflake
OCTOBER 17, 2024
Predictive machine learning continues to be a cornerstone of data-driven decision-making. However, as organizations accumulate more data in a wide variety of forms, and as modeling techniques continue to advance, the tasks of a data scientist and ML engineer are becoming increasingly complex. Oftentimes, more effort is spent on managing infrastructure, jumping through package management hurdles, and dealing with scalability issues than on actual model development.
Monte Carlo
OCTOBER 16, 2024
Measuring the progress of a data quality initiative is not as straightforward as it seems. One of the biggest reasons for this is because data quality is use case specific. For example, the accuracy required for a machine learning application may only need to be directional whereas a finance report may need to be accurate to the penny. To this end, we are happy to introduce our latest Data Quality Dashboard.
Precisely
OCTOBER 14, 2024
Key Takeaways: Only 12% of organizations report their data is of sufficient quality and accessibility for AI. Data analysis (57%) is the top-cited reason organizations are considering the use of AI. The top data challenge inhibiting the progress of AI initiatives is data governance (62%). The 2025 Outlook: Data Integrity Trends and Insights report is here!
Speaker: Aindra Misra, Senior Manager, Product Management (Data, ML, and Cloud Infrastructure) at BILL
Join us for an insightful webinar that explores the critical intersection of data privacy and AI governance. In today’s rapidly evolving tech landscape, building robust governance frameworks is essential to fostering innovation while staying compliant with regulations. Our expert speaker, Aindra Misra, will guide you through best practices for ensuring data protection while leveraging AI capabilities.
databricks
OCTOBER 16, 2024
AI/BI Genie is a conversational experience for business teams to self-serve insights from their data through natural language. Genie leverages generative AI tailored.
KDnuggets
OCTOBER 14, 2024
Creating custom educational podcasts with NotebookLM is straightforward. Simply upload your content, customize the audio output to highlight key topics, and export your final podcast for sharing with your audience.
Snowflake
OCTOBER 14, 2024
If there’s one thing you can count on with Advertising Week New York 2024, it is that you will leave with your head buzzing with ideas, insights and the latest industry trends. It is the advertising event of the year. With more than 60% of attendees director-level or above from leading brands, agencies and adtech companies, it is a thought leadership event of truly epic proportions.
DareData
OCTOBER 16, 2024
An aggressive and dominant communication style, unrealistic goals, inefficient structures where no one takes ownership and rigid hierarchies where people fear failure. Does it sound familiar? Throughout my career, I've encountered the same environment numerous times. In each new job, I've tried to make improvements, yet inevitably, I would find myself without any major victories, feeling tired, out of place, and anxious.
Speaker: Evelyn Chou
Choosing the right business intelligence (BI) platform can feel like navigating a maze of features, promises, and technical jargon. With so many options available, how can you ensure you’re making the right decision for your organization’s unique needs? 🤔 This webinar brings together expert insights to break down the complexities of BI solution vetting.
Confluent
OCTOBER 15, 2024
Third installment of the Producer/Consumer Internals series that covers preparing the consumer fetch: how consumers interact with brokers, coordinate partitions, and send requests.
databricks
OCTOBER 16, 2024
At Databricks, we are constantly innovating and optimizing our platform to ensure that our customers can maximize the value of their data and.
KDnuggets
OCTOBER 14, 2024
For developers working with LLMs, Intel’s article serves as a practical guide to navigating the complexities of fine-tuning and inference, offering valuable insights and techniques for optimizing both the development and deployment phases.
Monte Carlo
OCTOBER 15, 2024
Ever look at a dashboard and wonder “ How exactly did these numbers get here?” Yeah, you’re not the only one to think that. How quick and easy it is to come to an answer depends on how traceable the data is. Data traceability is the process of tracking data’s flow, transformations, and uses from its creation to its final destination. I’ll walk you through why data traceability is so important and how you can do it.
Advertisement
Entity Resolution Sometimes referred to as data matching or fuzzy matching, entity resolution, is critical for data quality, analytics, graph visualization and AI. Learn what entity resolution is, why it matters, how it works and its benefits. Advanced entity resolution using AI is crucial because it efficiently and easily solves many of today’s data quality and analytics problems.
Expert insights. Personalized for you.
130 
Let's personalize your content