Waitingforcode

DECEMBER 19, 2023

Even though nowadays data processing frameworks and data stores have smart query planners, they don't take our responsibility to correctly design the job logic.

Designing 130

Designing Data Process Process Data 130

Engineering at Meta

DECEMBER 19, 2023

On July 5, 2023, Meta launched Threads, the newest product in our family of apps, to an unprecedented success that saw it garner over 100 million sign ups in its first five days. A small, nimble team of engineers built Threads over the course of only five months of technical work. While the app’s production launch had been under consideration for some time, the business finally made the decision and informed the infrastructure teams to prepare for its launch with only two days’ advance notice.

Programming Language 142

Programming Language Engineering Database Coding 142

LinkedIn Engineering

DECEMBER 19, 2023

Co-authors: Max Kanat-Alexander and Grant Jenks Today we are open-sourcing the LinkedIn Developer Productivity & Happiness Framework (DPH Framework) - a collection of documents that describe the systems, processes, metrics, and feedback systems we use to understand our developers and their needs internally at LinkedIn. Now more than ever, developers are navigating so much change and new opportunity in this new era of Generative AI, so ensuring teams have the systems, processes, metrics and f

Data Schemas 121

Data Schemas Software Engineer Software Engineering Data Pipeline 121

ThoughtSpot

DECEMBER 19, 2023

2023 has been a year of breakthrough innovation for many, and a deer-in-headlights moment for others. I keep flashing back to the 90s when the Internet created new businesses and destroyed others—LLMs are doing the same, only with more velocity. From CDAOs to VCs alike, the rate of creative destruction is faster, but there is also an intense focus on value.

Entertainment 111

Entertainment Data Big Data Data Analytics 111

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

Engineering at Meta

DECEMBER 19, 2023

HawkEye is the powerful toolkit used internally at Meta for monitoring, observability, and debuggability of the end-to-end machine learning (ML) workflow that powers ML-based products. HawkEye supports recommendation and ranking models across several products at Meta. Over the past two years, it has facilitated order of magnitude improvements in the time spent debugging production issues.

Data Pipeline 111

Data Pipeline Algorithm Architecture Machine Learning 111

KDnuggets

DECEMBER 19, 2023

There are many data-learning materials locked up behind expensive books. These cheap books would bolster your skills without blowing up your savings.

Data Science 125

Data Science Data 125

KDnuggets

DECEMBER 19, 2023

Soon AI will become embedded into daily business processes, including cybersecurity controls. The author explains how to assess which processes make sense to automate.

Process 112

Process 112

LinkedIn Engineering

DECEMBER 19, 2023

Co-authors: Arjun Mohnot , Jenchang Ho , Anthony Quigley , Xing Lin , Anil Alluri , Michael Kuchenbecker LinkedIn operates one of the world’s largest Apache Hadoop big data clusters. These clusters are the backbone for storing and processing extensive data volumes, empowering us to deliver essential features and services to members, such as personalized recommendations, enhanced search functionality, and valuable insights.

Big Data 92

Big Data Hadoop Metadata Data 92

databricks

DECEMBER 19, 2023

Request a meeting with Databricks executives/thought leaders at NRF! Each January, thousands of leaders from retailers around the globe gather at Javits Center.

Retail 103

Retail Data 103

Cloudera

DECEMBER 19, 2023

Without a doubt, 2023 has shaped up to be generative AI’s breakout year. Less than 12 months after the introduction of generative AI large language models such as ChatGPT and PaLM, image generators like Dall-E, Midjourney, and Stable Diffusion, and code generation tools like OpenAI Codex and GitHub CoPilot, organizations across every industry, including government, are beginning to leverage generative AI regularly to increase creativity and productivity.

Government 90

Government Education Unstructured Data Algorithm 90

Speaker: Jay Allardyce, Deepak Vittal, and Terrence Sheflin

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

ArcGIS

DECEMBER 19, 2023

Here's how to fake an isometric 3D topo terrain in 2D! And stuff.

138

138

Snowflake

DECEMBER 19, 2023

The best marketing is truly data-driven, creating powerful product promotions and offers through an understanding of customer needs and preferences. But for many organizations, building this understanding is more akin to solving an ever-growing jigsaw puzzle (with no easy edge pieces!) than reading data insights from a beautiful dashboard. Every customer store interaction, online transaction, form fill, event participation, chatbot response, text request, like, review, complaint, and click creat

Building 82

Building Pipeline-centric Database-centric Digital Media 82

Workfall

DECEMBER 19, 2023

Reading Time: 16 minutes Introducing Amazon PartyRock, an innovative platform that redefines the landscape of app exploration and creation. For Part 1 of this blog, refer here. In this transformative hands-on implementation, we will guide you through the PartyRock playground, an exciting journey that encompasses navigating its free features, signing in to unlock personalized experiences, experimenting with suggested apps, exploring a myriad of pre-built applications, and culminating in the creat

Building 81

Building AWS Software Engineer Software Engineering 81

ArcGIS

DECEMBER 19, 2023

We will dive into our best practices for preparing and using training samples for object detection models.

Data Preparation 103

Data Preparation Data Deep Learning 103

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

Towards Data Science

DECEMBER 19, 2023

Using Airflow sensors to control the execution of DAGs on a different schedule Continue reading on Towards Data Science »

Data Science 74

Data Science Data Programming Data Engineering 74

dbt Developer Hub

DECEMBER 19, 2023

Overview ​ Over the course of my three years running the Partner Engineering team at dbt Labs, the most common question I've been asked is, How do we integrate with dbt? Because those conversations often start out at the same place, I decided to create this guide so I’m no longer the blocker to fundamental information. This also allows us to skip the intro and get to the fun conversations so much faster, like what a joint solution for our customers would look like.

Metadata 52

Metadata Cloud Accessible Accessibility 52

Rockset

DECEMBER 19, 2023

Overview In this guide, we will: Understand the Blueprint of any modern recommendation system Dive into a detailed analysis of each stage within the blueprint Discuss infrastructure challenges associated with each stage Cover special cases within the stages of the recommendation system blueprint Get introduced to some storage considerations for recommendation systems And finally, end with what the future holds for the recommendation systems Introduction In a recent insightful talk at Index confe

Systems 52

Systems Machine Learning Deep Learning Media 52

WeCloudData

DECEMBER 19, 2023

The Recap We continue with the unfolding saga of the data science jobs landscape, this time for the month of November. In my previous blog, I continued to compare Toronto’s data science jobs market against the rest of North America, and after a strong September, it didn’t look especially good for Toronto. It looked as […] The post Toronto’s Data Science Renaissance: A Tale of Two Markets appeared first on WeCloudData.

Data Science 52

Data Science Data IT 52

Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage

When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.

Systems

Ascend.io

DECEMBER 19, 2023

If you’re working with Snowflake or just starting to explore its capabilities, you might be wondering: Do I really need ETL for Snowflake? Is it possible to rely solely on Snowflake’s own features, or is there a strong case for bringing ETL into the mix? If so, where do I get started? In this article, we’re diving into these questions to clear up any confusion.

ETL Tools 52

ETL Tools IT Data Pipeline Data Warehouse 52

Confluent

DECEMBER 19, 2023

Legacy data systems often power travel experiences, such as on cruise lines, but modern customers want real-time experiences online. Here's how to think about data integration with data streaming for travelers.

Data Integration 52

Data Integration Systems Data 52

Scott Logic

DECEMBER 19, 2023

Many of us love a good podcast so I reached out to our project team to see what they were listening to in the tech and sustainability space. Here are their recommendations: Environment Variables If I were to pick one podcast to start with, then this would be the one. Published by the Green Software Foundation, each episode aims to bring listeners the latest news regarding how to reduce the emissions of software and how the industry is dealing with its own environmental impact.

Coding 52

Coding Building Technology Project 52

Picnic Engineering

DECEMBER 19, 2023

With operations in full swing to pull us through the busiest time of the year, the code slush we apply in some of our teams allow us to take a step back and reflect on another exciting year in the crazy little groceries roller coaster we call Picnic. In this blog, we’d like to give you a glimpse into some of the major developments in Picnic Tech in 2023.

Transportation 52

Transportation Pipeline-centric Database-centric Python 52

Speaker: Donna Laquidara-Carr, PhD, LEED AP, Industry Insights Research Director at Dodge Construction Network

In today’s construction market, owners, construction managers, and contractors must navigate increasing challenges, from cost management to project delays. Fortunately, digital tools now offer valuable insights to help mitigate these risks. However, the sheer volume of tools and the complexity of leveraging their data effectively can be daunting. That’s where data-driven construction comes in.

Project

Knowledge Hut

DECEMBER 19, 2023

Java, as the language of digital technology, is one of the most popular and robust of all software programming languages. It is ideal for cross-platform applications because it is a compiled language with object code that can work across more than one machine or processor. All programming is done using coding languages. Java, like Python or JavaScript, is a coding language that is highly in demand.

Java 98

Java Programming Language Database Architecture 98