Analytics Vidhya

APRIL 3, 2023

Introduction Companies can access a large pool of data in the modern business environment, and using this data in real-time may produce insightful results that can spur corporate success. Real-time dashboards such as GCP provide strong data visualization and actionable information for decision-makers. Nevertheless, setting up a streaming data pipeline to power such dashboards may […] The post Data Engineering for Streaming Data on GCP appeared first on Analytics Vidhya.

Data Engineering 251

Data Engineering Data Engineer Engineering Data 251

The Pragmatic Engineer

APRIL 6, 2023

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. We cover one out of five topics in today’s subscriber-only The Scoop issue. If you’re not yet a full subscriber, you missed this week’s deep-dive into Figma’s engineering culture. To get full newsletters twice a week, subscribe here.

Software Engineer 190

Software Engineer Software Engineering Datasets Database 190

Simon Späti

APRIL 3, 2023

Amidst the excitement and hype surrounding artificial intelligence, the significance of data engineering and its critical foundation—data modeling—can often be overlooked. This article is the first in a three-part series that will shine a spotlight on the fascinating world of data modeling, delving into its crucial importance within the broader context of data engineering.

Data Engineering 130

Data Engineering Data Engineer Engineering Data Architecture 130

Data Engineering Podcast

APRIL 2, 2023

Summary The data ecosystem has been building momentum for several years now. As a venture capital investor Matt Turck has been trying to keep track of the main trends and has compiled his findings into the MAD (ML, AI, and Data) landscape reports each year. In this episode he shares his experiences building those reports and the perspective he has gained from the exercise.

Hadoop 130

Hadoop Machine Learning Python Architecture 130

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

KDnuggets

APRIL 3, 2023

LangChain is a Python library that helps you build GPT-powered applications in minutes. Get started with LangChain by building a simple question-answering app.

Building 158

Building Python Process 158

Engineering at Meta

APRIL 6, 2023

Buck2, our new open source, large-scale build system , is now available on GitHub. Buck2 is an extensible and performant build system written in Rust and designed to make your build experience faster and more efficient. In our internal tests at Meta, we observed that Buck2 completed builds 2x as fast as Buck1. Buck2, Meta’s open source large-scale build system, is now publicly available via the Buck2 website and the Buck2 GitHub repository.

Building 144

Building Systems Java Coding 144

Waitingforcode

APRIL 7, 2023

Last time you discovered the Z-Order compaction in Delta Lake. But guess what? Apache Iceberg also has this feature!

130

130

KDnuggets

APRIL 3, 2023

This article will explain how RAPIDS can help you speed up your next data science workflow. RAPIDS cuDF is a GPU DataFrame library that allows you to produce your end-to-end data science pipeline development all on GPU.

Data Science 145

Data Science Data 145

LinkedIn Engineering

APRIL 4, 2023

Authors: Viktoras Truchanovicius and Selina Zhang At LinkedIn, developer productivity and happiness has always been a priority. It is critical for our engineering leaders to understand how efficiently and effectively their teams are operating to continuously deliver value-added features for our members and build an industry-leading engineering culture.

MySQL 122

MySQL Datasets Software Engineer Software Engineering 122

ArcGIS

APRIL 7, 2023

We're happy to announce the conda init command is now enabled for ArcGIS users of Python! Learn about how to use it, how it works, and benefits.

Python 127

Python IT 127

Speaker: Jay Allardyce, Deepak Vittal, and Terrence Sheflin

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

Confessions of a Data Guy

APRIL 6, 2023

The post QuickSort in Rust! appeared first on Confessions of a Data Guy.

Data 130

Data 130

KDnuggets

APRIL 6, 2023

With more and more companies integrating artificial intelligence into the workplace, what does this mean for employees' futures and careers?

147

147

Preset

APRIL 5, 2023

Entity-centric modeling is a data modeling approach focusing on enriching tabular datasets with useful "features" to enable segmentation, cohort creation, and complex classification analyses easier.

Datasets 111

Datasets Data 111

Snowflake

APRIL 7, 2023

Spring has sprung—and with it comes a new crop of Snowflake Startup Challenge semi-finalists! The 2023 submission pool was the largest to date—twice as many submissions as last year—with entries that spanned not just the globe but the breadth of the Snowflake platform. Our judges put a lot of careful consideration into selecting the top 10, and we offer our sincere thanks to every company that sent in an entry this year—we know how much hard work goes into these submissions, and we appreciate it

Raw Data 111

Raw Data Portfolio Building SQL 111

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

Confluent

APRIL 5, 2023

All the best data streaming resources, tips, and guides to help you learn introductory concepts, streaming architecture basics, common tools and technologies, and more.

Architecture 105

Architecture Data Technology 105

KDnuggets

APRIL 6, 2023

Discover the widely-used open-source frameworks and models for creating your ChatGPT like chatbots, integrating LLMs, or launching your AI product.

Process 145

Process 145

Tweag

APRIL 3, 2023

For a software team to be successful, you need excellent communication. That is why we want to build systems that foster cross-team communication. Using a monorepo is an excellent way to do that. A monorepo provides: Visibility: by seeing the pull requests (PRs) of colleagues, you are easily informed of what other teams are doing. Uniformity: by working in one central repository, it is easier to share the configuration of linters, formatters, etc.

Python 98

Python Coding Project Data Science 98

The Modern Data Company

APRIL 7, 2023

Technical debt is something that many companies are aware of and are attempting to address. It is a big enough issue that several of our recent blog posts ( Lessons in Technical Debt from Southwest Airlines , Start Paying Down Your Technical Debt Today , and A Better Way to Plan the Payoff of Technical Debt) discussed it at length. What about data debt?

Management 98

Management Government Data Data Management 98

Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage

When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.

Systems

Confluent

APRIL 4, 2023

Machine learning and data streaming are a perfect match, but have diverging tech stacks. How can we overcome the pitfalls of SQL and the gulf between languages?

Machine Learning 105

Machine Learning SQL Data 105

KDnuggets

APRIL 6, 2023

I will be sharing a couple of things I have learned in the past six months and tips that helped me stay dedicated and true to my journey in this article.

Data Science 132

Data Science Data 132

LinkedIn Engineering

APRIL 6, 2023

It’s been an exciting few months at LinkedIn, as our engineering and product teams have been working hard to build some new and advanced AI-powered experiences for our members and customers. I have the opportunity to sit at such a unique vantage point where I get to see first hand the work that went into setting the technology foundations - from the technical resources, tools, engineering playgrounds and guidelines - to make it all possible.

Accessible 94

Accessible Accessibility Engineering Technology 94

ArcGIS

APRIL 4, 2023

ArcGIS Pro 3.1 introduces a new layer type—catalog layers—and this blog covers how they could be used in your analytic workflows.

Building 111

Building Datasets Data Management Management 111

Speaker: Donna Laquidara-Carr, PhD, LEED AP, Industry Insights Research Director at Dodge Construction Network

In today’s construction market, owners, construction managers, and contractors must navigate increasing challenges, from cost management to project delays. Fortunately, digital tools now offer valuable insights to help mitigate these risks. However, the sheer volume of tools and the complexity of leveraging their data effectively can be daunting. That’s where data-driven construction comes in.

Project

Lyft Engineering

APRIL 3, 2023

Recommendation plays an important role in Lyft’s understanding of its riders and allows for customizing app experiences to better fulfill their needs. At times, recommendations are also leveraged to manage the marketplace, making sure there’s a healthy balance between ride demand and driver supply. This allows ride requests to be fulfilled with more desirable dispatch outcomes such as matching riders with the best driver nearby.

Systems 87

Systems Pipeline-centric Machine Learning Transportation 87

KDnuggets

APRIL 7, 2023

Utilizing the power of GPT-3.5 to develop a simple summarize generator application.

Python 145

Python Utilities Process 145

Towards Data Science

APRIL 3, 2023

Building a Mental Model for Engineers and Anyone in Between Stream Processing can be handled gently and with care, or wildly, and almost out of control! You be the judge of what future you’d rather embrace. credit: @psalms original_photo Introduction In many cases, processing data in-stream, or as it becomes available, can help reduce an enormous data problem (due to the volume and scale of the flow of data) into a more manageable one.

Process 84

Process Data Lake Systems Bytes 84

ArcGIS

APRIL 6, 2023

Organizations with IFC files can still reap the benefits of an ArcGIS Indoors deployment by following these recommendations.

Data Management 111

Data Management Architecture Management Engineering 111

Speaker: Evelyn Chou

Choosing the right business intelligence (BI) platform can feel like navigating a maze of features, promises, and technical jargon. With so many options available, how can you ensure you’re making the right decision for your organization’s unique needs? 🤔 This webinar brings together expert insights to break down the complexities of BI solution vetting.

Business Intelligence

Cloudera

APRIL 3, 2023

Cloudera Contributors: Ayush Saxena, Tamas Mate, Simhadri Govindappa Since we announced the general availability of Apache Iceberg in Cloudera Data Platform (CDP), we are excited to see customers testing their analytic workloads on Iceberg. We are also receiving several requests to share more details on how key data services in CDP, such as Cloudera Data Warehousing ( CDW ), Cloudera Data Engineering ( CDE ), Cloudera Machine Learning ( CML ), Cloudera Data Flow ( CDF ) and Cloudera Stream Proce

Data Warehouse 81

Data Warehouse Java Metadata Data 81

KDnuggets

APRIL 4, 2023

Learn how Bard, Bing, ChatGPT, GitHub Copilot, and Hugging Face are improving data scientists' work life.

Data Science 129

Data Science Data 129

databricks

APRIL 3, 2023

Introduction According to the latest reports from global consultancy EY, the future of insurance will become increasingly data-driven, and analytics enabled. The recent.

Insurance 86

Insurance Consulting Data 86

Towards Data Science

APRIL 6, 2023

Principles, practices, and examples for ensuring high quality data flows Source: DreamStudio (generated by author) Nearly 100% of companies today rely on data to power business opportunities and 76% use data as an integral part of forming a business strategy. In today’s age of digital business, an increasing number of decisions companies make when it comes to delivering customer experience, building trust, and shaping their business strategy begins with accurate data.

Unstructured Data 79

Unstructured Data Metadata Data Coding 79

Speaker: Aindra Misra, Senior Manager, Product Management (Data, ML, and Cloud Infrastructure) at BILL

Join us for an insightful webinar that explores the critical intersection of data privacy and AI governance. In today’s rapidly evolving tech landscape, building robust governance frameworks is essential to fostering innovation while staying compliant with regulations. Our expert speaker, Aindra Misra, will guide you through best practices for ensuring data protection while leveraging AI capabilities.

Government