Start Data Engineering
MAY 11, 2022
1. Introduction 2. Objective 3. Design 4. Setup 4.1 Prerequisite 4.2 AWS Infrastructure costs 4.3 Data lake structure 5. Code walkthrough 5.1 Loading user purchase data into the data warehouse 5.2 Loading classified movie review data into the data warehouse 5.3 Generating user behavior metric 5.4. Checking results 6. Tear down infra 7. Design considerations 8.
KDnuggets
MAY 13, 2022
This article is the first in a series of articles looking at the different aspects of k-means clustering, beginning with a discussion on centroid initialization.
Confluent
MAY 12, 2022
Internet connectivity is something we sometimes take for granted. For many, most places we visit, work, or reside have some form of connectivity whether it be cellular, Wi-Fi, fiber, etc. […].
Cloudera
MAY 9, 2022
Tuning Hive on Tez queries can never be done in a one-size-fits-all approach. The performance on queries depends on the size of the data, file types, query design, and query patterns. During performance testing, evaluate and validate configuration parameters and any SQL modifications. It is advisable to make one change at a time during performance testing of the workload, and would be best to assess the impact of tuning changes in your development and QA environments before using them in product
Advertisement
Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.
AltexSoft
MAY 12, 2022
We live in the world of sounds: Pleasant and annoying, low and high, quiet and loud, they impact our mood and our decisions. Our brains are constantly processing sounds to give us important information about our environment. But acoustic signals can tell us even more if analyze them using modern technologies. Today, we have AI and machine learning to extract insights, inaudible to human beings, from speech, voices, snoring, music, industrial and traffic noise, and other types of acoustic signals
KDnuggets
MAY 12, 2022
Learn about the free and easy-to-deploy hosting platform for your machine learning projects.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Data Engineering Podcast
MAY 8, 2022
Summary Many of the events, ideas, and objects that we try to represent through data have a high degree of connectivity in the real world. These connections are best represented and analyzed as graphs to provide efficient and accurate analysis of their relationships. TigerGraph is a leading database that offers a highly scalable and performant native graph engine for powering graph analytics and machine learning.
Cloudera
MAY 13, 2022
Cloudera Data Platform (CDP) unifies the technologies from Cloudera Enterprise Data Hub (CDH) and Hortonworks Data Platform (HDP). A few functionalities that existed in the legacy platforms (HDP and CDH) are substituted by other alternatives based on a detailed and careful analysis. CDH users would have used Fair Scheduler (FS), and HDP users would have used Capacity Scheduler (CS).
KDnuggets
MAY 9, 2022
Read this overview of 12 important machine learning concepts, presented in a no frills, straightforward definition style.
Teradata
MAY 12, 2022
The next generation of customers expects newer technologies & advanced self-service capabilities as the airline business becomes more competitive. How can airlines meet these expectations?
Speaker: Tamara Fingerlin, Developer Advocate
In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!
Data Engineering Podcast
MAY 8, 2022
Summary Dan Delorey helped to build the core technologies of Google’s cloud data services for many years before embarking on his latest adventure as the VP of Data at SoFi. From being an early engineer on the Dremel project, to helping launch and manage BigQuery, on to helping enterprises adopt Google’s data products he learned all of the critical details of how to run services used by data platform teams.
Rockset
MAY 12, 2022
This is the third post in a series by Rockset's CTO Dhruba Borthakur on Designing the Next Generation of Data Systems for Real-Time Analytics. We'll be publishing more posts in the series in the near future, so subscribe to our blog so you don't miss them! Posts published so far in the series: Why Mutability Is Essential for Real-Time Data Analytics Handling Out-of-Order Data in Real-Time Analytics Applications Handling Bursty Traffic in Real-Time Analytics Applications SQL and Complex Queries A
KDnuggets
MAY 10, 2022
This is a list of FREE data science resources and notes that are available online, some of which are provided by universities.
Rock the JVM
MAY 11, 2022
Scala generics are a breeze for Java developers, but what about those coming from Python or JavaScript?
Advertisement
Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.
Monte Carlo
MAY 11, 2022
If you’ve been a data analyst, BI analyst, or general business user of dashboards and reports, you’ve probably asked these questions (and more) before: What’s the most reliable field to use? When was the last time this table was updated? Should there be this many null entries in this column? Who can I reach out to figure out if this data is expected?
Palantir
MAY 11, 2022
(Scroll down for English translation below) Avec l’adoption des accords de Paris en 2016, les institutions du secteur public et privé ont considérablement renforcé leurs ambitions en matière de décarbonation. Plus particulièrement, la capacité des organisations à s’adapter et améliorer leur prise de décision va devenir un élément clé de différenciation et de compétitivité.
KDnuggets
MAY 12, 2022
By implementing the different NLP techniques into the production processes, compliance departments can maintain detailed checks and keep up with regulator demands.
Rockset
MAY 10, 2022
DynamoDB is a popular NoSQL database available in AWS. It is a managed service with minimal setup and pay-as-you-go costing. Developers can quickly create databases that store complex objects with flexible schemas that can mutate over time. DynamoDB is resilient and scalable due to the use of sharding techniques. This seamless, horizontal scaling is a huge advantage that allows developers to move from a proof of concept into a productionized service very quickly.
Advertisement
Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?
dbt Developer Hub
MAY 9, 2022
❓ Who is this for: This is for advanced users of dbt Cloud that are interested in expanding their knowledge of the dbt API via an interactive Postman Collection. We only suggest diving into this once you have a strong knowledge of dbt + dbt Cloud. You have a couple of options to review the collection: get a live version of the collection via. check out the collection documentation to learn how to use it.
KDnuggets
MAY 11, 2022
Save time and effort with this guide, which will show you how to do data join operations in Tableau.
KDnuggets
MAY 9, 2022
Check out this list of free resources, courses, and more to help you become a Data Scientist for free.
KDnuggets
MAY 13, 2022
Predict the performance of your model - before the ground truth is available.
Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali
As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.
KDnuggets
MAY 10, 2022
While it is true that Machine Learning today isn’t ready for prime time in many business cases that revolve around Document Analysis, there are indeed scenarios where a pure ML approach can be considered.
KDnuggets
MAY 11, 2022
If you aren't familiar with Kaggle, you should be. Hear why from two expert Kagglers in this article.
KDnuggets
MAY 11, 2022
The objective of data mesh is to establish coherence between data coming from different domains across an enterprise. The domains are handled autonomously to eliminate the challenges of data availability and accessibility for cross-functional teams.
KDnuggets
MAY 10, 2022
How To Tutorials with SAS data scientists and analytics instructors.
Speaker: Nikhil Joshi, Founder & President of Snic Solutions
Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.
KDnuggets
MAY 13, 2022
In this article, we will build a beginner-friendly machine learning model using TensorFlow.
KDnuggets
MAY 13, 2022
Discover "compound sparsification" and how to apply it to BERT models for 10x compression and GPU-level latency on commodity CPUs.
KDnuggets
MAY 11, 2022
SQL Notes for Professionals: The Free eBook Review; How To Structure a Data Science Project: A Step-by-Step Guide; Everything You Need to Know About Tensors; Free University Data Science Resources; Image Classification with Convolutional Neural Networks (CNNs).
Advertisement
Cloud Development Environments (CDEs) are changing how software teams work by moving development to the cloud. Our Cloud Development Environment Adoption Report gathers insights from 223 developers and business leaders, uncovering key trends in CDE adoption. With 66% of large organizations already using CDEs, these platforms are quickly becoming essential to modern development practices.
Expert insights. Personalized for you.
246 
Let's personalize your content