Simon Späti

JANUARY 3, 2023

I used VS Code, Sublime, Notepad++, TextMate, and others, but the shortcut with cmd(+shift)+end, jumping with option+arrow-keys from word to word, needed to be faster at some point. I was hitting my limits. Everything I was doing I did decently fast, but I didn’t get any faster. Vim is the only editor you get faster with time. Vim is based solely on shortcuts.

Data Engineer 195

Data Engineer Data Engineering Engineering Coding 195

The Pragmatic Engineer

JANUARY 5, 2023

Originally published on 5 January 2023. 👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. We cover one out of seven topics in today’s subscriber-only The Scoop issue. To get this newsletter every week, subscribe here. For most engineering teams, returning from the winter holiday usually involves gradually getting back into the swing of things.

Pipeline-centric 130

Pipeline-centric Database-centric Coding Accessible 130

KDnuggets

JANUARY 3, 2023

Matplotlib is the most famous and commonly used plotting library in Python. It allows you to create clear and interactive visualizations that make your data easier to understand and your results more concrete.

Python 158

Python IT Data Data Science 158

Confluent

JANUARY 6, 2023

Introducing fully managed Apache Kafka® + Flink for the most robust, cloud-native data streaming platform with stream processing, integration, and streaming analytics in one.

Kafka 145

Kafka Cloud Process Management 145

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

Simon Späti

JANUARY 3, 2023

Throughout my time as a developer, I’ve used VS Code, Sublime, Notepad++, TextMate, and others. But shortcuts like cmd(+shift)+end and jumping with option+arrow-keys from word to word needed to be faster at some point. I was hitting my limits. Everything I was doing I did decently fast, but I didn’t get any faster. I’ve since learned that Vim is the only editor that you get faster using with time.

Coding 130

Coding 130

Christophe Blefari

JANUARY 4, 2023

🎙 A few week ago I did my first podcast with Robin. We talked about data engineering and everything around doing a weekly curation. This is the first episode of Robin's podcast in English and you should follow him because more are coming! In the podcast we talked about 🔥 My journey before launching the newsletter 🔥 Why and how I write 🔥 My main challenges as a Data Engineer 🔥 My favorite contents 🔥 What I like about data 🔥 A few tips f

Data Engineer 100

Data Engineer Data Engineering Engineering Data 100

Precisely

JANUARY 3, 2023

Times are changing, and at a near-constant pace. With shifting customer preferences and disruptive world events shaking up the global supply chain market, many business leaders are left wondering whether they’ll be able to stay competitive. Supply chain automation technologies can have a big role to play when it comes to providing end-to-end visibility and risk mitigation for complex, data-intensive SAP processes in supply chain.

Process 59

Process Data Governance Algorithm Data Integration 59

Simon Späti

JANUARY 3, 2023

In this article, we are going to explore core open-source tools that are needed for any company to become data-driven. We’ll cover integration, transformation, orchestration, analytics, and ML tools as a starter guide to the latest open data stack. Let’s start with the Modern Data Stack. Have you heard of it or where the term came from?

Data 130

Data IT 130

Cloudyard

JANUARY 6, 2023

Read Time: 2 Minute, 32 Second During this post we will discuss multiple scenario on Clustering Tables. We will be analyzing and implementing the following scenarios in this post. Non Cluster to Cluster table : Create Clustering on Normal table and see the partitions pruning. CLONE Cluster table: CLONE the above Clustered table and analyze the Clustering.

IT 52

IT Data 52

KDnuggets

JANUARY 5, 2023

Register now for this webinar on Jan. 12 to learn how to simplify the process of using scientific literature within your AI, machine learning, and text mining projects.

Machine Learning 108

Machine Learning Project Process 108

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Monte Carlo

JANUARY 6, 2023

Understanding and aligning with each business domain’s unique incentives and workflows is what ultimately makes data teams not just efficient, but great. Part one of this series looked at everyone’s favorite spreadsheet power users: the finance team. This article will examine how data teams can better conduct product experimentation and better align with product teams.

Building 52

Building Project BI Data 52

Precisely

JANUARY 6, 2023

Environmental, social, and governance (ESG) initiatives are topics of discussion everywhere – in the workplace, social media, news outlets, and beyond. And for good reason. Recent public advocacy efforts around climate issues, diversity and inclusion, data privacy, and more have been driving forces in pushing ESG to the forefront. While stellar products and services used to be enough for businesses to attract new customers, investors, and employees – and win their loyalty over time – that’s not

Government 52

Government Education Data Integration Media 52

Teradata

JANUARY 6, 2023

As we start 2023, our product marketing team has compiled a list of the top 10 features in Teradata Vantage which have immensely helped our customers and are technological breakthroughs.

Technology 52

Technology 52

KDnuggets

JANUARY 4, 2023

Understanding the concepts behind the micro average, macro average, and weighted average of F1 score in multi-class classification with simple illustrations.

Machine Learning 112

Machine Learning 112

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud

Monte Carlo

JANUARY 4, 2023

As a radiologist-owned alliance built by physicians, Collaborative Imaging knows a thing or two about what it means to be healthy. And the same goes for their data. From revenue cycle management to telehealth, Collaborative Imaging ’s physician-conceived platform is solving some of the biggest technology challenges facing modern medical practices. And with hundreds of hospitals utilizing Collaborative Imaging’s data products to optimize their practices, data quality is paramount for the data tea

Hospitality 52

Hospitality Healthcare Medical Insurance 52

Yelp Engineering

JANUARY 4, 2023

Manually managing the lifecycle of Kubernetes nodes can become difficult as the cluster scales. Especially if your clusters are multi-tenant and self-managed. You may need to replace nodes for various reasons, such as OS upgrades and security patches. One of the biggest challenges is how to terminate nodes without disturbing tenants. In this post, I’ll describe the problems we encountered administering Yelp’s clusters and the solutions we implemented.

Building 52

Building Management IT 52

Precisely

JANUARY 5, 2023

Does your company have a formal data strategy? If so, does that strategy effectively lay out a path toward better business outcomes by helping you optimize your use of data? Although it is a given that data can be one of a company’s real differentiators if used properly, many organizations still do not have a comprehensive data strategy in place.

Data Governance 52

Data Governance Government Data Raw Data 52

KDnuggets

JANUARY 5, 2023

Data merge is a common data processing activity. Learn how Pandas provide various ways to merge our data.

Data Process 129

Data Process Process Data Data Science 129

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

Cloudera

JANUARY 5, 2023

In the past year, businesses who doubled down on digital transformation during the pandemic saw their efforts coming to fruition in the form of cost savings and more streamlined data management. Faced with even more pressure to remain resilient and agile amid looming global economic threats, Asia-Pacific (APAC) region businesses are looking to further mobilize emerging technologies such as artificial intelligence (AI) and machine learning that will optimize operational efficiencies and cost savi

Banking 52

Banking Machine Learning Insurance Data Architecture 52

Striim

JANUARY 4, 2023

Companies need to analyze large volumes of datasets, leading to an increase in data producers and consumers within their IT infrastructures. These companies collect data from production applications and B2B SaaS tools (e.g., Mailchimp). This data makes its way into a data repository, like a data warehouse (e.g., Redshift), and is shown to users via a dashboard for decision-making.

PostgreSQL 52

PostgreSQL Data Warehouse Data Data Lake 52

DoorDash Engineering

JANUARY 4, 2023

In order to inspire DoorDash consumers to order from the platform there are few tools more powerful than a compelling image, which raises the questions: what is the best image to show each customer, and how can we build a model to determine that programmatically using each merchant’s available images? Figure 1: Discovery surfaces with merchant images Out of all the different information presented on the home page (see Figure 1), studies with consumers have repeatedly shown that images play the m

Machine Learning 52

Machine Learning Food Algorithm Building 52

KDnuggets

JANUARY 6, 2023

Learn the syntax and uses of the lambda function, which is an alternative to the regular Python function.

Python 123

Python 123

Speaker: Jay Allardyce, Deepak Vittal, and Terrence Sheflin

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

Precisely

JANUARY 4, 2023

At Precisely, diversity is an inherent aspect of the company culture. Celebrating each other’s differences makes the team stronger as a whole. The Precisely Women in Technology (PWIT) program is a designated space for women in the company to learn from one another, support each other, provide career-growth advice, and share opportunities. Each month, a member of the PWIT group is featured to learn more about their experience as a woman in technology at Precisely.

Computer Science 52

Computer Science Programming Technology Programming Language 52

DataKitchen

JANUARY 4, 2023

Why do 78% of data engineers wish their job came with a therapist to help manage work-related stress? THEY DO NOT TEST. The post ON-DEMAND WEBINAR: Managing Stress in Data Engineering: Data Quality and Testing Techniques for Data Observability first appeared on DataKitchen.

Data Engineer 52

Data Engineer Data Engineering Engineering Management 52

Teradata

JANUARY 4, 2023

From advances in AI/ML to the expansion of satellite/cellular services to expand coverage to remote areas, our tech & industry experts weigh in on game-changing predictions for 2023.

52

52

KDnuggets

JANUARY 2, 2023

This рареr attempts to exploit primarily twо flaws in the Infо-GАN рареr while retаining the оther good qualities improvements.

Datasets 108

Datasets Machine Learning 108

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Workfall

JANUARY 3, 2023

Reading Time: 6 minutes In our previous blog, we demonstrated How to Write Unit Tests for Angular 15 Application Using Jasmine and Enforce Code Quality in a CI Workflow With Github Actions. We looked at general unit tests involving components that receive data from services. It is important to always ensure that Unit Tests are written in order to ensure that the quality of code being shipped to production is guaranteed to be free of any errors or bugs that can be avoided.

Coding 52

Coding SQL Data Validation Utilities 52

Scott Logic

JANUARY 2, 2023

Here are five books that influenced my coding style and working practices early in my career. A top list of anything is deeply personal, so I’ve tried to select books I found both inspirational and informative, with an equal focus on technical and non-technical skills. Personally, I feel the content of these books will most benefit those at the start of their careers.

Entertainment 52

Entertainment Java Coding Designing 52

Acceldata

JANUARY 3, 2023

We released the latest version of Acceldata Data Observability Cloud (ADOC). Version 2.4.1 is now generally available, and it provides enhancements to compute, data reliability, data pipeline management and orchestration, and introduces new monitoring and alerting capabilities.

Cloud 52

Cloud Data Pipeline Data Management 52

KDnuggets

JANUARY 2, 2023

Learn about the most commonly used Streamlit commands and build a customized web application.

Building 123

Building Python 123

Advertisement

Cloud Development Environments (CDEs) are changing how software teams work by moving development to the cloud. Our Cloud Development Environment Adoption Report gathers insights from 223 developers and business leaders, uncovering key trends in CDE adoption. With 66% of large organizations already using CDEs, these platforms are quickly becoming essential to modern development practices.

Cloud