The Pragmatic Engineer

OCTOBER 13, 2022

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. We cover one out of six topics in today’s subscriber-only The Scoop issue. To get this newsletter every week, subscribe. Between 2010 to 2021, one of the best strategies for maximizing your total compensation as a software engineer was to follow this recipe: Identify late-stage, fast-growing, private companies which seemed close to going public.

Software Engineer 130

Software Engineer Software Engineering Media Engineering 130

KDnuggets

OCTOBER 14, 2022

Leveraging sparse matrix representations for your data when appropriate can spare you memory storage. Have a look at the reasons why, see how to create sparse matrices in with Python, and compare the memory requirements for standard and sparse representations of the same data.

Python 160

Python Data 160

Data Engineering Podcast

OCTOBER 9, 2022

Summary For any business that wants to stay in operation, the most important thing they can do is understand their customers. American Express has invested substantial time and effort in their Customer 360 product to achieve that understanding. In this episode Purvi Shah, the VP of Enterprise Big Data Platforms at American Express, explains how they have invested in the cloud to power this visibility and the complex suite of integrations they have built and maintained across legacy and modern sy

Food 100

Food MongoDB Scala MySQL 100

Teradata

OCTOBER 12, 2022

ClearScape Analytics provides robust functionality giving people across the organization the ability to efficiently execute their roles in the analytics process on a common platform.

Process 105

Process 105

Advertisement

Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?

Business Intelligence

The Pragmatic Engineer

OCTOBER 10, 2022

Part of this article was originally published in The Scoop #27 , for subscribers of The Pragmatic Engineer Newsletter last week. I decided to publish this section for everyone to read after the Business Insider article claiming that 15% of Facebook employees - 12,000 people - may lose their jobs started to spread within the media. The Business Insider article was not specific to software engineers but still spread heavily within tech circles.

Engineering 130

Engineering Software Engineer Software Engineering Recruitment 130

KDnuggets

OCTOBER 11, 2022

A Data Science Enablement Team consists of people from various departments like marketing, sales, product development, etc. They are responsible for providing the necessary tools and resources to help the data scientists do their job more efficiently.

Data Science 160

Data Science Building Data 160

Cloudera

OCTOBER 11, 2022

A recent VentureBeat article , “4 AI trends: It’s all about scale in 2022 (so far),” highlighted the importance of scalability. I recommend you read the entire piece, but to me the key takeaway – AI at scale isn’t magic, it’s data – is reminiscent of the 1992 presidential election, when political consultant James Carville succinctly summarized the key to winning – “it’s the economy”.

Data Science 94

Data Science Aggregated Data Data Consulting 94

AltexSoft

OCTOBER 13, 2022

Take a look at the featured image above. Beautiful, isn’t it? The interesting thing is, it isn’t a painting drawn by some famous artist, nor is it a photo taken by a satellite. The image you see has been generated with the help of Midjourney — a proprietary artificial intelligence program that creates pictures from textual descriptions. Neural nets can create images, video, and audio content that not every person can.

Algorithm 90

Algorithm Deep Learning Machine Learning Datasets 90

KDnuggets

OCTOBER 10, 2022

The only cheat you need for a job interview and data professional life. It includes SQL, web scraping, statistics, data wrangling and visualization, business intelligence, machine learning, deep learning, NLP, and super cheat sheets.

Data Science 160

Data Science Deep Learning Business Intelligence Machine Learning 160

Teradata

OCTOBER 13, 2022

Implementing a cookieless identity solution will help businesses maintain advertising efforts amid the phaseout of third-party cookies.

98

98

Speaker: Jay Allardyce, Deepak Vittal, and Terrence Sheflin

As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.

Data

Netflix Tech

OCTOBER 12, 2022

A conversation between Travis Brooks, Netflix Product Manager for Experimentation Platform, and George Khachatryan, OfferFit CEO Note: I’ve known George for a little while now, and as we’ve talked a lot about the philosophy of experimentation, he kindly invited me to their office (virtually) for their virtual speaker series. We had a fun conversation with his team, and we realized that some parts of it might make a good blog post as well.

Building 77

Building Designing Management Algorithm 77

DataKitchen

OCTOBER 14, 2022

The post Podcast: Scaling DataOps first appeared on DataKitchen.

98

98

KDnuggets

OCTOBER 13, 2022

In NLP we must find a way to represent our data (a series of texts) to our systems (e.g. a text classifier). As Yoav Goldberg asks, "How can we encode such categorical data in a way which is amenable for us by a statistical classifier?" Enter the word vector.

Process 159

Process Data Systems 159

U-Next

OCTOBER 13, 2022

Introduction . The Product Manager is the visionary and leader of the product, who leads a team of designers, engineers, and other stakeholders to build a great product interaction design. It is estimated that companies could increase their profits by more than 34 percent when their Product Manager is “fully optimized.” . The role of a Product Manager goes beyond simply managing requirements and specifications.

Designing 52

Designing Management Certification Project 52

Speaker: Nikhil Joshi, Founder & President of Snic Solutions

Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.

Manufacturing

Rockset

OCTOBER 13, 2022

The climate changed and everyone quickly noticed how expensive Snowflake is. How Snowflake fails - Benn Stancil Why is Snowflake so expensive - Stas Sajin Snowflake performance challenges - Slim Baltagi Ok, so Snowflake is expensive. But what do I do about it? Avoid frequent updates Optimize for cost-per-query with apps running 24x7 Tune slow queries Reduce auto-suspend to 1 or 2 minutes Build Snowflake chargeback dashboards Try third-party cost analyzers Set resource monitors and spend threshol

MongoDB 52

MongoDB BI Kafka Database 52

Picnic Engineering

OCTOBER 13, 2022

We’re excited to announce that Picnic’s Error Prone Support project is now open-source! Last week, we already shared an in-depth overview of how Picnic has adopted Google’s Error Prone static analysis tool for Java code. In short, it allows us to: Improve the consistency and quality of our Java codebases. Introduce custom checks for code (anti-)patterns we value.

Java 52

Java Algorithm Coding Project 52

KDnuggets

OCTOBER 12, 2022

Do you want to learn PyTorch for machine learning and deep learning? Check out this 24 hour long video course with accompanying notes and courseware for free. Did I mention it's free?

Deep Learning 159

Deep Learning Machine Learning IT 159

phData: Data Engineering

OCTOBER 12, 2022

So you’ve heard all the talk around dbt , but now you’re working to determine if you should go with dbt Core or dbt Cloud and you’re wanting to know what advantages dbt Cloud has over the free dbt Core offering. Upon a quick trial and look at dbt Cloud, the primary things you might notice are the IDE as well as the ease of managing deployments. However, dbt Cloud offers you much more than that.

Cloud 52

Cloud Metadata SQL Data Warehouse 52

Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage

When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.

Systems

Striim

OCTOBER 12, 2022

I am excited to share with you that Striim is a proud participant in the Microsoft Intelligent Data Platform partner ecosystem as announced at Microsoft Ignite 2022. We have a history of working with Microsoft to help provide our mutual customers with access to enhanced data insights in real time, allowing them to make decisions the moment data is created.

BI 52

BI Data Integration Government Data 52

Lyft Engineering

OCTOBER 12, 2022

Pain Points of Traditional Automated UI Tests Creating a great modern-day software product requires a shift-left approach to testing by ensuring faster, more frequent, and earlier testing. Shift-left testing is an approach to software testing and system testing in which testing is performed earlier in the lifecycle (i.e., moved left on the project timeline).

Architecture 52

Architecture Coding Utilities Engineering 52

KDnuggets

OCTOBER 11, 2022

This article serves as a beginner’s guide to web scraping using Python and looks at the different frameworks and methods you can use, outlined in simple terms.

Python 160

Python 160

dbt Developer Hub

OCTOBER 11, 2022

IMPORTANT: This document serves as the temporary location for information on how to design and structure your metrics. It is our intention to take this content and turn it into a Guide, like How we structure our dbt projects , but we feel that codifying information in a Guide first requires that metrics be rigorously tested by the community so that best practices can arise.

Designing 52

Designing Finance Datasets Coding 52

Speaker: Donna Laquidara-Carr, PhD, LEED AP, Industry Insights Research Director at Dodge Construction Network

In today’s construction market, owners, construction managers, and contractors must navigate increasing challenges, from cost management to project delays. Fortunately, digital tools now offer valuable insights to help mitigate these risks. However, the sheer volume of tools and the complexity of leveraging their data effectively can be daunting. That’s where data-driven construction comes in.

Project

Pinterest Engineering

OCTOBER 12, 2022

Khubi Shah | (former) Software Engineer Intern, Shopping Content Mining This summer, I had the incredible opportunity to intern at the one and only Pinterest from the new engineering hub in Toronto! I am a final year undergraduate student from the University of Waterloo, majoring in Computer Science with an AI specialization. Growing up, Pinterest was always my go-to social media platform, as it inspired me with new ideas for food, fashion, design, or anything creative!

Software Engineer 52

Software Engineer Software Engineering Engineering Recruitment 52

DataKitchen

OCTOBER 12, 2022

Can you draw a map of all the paths data takes from source systems to production insight delivery? How many tools, technologies, configurations, and paths do your data take during its production process? What is the ‘run-time lineage’ of data in your organization? The post Map and Monitor Your Data Journey first appeared on DataKitchen.

Data 52

Data Technology Systems Process 52

KDnuggets

OCTOBER 14, 2022

Check out this free ebook covering the fundamentals of mathematics for machine learning, as well as its companion website of exercises and Jupyter notebooks.

Machine Learning 160

Machine Learning IT 160

U-Next

OCTOBER 11, 2022

The work on IDS or Intrusion Detection System was done during the years 1984 and 1986. Dorothy Denning and Peter Neumann created the Intrusion Detection Expert System with the initial iteration of the IDS (IDES). IDS is a term used to describe a method that may recognize or detect the existence of invasive activity. . In a larger sense, this refers to all the procedures used to identify the unlawful computer or network usage.

IT 52

IT Systems Machine Learning Data Collection 52

Speaker: Evelyn Chou

Choosing the right business intelligence (BI) platform can feel like navigating a maze of features, promises, and technical jargon. With so many options available, how can you ensure you’re making the right decision for your organization’s unique needs? 🤔 This webinar brings together expert insights to break down the complexities of BI solution vetting.

Business Intelligence

WeCloudData

OCTOBER 11, 2022

Objectives This tutorial is one part of a containers series of tutorials that will walk the reader through installation of tools that can run applications in containers. By the end of these tutorials the reader will be able to Install services (container engines) that can run containers using tools such as LXD/LXC, Docker, or Podman. […] The post Install and Run Containers on Linux Virtual Machines – LXD/LXC appeared first on WeCloudData.

Engineering 52

Engineering 52

Rockset

OCTOBER 11, 2022

Introduction Managing streaming data from a source system, like PostgreSQL, MongoDB or DynamoDB, into a downstream system for real-time analytics is a challenge for many teams. The flow of data often involves complex ETL tooling as well as self-managing integrations to ensure that high volume writes, including updates and deletes, do not rack up CPU or impact performance of the end application.

Data Ingestion 52

Data Ingestion Kafka Relational Database PostgreSQL 52

KDnuggets

OCTOBER 11, 2022

The post explains three popular frameworks, PySpark, Dask, and Ray, and discusses various factors to select the most appropriate one for your project.

Coding 155

Coding Python Project 155

Monte Carlo

OCTOBER 11, 2022

Precise endeavors must be done to exacting standards in clean environments. Surgeons scrub in, rocket scientists work in clean rooms, and data scientists…well we try our best. We’ve all heard the platitude, “garbage in, garbage out,” so we spend most of our time doing the most tedious part of the job: data cleaning. Unfortunately, no matter how hard we scrub, poor data quality is often too pervasive and invasive for a quick shower.

IT 52

IT Datasets Data Warehouse Data Analysis 52

Speaker: Aindra Misra, Senior Manager, Product Management (Data, ML, and Cloud Infrastructure) at BILL

Join us for an insightful webinar that explores the critical intersection of data privacy and AI governance. In today’s rapidly evolving tech landscape, building robust governance frameworks is essential to fostering innovation while staying compliant with regulations. Our expert speaker, Aindra Misra, will guide you through best practices for ensuring data protection while leveraging AI capabilities.

Government