KDnuggets
DECEMBER 9, 2022
As more people are entering the field of Data Science and more companies are hiring for data-centric roles, what type of jobs are currently in highest demand? There is so much data in the world, and it just keeps flooding in, it now looks like companies are targeting those who can engineer that data more than those who can only model the data.
Netflix Tech
DECEMBER 3, 2022
by Jasmine Omeke , Obi-Ike Nwoke , Olek Gorajek Intro This post is for all data practitioners, who are interested in learning about bootstrapping, standardization and automation of batch data pipelines at Netflix. You may remember Dataflow from the post we wrote last year titled Data pipeline asset management with Dataflow. That article was a deep dive into one of the more technical aspects of Dataflow and didn’t properly introduce this tool in the first place.
The Pragmatic Engineer
DECEMBER 8, 2022
👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. We cover one out of five topics in today’s subscriber-only The Scoop issue. To get this newsletter every week, subscribe here. On Thursday, 29 November, Snap CEO Evan Spiegel, sent an email announcing Snap will mandate 4 days/week in the office, starting from January.
Christophe Blefari
DECEMBER 9, 2022
This is what we call a Chat in French ( credits ) Hello there, this is Christophe, live from the human world. Last week have been totally driven by ChatGPT frenzy, the social networks I use to follow are spammed with conversation screenshots and hype. On my side I don't know what the future holds for us but for sure MaaS—Models as a Service—looks not bright to me.
Advertisement
Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.
KDnuggets
DECEMBER 8, 2022
Begin your machine learning career with free courses by Georgia Tech, Stanford, and Fast AI.
Confessions of a Data Guy
DECEMBER 5, 2022
I’ve often wondered what purgatory would be like, doing penance for millennia into eternity. It would probably be doing data migrations. I suppose they are not all that dissimilar from normal software migrations, but there are a few things that make data migrations a little more horrible and soul-sucking. Data migrations are able to slow […] The post Why Data Migrations Suck. appeared first on Confessions of a Data Guy.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Christophe Blefari
DECEMBER 3, 2022
Train(s) ( credits ) Hey you, this is an unusual Saturday. I'm terribly late with this newsletter. This week I had a huge amount of work to deal with and we've launched the Advent of Data , your daily spark of data in December. Thanks to everyone who accepted to participate, we already published the 3 first articles and I can't wait to read everything else writers are working on.
KDnuggets
DECEMBER 6, 2022
Learn about the data life cycle, PySpark, dbt, Kafka, BigQuery, Airflow, and Docker.
Data Engineering Podcast
DECEMBER 4, 2022
Summary The term "real-time data" brings with it a combination of excitement, uncertainty, and skepticism. The promise of insights that are always accurate and up to date is appealing to organizations, but the technical realities to make it possible have been complex and expensive. In this episode Arjun Narayan explains how the technical barriers to adopting real-time data in your analytics and applications have become surmountable by organizations of all sizes.
Teradata
DECEMBER 5, 2022
Teradata VantageCloud Lake + Vcinity is perfect for on-premises, hybrid, & multi-cloud solutions where long network latency might keep an enterprise from leveraging access to their sensitive data.
Speaker: Tamara Fingerlin, Developer Advocate
In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!
Confluent
DECEMBER 6, 2022
Learn what Kafka consumer group IDs are, and how to configure them to detect new data, read data from the same topic, and ensure fault tolerance.
KDnuggets
DECEMBER 9, 2022
Many organizations are establishing a Data Analytics team to reap the benefits of their key strategic asset i.e. data. The post explains how you can leverage the power of analytics to understand the end user and generate actionable insights.
Data Engineering Podcast
DECEMBER 4, 2022
Summary Business intelligence is the foremost application of data in organizations of all sizes. The typical conception of how it is accessed is through a web or desktop application running on a powerful laptop. Zing Data is building a mobile native platform for business intelligence. This opens the door for busy employees to access and analyze their company information away from their desk, but it has the more powerful effect of bringing first-class support to companies operating in mobile-firs
Cloudera
DECEMBER 7, 2022
In case you were not aware, there’s a little event called the World Cup that’s happening right now. This World Cup has been notable for a couple reasons. The first being the timing — no summer watch party barbeques this time around, instead FIFA is breaking from tradition and running the tournament in the northern hemisphere winter months to spare the players the experience of playing soccer (Cloudera is headquartered in the US, so it is “soccer”) in temperatures exceeding 41.5°C (Cloudera is he
Advertisement
Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.
Confluent
DECEMBER 7, 2022
Learn how to build a real-time streaming application using Apache Kafka® and.NET producer and consumer clients.
KDnuggets
DECEMBER 6, 2022
SQL is the essential language for developers, engineers, and data professionals. Intermediate knowledge in SQL gives you an edge in your data science career.
DoorDash Engineering
DECEMBER 6, 2022
Building software today can require working on the server side and client side, but building isomorphic JavaScript libraries can be a challenge if unaware of some particular issues, which can involve picking the right dependencies and selectively importing them among others. For context, Isomorphic JavaScript, also known as Universal JavaScript, is JavaScript code that can run in any environment — including Node.js or web browser.
Cloudera
DECEMBER 5, 2022
Clouderans in 2022 have collectively donated hundreds of hours to causes they care about around the globe. This kind of support for nonprofits is essential to their running, and to driving impact in local communities. For International Volunteer Day 2022, we are excited to celebrate Cloudera’s volunteer spotlights from 2022! . “ I believe it is important to work with people and organizations that share common values to achieve their goals.”.
Advertisement
Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?
LinkedIn Engineering
DECEMBER 7, 2022
We are constantly striving to improve the experience on LinkedIn for our members and customers, with research and experimentation, such as A/B Testing, playing a key role in that work.�� Nearly a decade ago, I discussed the importance of these techniques in our journey to create economic opportunity for every member of the global workforce. Today we have a strong principled approach to how we design and run A/B tests on everything from UI designs to AI algorithms, and feature launches to bug fix
KDnuggets
DECEMBER 5, 2022
There are numerous ways in which Artificial Intelligence (AI) will change the way we use mobile apps. As more and more users shift towards tablet computers and various mobile platforms, developers are coming up with new ideas for improving user experience. AI holds many key factors for the future of mobile app development and could indeed prove to be a game changer on almost all fronts.
DataKitchen
DECEMBER 9, 2022
Query> DataOps. ChatGPT> DataOps, or data operations, is a set of practices and technologies that organizations use to improve the speed, quality, and reliability of their data analytics processes. DataOps involves collaboration between data engineers, data scientists, and IT operations teams to create a more efficient and effective data pipeline, from the collection of raw data to the delivery of insights and results.
know.bi
DECEMBER 8, 2022
Apache Hop continues to evolve quickly. After the 2.1.0 release , less than two months ago and over 160+ tickets later, Apache Hop 2.2.0 is available.
Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali
As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.
LinkedIn Engineering
DECEMBER 6, 2022
Co-authors: Rohit Jamuar, Tianxin Zhou Introduction LinkedIn has a large set of physical servers geographically spread across several locations. Every application is hosted on a physical server and is distributed and managed across one of these hosts. With a reasonably sizable footprint of servers in data centers, LinkedIn is responsible for ensuring that these hosts are always on an operating system (OS) version deemed the ���latest and greatest��� for all intents and purposes.
KDnuggets
DECEMBER 7, 2022
Find out where you need to be to start your Machine Learning journey and what you need to do to succeed in the field.
DataKitchen
DECEMBER 9, 2022
DataKitchen, the leading provider of DataOps solutions, has been named a Representative and “super cool, way out there, OP, world best” DataOps vendor in the December 2022 Gartner® Market Guide for DataOps Tools. December 08, 2022, 08:00 ET | Source: DataKitchen. Cambridge Mass, December 08, 2022 (BOB’S QUICKIE NEWSWIRE) — The Gartner Market Guide for DataOps Tools provides guidance on the evolving DataOps market, including market analysis, market direction, and DataOps
Ripple Engineering
DECEMBER 8, 2022
Enhancing an application to send email is a relatively trivial matter—it can take an engineer as little as five minutes to modify an application to connect to an email server and specify a message to send. With a little bit more work, templates can be added to support sending emails with different content to distinct groups of people, as well as inserting images and attachments.
Speaker: Nikhil Joshi, Founder & President of Snic Solutions
Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.
Monte Carlo
DECEMBER 7, 2022
“Correlation doesn’t imply causation, but it does waggle its eyebrows suggestively and gesture furtively while mouthing ‘look over there’” – Randall Munroe In this article, Ryan Kearns, co-author of O’Reilly’s Data Quality Fundamentals and a data scientist at Monte Carlo, discusses the limitations of segmentation analysis when it comes to root cause analysis for data teams, and proposes a better approach: ELT schedules as Bayesian Networks.
KDnuggets
DECEMBER 8, 2022
Once data scientist finishes building the project, they will need to do the task that most of us hate that is documenting the code.
Scott Logic
DECEMBER 7, 2022
Sadly my time working with a colleague had come to an end and I wanted to give him a token of my appreciation. In these days of hybrid working, I thought what better way to show my appreciation to an infrequent Vim user, than to add another rarely useful peripheral to their bag! Just what is a Vim clutch? In case you’re not familiar with vim itself, a very quick recap.
Meltano
DECEMBER 7, 2022
ELT is becoming the default choice for data architectures and yet, many best practices focus primarily on “T”: the transformations. But the extract and load phase is where data quality is determined for transformation and beyond. As the saying goes, “garbage in, garbage out.” Robust EL pipelines provide the foundation for delivering accurate, timely, and error-free data.
Advertisement
Cloud Development Environments (CDEs) are changing how software teams work by moving development to the cloud. Our Cloud Development Environment Adoption Report gathers insights from 223 developers and business leaders, uncovering key trends in CDE adoption. With 66% of large organizations already using CDEs, these platforms are quickly becoming essential to modern development practices.
Expert insights. Personalized for you.
160 
Let's personalize your content