Confessions of a Data Guy
JULY 24, 2024
I’ve had something rattling around in the old noggin for a while; it’s just another strange idea that I can’t quite shake out. We all keep hearing about Arrow this and Arrow that … seems every new tool built today for Data Engineering seems to be at least partly based on Arrow’s in-memory format. So, […] The post PyArrow vs Polars (vs DuckDB) for Data Pipelines. appeared first on Confessions of a Data Guy.
KDnuggets
JULY 24, 2024
From the soft tools to the hard tools, these are what make a data scientist successful.
databricks
JULY 24, 2024
Dataricks is thrilled to announce the General Availability (GA) of Primary Key (PK) and Foreign Key (FK) constraints, starting in Databricks Runtime 15.2.
KDnuggets
JULY 24, 2024
Setup the environment, load the data, perform data analysis and visualization, and create the data pipeline all using Julia programming language.
Advertisement
Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.
Zalando Engineering
JULY 24, 2024
A disrupted gaming night I do not usually read code when dealing with production incidents, as it is one of the slower ways to understand and mitigate what is happening. But on that Friday night, I was glad I did. I was about to start another session of Elden Ring (a video game in which everything is pretty much trying to kill the player) when I was paged with the following: "campaign service is consuming all resources we throw at it".
Snowflake
JULY 24, 2024
Let’s say you are building a house that you plan to put up for sale. You focus on an amazing design, beautiful entry, large windows for plenty of sunlight — things that will create a delightful experience for your future buyer. At the same time, the house also needs less glamorous but vitally important infrastructure, like plumbing, running water, electricity, heating, cooling and so on.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Towards Data Science
JULY 24, 2024
Hierarchies play a crucial role in dimensional modeling for Data Warehouses. See my recommendations on how to handle them effectively.
Snowflake
JULY 24, 2024
Let’s say you are building a house that you plan to put up for sale. You focus on an amazing design, beautiful entry, large windows for plenty of sunlight — things that will create a delightful experience for your future buyer. At the same time, the house also needs less glamorous but vitally important infrastructure, like plumbing, running water, electricity, heating, cooling and so on.
Precisely
JULY 24, 2024
Oracle’s recent decision to shut down its advertising business marks a significant shift in the ad tech landscape. This move, driven by declining revenues and increasing regulatory pressures, leaves many advertisers seeking alternative solutions to maintain their marketing momentum while leveraging data that adheres to privacy requirements and aligns with consumers’ expectations.
phData: Data Engineering
JULY 24, 2024
Effective marketing is crucial for business growth, yet achieving cost-effective and impactful results from marketing can be challenging for companies of all sizes. Marketing leaders are tasked with driving results and determining the best course of action for their team by asking questions like: How much should we spend on this new campaign? Should we focus on retaining our customers or trying to find new ones?
Speaker: Tamara Fingerlin, Developer Advocate
In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!
Hevo
JULY 24, 2024
Searching for data manually in Snowflake can be very challenging, time-consuming and sometimes frustrating. Snowflake identifies these problems and has developed Universal Search to change the way we search for data. The universal search, built on a powerful snowflake cortex, is designed to make finding data straightforward.
Edureka
JULY 24, 2024
For almost any business to remain competitive in their industry, modern technology is a must. This may even involve storing confidential business data and processes on cloud computing systems. Although digital transformation presents benefits for businesses, it also introduces new risks that require effective management. The possibility of a ransomware attack is one risk that needs to be controlled with Cybersecurity awareness.
Hevo
JULY 24, 2024
While working with huge amounts of data, Data serialization plays an important role in the performance of the system. Data Serialization converts complex data structures, such as graphs, trees, etc., into a format that can be easily stored or transmitted over the network or across different distributed systems and programming languages.
Snowflake
JULY 24, 2024
Over the last year, as Snowflake has focused on putting AI tools in the hands of its customers, we have prioritized easy, efficient and safe enterprise generative AI. With that in mind, we’re happy to announce the general availability of safety guardrails for Snowflake Cortex AI with Cortex Guard, a new feature that enables enterprises to easily implement safeguards that filter out potentially inappropriate or unsafe large language model (LLM) responses.
Advertisement
Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.
Robinhood
JULY 24, 2024
Robinhood was founded on a simple idea: that our financial markets should be accessible to all. With customers at the heart of our decisions, Robinhood is lowering barriers and providing greater access to financial information and investing. Together, we are building products and services that help create a financial system everyone can participate in. … Recently, Robinhood Markets hosted its highly anticipated Annual Women in Tech (WIT) Conference, a day-long event designed to empower and inspi
Snowflake
JULY 24, 2024
Over the last year, as Snowflake has focused on putting AI tools in the hands of its customers, we have prioritized easy, efficient and safe enterprise generative AI. With that in mind, we’re happy to announce the general availability of safety guardrails for Snowflake Cortex AI with Cortex Guard, a new feature that enables enterprises to easily implement safeguards that filter out potentially inappropriate or unsafe large language model (LLM) responses.
Expert insights. Personalized for you.
162 
Let's personalize your content