KDnuggets

APRIL 18, 2024

Join us on a journey of becoming a professional MLOps engineer by mastering essential tools, frameworks, key concepts, and processes in the field.

Engineering 145

Engineering Process 145

databricks

APRIL 18, 2024

We are excited to partner with Meta to release the latest state-of-the-art large language model, Meta Llama 3 , on Databricks. With Llama.

Building 143

Building Data Science Data 143

KDnuggets

APRIL 18, 2024

Learning a new skill can be expensive, but it doesn’t have to be.

IT 141

IT 141

Waitingforcode

APRIL 18, 2024

Streaming jobs are supposed to run continuously but it applies to the data processing logic. After all, sometimes you may need to release a new job package with upgraded dependencies or improved business logic. What happens then?

Data Process 130

Data Process Process IT Data 130

Advertisement

Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.

Architecture

Cloudera

APRIL 18, 2024

Artificial intelligence (AI) is something that, by its very nature, can be surrounded by a sea of skepticism but also excitement and optimism when it comes to harnessing its power. With the arrival of the latest AI-powered technologies like large language models (LLMs) and generative AI (GenAI), there’s a vast amount of opportunities for innovation, growth, and improved business outcomes right around the corner.

Metadata 115

Metadata Government Data Governance Data 115

Confessions of a Data Guy

APRIL 18, 2024

It’s probably what every single person wants to accomplish first after they’ve been writing code for a year professionally. How do I get to Senior Engineer? What skills do I need? “I am a good coder, give me the Senior Engineer title.” Sadly, most Junior and Mid-Level Engineers think that being a Senior Engineer is […] The post Transitioning to Senior Engineer appeared first on Confessions of a Data Guy.

Engineering 100

Engineering Coding IT Data 100

Cloudera

APRIL 18, 2024

Regulations often get a bad rap. You may have heard the old idiom “cut the red tape” which means to circumvent obstacles like regulations or bureaucracy. But in many – if not most )– cases the underlying need for regulations outweighs the burden of compliance. In the financial sector, regulations are essential for financial institutions to maintain stability by preventing excessive risk-taking, ensuring adequate capitalization and reducing the likelihood of failures or financial crises.

Government 82

Government Data Security Cloud Banking 82

Confluent

APRIL 18, 2024

Learn how to use kafka-docker-composer, a simple tool to create a docker-compose.yml file for failover testing, to understand cluster settings like Kraft, and for app development.

Kafka 69

Kafka 69

Scott Logic

APRIL 18, 2024

In this episode, Oliver Cronk, Andrew Carr and David Hope talk about the ever-changing world of data, with conversations moving from data warehouse to data lake, and data mesh to data fabric. They discuss the importance of data ownership and common tooling, and their view that data mesh is an approach rather than an architecture.

Data Lake 59

Data Lake Data Warehouse Data Architecture Architecture 59

Confluent

APRIL 18, 2024

Siemens and Brose revolutionized global manufacturing operations with Confluent’s data streaming, increasing its IoT integration and the future of Industry 4.0.

Manufacturing 59

Manufacturing Data IT 59

Speaker: Tamara Fingerlin, Developer Advocate

In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!

Data

Scott Logic

APRIL 18, 2024

The Internal App Portal Project What do you do when you’re a software consultancy that uses small, internally developed applications, and you need to be able to spin those applications up on demand? You build something! Part of life at a consultancy means stints on the bench, allowing us to participate in internal projects. One of those projects is the ‘Internal Application Portal’ (IAP).

AWS 52

AWS Building Consulting Architecture 52

Cloudyard

APRIL 18, 2024

Read Time: 2 Minute, 59 Second Consider a Scenario where the Client manages a large Customer-Invoice details with constantly changing Invoice data. The company uses Snowflake to manage Invoice data in a table named S_INVOICE. This table receives daily loads(history + Current) to reflect changes in customer and invoice partial payment. The goal is to efficiently merge this dynamic data into a target table named S_INVOICE_TARGET while performing Insert operations using stored procedure.

Utilities 52

Utilities Management IT Data 52

RandomTrees

APRIL 18, 2024

In today’s dynamic business landscape, data integration has become a critical component for enterprises to derive meaningful insights and make informed decisions. Among the various tools available for data integration, Informatica and Talend stand out as popular choices, each with its strengths and capabilities. However, migrating from one platform to another can be a daunting task, especially when it involves converting existing code.

Coding 52

Coding Retail Metadata Python 52

DataKitchen

APRIL 18, 2024

Introducing DataKitchen’s Open Source Data Observability Software Today, we announce that we have open-sourced two complete, feature-rich products that solve the data observability problem: DataOps Observervability and DataOps TestGen. With these two products, you will know if your pipelines are running without error and on time and can finally trust your data.

Data Analytics 49

Data Analytics Data SQL Software Engineer 49

Advertisement

Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.

Cloud