Data Engineering Podcast

JULY 10, 2022

Summary Building and maintaining reliable data assets is the prime directive for data engineers. In order to make this a tractable problem it is essential that engineers embrace automation at every opportunity. If you’re a data engineering podcast listener, you get credits worth $5,000 when you become a customer.

Data Engineer 100

Data Engineer Data Engineering Engineering MongoDB 100

Data Engineering Podcast

JULY 17, 2022

Summary Data engineering is a large and growing subject, with new technologies, specializations, and "best practices" emerging at an accelerating pace. Sign up free… or just get the free t-shirt for being a listener of the Data Engineering Podcast at dataengineeringpodcast.com/rudder. why not a data science podcast?

Data Engineer 100

Data Engineer Data Engineering Engineering MongoDB 100

Data Engineering Podcast

OCTOBER 30, 2022

It’s hard to have a conversation about data engineering or analysis without mentioning it. With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs.

Engineering 100

Engineering MongoDB Scala MySQL 100

Data Engineering Podcast

OCTOBER 23, 2022

In this episode field CTO Manjot Singh shares his experiences as an early user of MySQL and MariaDB and explains how the suite of products being built on top of the open source foundation address the growing needs for advanced storage and analytical capabilities. Data teams are increasingly under pressure to deliver.

Database 100

Database MySQL Cloud MongoDB 100

Knowledge Hut

SEPTEMBER 24, 2024

We are all aware of how quickly technology is changing the world and that a career as a software engineer will always provide you with new opportunities as you acquire experience and develop your technical skills and abilities. But is software engineering a good career? What is Software Engineering?

Software Engineering 52

Software Engineering Software Engineer Engineering Programming Language 52

Christophe Blefari

FEBRUARY 23, 2024

This week I've participated to a round table about data and did a cool presentation about Engines. The idea was to depict the history of engines over the last 40 years and what leads to polars and DuckDB. Engines evolution (me) There are 3 points that have triggered discussion about the visualisation I done What about Arrow?

Data Lake 130

Data Lake PostgreSQL MongoDB Scala 130

Marc Lamberti

SEPTEMBER 22, 2023

With the PythonOperator, we can use the keyword return along with the value in the Python callable function to push the value we want to share into the database and create the corresponding XCom automatically. With the PythonOperator you can access it by passing the parameter ti to the Python callable function. In Airflow 1.10.x,

MySQL 246

MySQL Data Pipeline Database Python 246

Knowledge Hut

SEPTEMBER 24, 2024

A software engineer resume is a resume that specifically highlights the skills and experience related to the field of software engineering. It is important for a software engineer's resume to be clear and concise, as well as highlight the most relevant skills and experience for the position.

Software Engineering 52

Software Engineering Software Engineer Engineering Programming Language 52

Christophe Blefari

JUNE 21, 2024

One way to read data platforms When we look at platforms history what characterises evolution is the separation (or not) between the engine and the storage. A UX where you buy a single tool combining engine and storage, where all you have to do is flow data in, write SQL, and it's done. —with Databricks you buy an engine.

Metadata 147

Metadata Data Warehouse BI Scala 147

Christophe Blefari

JULY 26, 2024

Joe is a great speaker, he wrote Fundamentals of Data Engineering , which is one of the bibles in data engineering and I can't wait to hear him at Forward Data. Querying 1TB on a laptop with Python dataframes — A benchmark on a 96 GB of memory laptop using DuckDB, DataFusion and Polars. Crazy what we can do nowadays.

MySQL 130

MySQL Data PostgreSQL Datasets 130

Knowledge Hut

MARCH 20, 2024

Technology is advancing so quickly that there will always be chances in tech industries like software engineering for employment and financial gain. There are always positions available for software engineers who perform various duties and responsibilities in multiple businesses. Who is a Software Engineer?

Software Engineering 52

Software Engineering Software Engineer Engineering Java 52

Knowledge Hut

FEBRUARY 6, 2024

What are the tasks of a DevOps Engineer? A Summary of day-to-day tasks carried out by a DevOps engineer - Design, build, test and deploy scalable, distributed systems from development through production Manage the code repository(such as Git, SVN, BitBucket, etc.) Who can become a DevOps Engineer?

Engineering 98

Engineering Programming Language Amazon Web Services Google Cloud 98

Knowledge Hut

DECEMBER 26, 2023

Most Popular Programming Certifications C & C++ Certifications Oracle Certified Associate Java Programmer OCAJP Certified Associate in Python Programming (PCAP) MongoDB Certified Developer Associate Exam R Programming Certification Oracle MySQL Database Administration Training and Certification (CMDBA) CCA Spark and Hadoop Developer 1.

Certification 98

Certification Programming MongoDB R (Programming) 98

Knowledge Hut

NOVEMBER 28, 2023

Software engineering has become a prevalent and rewarding career option in the past two to three decades. This has heightened the demand, along with an increase in software engineer roles and responsibilities. What is Software Engineering? Standard engineering principles are applied during the software engineering process.

Software Engineering 52

Software Engineering Software Engineer Engineering Python 52

Data Engineering Podcast

AUGUST 28, 2022

Summary The dream of every engineer is to automate all of their tasks. For data engineers, this is a monumental undertaking. Orchestration engines are one step in that direction, but they are not a complete solution. __init__ covers the Python language, its community, and the innovative ways it is being used.

Data Engineer 100

Data Engineer Data Engineering MongoDB Metadata 100

Knowledge Hut

JUNE 23, 2023

Data engineers make a tangible difference with their presence in top-notch industries, especially in assisting data scientists in machine learning and deep learning. Let us understand here the complete big data engineer roadmap to lead a successful Data Engineering Learning Path.

Data Engineer 52

Data Engineer Data Engineering Engineering Non-relational Database 52

Data Engineering Podcast

DECEMBER 11, 2022

Planetscale is a serverless option for your MySQL workloads that lets you focus on your applications without having to worry about managing the database or fight with differences between development and production. Datafold built automated regression testing to help data and analytics engineers deal with data quality in their pull requests.

Database 100

Database MySQL Data Lake MongoDB 100

Data Engineering Podcast

MARCH 25, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Businesses that adapt well to change grow 3 times faster than the industry average. RudderStack Transformations lets you customize your event data in real-time with your own JavaScript or Python code.

MySQL 130

MySQL Python Architecture Machine Learning 130

LinkedIn Engineering

APRIL 18, 2023

At LinkedIn, site engineers like to automate operational tasks at various infrastructure layers to minimize manual interventions, which can scale well and be easy to operate. The Zen of python) by Tim Peters. Certain automations are performed via onDemand job executions. cfengine & puppet.

MySQL 103

MySQL Python Bytes Kafka 103

Data Engineering Podcast

NOVEMBER 20, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. Closing Announcements Thank you for listening!

Data Lake 130

Data Lake Data Ingestion MongoDB Google Cloud 130

Data Engineering Podcast

NOVEMBER 6, 2022

Summary Despite the best efforts of data engineers, data is as messy as the real world. With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs.

MongoDB 130

MongoDB Scala MySQL Machine Learning 130

Data Engineering Podcast

JUNE 12, 2022

From a data engineering perspective that often means things like JSON files, audio or video recordings, images, etc. With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs.

Unstructured Data 130

Unstructured Data MongoDB Scala MySQL 130

Data Engineering Podcast

AUGUST 6, 2022

In this episode Frank Liu explains how the open source Milvus vector database is implemented to speed up machine learning development cycles, how to think about proper storage and scaling of these vectors, and how data engineering and machine learning teams can collaborate on the creation and maintenance of these data sets.

Machine Learning 100

Machine Learning Database MySQL PostgreSQL 100

Engineering at Meta

OCTOBER 24, 2023

We discussed how SCARF saves engineering time by identifying the correct order of tasks for cleaning up a product and how it can be blocked from automating the cleanup when there are intersystem dependencies. Dead code removal at scale SCARF uses CodemodService to create code change requests for engineers to review.

Coding 133

Coding Programming Language Python MySQL 133

Data Engineering Podcast

NOVEMBER 13, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. Closing Announcements Thank you for listening!

Building 130

Building Metadata MongoDB MySQL 130

Data Engineering Podcast

JUNE 19, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. Closing Announcements Thank you for listening!

Metadata 130

Metadata MongoDB Scala MySQL 130

Databand.ai

DECEMBER 13, 2022

The Top 25 Data Engineering Influencers and Content Creators on LinkedIn Ryan Yackel 2022-12-13 10:23:19 Interested in data engineering? LinkedIn is full of influencers sharing new ideas and sparking conversations on all kinds of topics, and data engineering is no exception. You’ve come to the right place. Happy following!

Data Engineer 52

Data Engineer Data Engineering Engineering AWS 52

Knowledge Hut

DECEMBER 5, 2023

If you have a passion for information technology (IT) and dream of turning it into a fulfilling career, ponder the path of a systems engineer. Join us on a detailed exploration of who can pursue a career as a systems engineer and the steps to become one in the year 2024. Who is a System Engineer, and What Do They Do?

Systems 52

Systems Engineering Recruitment Electronics 52

Data Engineering Podcast

JANUARY 22, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Truly leveraging and benefiting from streaming data is hard - the data stack is costly, difficult to use and still has limitations. __init__ covers the Python language, its community, and the innovative ways it is being used.

PostgreSQL 130

PostgreSQL MongoDB MySQL Machine Learning 130

Data Engineering Podcast

DECEMBER 18, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you're ready to build your next pipeline, or want to test out the projects you hear about on the show, you'll need somewhere to deploy it, so check out our friends at Linode. Struggling with broken pipelines? Stale dashboards?

Metadata 130

Metadata Business Intelligence Data Lake BI 130

Snowflake

SEPTEMBER 7, 2023

Snowpark is the set of libraries and runtimes that enables data engineers, data scientists and developers to build data engineering pipelines, ML workflows, and data applications in Python, Java, and Scala. Airbyte enables customers to continuously load data from external APIs and databases into Snowflake.

Amazon Web Services 118

Amazon Web Services AWS Government Python 118

Data Engineering Podcast

JULY 31, 2022

The Arkouda project is a Python interface built on top of the Chapel compiler to bring back those interactive speeds for exploratory analysis on horizontally scalable compute that parallelizes operations on large volumes of data. If you’re a data engineering podcast listener, you get credits worth $5,000 when you become a customer.

Data Analysis 100

Data Analysis MongoDB Algorithm Scala 100

Data Engineering Podcast

OCTOBER 2, 2022

Alvin is a platform that aims to provide a low effort solution for data lineage capabilities focused on simplifying the work of data engineers. Sign up free… or just get the free t-shirt for being a listener of the Data Engineering Podcast at dataengineeringpodcast.com/rudder.

IT 100

IT Food PostgreSQL MongoDB 100

Data Engineering Podcast

JUNE 22, 2021

We spend countless hours designing the data model, updating engine versions, and tuning performance. In this episode he explains how the system works, the challenge of scaling it to work across different database engines, and his hopes for the future of database systems. Write some Python scripts to automate it?

Database 100

Database MySQL PostgreSQL Data Warehouse 100

Striim

SEPTEMBER 26, 2024

By carefully analyzing the data, engineers can extract key features that define a fraudulent user profile and behavior, such as personal information, number of orders, order content, payment history, geolocation, and network activity. Once the data load is completed, the application will change its status from RUNNING to COMPLETED.

MySQL 52

MySQL Datasets Database Retail 52

Data Engineering Podcast

SEPTEMBER 11, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. Sifflet also offers a 2-week free trial.

Data Pipeline 130

Data Pipeline Building MongoDB Scala 130

Edureka

AUGUST 18, 2024

A full-stack Web Developer is a developer or engineer who can develop both the front end and the back end of a website. Databases are utilized in back-end engineering to store and process information. In essence, with a line of code, a full-stack web developer can construct creative ideas from his mind on the computer screen.

Programming Language 98

Programming Language Java Non-relational Database Database 98