Uber Engineering
NOVEMBER 1, 2016
Uber Engineering’s Schemaless storage system powers some of the biggest services at Uber, such as Mezzanine. Schemaless is a scalable and highly available datastore on top of MySQL ¹ clusters. Managing these clusters was fairly easy when we had … The post Dockerizing MySQL at Uber Engineering appeared first on Uber Engineering Blog.
ProjectPro
FEBRUARY 8, 2016
Different types, types, and stages of data analysis have emerged due to the big data revolution. Data analytics is booming in boardrooms worldwide, promising enterprise-wide strategies for business success. But, what do these imply for businesses? Gaining the correct information, which produces knowledge, allows businesses to create a competitive edge, which is essential to companies successfully leveraging Big Data.
Zalando Engineering
DECEMBER 18, 2016
It’s the event that many of our teams anticipate throughout the entire year – our annual Zalando Hack Week. 2016 sees us branching out even further than past events , hosting parallel locations in Berlin , Dortmund , Dublin , and Helsinki. Over 1,600 technologists across three countries are putting their heads together to hack and innovate the fashion ecosystem, draped in this year’s playful theme of Gaming Allstars.
nodeSWAT
MARCH 30, 2016
Most web applications rely on some sort of access control to keep users from accessing information not meant for them. If authentication is a lock on the main door of the hotel, then access control is the individual access card they give to each user for accessing their room. We have spent quite a few blog posts on various theories about security mechanisms for web applications ( Set Up a Secure Node.js Web Application , Unvalidated Redirects , What Do You Know About Clickjacking?
Advertisement
Whether you’re creating complex dashboards or fine-tuning large language models, your data must be extracted, transformed, and loaded. ETL and ELT pipelines form the foundation of any data product, and Airflow is the open-source data orchestrator specifically designed for moving and transforming data in ETL and ELT pipelines. This eBook covers: An overview of ETL vs.
Uber Engineering
JULY 26, 2016
By Evan Klitzke. Introduction. The early architecture of Uber consisted of a monolithic backend application written in Python that used Postgres for data persistence. Since that time, the architecture of Uber has changed significantly, to a model of microservices and … The post Why Uber Engineering Switched from Postgres to MySQL appeared first on Uber Engineering Blog.
Uber Engineering
MAY 17, 2016
Our last article on Uber for Business (U4B) discussed implementing our ride profiles feature. Today, we want to share more insight into the kind of technical challenges we’re solving. Just because our mission is to improve businesses’ experiences doesn’t … The post How Uber for Business Engineering Verifies Your Ride in Real Time appeared first on Uber Engineering Blog.
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
Uber Engineering
JUNE 23, 2016
In 2014, Uber began expanding ever rapidly. Our platform grew from about 60 cities to 100 in the spring, and then to 200 in the fall. Meanwhile, our fastest growing cities were among our oldest. As the number of additional … The post Uber Engineering’s Micro Deploy: Deploying Daily with Confidence appeared first on Uber Engineering Blog.
Uber Engineering
APRIL 20, 2016
A few months back, we discussed Uber’s decision to abandon its monolithic codebase in favor of a modular, flexible microservice architecture. Since then, we’ve devoted many thousands of engineering hours to expanding this ecosystem of Uber microservices (several hundred … The post Rewriting Uber Engineering: The Opportunities Microservices Provide appeared first on Uber Engineering Blog.
Uber Engineering
JANUARY 19, 2016
The details and examples of Schemaless triggers, a key feature of the datastore that’s kept Uber Engineering scaling since October 2014. This is the third installment of a three-part series on Schemaless; the first part is a design overview … The post Using Triggers On Schemaless, Uber Engineering’s Datastore Using MySQL appeared first on Uber Engineering Blog.
Uber Engineering
DECEMBER 20, 2016
Why Uber Started Over. Uber is based on a simple concept: push a button, get a ride. What started as a way to request premium black cars now offers a range of products, coordinating millions of rides per day across … The post Engineering the Architecture Behind Uber’s New Rider App appeared first on Uber Engineering Blog.
Speaker: Tamara Fingerlin, Developer Advocate
In this new webinar, Tamara Fingerlin, Developer Advocate, will walk you through many Airflow best practices and advanced features that can help you make your pipelines more manageable, adaptive, and robust. She'll focus on how to write best-in-class Airflow DAGs using the latest Airflow features like dynamic task mapping and data-driven scheduling!
Uber Engineering
DECEMBER 6, 2016
Cherami is a distributed, scalable, durable, and highly available message queue system we developed at Uber Engineering to transport asynchronous tasks. We named our task queue after a heroic carrier pigeon with the hope that this system would be just … The post Cherami: Uber Engineering’s Durable and Scalable Task Queue in Go appeared first on Uber Engineering Blog.
Uber Engineering
JANUARY 15, 2016
How Uber’s infrastructure works with Schemaless, the datastore using MySQL that’s kept Uber Engineering scaling since October 2014. This is part two of a three- part series on Schemaless; part one is on designing Schemaless. In Project Mezzanine: … The post The Architecture of Schemaless, Uber Engineering’s Trip Datastore Using MySQL appeared first on Uber Engineering Blog.
Uber Engineering
DECEMBER 8, 2016
As Uber continues to scale, our systems generate continually more events, interservice messages, and logs. Those data needs go through Kafka to get processed. How does our platform audit all these messages in real time? To monitor our Kafka pipeline … The post Introducing Chaperone: How Uber Engineering Audits Apache Kafka End-to-End appeared first on Uber Engineering Blog.
ProjectPro
DECEMBER 22, 2016
This list of data analyst interview questions is based on the responsibilities handled by data analysts.However, the questions in a data analytic job interview may vary based on the nature of work expected by an organization. If you are planning to appear for a data analyst job interview, these interview questions for data analysts will help you land a top gig as a data analyst at one of the top tech companies.
Advertisement
Many software teams have migrated their testing and production workloads to the cloud, yet development environments often remain tied to outdated local setups, limiting efficiency and growth. This is where Coder comes in. In our 101 Coder webinar, you’ll explore how cloud-based development environments can unlock new levels of productivity. Discover how to transition from local setups to a secure, cloud-powered ecosystem with ease.
ProjectPro
DECEMBER 6, 2016
News on Hadoop-November 2016 Microsoft's Hadoop-friendly Azure Data Lake will be generally available in weeks. TechRepublic.com, November 2, 2016. Microsoft's cloud-based Azure Data Lake will soon be available for big data analytic workloads. Azure Data Lake will have 3 important components -Azure Data Lake Analytics, Azure Data Lake Store and U-SQL.
Zalando Engineering
DECEMBER 6, 2016
I recently had the opportunity to be a mentor at the Hack Like A Girl Hackathon. It was organized by Geek Girls Carrots Berlin , with Zalando on board as a sponsor. The hackathon had a health and fitness theme, and I jumped at the chance to volunteer for the event. Before the hacking was to begin, participants and mentors met up at the Native Intruments offices on Friday evening to brainstorm their health and fitness related hacks for the weekend.
Zalando Engineering
NOVEMBER 15, 2016
After seeing the State of JavaScript survey make headway online, we thought it would be interesting to do the same for Zalando to get a better understanding about what technologies and frameworks we’re using here. For the JavaScript survey, participating developers answered questions on topics ranging from frontend frameworks and state management, to build tools and testing libraries.
ProjectPro
NOVEMBER 10, 2016
Having worked your way up in the IT totem pole in the same job role, you have decided this is the best to find new horizons, new environment and a new gig in the big data domain. Starting a new career is exciting but it is not easy as lot of analysis goes into choosing a new career path. Let’s help you out with some detailed analysis on the career path taken by hadoop developers so you can easily decide on the career path you should follow to become a Hadoop developer.
Advertisement
Large enterprises face unique challenges in optimizing their Business Intelligence (BI) output due to the sheer scale and complexity of their operations. Unlike smaller organizations, where basic BI features and simple dashboards might suffice, enterprises must manage vast amounts of data from diverse sources. What are the top modern BI use cases for enterprise businesses to help you get a leg up on the competition?
Zalando Engineering
NOVEMBER 9, 2016
This year was RecSys’ 10th anniversary, and we were lucky enough to be in attendance. The evolution of the conference is great to see – it has grown from a mid-size event beginning in 2009, to an ever-sold-out conference in the past few years, with one of the bests mixes of industry and academia you can find. It also has an open and welcoming community, where the standard of work being presented is incredibly high.
Zalando Engineering
NOVEMBER 3, 2016
Our team have been building data science products together for almost one year now. We believe that data scientists and data engineers should work closely together, yet we understand the differences in the environments and tasks that each of us perform. In this vein, we have iterated the way we do data science, specially early stage exploratory data science with large datasets.
ProjectPro
OCTOBER 31, 2016
Having complete diverse big data hadoop projects at ProjectPro, most of the students often have these questions in mind – “How to prepare for a Hadoop job interview?” “Where can I find real-time or scenario-based hadoop interview questions and answers for experienced?” While we have answered the first question for our students through a series of blog posts on commonly asked Hadoop Interview Questions and tips for preparing a Hadoop resume.
Zalando Engineering
OCTOBER 24, 2016
At Zalando AdTech Lab Hamburg, we develop techniques to predict the future behavior of consumers based on their interaction history on Zalando with the goal of delivering better consumer experiences. Recurrent neural networks (RNNs) , one of the major classes of deep learning methods, offer us two benefits: (1) By processing raw consumer histories, RNNs provide accurate prediction models without requiring tedious feature engineering efforts. (2) Furthermore, we can interpret their reasoning when
Speaker: Jay Allardyce, Deepak Vittal, Terrence Sheflin, and Mahyar Ghasemali
As we look ahead to 2025, business intelligence and data analytics are set to play pivotal roles in shaping success. Organizations are already starting to face a host of transformative trends as the year comes to a close, including the integration of AI in data analytics, an increased emphasis on real-time data insights, and the growing importance of user experience in BI solutions.
ProjectPro
OCTOBER 14, 2016
Hadoop certifications are recognized in the industry as a confident measure of capable and qualified big data experts. Career Counsellors at ProjectPro often receive at least one question a week relating to the specific topic of Hadoop certifications whether it is from prospects who want to learn hadoop or from students who are learning hadoop. Some of the commonly asked questions are - “Is hadoop certification worth the investment?
Zalando Engineering
OCTOBER 13, 2016
With users in 15 European countries, from the southwest of Spain to the northeast of Norway, we have to cater to a variety of shopping behaviours and needs. At Onsite Management, we strive to localise our international websites and apps to build an experience that our users can trust and relate to, culturally. Localisation spans an array of projects and processes, such as understanding the local users performing qualitative and quantitative research, or optimizing elements such as navigation or
ProjectPro
OCTOBER 3, 2016
News on Hadoop-September 2016 HPE adapts Vertica analytical database to world with Hadoop, Spark.TechTarget.com,September 1, 2016. To compete in a field of diverse data tools, Vertica 8.0 has expanded its analytical database support for Apache Hadoop and Spark integration and also to enhance Apache Kafka management pipeline. Vertical analytic platform could access hadoop data before but with Vertica 8.0 the analytical engine can now directly work with data stored in hadoop without worrying about
Zalando Engineering
SEPTEMBER 20, 2016
React has become widely popular within Zalando, making our attendance at this year’s ReactEurope Conference in Paris a no-brainer. With around 800 like-minded developers and techies, this was set to be a great place to share some hands-on experiences with React. A lot of this year’s talks were centered around React Native and GraphQL. Let’s take a quick look into them below.
Speaker: Nikhil Joshi, Founder & President of Snic Solutions
Is your manufacturing operation reaching its efficiency potential? A Manufacturing Execution System (MES) could be the game-changer, helping you reduce waste, cut costs, and lower your carbon footprint. Join Nikhil Joshi, Founder & President of Snic Solutions, in this value-packed webinar as he breaks down how MES can drive operational excellence and sustainability.
ProjectPro
SEPTEMBER 16, 2016
With increased enterprise adoption of Hadoop, organizations are in need of hadoop administrators to take care of the large hadoop clusters they have. The job role of a Hadoop administrator is strong and the job’s outlook is healthy, with an average of 4300 hadoop admin jobs in US as of 13 th Sept, 2016. A hadoop administrator’s job role is unspectacular but is necessary to keep hadoop clusters running smoothly in production, so a hadoop admin job involves being the nuts and bolts of
Zalando Engineering
AUGUST 15, 2016
Where in the world should you head to if you want to soak up the energy, innovation, and entrepreneurial success of the tech industry? If you’re looking to get amongst it, consider this your personal travel guide to the cities that are making waves in tech right now. Moscow If low living costs are still a priority, then Moscow should definitely be at the top of your tech hub list.
ProjectPro
AUGUST 9, 2016
After taking comprehensive hands-on hadoop training, the placement season is finally upon you. You applied for a Cognizant Hadoop Job interview and fortunately, were shortlisted. It is just the technical hadoop job interview that separates you from your big data career. At this step, the help that you can get from your hadoop developer resume is over and it’s up to you how you drive your hadoop developer interview and whether you get selected for the hadoop job.
Zalando Engineering
AUGUST 2, 2016
The Ruhr Area is the biggest metropolitan area in Germany, with nearly 5.4 million inhabitants and of course, home to Zalando Tech Dortmund. Being the former industrial heart of Germany, it has made a terrific structural change towards services and information technologies in the last few years. To reflect the growing impact of the Ruhr Area in the IT-sector and to strengthen the frontend community, RuhrJS was founded, an international JavaScript conference taking place from 2-3 July.
Speaker: Anindo Banerjea, CTO at Civio & Tony Karrer, CTO at Aggregage
When developing a Gen AI application, one of the most significant challenges is improving accuracy. This can be especially difficult when working with a large data corpus, and as the complexity of the task increases. The number of use cases/corner cases that the system is expected to handle essentially explodes. 💥 Anindo Banerjea is here to showcase his significant experience building AI/ML SaaS applications as he walks us through the current problems his company, Civio, is solving.
Expert insights. Personalized for you.
111 
Let's personalize your content