This site uses cookies to improve your experience. To help us insure we adhere to various privacy regulations, please select your country/region of residence. If you do not select a country, we will assume you are from the United States. Select your Cookie Settings or view our Privacy Policy and Terms of Use.
Cookie Settings
Cookies and similar technologies are used on this website for proper function of the website, for tracking performance analytics and for marketing purposes. We and some of our third-party providers may use cookie data for various purposes. Please review the cookie settings below and choose your preference.
Used for the proper function of the website
Used for monitoring website traffic and interactions
Cookie Settings
Cookies and similar technologies are used on this website for proper function of the website, for tracking performance analytics and for marketing purposes. We and some of our third-party providers may use cookie data for various purposes. Please review the cookie settings below and choose your preference.
Strictly Necessary: Used for the proper function of the website
Performance/Analytics: Used for monitoring website traffic and interactions
Summary Collaboration, distribution, and installation of software projects is largely a solved problem, but the same cannot be said of data. Every data team has a bespoke means of sharing data sets, versioning them, tracking related metadata and changes, and publishing them for use in the software systems that rely on them. The CEO and founder of Quilt Data, Kevin Moore, was sufficiently frustrated by this problem to create a platform that attempts to be the means by which data can be as collabo
Computing frameworks like Apache Spark have been widely adopted to build large-scale data applications. For Uber, data is at the heart of strategic decision-making and product development. To help us better leverage this data, we manage massive deployments of Spark … The post JVM Profiler: An Open Source Tool for Tracing Distributed JVM Applications at Scale appeared first on Uber Engineering Blog.
AWS certifications are the most in-demand cloud computing certifications in the IT industry today, with an overwhelming growth in cloud computing. So, for those looking for a career in Amazon Web Services, this blog lists the best AWS certifications available today, including the cost, duration, and topics covered in each certification exam. With everyone from Netflix to American Airlines signing up to the cloud to keep things from crumbling into pieces, organizations are running into a signific
The data science life cycle is generally comprised of the following components: data retrieval data cleaning data exploration and visualization statistical or predictive modeling While these components are helpful for understanding the different phases, they don’t help us think about our programming workflow. Often, the entire data science life cycle ends up as an arbitrary mess of notebook cells in either a Jupyter Notebook or a single messy script.
In Airflow, DAGs (your data pipelines) support nearly every use case. As these workflows grow in complexity and scale, efficiently identifying and resolving issues becomes a critical skill for every data engineer. This is a comprehensive guide with best practices and examples to debugging Airflow DAGs. You’ll learn how to: Create a standardized process for debugging to quickly diagnose errors in your DAGs Identify common issues with DAGs, tasks, and connections Distinguish between Airflow-relate
Authors: Mai N. Nguyen, Accenture & Mitch Gomulinski, Cloudera. Imagine storing the DNA of the entire population of the US – and then cloning them, twice. That’s the equivalent of 1 petabyte ( ComputerWeekly ) – the amount of unstructured data available within our large pharmaceutical client’s business. Then imagine the insights that are locked in that massive amount of data.
The evolution and future of open source at Zalando Open source software has been the core of Zalando’s tech stack since the company’s humble beginnings, selling flip-flops from a basement 10 years ago; it’s part of our DNA as a tech company. For engineering teams at Zalando, open source is a natural part of how we solve problems, we consult and share the TechRadar for guidance on appropriate technologies to use, we contribute to projects such as Kubernetes , and work in the open on a very large
Summary Web and mobile analytics are an important part of any business, and difficult to get right. The most frustrating part is when you realize that you haven’t been tracking a key interaction, having to write custom logic to add that event, and then waiting to collect data. Heap is a platform that automatically tracks every event so that you can retroactively decide which actions are important to your business and easily build reports with or without SQL.
Summary Web and mobile analytics are an important part of any business, and difficult to get right. The most frustrating part is when you realize that you haven’t been tracking a key interaction, having to write custom logic to add that event, and then waiting to collect data. Heap is a platform that automatically tracks every event so that you can retroactively decide which actions are important to your business and easily build reports with or without SQL.
Summary With the increased ease of gaining access to servers in data centers across the world has come the need for supporting globally distributed data storage. With the first wave of cloud era databases the ability to replicate information geographically came at the expense of transactions and familiar query languages. To address these shortcomings the engineers at Cockroach Labs have built a globally distributed SQL database with full ACID semantics in Cockroach DB.
Summary Using a multi-model database in your applications can greatly reduce the amount of infrastructure and complexity required. ArangoDB is a storage engine that supports documents, dey/value, and graph data formats, as well as being fast and scalable. In this episode Jan Steeman and Jan Stücke explain where Arango fits in the crowded database market, how it works under the hood, and how you can start working with it today.
News on Hadoop - May 2018 Data-Driven HR: How Big Data And Analytics Are Transforming Recruitment.Forbes.com, May 4, 2018. With platforms like LinkedIn and Glassdoor giving every employer access to valuable big data, the world of recruitment transforming to intelligent recruitment.HR teams that make use of big data in future are likely to be successful in recruiting the right talent in the coming years.
Gartner’s recently released report “Master Data Management Forms the Basis of a Trusted 360-Degree View of the Customer,” shares the results of an executive survey highlighting several key points, including that customer initiatives, are among CEOs’ top five priorities in 2018. The report includes numerous strategic recommendations and outlines the impact of a Master Data Management (MDM) strategy.
Apache Airflow® 3.0, the most anticipated Airflow release yet, officially launched this April. As the de facto standard for data orchestration, Airflow is trusted by over 77,000 organizations to power everything from advanced analytics to production AI and MLOps. With the 3.0 release, the top-requested features from the community were delivered, including a revamped UI for easier navigation, stronger security, and greater flexibility to run tasks anywhere at any time.
Over the past decade, Cloudera University has taught more than 50,000 developers, administrators, analysts, and data scientists how to apply big data technologies. Developers are learning the APIs, so they can create new applications that were never before possible. Administrators learn to plan, install, monitor, and troubleshoot clusters. And analysts discover the power of SQL over large, diverse datasets.
Sharing our innovation stories: success, failures, and learnings Franzi, Humberto, Neil, Lenia, Vivek. These are just some names of the people who are willing to put in the extra effort and run the additional mile to impact the organization in a way they haven’t done before. The stories of these Zalando intrapreneurs are the ones I summarized at the Innov8rs conference in Madrid.
What new tech employees can expect from Zalando onboarding So, you’ve applied for a technical role at Zalando and you’ve just accepted the offer! If you’re wondering what to expect, look no further. We are excited to share a peek behind the scenes, so you can see what awaits you in the first few weeks of this journey, regardless of whether you’re joining in Berlin, Dortmund, Dublin, Hamburg, Helsinki or Lisbon, to make sure you’re well-equipped to dive into life at Zalando.
How Zalando's overall site speed improved by more than 25% in five months We all know that providing a fast user experience is key. Still, it was somewhat a wake-up call for us last fall when we saw our aggregated loading time increasing; not because we had increased latency in our systems but simply because the share of mobile visits kept increasing.
Speaker: Alex Salazar, CEO & Co-Founder @ Arcade | Nate Barbettini, Founding Engineer @ Arcade | Tony Karrer, Founder & CTO @ Aggregage
There’s a lot of noise surrounding the ability of AI agents to connect to your tools, systems and data. But building an AI application into a reliable, secure workflow agent isn’t as simple as plugging in an API. As an engineering leader, it can be challenging to make sense of this evolving landscape, but agent tooling provides such high value that it’s critical we figure out how to move forward.
We organize all of the trending information in your field so you don't have to. Join 37,000+ users and stay up to date on the latest articles your peers are reading.
You know about us, now we want to get to know you!
Let's personalize your content
Let's get even more personalized
We recognize your account from another site in our network, please click 'Send Email' below to continue with verifying your account and setting a password.
100 
Let's personalize your content