Senior Hadoop Developer

Experience: 8 Years

Budget: 1.1L-1.6L

JD:

Overview

We are seeking a highly experienced and dedicated Senior Hadoop Developer with a minimum of 8 years of experience in big data technologies. The ideal candidate will be responsible for providing dedicated support to our customers, leveraging their expertise in Hadoop, Spark, Trino, Hive Metastore, and Kubernetes (K8S) to ensure robust and efficient data solutions. This is a 100% Remote position, offering the flexibility to work from anywhere.

Responsibilities

•Provide expert-level dedicated support to customers for big data platforms and applications.

•Design, develop, and maintain scalable and high-performance data processing solutions using Hadoop and Spark.

•Implement and optimize queries and data pipelines using Trino for efficient data access and analysis.

•Manage and configure Hive Metastore for metadata management and integration with various data tools.

•Deploy, manage, and troubleshoot big data applications and services on Kubernetes (K8S) clusters.

•Collaborate with cross-functional teams to understand data requirements and translate them into technical specifications.

•Monitor system performance, identify bottlenecks, and implement solutions for continuous improvement.

•Ensure data quality, security, and compliance across all big data initiatives.

•Participate in code reviews, mentor junior team members, and contribute to best practices in big data development.

•Document technical designs, operational procedures, and troubleshooting guides.

Qualifications

•Minimum of 8+ years of hands-on experience as a Hadoop Developer or in a similar big data role.

•Mandatory Expertise in:

•Hadoop: HDFS, YARN, MapReduce, and other core Hadoop ecosystem components.

•Spark: Spark Core, Spark SQL, Spark Streaming, and PySpark/Scala.

•Trino (formerly PrestoSQL): Experience with query optimization and connector configuration.

•Hive Metastore: Deep understanding of metadata management and integration.

•Kubernetes (K8S): Experience in deploying, managing, and scaling big data applications on K8S.

•Proficiency in programming languages such as Java, Scala, or Python.

•Strong understanding of data warehousing concepts, ETL processes, and data modeling.

•Experience with cloud platforms (AWS, Azure, GCP) and their big data services is a plus.

•Excellent problem-solving and analytical skills.

•Strong communication and interpersonal skills, with the ability to provide clear technical guidance to customers.

•Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent practical experience.