Senior Data Engineer (Remote - California) - Jobgether : Job Details

We are currently looking for a Senior Data Engineer in San Francisco, California.

We are seeking a Senior Data Engineer to design, build, and optimize large-scale data solutions in a fast-paced, innovative environment. This role involves working on cloud-native architectures, modernizing on-premise systems, and delivering scalable, high-performance data pipelines that support critical business processes. You will collaborate with cross-functional teams to translate business requirements into robust technical solutions, ensuring data integrity, accessibility, and security. The position offers the opportunity to influence enterprise data strategy while working with cutting-edge technologies like AWS, Spark, and Hadoop. Your work will directly impact the efficiency and effectiveness of analytics and operational systems, enabling data-driven decision-making across the organization.

Accountabilities

• Design, develop, and operationalize large-scale enterprise data solutions using AWS services such as S3, Glue, Athena, Lambda, RedShift, EMR, Spark, and DynamoDB.
• Analyze, redesign, and migrate on-premise Cloudera Hadoop data solutions to cloud-native architectures.
• Build, deploy, and maintain data pipelines from ingestion to consumption using Java, Python, or Scala.
• Participate in architecture and design discussions to translate business requirements into technical solutions.
• Take ownership of user stories through analysis, development, testing, and deployment.
• Develop and maintain scripts to automate batch jobs and support existing applications.
• Ensure compliance with technical best practices, internal processes, and information security standards.
• Maintain Hadoop-based data solutions (HDFS, Oozie, Impala, Hive) until full cloud migration.