Job Title: Senior DevOps Engineer – Azure, Databricks (Financial Industry)
Location: Iselin, NJ (Day 1 Onsite, Hybrid)
Employment Type: Contract
Job Summary:
We are seeking a highly skilled and motivated Senior DevOps Engineer with proven experience in the financial industry and strong expertise in Databricks, Azure, Terraform, and DevOps practices. The ideal candidate will design, implement, and manage scalable, secure cloud solutions, ensuring robust infrastructure, automation, and CI/CD pipelines to meet business and data processing needs.
Key Responsibilities:
- Design and implement scalable, secure, and high-performance cloud infrastructure using Azure services.
- Manage and optimize cloud resources to ensure high availability, cost-efficiency, and performance.
- Develop, manage, and optimize Databricks environments for data processing and analytics.
- Implement ETL pipelines using Databricks and integrate with multiple data sources.
- Write and maintain Terraform scripts to automate infrastructure provisioning and manage IaC.
- Integrate Terraform into CI/CD pipelines and collaborate with development teams for seamless deployments.
- Implement and manage CI/CD pipelines using Jenkins, GitHub Actions, or Azure DevOps.
- Utilize Azure services such as VMs, AKS, SQL Database, Blob Storage, and manage networking (VNETs, VPNs, ExpressRoute).
- Manage Azure Active Directory and RBAC for secure access controls.
- Monitor, troubleshoot, and document infrastructure designs, configurations, and procedures.
- Collaborate with developers, data scientists, and IT operations teams to support business objectives.
- Provide technical guidance and training to team members on best practices in cloud and DevOps.
Required Qualifications:
- 8+ years of experience in cloud engineering, with a primary focus on Azure.
- Proven hands-on experience with Databricks for data engineering and analytics.
- Strong knowledge of Terraform and infrastructure-as-code principles.
- Solid experience with DevOps tools and CI/CD practices.
- Expertise in Docker and Kubernetes for containerization and orchestration.
- Proficiency in scripting languages such as Python, PowerShell, or Bash.
- Strong understanding of networking concepts and cloud security best practices.
- Excellent analytical and problem-solving skills, with the ability to work in a fast-paced environment.
Preferred Qualifications:
- Azure certifications (e.g., Azure Solutions Architect, Azure DevOps Engineer).
- Familiarity with Big Data technologies and frameworks.