Senior Databricks Developer (Contract)
Location: Houston, TX
Duration: 3-6 months
Company Overview:
COSOURCING PARTNERS is a leading provider of innovative data solutions, serving clients across various industries. We are currently seeking a Senior Databricks Developer to join our dynamic team for a contract opportunity based in Houston, TX. The successful candidate will play a crucial role in supporting a standup, setting up, and migrating databases into a data lake environment.
Responsibilities:
Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions using Databricks.
Support the setup of Databricks environments, ensuring optimal configurations and performance.
Lead the migration of databases and data pipelines into Databricks and data lake environments.
Design, develop, and implement ETL processes, data ingestion pipelines, and data transformations using Apache Spark and Databricks.
Develop and maintain data quality and governance frameworks within Databricks, ensuring data integrity and reliability.
Optimize data processing workflows for performance, scalability, and efficiency.
Provide technical guidance and mentorship to junior team members on Databricks best practices and development methodologies.
Collaborate with data architects, engineers, and analysts to design and implement scalable data solutions.
Troubleshoot and resolve technical issues related to Databricks environments, data pipelines, and data quality.
Stay updated on emerging technologies and best practices in big data, cloud computing, and data engineering.
Requirements:
Bachelor’s degree in computer science, Engineering, or related field.
Proven experience as a Databricks developer or data engineer, with at least 5+ years of hands-on experience.
Strong expertise in Apache Spark, Databricks, and Spark SQL for big data processing and analytics.
Experience in setting up and configuring Databricks environments in cloud platforms (e.g., AWS, Azure, GCP).
Solid understanding of data lake architectures, data modeling, and data warehousing concepts.
Proficiency in programming languages such as Python, Scala, or Java for developing ETL processes and data pipelines.
Experience with version control systems (e.g., Git) and CI/CD pipelines for code deployment.
Excellent analytical and problem-solving skills, with the ability to troubleshoot complex technical issues.
Strong communication and collaboration skills, with the ability to work effectively in a team environment.
Proven track record of delivering high-quality solutions within scheduled timelines.
Preferred Qualifications:
Certification in Databricks or related technologies.
Experience with containerization technologies such as Docker and Kubernetes.