وصف الوظيفة
Candidate Requirements for Databricks + CI/CD + Git + GitHub Actions
Databricks Expertise
Hands-on experience with Databricks notebooks, jobs, clusters, DBFS, and Repos.
Strong knowledge of Apache Spark (especially PySpark).
Experience building ETL/ELT pipelines using Delta Lake and DataFrames.
Familiarity with Databricks REST APIs for automation.
CI/CD & Automation
Proven experience designing and maintaining CI/CD pipelines using GitHub Actions.
Custom workflows (.github/workflows) for linting, testing, deploying notebooks, and triggering Databricks jobs.
Using Databricks CLI and REST API in pipelines.
Experience with workflow orchestration and automation for data jobs.
Familiar with environment-based deployments (dev/stage/prod).
Git Version Control
Strong knowledge of Git fundamentals: branching strategies, PRs, tags, conflict resolution.
Familiarity with Git workflows in collaborative environments (e.g., GitFlow, trunk-based).
Experience with Databricks Repos and how Git integrates with Databricks environments.
Scripting & Tooling
Experience writing Python or Bash scripts for automation.
Familiar with Docker for replicating environments locally (optional but helpful).
Cloud & Infrastructure
Depending on your cloud environment:
Azure Databricks or AWS Databricks
CI/CD with GitHub Actions + cloud runners, secrets, and service principals (Azure AD or AWS IAM)
Bonus: Infrastructure-as-Code (Terraform) to automate Databricks resources
Tools & Technologies
Git + GitHub
GitHub Actions (custom workflows, secrets, reusable workflows)
Databricks CLI and Databricks REST API
Python, Spark (PySpark)
Delta Lake, Databricks Repos
متطلبات الوظيفة
5-8 Years of experience , preferably 5 years min using darabricks
Bonus Qualifications
Used GitHub Actions matrix builds to test across different environments.
Experience with Databricks Asset Bundles (DAB) for configuration-as-code deployments
