Sr. Developer

Job Summary

This hybrid Sr Developer role focuses on designing and implementing secure and scalable data and GenAI solutions using Databricks Python and cloud storage. The role involves building resilient pipelines optimizing analytics workloads and operationalizing AI driven insights while ensuring data governance quality and reliability to support critical business intelligence needs for global stakeholders.

Responsibilities

Design and implement robust Databricks based data solutions that leverage Delta Lake Databricks SQL and Python to support analytics and GenAI driven applications across multiple business domains.
Develop and maintain scalable Delta Live Pipelines that ensure reliable data ingestion transformation and enrichment while meeting stringent performance and data quality requirements for enterprise reporting.
Configure and administer Unity Catalog in Databricks to enforce consistent access control data classification and audit policies that protect sensitive information and support regulatory compliance.
Build and optimize PySpark based data processing jobs that handle large scale structured and semi structured datasets while improving execution efficiency and resource utilization within the Databricks environment.
Implement secure and efficient integrations with Amazon S3 to manage raw and curated data zones while ensuring consistent data lifecycle management backup strategies and recovery processes for critical assets.
Create and fine tune Databricks SQL queries dashboards and analytical models that deliver accurate and timely insights to business partners and enhance decision making effectiveness.
Use GenAI basics and AI BI Genie capabilities to design intelligent analytics features that surface contextual insights automated summaries and recommendations to end users in a responsible manner.
Apply AgentBricks and Mosaic AI features to build and orchestrate AI agents that automate routine analytics workflows support conversational insights and improve consumer experience with governed AI outputs.
Develop and orchestrate Databricks Workflows that coordinate complex dependency chains across notebooks jobs and pipelines while ensuring reliable scheduling monitoring and alerting for production operations.
Implement comprehensive testing observability and logging practices for all data and AI solutions using Python CLI tools and Databricks utilities to ensure resilience traceability and rapid incident diagnosis.
Use Databricks CLI to automate environment setup job deployment and configuration management while promoting repeatable and consistent delivery across development testing and production workspaces.
Collaborate proactively with data engineers data scientists and analytics partners in a hybrid work model to refine requirements validate deliverables and ensure high value outcomes from data and AI investments.
Document solution designs coding standards data dictionaries and operational runbooks in a clear and maintainable manner to support knowledge sharing smooth onboarding and sustainable long term operations.

Qualifications

Showcase extensive hands on experience in Databricks platform components including Unity Catalog Databricks SQL Delta Lake Delta Live Pipelines and Databricks Workflows with a strong focus on production grade solutions.
Demonstrate advanced proficiency in Python and PySpark for building modular testable and high performing data processing and AI enablement code within large scale distributed environments.
Exhibit practical understanding of GenAI concepts AI BI Genie and AgentBricks or Mosaic AI to design safe governed and value driven AI capabilities that complement existing analytics solutions.
Display strong expertise in cloud storage integration with Amazon S3 including data layout design security configuration and performance optimization for analytic workloads.
Present proven experience using Databricks CLI and related automation tools to implement infrastructure as code practices and consistent environment management across multiple workspaces.
Highlight a track record of at least twelve years in data engineering analytics or related development roles delivering resilient pipelines dashboards and AI driven insights for enterprise consumers.
Illustrate effective communication and collaboration skills that enable partnership with cross functional teams clear translation of requirements and delivery of solutions aligned to organizational objectives.
Show more Show less