Job Summary
We are looking for a Senior Developer with strong expertise in Python, Azure Databricks (ADB), Delta Lake, and AI/LLM technologies to design and build Databricks jobs, reusable scripts, and cloud-native microservices. The role requires hands-on experience with Copilot agentic mode (or equivalent), building LLM-driven AI use cases, and the ability to work independently by taking full ownership of deliverables without the need for micromanagement.
Roles and Responsibilities
Azure Databricks & Data Engineering
- Design, develop, and manage Azure Databricks jobs, workflows, and notebooks.
- Build reusable Python/PySpark scripts for data ingestion, transformation, and analytics.
- Implement Delta Lake–based lakehouse architectures with ACID transactions and versioning.
- Optimize Databricks workloads for performance, scalability, and cost efficiency.
- Implement batch and streaming pipelines using Databricks.
Microservices & Application Development
- Develop Python-based microservices to expose data and AI capabilities via APIs.
- Build and integrate RESTful services for data access, AI inference, and orchestration.
- Ensure secure, scalable, and resilient service design using cloud-native patterns.
- Drive development tasks independently, from design through deployment, with minimal supervision.
AI, LLMs & Copilot Agentic Mode
- Design and build LLM-powered AI use cases such as copilots, intelligent assistants, and automation agents.
- Implement Copilot agentic mode workflows, including tool calling, task planning, and multi-step reasoning.
- Develop prompt engineering strategies, system prompts, and guardrails for enterprise use.
- Integrate LLMs with Databricks, data lakes, and microservices for real-time and batch inference.
- Build RAG (Retrieval-Augmented Generation) pipelines using enterprise data.
- Evaluate, fine-tune, and monitor LLM performance, accuracy, and cost.
Architecture, Governance & Best Practices
- Contribute to solution architecture and technical design decisions.
- Ensure data security, governance, and compliance in data and AI solutions.
- Implement logging, monitoring, and observability for Databricks jobs and AI services.
- Perform code reviews and enforce engineering best practices.
Leadership & Collaboration
- Mentor junior developers and provide technical guidance when needed.
- Collaborate effectively with data scientists, architects, DevOps teams, and business stakeholders.
- Clearly communicate technical concepts, progress, and risks through excellent verbal and written communication skills.
- Take full ownership of assigned workstreams and drive them to completion without requiring micromanagement.
Required Technologies & Skills
Programming & Frameworks
- Python (advanced)
- PySpark / Spark SQL
- REST APIs and microservices frameworks (FastAPI, Flask, etc.)
Azure & Data Platforms
- Azure Databricks (ADB)
- Delta Lake
- Azure Data Lake Storage (ADLS Gen2)
- Azure cloud services and security fundamentals
AI / LLM & Copilot
- Hands-on experience with LLMs (Azure OpenAI, OpenAI, or similar)
- Experience building Copilot solutions and agentic workflows
- Prompt engineering and orchestration frameworks (LangChain, Semantic Kernel, etc.)
- Experience with RAG architectures, vector databases, and embeddings
- Understanding of AI safety, guardrails, and responsible AI practices
DevOps & Engineering
- CI/CD pipelines (Azure DevOps, GitHub Actions)
- Git version control
- Containerization (Docker preferred)
- Infrastructure-as-code concepts (Terraform/Bicep – preferred)
Preferred Qualifications
- 5+ years of experience in Python, data engineering, or backend development
- Proven experience delivering production-grade AI/LLM solutions
- Experience in enterprise-scale cloud environments
- Exposure to streaming platforms (Kafka, Event Hubs – preferred)
Soft Skills
- Excellent verbal and written communication skills
- Strong sense of ownership and accountability
- Ability to work independently and proactively drive outcomes
- Strong problem-solving and analytical skills
- Collaborative mindset with leadership capabilities
The Cognizant community:
We are a high caliber team who appreciate and support one another. Our people uphold an energetic, collaborative and inclusive workplace where everyone can thrive.
- Cognizant is a global community with more than 300,000 associates around the world.
- We don’t just dream of a better way – we make it happen.
- We take care of our people, clients, company, communities and climate by doing what’s right.
- We foster an innovative environment where you can build the career path that’s right for you.
About us:
Cognizant is one of the world's leading professional services companies, transforming clients' business, operating, and technology models for the digital era. Our unique industry-based, consultative approach helps clients envision, build, and run more innovative and efficient businesses. Headquartered in the U.S., Cognizant (a member of the NASDAQ-100 and one of Forbes World’s Best Employers 2025) is consistently listed among the most admired companies in the world. Learn how Cognizant helps clients lead with digital at www.cognizant.com
Cognizant is an equal opportunity employer. Your application and candidacy will not be considered based on race, color, sex, religion, creed, sexual orientation, gender identity, national origin, disability, genetic information, pregnancy, veteran status or any other characteristic protected by federal, state or local laws.
Disclaimer:
Compensation information is accurate as of the date of this posting. Cognizant reserves the right to modify this information at any time, subject to applicable law.
Applicants may be required to attend interviews in person or by video conference. In addition, candidates may be required to present their current state or government issued ID during each interview.