S&C GN - TS&T – Cloud AI Infra Architect - Senior Manager

About Accenture

Accenture is a leading global professional services company, providing a broad range of services and solutions in strategy, consulting, digital, technology and operations. Combining unmatched experience and specialized skills across more than 40 industries and all business functions - underpinned by the world’s largest delivery network - Accenture works at the intersection of business and technology to help clients improve their performance and create sustainable value for their stakeholders. With 750K people serving clients in more than 120 countries, Accenture drives innovation to improve the way the world works and lives. Visit us at www.accenture.com

About Global Network

Accenture Strategy shapes our clients’ future, combining deep business insight with the understanding of how technology will impact industry and business models. Our focus on issues such as digital disruption, redefining competitiveness, operating and business models as well as the workforce of the future helps our clients find future value and growth in a digital world. Today, every business is a digital business. Digital is changing the way organisations engage with their employees, business partners, customers, and communities - how they manufacture and deliver products and services, and how they run their organisations. This is our unique differentiator. We seek people who recognise and understand the impact that digital and technology have on every industry and every sector, and share our passion to shape unique strategies that allow our clients to succeed in this environment.

To bring this global perspective to our clients, Accenture Strategy’s services include those provided by our Global Network - a distributed management consulting organisation that provides management consulting and strategy expertise across the client lifecycle. Approximately 10,000 consultants are part of this rapidly expanding network, providing specialised and strategic industry and functional consulting expertise from key locations around the world. Our Global Network teams complement our in-country teams to deliver cutting-edge expertise and measurable value to clients all around the world. For more information visit www.accenture.com/capabilitynetwork

Global Network Videos

Video Title

External Link

Accenture Global Network

https://www.youtube.com/watch?v=-92pvOH1d_k

Accenture in One Word

https://www.youtube.com/watch?v=t1Fo8uNWZ-0

MBA Careers: What makes Accenture different

https://www.youtube.com/watch?v=5bg4u5Sczm8

Accenture Inclusion & The Power of Diversity

https://www.youtube.com/watch?v=2g88Ju6nkcg

AI Infrastructure Architect - Senior Manager

Practice Overview

Skill / Operating Group

Technology Consulting - AI Infrastructure Advisory

Level

Senior Manager

Location

Gurugram / Mumbai / Bangalore / Pune / Kolkata

Travel

Expected travel could be anywhere between 0–100%

Why Technology Consulting

The Technology Consulting business within Global Network invents the future for clients by providing them with the right guidance, design thinking and innovative solutions for technological transformation. Specialise in AI infrastructure advisory to transform the world’s leading organisations by: Helping Clients rethink their AI compute, networking, retrieval, and agent orchestration estates to scale enterprise AI reliably and cost-efficiently. Enhancing your Skillset with hyperscaler AI infrastructure (AWS, Azure, GCP), Kubernetes, Terraform, agentic AI architecture, and enterprise AI governance. Transforming Businesses by defining next-generation AI infrastructure strategies that move clients from AI pilots to resilient, governed, enterprise-scale production deployments.

Overview

Principal Duties & Responsibilities

Own and lead large-scale AI infrastructure architecture advisory programs for enterprise clients - setting the architecture vision, governance approach, and delivery strategy for the compute, networking, retrieval, agent orchestration, and tooling layers that power every AI and agentic workload.

Serve as the senior trusted technical advisor to client CTOs, CIOs, and senior technology leaders, translating complex AI infrastructure decisions into clear outcomes across cost, performance, scalability, resilience, and security.

Define the target-state AI infrastructure architecture, set standards that engineering teams work within, and govern architecture quality across large delivery programs.

This is a senior architecture advisory role, not a hands-on build role. The mandate is to set the architecture vision, define standards and reference architectures, and lead practice development.

This role focuses on AI infrastructure, platform engineering, governance, and architecture. It does not focus on model development, prompt engineering, model fine-tuning, or AI application development. Applications from AI Engineers, Prompt Engineers, or ML practitioners without infrastructure architecture experience are unlikely to be a strong fit.

Key Responsibilities

AI Compute & Infrastructure Architecture
Own the enterprise-wide AI infrastructure architecture vision - GPU/CPU compute, Kubernetes (AKS, EKS, GKE), managed ML compute, networking, identity, and security across AWS, Azure, or GCP.
Set reference architecture standards for AI Center of Excellence infrastructure - model hosting, agent environments, and the compute and networking beneath them.
Define the compute strategy - GPU SKU selection, GPU economics and capacity planning, provisioned vs pay-as-you-go inference, and shared vs dedicated capacity governance across business units.
Design AI-specific landing zones and govern their adoption - network isolation, private connectivity for AI services, and AI guardrails at the workload boundary.
AI Platform Architecture
Design enterprise AI platform architectures using Azure AI Foundry, AWS Bedrock, and GCP Vertex AI.
Define model catalog governance, model lifecycle management, platform operating models, and AI self-service enablement patterns.
Define the integration architecture between Azure AI Foundry, AWS Bedrock, and GCP Vertex AI and the underlying Kubernetes (AKS / EKS / GKE) compute, networking, and identity layers.
Inference Layer Architecture
Set the model serving strategy across managed cloud endpoints and self-hosted inference - defining architecture standards for serving patterns, routing, and load distribution at enterprise scale.
Define inference routing standards - model fallback chains, multi-model routing, and load balancing across provisioned and on-demand deployments.
Own the latency and throughput architecture framework for real-time and batch inference, including token-level latency budgets and streaming response design.
Define caching strategy at the inference layer - prompt caching and semantic caching standards to control token economics and reduce redundant model calls enterprise-wide.
Retrieval / RAG Architecture
Design vector database architecture for RAG - select and size across cloud AI search, vector-enabled databases, and dedicated vector stores based on scale and latency needs.
Architect embedding pipeline infrastructure - embedding model selection, reindexing strategy, and compute/storage patterns for chunking at enterprise scale.
Define hybrid search architecture (vector, keyword, metadata filtering) and the infrastructure to support it at scale.
Architect multi-tenant vector store isolation - ensuring one business unit’s embedded data cannot leak into another’s retrieval results through index- and access-level boundaries.
Design knowledge graph and GraphRAG infrastructure - graph database hosting and its integration pattern with the vector retrieval layer.
Agentic & Tool Calling Architecture
Define the enterprise tool calling infrastructure architecture - how agents securely discover, authenticate to, and invoke internal APIs, MCP servers, and third-party connectors.
Set MCP server hosting and governance standards - versioning, security, and reuse across multiple agents and business units.
Define the enterprise integration security boundary standard - scoped credentials, rate limiting, and audit logging frameworks.
Define agent orchestration environment standards - hosting for agent orchestration frameworks on Kubernetes, serverless containers, and managed agent services.
Set multi-agent orchestration infrastructure standards, including agent-to-agent communication patterns and agent memory storage architectures.
Identity, Access & AI Governance Architecture
Own the identity architecture standard for AI workloads - Managed Identity strategy, RBAC governance, and conditional access for AI services.
Define the private connectivity standard for AI services across the enterprise - ensuring inference traffic does not traverse the public internet.
Set the network segmentation standard between inference, training, and agent orchestration workloads.
Define the infrastructure-level Responsible AI security boundary and AI guardrails - content filtering integration points, prompt-injection mitigation, and data residency/sovereignty controls.
Own the model governance framework - model registry architecture, version control, promotion gates, and audit trails for deployed models and agents enterprise-wide.
Resilience, Scalability & Cost Optimisation
Define multi-region resilience standards for AI workloads - endpoint failover, model endpoint high availability, and project replication across hyperscalers.
Set the autoscaling architecture standard for inference workloads, accounting for GPU cost sensitivity and cold-start latency.
Own the capacity and quota management framework across cloud subscriptions for shared AI services.
Apply Well-Architected cost optimisation principles to AI infrastructure at enterprise scale - rightsizing compute, reserved capacity planning, GPU utilisation efficiency, and token-level consumption awareness across business units.
AI Observability, Governance & MLOps Architecture
Define the enterprise technical governance framework - model registry architecture, endpoint lifecycle management, and model governance standards for models and agents.
Set the MLOps/LLMOps foundation architecture - CI/CD for model and agent deployment using Terraform and native hyperscaler tooling, prompt versioning, and evaluation pipeline standards.
Own the AI observability architecture standard - token usage, per-inference latency, model drift signals, GPU utilisation monitoring, and hallucination-tracking hooks.
Define enterprise-wide standards for how agents are hosted, secured, and connected to systems.
Provide architecture design review for AI use cases proposed by business units, ensuring infrastructure is scalable, reusable, and aligned to enterprise standards.
Track hyperscaler AI roadmaps (AWS, Azure, GCP) and proactively evolve the reference architecture as the landscape changes.
Business Development, Thought Leadership & Executive Advisory
Engage client CTOs, CIOs, and senior technology leaders as a trusted AI infrastructure advisor - shaping strategy, framing investment decisions, and translating architecture into board-level narratives.
Lead RFP and RFI responses, solutioning, and executive presentations for large AI infrastructure programs.
Build quantified AI infrastructure business cases and value realisation frameworks for enterprise modernisation programs.
Develop and publish AI infrastructure points of view, reference architectures, and thought leadership assets that position the practice in the market.
Represent Accenture at industry forums, client events, and analyst briefings as an AI infrastructure subject matter authority.
Identify and develop new business opportunities - driving account growth through trusted advisory relationships and differentiated AI infrastructure offerings.


Practice Building

Codify methods and frameworks into scalable AI infrastructure assets for replication across engagements.
Create differentiated infrastructure offerings, accelerators, and reference implementations for the market.
Mentor and develop team members across the AI infrastructure architecture capability.
Manage engagement budgets, forecasting, and financial proposals.


Qualifications

Qualifications & Certifications

Educational Background

Bachelor’s degree in Computer Science, Information Technology, Engineering, or a closely related technical discipline. A Master’s degree is preferred.

Mandatory

AWS Certified Solutions Architect – Professional, OR
Microsoft Certified: Azure Solutions Architect Expert (AZ-305), OR
Google Cloud Professional Cloud Architect


Preferred

AWS Machine Learning Specialty, OR Azure AI Engineer Associate, OR Google Professional Machine Learning Engineer
Azure OpenAI Service / AWS Bedrock / GCP Vertex AI specialisation


Key Competencies & Skills

Competencies

FUNCTIONAL COMPETENCIES

Senior executive advisory - advise CTO, CIO, and board-level stakeholders on AI infrastructure strategy, platform choices, and investment decisions; translate technical complexity into clear executive narratives.
Architecture strategy and roadmap leadership - assess current-state AI estates, define target-state designs, and build transformation roadmaps at enterprise scale.
Structured problem-solving under ambiguity - frame AI infrastructure challenges, evaluate trade-offs, and produce actionable recommendations at program and enterprise level.
Cloud cost optimisation advisory - apply Well-Architected cost principles to AI infrastructure investment decisions; provide guidance on compute rightsizing, capacity strategy, and GPU utilisation efficiency.
Business development leadership - shape RFP responses, lead large solutioning efforts, drive account growth, and build AI infrastructure advisory offerings that differentiate the practice.
Industry thought leadership - develop and publish AI infrastructure points of view, represent Accenture at industry forums and analyst briefings, and build external credibility as an AI infrastructure authority.


TECHNICAL COMPETENCIES

Hyperscaler AI infrastructure - expert-level mastery in at least one of: AWS (Bedrock, SageMaker, EKS, EC2 GPU, Lambda), Azure (AI Foundry, Azure OpenAI, Azure ML, AKS, Azure Container Apps), or GCP (Vertex AI, Gemini, GKE, Cloud Run). Working knowledge of the other two for cross-cloud advisory.
AI platform architecture - Azure AI Foundry, AWS Bedrock, GCP Vertex AI; model catalog governance, model lifecycle management, and self-service enablement at enterprise scale.
Container orchestration - Kubernetes (AKS, EKS, GKE) and cloud-native infrastructure design for AI workloads at enterprise scale; able to define Kubernetes governance standards for large programs.
Infrastructure as code - expert advisory fluency across Terraform, Ansible, and native hyperscaler IaC tooling; able to set IaC governance standards for large programs.
Inference layer architecture - expert design of model serving, routing, caching, latency budgets, and cost optimisation at enterprise scale.
Retrieval and RAG architecture - expert design of vector databases, embedding infrastructure, hybrid search, GraphRAG, and multi-tenant isolation at enterprise scale.
Agentic architecture - tool calling infrastructure, MCP server governance, agent orchestration frameworks, multi-agent systems, and agent memory architectures at enterprise scale.
MLOps/LLMOps architecture - expert knowledge of CI/CD for models, pipeline automation, model versioning, model governance, evaluation pipelines, and developer self-service.
AI governance and Responsible AI - model governance frameworks, AI guardrails, Managed Identity, RBAC, private connectivity, Responsible AI controls, and compliance frameworks for AI workloads.
AI observability - expert design of token usage monitoring, per-inference latency, model drift signals, GPU utilisation tracking, and cost attribution frameworks.
Cloud cost optimisation - Well-Architected cost principles applied to AI workloads at enterprise scale; compute rightsizing, reserved capacity, GPU utilisation efficiency, and consumption governance.
AI Compute Infrastructure - familiarity with NVIDIA GPU ecosystem, NVIDIA NIM, DGX platforms, GPU sizing strategies, AI accelerator technologies, and enterprise AI deployment patterns.
AI roadmap awareness - deep familiarity with AWS, Azure, and GCP AI roadmaps; able to proactively evolve enterprise reference architectures as the landscape evolves.


Additional Information

Equal Opportunities

Accenture is an equal opportunities employer and welcomes applications from all sections of society and does not discriminate on grounds of race, religion or belief, ethnic or national origin, disability, age, citizenship, marital, domestic or civil partnership status, sexual orientation, gender identity, or any other basis as protected by applicable law.

,

Experience

14–22 years of combined experience across enterprise architecture, cloud infrastructure, and/or AI infrastructure delivery.

5+ years leading AI infrastructure architecture strategy and advisory programs for enterprise clients.
Strong expertise in at least ONE hyperscaler (AWS, Azure, or GCP) is mandatory; second hyperscaler is a strong advantage at this level.
Proven experience owning end-to-end AI infrastructure architecture for large transformation programs - setting target-state designs and governing quality across delivery teams.
Deep experience with Kubernetes (AKS, EKS, GKE) and container orchestration in production AI environments.
Expert-level proficiency in IaC tooling (Terraform, Ansible) and setting IaC governance standards for large programs.
Deep experience designing inference serving patterns, vector database architectures, and RAG pipeline infrastructure at enterprise scale.
Experience designing tool calling and MCP architecture for agentic systems, including security and governance of tool access at enterprise scale.
Experience defining multi-agent orchestration infrastructure and agent orchestration standards.
Experience defining MLOps/LLMOps foundations - CI/CD for models, model governance, model registries, and evaluation pipelines.
Deep knowledge of enterprise network and identity architecture (private endpoints, VNets/VPCs, IAM, RBAC) as applied to AI workloads.
Strong understanding of Well-Architected cost optimisation principles applied to AI infrastructure - compute rightsizing, reserved capacity, GPU utilisation efficiency, and consumption governance at enterprise scale.
Experience defining AI observability architecture - token usage, inference latency, model drift, and GPU utilisation monitoring.
Deep understanding of Responsible AI principles, AI guardrails, and model governance at enterprise scale.
Enterprise Architecture knowledge (TOGAF or equivalent).
Demonstrated track record leading business development - shaping RFPs, driving account growth, and developing AI infrastructure advisory offerings.
Architecture design skills at L1–L3 levels; proficiency in draw.io, Visio, or Lucid Chart.
Show more Show less