Software Engineer II
Software Engineering
India · Bengaluru, Karnataka, India · Hyderabad, Telangana, India
We are hiring a Software Engineer II to build new AI platform capabilities for Windows Server across on-premises and hybrid customer deployments. You will be at the forefront of AI inference in a novel environment, and your work will infuse cutting edge AI concepts into existing Server platforms and workflows.
AI technologies evolved in the cloud, and you will translate those cloud-connected ideas to air-gapped, on-premises and hybrid environments. You will work at the intersection of AI models, agents, identity, and management surfaces on our most hardened OS platform. Experience or demonstrated interest in operating system fundamentals is a plus; experience with existing AI inference toolchains is valued.
Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
Responsibilities
- Design and implement how we track usage, policy and efficiency for artificial intelligence capabilities.
- Build observability pipelines through telemetry insights, diagnostics, and health signals needed to measure how this on-prem infrastructure is performing.
- Define and develop the management layer for AI inference toolchains from model lifecycles and catalogs to language and protocol gateways.
- Design and implement validation plans to assess the quality of this infrastructure and evaluate the end to end workflows and life cycle experiences of such a solution.
- Coordinate with security and compliance teams to ensure this toolchain meets or exceeds platform release requirements.
Qualifications
Required/Minimum Qualifications:
Bachelor's Degree in Computer Science, Engineering, or a related technical field, and 5+ years of software engineering experience. Strong development experience in C#/.NET or C++ and experience building Windows Server-based services and components. Experience designing and developing distributed or multi-tier service architectures, including areas such as reverse proxies, policy enforcement, telemetry, usage tracking, logging, and observability systems.
- Experience with AI/LLM inference platforms and model-serving technologies (e.g., vLLM, SGLang, OpenVINO, Triton, or similar), model lifecycle management, and API gateway architectures supporting AI workloads.
Other Requirements: Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings:
- Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.
Preferred Qualifications:
- Bachelor’s degree in computer science, a related technical field, or equivalent practical experience. AND 5+ years of professional software engineering experience with coding in one or more languages such as C, C++ including designing automated tests and validation for the software you ship.
- Experience or demonstrated interest in operating system fundamentals is a plus; experience with existing AI inference toolchains is valued.
- Demonstrated knowledge of Server administration and management features.
- Familiarity with Server deployment and usage strategies in air-gapped or on-premises environments.
- Understanding of AI model catalogs, model serving platforms, inference protocols, and AI orchestration layers.
- Familiarity with modern observability, diagnostics, distributed tracing, logging, and telemetry solutions.
#W+DJOBS
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.