Senior QA Automation Engineer

About the role

Netsmart is seeking a Senior QA Automation Engineer to join its team in Bengaluru, India, focusing on test automation for software solutions integrated with AI/ML/GenAI workflows. Day-to-day, the role involves designing and maintaining modular automation frameworks, validating AI-driven features and model outputs, creating synthetic test data, and collaborating with developers and product teams to ensure quality standards. This is a full-time, on-site position requiring 5–7 years of hands-on experience.

What you’ll do

• Design, develop, and maintain modular, extensible, and reusable test automation frameworks and scripts while maintaining high stability
• Set up, configure, and maintain the automation environment and data to ensure smooth testing processes
• Develop and execute efficient automated test scripts to validate stability and responsiveness
• Create and maintain a comprehensive test automation library with scenarios to ensure thorough requirements and regression coverage
• Perform validation and create test data for applications/APIs/agents integrated with AI/ML/GenAI workflows and models
• Implement Risk Gating processes to validate outputs for accuracy, bias, toxicity, correctness, and traceability
• Identify, report, and track product defects, providing clear and concise documentation of issues and potential improvements
• Participate in product design reviews to provide input on requirements, automation feasibility, and testing implications

What you’ll bring

• 5–7 years of hands-on experience developing and executing automated test cases and test plans/scripts
• Proficiency with Selenium/WebDriver, Playwright, or Postman for API automation
• Strong programming skills in Python, Java, JavaScript/TypeScript, or C#
• Strong understanding of software QA standards, practices, and methodologies
• Hands-on experience validating end-to-end workflows, edge cases, and non-functional aspects
• Ability to design checks for accuracy, hallucinations, bias, correctness, and traceability of model outputs
• Familiarity with model evaluation metrics (precision/recall, BLEU/ROUGE, toxicity/bias scores) and guardrail patterns
• Strong skills in synthetic test data generation, data masking, and scenario design using SQL or CSVs

Nice to have

• Experience testing AI-driven products and LLM-integrated workflows
• Exposure to ML pipelines and data validation (e.g., feature store checks, dataset drift detection)
• Knowledge of data visualization tools (e.g., Matplotlib, Seaborn)
• Knowledge of cloud platforms (AWS, Azure, or GCP)

Skills