Manual QA Engineer

Editorial Overview & Role Snapshot

Alivo is hiring a Manual QA Engineer for a full-time position in Portland, Maine. The role is centered on traditional software quality assurance, but it also adds an interesting AI-focused responsibility: evaluating how well AI-powered phone and chat agents communicate with customers.

For a job seeker, this means the position is broader than simply running test cases and reporting bugs. You would manually test new features and workflows, perform exploratory and regression testing, review releases, and work with engineering and product teams to identify problems before they reach customers. You would also evaluate AI conversations for accuracy, naturalness, effectiveness, objection handling, and unexpected behavior.

The strongest candidates are likely to be QA professionals or software testers with 2–5+ years of experience who are detail-oriented, methodical, and comfortable identifying edge cases. Good written and verbal communication is important because the role involves creating clear bug reports and working across Engineering, Product, Customer Success, and Leadership.

AI or LLM testing experience is listed as a nice-to-have rather than a core requirement. Experience with tools such as Jira, Linear, TestRail, or Postman can also strengthen an application, as can basic SQL, scripting, technical troubleshooting, or previous work in SaaS, customer support, or other customer-facing environments.

The role may be particularly appealing to QA professionals who want to move into AI quality assurance and conversational AI evaluation. Alivo describes itself as a Maine startup focused on AI-powered communication for roofing and home-service businesses, making this an opportunity to apply conventional QA skills to a growing AI product area.

Job at a Glance

Detailed Role Requirements & Responsibilities

Manual Software Testing

The core of the position is manual quality assurance. You would help make sure new product features and workflows work correctly before they are released.

Responsibilities include:

• Manually testing new features and workflows.
• Performing exploratory testing to find unexpected bugs, usability problems, and edge cases.
• Following structured test plans and release checklists.
• Performing regression testing after bugs have been fixed.
• Reviewing release candidates before they reach customers.

Bug Reporting and Defect Tracking

Finding a problem is only part of QA work. The employer also expects the engineer to communicate defects clearly so developers can reproduce and fix them.

You would:

• Identify software defects and customer experience issues.
• Create clear and reproducible bug reports.
• Verify that reported issues have been fixed.
• Work with Engineering when reviewing release candidates.
• Look for failure modes that may not appear during normal testing.

The posting mentions familiarity with tools such as Linear, Jira, or TestRail as a preferred qualification.

AI Testing

One of the more distinctive parts of this Manual QA Engineer position is testing AI-powered conversations.

You may evaluate phone and chat interactions for:

• Accuracy
• Natural and understandable responses
• Effectiveness
• Conversational flow
• Objection handling
• Off-script behavior
• AI failure modes
• Hallucinations

You would also help establish quality standards for AI-powered interactions and monitor how AI performance changes over time.

Quality Evaluation

The position goes beyond individual test cases. The engineer will help build repeatable ways to measure AI quality.

This includes:

• Building and maintaining evaluation datasets.
• Creating scoring rubrics.
• Developing quality frameworks.
• Tracking AI performance over time.
• Identifying patterns in AI failures.
• Helping define what constitutes a high-quality AI interaction.

This makes the role relevant to candidates interested in the growing area of AI evaluation and AI quality assurance.

Team Collaboration

The QA Engineer will work with several teams rather than operating independently.

The job posting specifically mentions collaboration with:

• Engineering
• Product
• Customer Success
• Leadership

Strong communication is therefore an important part of the position, especially when explaining defects, discussing release quality, and helping teams prioritize improvements.

Required Skills and Experience

Core Requirements

The main qualifications listed by Alivo are:

• 2–5+ years of experience in QA, software testing, product operations, or a similar role.
• Strong attention to detail.
• A structured and methodical approach to testing.
• Strong written communication skills.
• Strong verbal communication skills.
• Ability to identify edge cases and failure modes.
• Ability to recognize customer experience problems.
• Comfort working in a fast-moving and changing startup environment.

For applicants, the key point is that the employer is looking for someone who can think systematically about quality rather than simply follow a checklist.

Preferred Qualifications

The following are listed as nice-to-have qualifications rather than core requirements:

• Experience evaluating AI, LLM, or conversational AI systems.
• Familiarity with Linear, Jira, or TestRail.
• Experience with API testing tools such as Postman.
• Basic SQL knowledge.
• Basic scripting skills.
• Technical troubleshooting experience.
• Customer-facing, sales, or support experience.
• Experience working with SaaS products.
• Experience at a high-growth startup.

Skills That Can Strengthen Your Application

If you already meet the core QA requirements, the following skills could make your application stronger:

• AI/LLM testing: Experience testing AI-generated responses or evaluating conversational AI.
• Conversational AI: Understanding how voice or chat agents should handle different customer scenarios.
• API testing: Postman or similar tools.
• Bug tracking: Jira, Linear, TestRail, or comparable platforms.
• SQL: Basic ability to inspect or validate application data.
• Scripting: Basic scripting experience that can support testing or troubleshooting.
• SaaS testing: Experience testing web-based software products.
• Customer-facing experience: Experience in support, sales, or customer operations can help because the role considers customer experience when evaluating quality.

These should be presented as additional strengths. The job posting does not make them mandatory requirements.

Industry Context & Career Advice

QA is changing as software products increasingly include AI-generated or AI-driven functionality. Traditional testing remains important, but AI products introduce additional questions: Is the response accurate? Does it stay within the intended conversation? Does it handle unusual customer input? Does it produce an unsafe or incorrect answer? Can the quality be measured consistently?

That makes this role particularly relevant for QA professionals who want to develop experience in AI testing, LLM evaluation, and conversational AI quality.

If you apply, do not focus only on the number of test cases you have executed. Show how you have found difficult bugs, identified edge cases, improved release quality, or prevented customer-facing problems.

Candidates can also strengthen their long-term QA career by developing:

• API testing skills
• Basic SQL
• Test automation fundamentals
• AI and LLM evaluation techniques
• Data-driven quality measurement
• Strong defect reporting
• Exploratory testing skills
• Product and customer experience awareness

Alivo’s own careers page describes the company as a growing Maine startup and highlights growth opportunities as part of its employment offering.

What to Highlight on Your Resume

For this particular position, your resume should make it easy for the recruiter to see evidence of manual QA, software testing, defect detection, regression testing, and release validation.

Consider highlighting:

• Years of QA or software testing experience.
• Functional and exploratory testing experience.
• Regression testing.
• Test plans and test cases.
• Release validation.
• Bug reporting and defect tracking.
• Experience identifying edge cases.
• Jira, Linear, TestRail, or similar tools.
• Postman or API testing.
• SQL or scripting.
• SaaS testing.
• Customer-facing or product operations experience.
• AI, LLM, chatbot, voice-agent, or conversational AI testing if you have it.

Whenever possible, include measurable results rather than simply listing responsibilities.

For example:

Performed functional, exploratory, and regression testing for SaaS releases, identified edge cases and customer-impacting defects, and created reproducible bug reports to support timely fixes.

If you have AI testing experience, make it visible near the top of your resume rather than hiding it in a general skills section.

Interview Preparation

Prepare for questions that test both your technical QA thinking and your ability to evaluate customer-facing software.

Manual Testing

Be ready to explain:

• How you approach testing a new feature.
• How you decide what to test first.
• How you perform exploratory testing.
• How you approach regression testing before a release.
• How you identify edge cases that a normal test plan might miss.

A strong answer should demonstrate a logical testing process rather than simply saying that you would “test everything.”

Bug Reporting

You may be asked how you would report a difficult bug.

Think about how you would provide:

• A clear description of the problem.
• Steps to reproduce it.
• Expected behavior.
• Actual behavior.
• Relevant environment or test information.
• Evidence that helps Engineering investigate the issue.
• The potential customer impact.

AI and Conversational Testing

Because AI evaluation is a major part of this role, think about how you would test an AI phone or chat agent.

For example, consider:

• How would you test whether an AI response is accurate?
• How would you identify hallucinations?
• How would you test unexpected customer questions?
• How would you evaluate whether a conversation sounds natural?
• How would you test objection handling?
• How would you determine whether an AI agent has gone off-script?
• How would you create a scoring system for conversation quality?

You do not necessarily need previous AI testing experience to think through these questions. If you are new to AI testing, demonstrate that you understand the importance of repeatable evaluation criteria and real-world customer scenarios.

Prioritization and Collaboration

Also prepare for questions about:

• How you prioritize multiple bugs.
• How you decide whether an issue should block a release.
• How you communicate a serious defect to developers.
• How you handle disagreements about whether something is a bug.
• How you balance testing deadlines with product release schedules.

Use examples from your previous work where possible.

Advice for Applicants

This position is a good match for QA professionals who enjoy finding problems, investigating edge cases, and thinking about how real customers will use a product.

Do not automatically rule yourself out because you have limited AI experience. AI, LLM, or conversational AI experience is listed as a nice-to-have, while QA/software testing experience is part of the core requirement.

If you have strong manual testing experience but have not worked with AI, emphasize your ability to learn new products, identify failure modes, communicate clearly, and evaluate software from the customer’s perspective.

If you do have AI experience, make it a prominent part of your application. Explain exactly what you tested or evaluated rather than simply writing “AI experience” in your skills list.

How to apply: Candidates should email their resume and details directly to contact@alivo.ai. The Jobsinator listing does not provide an application deadline.

The strongest way to position yourself is to show that you can do more than execute test cases: you can identify risks, explain defects clearly, protect the customer experience, and help establish reliable quality standards for AI-powered products.