Review Queue
Human-in-the-loop review for AI-triaged feedback.
The Review Queue lets your team validate AI-triaged insights before they flow into analytics or sync to external tools. This human-in-the-loop step ensures data quality and gives your team control over what gets actioned.
How it works
- Feedback arrives and is processed by the AI triage pipeline
- Items appear in the Review Queue with status Triaged
- A team member reviews the AI classification, sentiment, and priority
- They either Approve or Reject the insight
Reviewing an insight
Open any insight from the queue to see:
- The original feedback content
- AI-assigned category, sentiment, and priority score
- Extracted entities (category-specific structured data)
- Similar insights that may be duplicates
Approving
Click Approve to confirm the AI triage is accurate. Approved insights:
- Appear in analytics dashboards
- Are eligible for sync to Jira Product Discovery or Productboard
- Count toward trend metrics and reports
Rejecting
Click Reject if the feedback is spam, irrelevant, or incorrectly classified. You must select a rejection reason from the dropdown before the rejection is accepted. You can also add an optional note. Rejected insights:
- Are hidden from default views (but not deleted)
- Don't count in analytics
- Won't sync to external tools
Adding notes
Use the notes field to add context before approving or rejecting. Notes are visible to all team members and help maintain an audit trail of review decisions.
Override reasons
When approving or rejecting, select an override reason (required for rejections). Available reasons:
- AI result confirmed
- Incorrect classification
- Missing context
- Wrong priority/urgency
- Duplicate or noise
- Policy or scope decision
ProductSights uses these reasons to improve triage prompts and rules over time.
Best practices
- Review regularly — check the queue daily to keep feedback flowing
- Focus on high-priority items first — sort by priority score to tackle the most important items
- Use rejection sparingly — only reject obvious spam or irrelevant content; miscategorized items can be useful for improving future triage accuracy
- Add notes — brief notes like "duplicate of #123" or "not actionable yet" help teammates understand past decisions
- Tag the right override reason — this is the fastest way to improve future AI triage quality