Position: LLM – AI Quality Analyst (Personalization) – Indonesian
Type: Short-Term Contract
Location: Remote
Commitment: 30–40 hours/week, 4-hour overlap with PST
Engagement Length: 2 months
Start Date: Immediate
Role Responsibilities
- Design multi-turn conversational prompts (1–5 turns) using personal context
- Evaluate personalized AI responses for grounding, integration, and helpfulness
- Assess correct usage of personal data and identify flawed inferences or hallucinations
- Review integration quality to ensure personalization feels natural and not over-narrated
- Conduct side-by-side (SxS) evaluation and ranking of model responses
- Identify personalization errors, reasoning gaps, and grounding issues
- Write clear, structured rationales referencing specific conversation turns
- Extract and verify “Debug Info” to confirm correct data source utilization
- Maintain strict data hygiene by deleting evaluation conversations
Requirements
- Indonesian proficiency (reading and writing)
- Strong experience in data annotation, AI quality evaluation, content moderation, or related roles
- Strong analytical skills for evaluating nuanced and ambiguous AI outputs
- Experience with creative prompt engineering and multi-turn conversations
- Understanding of personalization concepts and AI response evaluation
- High attention to detail for SxS comparisons
- Excellent written communication and feedback documentation skills
- BS/BA degree or equivalent experience in a relevant field
- Willingness to use a primary personal Google account with enabled personal data sources
- Self-motivated and able to work independently in a remote setup
- Desktop/laptop with reliable internet connection
Application Process
- Fill out the application form
- Complete the ICF
- Complete the assessment
