Première détection par notre source: il y a 25 jours
Health AI Conversation Evaluator
@ Mercor
Toronto, Ontario
En bref
Mercor is hiring a remote contract Generalist Annotator to evaluate health AI conversations for relevance, clarity, sycophancy and user outcome. The work involves reviewing 1 to 7 turn conversations, rating them against a detailed rubric, and writing short human justifications for ratings below the top option. Pay is $20 to $160 per hour.
Responsabilités :
- Evaluate relevance and clarity of AI health responses for non-experts
- Detect sycophancy where AI tells users what they want to hear
- Assess tone and outcome of conversations
- Write justifications for ratings below the top option
- Review 1 to 7 turn conversations in 2 to 13 minutes each
Résumé des exigences :
- Strong written English skills
- Explain judgments in two to four specific sentences
- Careful reading to notice omissions
- Consistent application of a detailed rubric
- Reliable availability for time-boxed batches
- Annotation or evaluation experience preferred
Outils et compétences
Source et dates
Type de source non vérifié · manatal
- Date publiée par l’employeur
- Non fourni
- Première détection par notre source
- 2026-10-16
- Mise à jour du dossier source
- 2026-10-16
Ces dates décrivent le dossier source, pas une vérification en direct du recrutement. Consultez l’offre originale.
Détails du poste
| Déplacements | avion : aucun · route : aucun |
| Milieu de travail | Domicile |
| Position | Surtout assis |
| Exigences physiques | physique faible · ordinateur élevé |
| Langues | English |
Résumé et détails rédigés par l’IA à partir de l’offre; l’offre de l’employeur fait foi.
Mercor
Profil d’entreprise sauvegardé
Renseignements historiques provenant d’un profil The Jobs API sauvegardé (champs LinkedIn/Crunchbase). Ils ne sont pas vérifiés en temps réel; les chiffres peuvent avoir changé. Les valeurs manquantes restent inconnues.
We connect human expertise with leading AI labs and enterprises to train frontier models.
Description complète de l’entreprise
Mercor is defining the future of work. We connect human expertise with leading AI labs and enterprises to train frontier models.
- Abonnés LinkedIn
- 654 507
- Personnes associées sur LinkedIn
- 5 250
- Bureaux déclarés
- San Francisco, California 94105, US
- Catégories d’activité
- Employment
- Software
- Recruiting
- Data Collection and Labeling
- Artificial Intelligence (AI)
- Machine Learning
- Autres noms enregistrés
- Mercor
- Le profil indique une agence de recrutement
- Oui
- Investissement total déclaré
- 483 600 000
La devise et la date d’observation ne sont pas fournies; ce n’est ni une valorisation actuelle ni une ronde de financement.
Mercor is defining the future of work. We connect human expertise with leading AI labs and enterprises to train frontier models.
| Champ | Valeur |
|---|---|
| Secteur | Software Development |
| Taille | 51-200 employees |
| Siège social | San Francisco, California |
| Type d’organisation | Privately Held |
| Fondée en | 2023 |
| Offres sur Jobsy | 1 626 |
| Voir la page |
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.
Position: Generalist Annotator — Health AI Conversation Quality Evaluation
Type: Contract
Compensation: $20–$160/hour
Location: Remote
Role Responsibilities
- Evaluate the relevance and clarity of AI responses. Ensure the AI answers the question in understandable language for non-experts.
- Detect sycophancy in AI interactions. Identify when the AI tells users what they want to hear instead of what they need.
- Assess the tone and outcome of conversations. Determine if the user leaves informed and calm or anxious and uninformed.
- Provide written justification for ratings below the top option. Explain what went wrong and suggest improvements.
- Work independently to review conversations ranging from 1 to 7 turns. Spend 2–13 minutes per conversation based on length.
Qualifications
Must-Have
- Strong written English skills.
- Ability to explain judgments in two to four specific sentences.
- Careful reading to notice omissions in responses.
- Comfort with applying a detailed rubric consistently.
- Reliable availability for time-boxed batches.
Preferred
- Experience in annotation, human-feedback, evaluation, or content review.
- Background in explaining technical concepts to non-experts.
- Experience with rubric-based grading or quality assurance.
Important
- Using an AI tool to write comments is prohibited. This work captures human judgment that AI lacks. AI-written feedback corrupts the dataset and is checked for.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome
- For any help or support, reach out to: [email protected]
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Journal
Visible seulement par vous. Conservé sur cet appareil, et dans votre compte une fois connecté.