Job Description

Use your administrative expertise to create realistic evaluation tasks that help improve next-generation AI systems. This remote contract role focuses on translating the complexity, judgment, and practical workflows of experienced administrative professionals into high-quality benchmark scenarios. No prior AI experience is required.

Key Responsibilities

• Design realistic, process-driven administrative evaluation tasks involving multi-tool coordination, such as calendar reconciliation, travel and expense workflows, records management, and professional correspondence.
• Create supporting scenario materials, including emails, calendar entries, receipts, meeting notes, and contact records.
• Define the correct outcome for each task and document the step-by-step decisions an experienced professional would make.
• Develop detailed evaluation rubrics with 35 or more criteria that assess process adherence and nuanced judgment in AI agent responses.
• Document administrative workflows and recommend improvements based on current professional best practices.
• Work with project leads to calibrate task complexity against real-world administrative challenges.
• Apply your domain expertise to help train and evaluate AI systems through authentic, expert-authored input.

Qualifications

• At least 5 years of professional administrative experience supporting teams or departments in mid-sized to large organizations.
• Demonstrated ownership of ongoing, recurring processes rather than only ad hoc assignments.
• Experience documenting, assessing, and improving inherited administrative procedures.
• Strong written and verbal communication skills, including professional correspondence standards.
• Excellent attention to detail, written judgment, process ownership, and multi-tool coordination skills.
• Comfort working across digital tools, platforms, and data formats including CSVs, PDFs, spreadsheets, and technical documents.
• Interest in applying administrative knowledge and best practices to intelligent systems.

Work Terms

• Remote independent-contractor engagement.
• Work is output-based, with payment tied to tasks that meet project specifications. Time required may vary based on experience and workflow.
• Minimum submission requirements apply, including a required minimum number of task submissions each week.
• Selected candidates are expected to begin their first tasks within 24 to 48 hours after completing onboarding.
• Openings are generally filled within 48 hours, so immediate availability is preferred.

Compensation

• Advertised compensation is $20 to $40 per hour.
• Payments are made per completed task that satisfies project specifications.

Application Process

• Apply to be considered for the project. Candidates who are selected will complete onboarding before beginning task work.
• By continuing an application, candidates acknowledge the applicable terms and privacy policy.

Job Tags

Similar Jobs