Shopify Expert - Fully Remote | Upto $50/hr
TypeContract
LocationUnited States
Posted1 hour ago
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.
Position: Shopify Storefront Agent Evaluator
Type:Contract
Compensation:$40–$50/hour
Location:Remote
Commitment:up to 5 hours/week
Role Responsibilities
Evaluate the Storefront Agent by simulating real shopper interactions, including product discovery, comparisons, and inquiries about sizing, stock, shipping, and returns.
Grade responses based on Relevance, Accuracy, and Safety to ensure they meet shopper expectations and adhere to store policies.
Provide concise feedback on any failures to ensure clarity for team members not present during evaluation.
Work independently to deliver precise and repeatable judgments on agent performance.
Qualifications
Must-Have
Consumer e-commerce fluency — you shop online regularly and understand effective product recommendations.
Careful, consistent judgment — apply the rubric uniformly across responses.
Clear, concise written English — articulate reasons for response failures in a sentence or two.
Preferred
Hands-on Shopify merchant experience — familiarity with shopper behavior on Shopify stores.
Experience using AI shopping assistants or chat agents — understand common failure points.
Prior work evaluating, red-teaming, or annotating AI model outputs.
Familiarity with retail operations: catalog and variant structure, inventory, shipping, and returns policy.
Application Process (Takes 20–30 mins to complete)
Upload resume
AI interview based on your resume
Submit form
Resources & Support
For details about the interview process and platform information, please check:
For any help or support, reach out to:
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Originally posted on Himalayas
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.
Position: Shopify Storefront Agent Evaluator
Type:Contract
Compensation:$40–$50/hour
Location:Remote
Commitment:up to 5 hours/week
Role Responsibilities
Evaluate the Storefront Agent by simulating real shopper interactions, including product discovery, comparisons, and inquiries about sizing, stock, shipping, and returns.
Grade responses based on Relevance, Accuracy, and Safety to ensure they meet shopper expectations and adhere to store policies.
Provide concise feedback on any failures to ensure clarity for team members not present during evaluation.
Work independently to deliver precise and repeatable judgments on agent performance.
Qualifications
Must-Have
Consumer e-commerce fluency — you shop online regularly and understand effective product recommendations.
Careful, consistent judgment — apply the rubric uniformly across responses.
Clear, concise written English — articulate reasons for response failures in a sentence or two.
Preferred
Hands-on Shopify merchant experience — familiarity with shopper behavior on Shopify stores.
Experience using AI shopping assistants or chat agents — understand common failure points.
Prior work evaluating, red-teaming, or annotating AI model outputs.
Familiarity with retail operations: catalog and variant structure, inventory, shipping, and returns policy.
Application Process (Takes 20–30 mins to complete)
Upload resume
AI interview based on your resume
Submit form
Resources & Support
For details about the interview process and platform information, please check:
For any help or support, reach out to:
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Originally posted on Himalayas
Apply on Himalayas →
Job sourced from Himalayas. Applications happen directly on the original platform — we never collect your data.