Prompt Engineer - Checkmate Services : Job Details

In this hybrid role, you will blend traditional software engineering with natural language processing expertise to develop and maintain systems that leverage LLMs to solve real-world challenges in the restaurant tech domain. You'll collaborate with cross-functional teams to implement AI-driven features that support everything from customer service automation to order management, reporting, and staff training.

Essential Job Functions:

• Prompt Development & Optimization: Design, build, test, and refine prompts for AI tasks like Q&A, classification, extraction, and synthetic data creation.
• Evaluation & Testing Build robust test suites using tools like CoVal, LangFuse, and Galileo to measure prompt accuracy, compliance, and performance at scale.
• Performance Analysis & Improvement: Analyze prompt outputs using metrics, product feedback, and business goals. Iterate to maximize accuracy and reliability.
• Collaboration: Work closely with operations, product, data science, and engineering teams to ensure prompts meet quality standards
• Model Adaptation: By understanding their unique behaviors, create effective prompts across different models (GPT, LLaMA, Gemini, and internally fine-tuned LLMs).
• End-to-End Ownership Drive prompts projects from planning to deployment, including monitoring and continuous improvement.
• Team Leadership & Mentorship Lead a team of analysts, guiding their evaluations and data work to support prompt development. Mentor and collaborate with ops teams to ensure seamless integration of prompts into production.

100% Remote

Minimum of three years of experience in the areas of AI, ML, NLP, and prompt engineering.

$125,000 to $160,000

Requirements

Requirements:

• Strong Python programming skills
• Solid data and computer science fundamentals
• Experience with relational databases (e.g., MySQL, PostgreSQL, Oracle, MS SQL)
• Hands-on development experience, preferably full-stack
• Good data processing and testing skills
• Ability to handle ambiguous problems and drive projects from start to finish
• Strong problem-solving mindset with a drive to improve and optimize systems
• Ability to use scientific and analytical approaches to measure and evaluate designs
• Excellent communication skills
• Bachelor's degree in Computer Science, Engineering, Mathematics, or a related field
• Flexible to work US hours until at least 6 pm ET with a strong remote setup

Preferred Qualifications

• Experience with multiple LLMs and tools like LangFuse and Galileo
• Knowledge of cloud platforms (AWS, GCP, Azure, OpenStack) and tools like Docker and Kubernetes
• Focus on scalability, performance, and cost optimization in cloud environments
• Broad technical expertise across front-end and back-end systems
• Familiarity with machine learning concepts and workflows
• Master's or PhD in Computer Science, Engineering, Math, Physics, or related field

Benefits

• Health Care Plan (Medical, Dental & Vision)
• Retirement Plan (401k)
• Life Insurance (Basic, Voluntary & AD&D)
• Flexible Paid Time Off
• Family Leave (Maternity, Paternity)
• Short Term & Long Term Disability
• Training & Development
• Work From Home
• Stock Option Plan