Software Engineer, AI Coding Tasks
- US$1,000 per task
- Remote, any location
We're looking for experienced software engineers to build AI coding tasks. You'll work from a codebase you know well. We pay US$1,000 per accepted task, not per hour, and you can do as many tasks as you want.
About the opportunity
You'll turn a real change from a codebase you know well into a task that today's best AI coding models struggle with: hard for the models, but fair for a skilled engineer.
Your tasks are used to measure and train AI coding models.
How it works
- 1. Apply
- Complete a short screener interview and upload your resume.
- 2. Technical task
- Once you're selected, complete a short technical task in our platform. It takes about an hour.
- 3. Build
- In our platform:
- set up the repository as it was before the change, so it builds and runs offline
- write the brief
- write the correct solution
- write the grading checks: tests, build commands and pass/fail review criteria, marking which are must-pass
- 4. Stress-test
- Try lazy, incorrect and shortcut solutions and make sure your checks catch them. Then confirm that a different correct solution still passes.
- 5. Model testing
- Run your task against leading AI models in our platform. The goal is a task that strong models often fail, but that a skilled engineer can clearly solve.
- 6. Review
- A reviewer checks your task, and you revise it based on their feedback. Most tasks take more than one round.
- 7. Next task
- Once your task is accepted, start another.
Requirements
- Experience
- 2+ years of professional software engineering.
- Open source
- Code merged into a public open-source repository, ideally one you know well. Code from a private codebase you own may also qualify. Employer or client code does not.
- Languages
- Strong in at least one of Python, TypeScript, Go, Rust, Java or C++.
- Tooling
- Solid with Docker, Linux, Git and automated testing.
- AI tools
- You use an AI coding tool regularly.
- Evaluation mindset
- Experience writing tests, rubrics or evaluation criteria, ideally for AI models. You enjoy finding the gaps in a specification.
- Writing
- You can explain a change clearly and concisely, in your own words.
- Availability
- At least 10 hours a week, starting now.
- Device security
- You work on a secure personal or work computer. Public or shared machines are not permitted.
Nice to have
Not required. Worth mentioning if it applies to you.
- Deep expertise in one area, such as databases, compilers, distributed systems, security or infrastructure.
- Experience building agent tasks or RL environments.
Contract and payment
- Once you pass the technical task, you'll sign Askable's contract and accept its terms.
- Fully remote, from any location.
- You're paid US$1,000 for each task that passes our quality review. We pay you directly, not through an agency.
- Pay is per accepted task, not per hour. How long a task takes varies, and most go through more than one review round.
- Do as many tasks as you want. We ask you to commit at least 10 hours a week.
- The work will be used for AI training. The change and your solution must be your own, and you must be free to license them to us.
- Don't publish the work anywhere. That means no public pull request and no public fork.
About Askable
Askable is the research infrastructure powering global enterprises since 2017. Askable Labs, a division of Askable Pty Limited, produces primary data for frontier AI labs from experts doing real work. We recruit those experts ourselves, verify them, and pay them directly.
We consider every qualified applicant without regard to legally protected characteristics, and we make reasonable adjustments on request.
Other open roles
- Accounting / Bookkeeping Expert (QuickBooks) $100 - $150 per hour
- Sales / RevOps Expert (QuickBooks) $100 - $150 per hour
- Voice Actor / Conversational Assistant US$50 per hour + US$15 per approved task
- Turkish Speaker, Conversation Project US$15 to US$70 per task
- Korean Speaker, Conversation Project US$15 to US$70 per task