All roles

Software Engineer, AI Coding Tasks

  • US$1,000 per task
  • Remote, any location

We're looking for experienced software engineers to build AI coding tasks. You'll work from a codebase you know well. We pay US$1,000 per accepted task, not per hour, and you can do as many tasks as you want.

About the opportunity

You'll turn a real change from a codebase you know well into a task that today's best AI coding models struggle with: hard for the models, but fair for a skilled engineer.

Your tasks are used to measure and train AI coding models.

How it works

1. Apply
Complete a short screener interview and upload your resume.
2. Technical task
Once you're selected, complete a short technical task in our platform. It takes about an hour.
3. Build
In our platform:
  • set up the repository as it was before the change, so it builds and runs offline
  • write the brief
  • write the correct solution
  • write the grading checks: tests, build commands and pass/fail review criteria, marking which are must-pass
4. Stress-test
Try lazy, incorrect and shortcut solutions and make sure your checks catch them. Then confirm that a different correct solution still passes.
5. Model testing
Run your task against leading AI models in our platform. The goal is a task that strong models often fail, but that a skilled engineer can clearly solve.
6. Review
A reviewer checks your task, and you revise it based on their feedback. Most tasks take more than one round.
7. Next task
Once your task is accepted, start another.

Requirements

Experience
2+ years of professional software engineering.
Open source
Code merged into a public open-source repository, ideally one you know well. Code from a private codebase you own may also qualify. Employer or client code does not.
Languages
Strong in at least one of Python, TypeScript, Go, Rust, Java or C++.
Tooling
Solid with Docker, Linux, Git and automated testing.
AI tools
You use an AI coding tool regularly.
Evaluation mindset
Experience writing tests, rubrics or evaluation criteria, ideally for AI models. You enjoy finding the gaps in a specification.
Writing
You can explain a change clearly and concisely, in your own words.
Availability
At least 10 hours a week, starting now.
Device security
You work on a secure personal or work computer. Public or shared machines are not permitted.

Nice to have

Not required. Worth mentioning if it applies to you.

  • Deep expertise in one area, such as databases, compilers, distributed systems, security or infrastructure.
  • Experience building agent tasks or RL environments.

Contract and payment

  • Once you pass the technical task, you'll sign Askable's contract and accept its terms.
  • Fully remote, from any location.
  • You're paid US$1,000 for each task that passes our quality review. We pay you directly, not through an agency.
  • Pay is per accepted task, not per hour. How long a task takes varies, and most go through more than one review round.
  • Do as many tasks as you want. We ask you to commit at least 10 hours a week.
  • The work will be used for AI training. The change and your solution must be your own, and you must be free to license them to us.
  • Don't publish the work anywhere. That means no public pull request and no public fork.

About Askable

Askable is the research infrastructure powering global enterprises since 2017. Askable Labs, a division of Askable Pty Limited, produces primary data for frontier AI labs from experts doing real work. We recruit those experts ourselves, verify them, and pay them directly.

We consider every qualified applicant without regard to legally protected characteristics, and we make reasonable adjustments on request.

Apply Earn US$400 by referring