Home › Blog › Job ads

A/B Testing Job Ads: What to Test, What to Ignore, and When to Stop

Job ads7 min read
Two phones side by side on a bright desk showing two versions of the same advert

Testing job adverts is worth doing, and it is easy to do in a way that produces confident nonsense. The two common failures are testing five things at once and declaring a winner after nine clicks.

Key takeaways
  • Test one thing at a time, and test the big things first: image, then opening line, then call to action.
  • Click-through is a vanity winner. Decide on qualified applications per euro.
  • Below a few hundred impressions per variant you are reading noise, not a result.
  • Most of the gain comes from the first two tests. After that, spend your attention elsewhere.

What should you test first?

In descending order of impact, which is roughly the order in which a candidate's attention meets your ad.

  1. The image. Nothing else comes close on a social feed. A real photograph of the actual workplace against a stock image is often the single largest difference you will ever measure - the point we make in job ad examples that actually get applications.
  2. The first line. The candidate reads about seven words before deciding. A concrete fact beats an adjective almost every time.
  3. The call to action. Short and specific beats long and enthusiastic. Three words is usually enough.
  4. The pay presentation. Hourly against monthly, gross against net, with or without the extras attached.
  5. The role title. Worth testing when the job has two common names in your market - and never worth stuffing with the location.

Notice that body copy is not near the top. Employers love rewriting the paragraph in the middle. Candidates mostly never reach it.

What is the right metric to decide on?

Qualified applications per euro spent. Not impressions, not clicks, not raw applications.

This matters because the metrics disagree with each other more often than you would expect. A playful advert with a vague first line frequently wins on click-through and loses badly on qualified applications, because it pulled in people who did not understand the job. If you declare winners on clicks, you will systematically select the ads that waste your screening time.

If you are scoring candidates, use the score. "Applications scoring above 60" is a far better target than "applications", and it is the number that connects advertising spend to actual hiring outcomes. We laid out which numbers deserve tracking in the hiring metrics that actually matter.

How long should a test run?

Long enough to cross two thresholds: a few hundred impressions per variant, and at least three or four full days.

The time threshold matters as much as the volume one because audiences behave differently by day. An advert that wins on Tuesday morning and loses on Saturday night has told you nothing yet. Frontline candidates in particular apply heavily in the evening and at weekends.

For most small campaigns, the honest answer is that you will not reach statistical certainty and do not need to. You are looking for differences large enough to see plainly - a variant pulling twice the qualified applications is a real signal. A variant ahead by eleven per cent is not.

What should you not bother testing?

When should you stop?

When the difference between your best variant and your next test stops being visible to the naked eye - usually after two or three rounds.

This is the part most guides omit. The first test typically produces a large gain, the second a moderate one, and the fourth is a rounding error you will spend a fortnight chasing. Once you have a strong image and a strong opening line, your remaining leverage is almost always in the offer, the speed of your reply or the length of your application - not in the advert. On speed specifically, see why candidates ghost you.

A simple test plan

  1. Round one: real photo against your current image. Everything else identical. Four days.
  2. Round two: winning image, then two opening lines - one concrete fact, one benefit statement. Four days.
  3. Round three: winning combination, then two calls to action. Four days.
  4. Then stop testing the ad and start testing the process: reply within an hour, shorten the application, and see what that does.

The takeaway

Change one thing, start with the image, judge on qualified applications per euro, and give it four days. Take the two big wins and then stop - the remaining gains live somewhere other than the advert.

Qwiza generates several ad variants from one job description and scores every applicant, so the comparison you end up making is between qualified candidates rather than between clicks.

Want to know which ad actually works?

Qwiza writes the variants, runs them on Instagram and Facebook and scores every applicant, so you compare qualified candidates and not clicks.

See how Qwiza works

Frequently asked questions

How many applications do I need before a test result means something?

Aim for a few hundred impressions per variant and at least three or four days of running. For small campaigns you will rarely reach statistical certainty, so act only on differences large enough to see plainly - roughly double, not eleven per cent.

Should I test the job title?

Only when the role genuinely has two common names in your market, such as warehouse operative and warehouse associate. Otherwise leave it alone, and never add the location into the title - that belongs in the targeting and in the ad body.

Is click-through rate a good measure for job ads?

No. A vague, playful ad often wins on clicks and loses on qualified applications because it attracts people who misunderstood the job. Decide on qualified applications per euro spent, and if you score candidates, use the score as the qualifier.

Keep reading