Listing Screening at a Second-Hand Marketplace
An example project: a second-hand marketplace checks all 400,000 new listings a day against its policies before they go live, with a small open model on two dedicated GPUs, and cuts the number of bad listings that slip through by three quarters while halving the review queue.
This is an example project, not a customer story. The company is made up, and every number on this page is an assumption stated so you can replace it with your own.
Review costs about $800,000 a year less, and 900 fewer violating listings go live each day
Build cost assumes engineers at $180,000 a year fully loaded. Time from other staff, such as reviewers labeling examples, is not counted. Payback is counted from go-live. See the arithmetic.
Keyword rules that flag too much and catch too little
An online marketplace for used goods (furniture, electronics, clothing, tools, baby gear) receives about 400,000 new listings a day from private sellers and small dealers. Every listing has to follow its policies: no prohibited items such as weapons, prescription drugs or recalled products, no scams such as advance-payment requests or listings copied from other sites, and the listing has to be in the right category.
Today a set of about 3,000 keyword and regular-expression rules screens each listing before it goes live. The rules flag about 5% of listings, 20,000 a day, and hold them for a trust-and-safety team of about 30 reviewers working shifts seven days a week. Each flag takes about 30 seconds to clear. Only about one flag in eleven turns out to be a real violation, because the rules cannot tell “airsoft gun” from “glue gun” or a seller describing a scam warning from a seller running one.
The rules also miss things. Of an estimated 3,000 violating listings a day, they catch about 1,800. The other 1,200 go live and are removed later, after buyers report them or after a scam has already taken someone’s money. Writing new rules for each new scam pattern takes the policy team days.
The goal is to check every listing against the written policy before it goes live, with the listing visible within two minutes of the seller pressing publish, and to send reviewers fewer flags that are more often right.
The model reads the listing against the policy
The model makes one decision per listing. Account checks, photo checks and the final routing stay in the marketplace’s own systems. Photos are checked by the image system the marketplace already runs and are out of scope here.
Hard rules first
marketplace's codeExact matches that need no judgment are handled before the model: sellers already banned, phone numbers and payment links from known scams, and a short list of terms that are always prohibited. These listings are rejected without a model call.
Build the request
marketplace's codeCode assembles the listing’s title, description, chosen category and price, plus a few account facts written as text: account age, number of past listings, and whether earlier listings were removed. The written policy, about 1,500 tokens, goes first in the prompt and is identical for every request.
Screen the listing
modelThe model returns a small JSON object: allow or review, the policy section that applies, the suggested category if the chosen one is wrong, and one sentence of reason. Thinking is switched off, so the output is about 40 tokens. Constraining output to a JSON schema depends on the model and engine; confirm it for your deployment.
Route
marketplace's codeA listing goes live when the model says allow and the image system passes its photos. Review holds it in the trust-and-safety queue with the policy section and reason shown, so the reviewer starts from the model’s reading. A wrong category alone is fixed automatically without review. If the endpoint does not answer within the time limit, the listing waits in the queue. It never goes live unchecked.
Record decisions
marketplace's codeEvery reviewer decision on a flag, every buyer report on a live listing, and a daily random sample of listings the model allowed are stored with the model’s output. This is the data for measuring the model and, later, for fine-tuning it.
Two GPUs cover the evening peak with room to spare
The arithmetic we would start from. Profiling on real traffic replaces these estimates before anything goes into a contract.
| New listings per day | 400,000 |
|---|---|
| Input tokens per listing (about 1,500 of policy, plus about 350 of listing and account facts) | 1,850 |
| Output tokens per listing (decision, policy section, category, reason) | 40 |
| Tokens per day | 740M in, 16M out |
| Busiest hour, if 9% of the day’s listings arrive in it | 36,000 listings, 10 a second, about 18,500 input and 400 output tokens per second |
| Input that is new for each request, since the policy prefix is cached | about 3,500 tokens per second at the peak |
| Starting configuration to profile | Qwen3-8B, thinking off, on L40S (48 GB) GPUs |
| If one GPU sustains 15 listings a second | about 7.4 GPU-hours of work a day; 2 GPUs, so the peak uses about a third of capacity |
Each request is short in and very short out, so throughput decides the cost. The policy text is the same at the start of every prompt, and prefix caching means it is not recomputed for each listing. Most of the 740 million input tokens a day are that cached prefix.
The 8B model’s weights are about 16 GB in 16-bit precision, which leaves about 30 GB of a 48 GB L40S for the cache. A two-minute target is short enough that the endpoint should keep two minimum workers. Starting a new worker takes minutes, so autoscaling from one would not react to an evening surge in time, and a single worker restarting would hold every new listing.
The rate of 15 listings a second per GPU is the assumption to check first. If it is half that, the peak needs both GPUs fully used and a third worker is the safe choice. If it is much higher, two workers are still worth keeping for availability.
How the numbers add up
| Reviewing today’s 20,000 flags a day, at 30 seconds each and $30 an hour | $1,825,000 a year |
|---|---|
| Violating listings that go live today (1,200 of an estimated 3,000 a day) | about 438,000 a year |
| With the model: 10,000 flags a day, if it catches 90% of violations (the target; shadow mode tells you the real rates) | $912,500 a year |
| Reviewing a random 1,000 allowed listings a day, to measure what the model misses | $91,250 a year |
| Two L40S GPUs running all year, at our reference price of $1.10 per GPU-hour | $19,272 a year |
| Ongoing review and GPU cost with the model | about $1,023,000 a year |
| Violating listings that go live with the model (300 a day) | about 110,000 a year |
| Saving a year | about $802,000 |
| One-time build: two engineers for three months, at $180,000 a year loaded | $90,000 |
| Payback on the build, from go-live | about 6 weeks |
At these assumptions, review and GPU cost falls from about $1.83 million to about $1.02 million a year, and the number of violating listings that go live falls from about 1,200 a day to about 300. Of the model’s 10,000 daily flags, 2,700 are real violations, so about one flag in four is right, against one in eleven today. The first year also carries the build, about $90,000, which the saving covers in about six weeks.
The two rates that matter are recall (the share of violations the model flags) and precision (the share of flags that are real). They trade against each other: a stricter prompt or threshold flags more. If precision is half the target and the model flags 20,000 listings a day to reach 90% recall, review costs the same as today, about $1.94 million a year with the audit sample and GPUs, and the gain is entirely in the 900 fewer bad listings a day. If recall is 75% instead of 90%, 750 bad listings a day go live, still fewer than today. Measure both on the random audit sample during shadow mode, and choose the operating point with the trust-and-safety lead.
The GPUs are about 2% of the ongoing cost. A larger model that raises precision by a few points would pay for its extra GPUs in reviewer time. Test it on the evaluation set before assuming it does.
The benefits that do not show in the table are the scams that never reach a buyer and the time the policy team saves: a new scam pattern becomes a paragraph in the policy text instead of a set of new keyword rules.
How the project would run
Build the evaluation set
weeks 1 to 3Past reviewer decisions only cover listings the keyword rules flagged, so they say nothing about what the rules missed. Add buyer-reported listings that were removed, and have senior reviewers label a random sample of 5,000 listings that went live. Hold back 1,000 of the total that no prompt or fine-tune ever sees.
Write the policy prompt and compare models
weeks 3 to 5Turn the written policy into the prompt, with a short example per section. Run three or four candidate models on a trial endpoint and score precision and recall per policy section on the held-out set. Pick the smallest model that meets the bar.
Shadow mode
weeks 6 to 9The model screens every listing while the keyword rules keep deciding. Reviewers clear a daily sample of listings the model would flag and the rules would not, and the other way round. This gives the real precision and recall, and the peak-hour throughput on real traffic.
Model decides, one category group at a time
week 10 onwardStart with electronics and phones, where most scams are, and keep the keyword rules as a second check there for two weeks. Move the next category group when the audit sample shows recall holding.
Retire most keyword rules
after full launchKeep the hard rules that need no judgment. Remove keyword rules whose flags the model already catches, using the shadow-mode logs to show which ones.
What to measure
Recall on the random audit sample
The share of violating listings the model flagged, from reviewers checking 1,000 allowed listings a day. Buyer reports alone undercount it, since many bad listings are never reported.
Precision of flags
The share of flags reviewers confirm, per policy section. It sets the review team’s workload and shows which sections of the prompt need work.
Time from publish to live
The p95 for listings the model allows, measured at the evening peak. Sellers notice when it goes past two minutes.
Violating listings reported after going live
Buyer reports that end in removal, per 10,000 live listings. It lags, but it is the number customers experience.
Models to test first
Candidates from our catalog, not a ranking. The project picks one by testing them on its own labeled examples.
Qwen3-8B
The starting choice. 8.2B parameters, small enough for one L40S with room for the cache. Thinking mode can be switched off for short, direct answers, and switched on to test whether it helps on borderline sections of the policy. 32,768-token native context, far more than a listing and policy need. Apache 2.0.
Qwen3-4B
Half the size of the starting choice. If it scores close on the held-out set, it roughly halves the compute per listing. The pretrained base, Qwen/Qwen3-4B-Base, is also in the catalog as a starting point for fine-tuning.
granite-4.0-h-tiny
About 1B of its 7B parameters are active per token, so each check needs little compute. A candidate if throughput turns out to be the constraint. Apache 2.0.
gpt-oss-20b
For policy sections that need careful reading, such as scam patterns. Its reasoning effort can be raised for those and kept low for the rest. Worth testing on the sections where the smaller models have low precision. Apache 2.0.
None of these models was trained as a dedicated safety classifier. Screening here works by giving an instruction model the marketplace’s written policy in the prompt, or by fine-tuning on its reviewers' past decisions.
Sizes are total parameters as listed in our catalog. Mixture-of-experts models use only part of them for each token. Check each model's license for your use. Several, such as the Llama and Gemma licenses, carry their own terms.
When fine-tuning helps
This project has labels at a volume few teams have: tens of thousands of reviewer decisions a week, plus buyer reports. A small model fine-tuned on those may beat a larger prompted model on this marketplace’s listings, at a lower serving cost. Measure that on the held-out set before switching.
Fine-tune after shadow mode, once the logs show which policy sections the prompted model gets wrong. Keep the random audit sample in the training data so the model learns from what reviewers allowed, not only from what they removed. Retrain when the policy changes, and test that older sections did not get worse.
When the data has to stay put
Listings themselves are public once live, but the request also carries account history and listings that were rejected, some of which contain phone numbers, addresses and payment details that scammers paste in. Consumer protection and privacy rules may limit where that is processed. The endpoint can run in the marketplace’s own environment or in a specific region.
Tell us about your project
Bring us your volumes and a sample of your data. We will tell you which models to test, what the endpoint would need, and what a performance commitment would cover. Self-service accounts can create inference endpoints to try the platform, but not on our top-tier GPU capacity.