r/datasets • u/RoofProper328 • 15h ago
question How do teams keep annotation consistent when different people label the same data?
5
Upvotes
I was looking through a public dataset yesterday and realized something.
Some images felt like they could reasonably have two different labels depending on who's annotating them.
Do companies just write really detailed guidelines, or is there another process for keeping everyone consistent?
I'm curious how this works in practice because it seems like even small inconsistencies could affect the model later.