False Ad detection system

Problem Statements

One of the more memorable projects I got a chance to work on as a freelancer was a custom computer vision model that could be integrated into an eCommerce site to scan pending listings by vendors, and in conjunction with other data signals, such as age of account, IP location vs Declared location, post description, etc., flag potentially fraudulent or misleading postings and flag them for review. The project ended up being broken into two milestones - the first was a crude and basic model I made as a proof of concept, and second was a fully scalable and tested implementation onto the site.

Though the model alone wouldn't necessarily stop or prevent misuse of the platform, its primarily purpose was to serve as a first line of defense against obvious cases and to automate part of the moderation process for the platform, enabling admins to focus on the most important or borderline cases. I was faced with several hurdles when designing and developing this system: 

Design Challenges

1. User-uploaded images were often of much lower quality or contained a lot of artifacts that weren't present in any readily-accessible training data

2. The model had to be integrated into the user's existing workflow without compromising experience

3. We had to maintain a fine balance between having an oversensitive model that deters vendors and an ineffective model that would be trivial to circumvent in practice

4. It was difficult to predict how sensitive the model would function in practice without actually trying it out on the real marketplace - it was difficult to simulate conditions