Strategy to stop spambots (from another forum)

[Haelbarde he/him]

Curious, does stuff tend to come at a particular time, or is it usually fairly regular throughout the day/various at what point of the day it comes in. Has the more recent stuff just been stuff that's managed to find a dead zone in when staff is online, or is it actually just more frequent?

[Mestiv he/him]

For last couple of days I noticed, that when I come to work at about 10-12 AM (Central Europe Time) there are no staff members online to delete the spambots topics. I wonder if the spammers learned that during this time of day all the mods are asleep and their topics will stay up longer

[Kestrel she/her]

For me it happens usually starting at 11 pm (central time) and then continues on until about 6 am. Sometimes there's a mod that comes online to clear it all off, sometimes there isn't. I stalk around the recent topics section a lot so myself or someone else would probably still notice if there were spambots coming on at 3 pm or so, but the messages were being removed faster.

[Moogle]

For last couple of days I noticed, that when I come to work at about 10-12 AM (Central Europe Time) there are no staff members online to delete the spambots topics. I wonder if the spammers learned that during this time of day all the mods are asleep and their topics will stay up longer

 

I think you're right. I can't even remember seeing any spam topics during regular working hours, it's all been during the dead of night (at least for me in North America). Sometimes I'm online past midnight (my sleep schedule has been awful of late, I'm usually until 4-5 AM) and there's like ten spambots I need to ban.

[Titan Arum]

Most of the spam has Hindi and Bengali script, which leads me to assume that the spammers reside in India. It's no surprise that they come on in the dead of night relative to the US. It's a 9.5 hour time difference between EST and IST and 12.5 hour difference between PST and IST. 

[Haelbarde he/him]

Solution: More international mods!

[Mestiv he/him]

Solution: More international mods!

 

I agree, I'd love to see some mods from Europe/Asia that could stop those bots, as there seems to be no other way to manually fight them every day :/

[Peng the Just he/him]

I've noticed they always post in General Brandon discussion. This may be weird, but what about making a new board in that same place and moving General Brandon discussion, then locking that board? they would try to post to that board and fail.

[Kaymyth she/her]

I've noticed they always post in General Brandon discussion. This may be weird, but what about making a new board in that same place and moving General Brandon discussion, then locking that board? they would try to post to that board and fail.

 

Ugh, no.  Moving topics/boards deletes the history that shows what you've read and what you haven't.  Picking back up in the right spot for a dozen different topics would be a nightmare.

[Rubix he/him]

I've noticed they always post in General Brandon discussion.

 

They actually post in many boards, not just that one.

[Comatose he/him]

Most of the spam has Hindi and Bengali script, which leads me to assume that the spammers reside in India. It's no surprise that they come on in the dead of night relative to the US. It's a 9.5 hour time difference between EST and IST and 12.5 hour difference between PST and IST. 

 

According to their IP addresses most are indeed from India.  

 

On the subject of banning according to IP address, I don't think I've ever seen the same address twice.  There are probably 5 or 6 common variations of the first 6 numbers in the address, but I'd be concerned that wildcard banning all of them would prevent genuine fans from certain regions of India from posting.  

 

On the subject of the spam bots timing their attacks, this may be true (or it may be actual people running the spam accounts).  It may be because more staff are online during North American daylight hours, but the most severe cases of spam I've encountered and dealt with have usually been near the end of the night if I'm up late, or early in the morning.  

[Herald he/him]

Do the spammers always have numbers in their post? I observed +, 91, few more digits in every post title (along with vashikaran). In such case, you can put a captcha after they hit submit or just prevent the post from being displayed until it's approved by the moderators.

 

Another way I think is to check the number of posts posted as soon as a member joins within 5 minutes or so. In case of spambots, there will be almost a post every minute for atleast 5 minutes with no other comments on any other thread. In this case, you can ban the user and purge all his posts immediately. But to use this, it needs to be confirmed if there can be a genuine active user who can fit this activity profile.

Edited September 16, 2015 by Jezerezeh

[Chaos he/him]

Do the spammers always have numbers in their post? I observed +, 91, few more digits in every post title (along with vashikaran). In such case, you can put a captcha after they hit submit or just prevent the post from being displayed until it's approved by the moderators.

 

Another way I think is to check the number of posts posted as soon as a member joins within 5 minutes or so. In case of spambots, there will be almost a post every minute for atleast 5 minutes with no other comments on any other thread. In this case, you can ban the user and purge all his posts immediately. But to use this, it needs to be confirmed if there can be a genuine active user who can fit this activity profile.

Those would all be wonderful things that I wish this software suite had.

[Sera]

 

[september 20 edit]: Now I've seen the posts here I'm pretty sure it's not manual spam, and thanks to the spam content, naming patterns and other details I suspect there's a single botmaster behind the current attacks. He added this forum registration Q&A answer to his bots DBs or has these updated bots I mentioned. In any case, changing the Q&A to a stronger one should completely stop the attacks. (:

 

 

Risking to repeat something already said in this topic because I'm a lazyass who skimmed over parts of this topic.
 
I'm a hobbyist programmer who has random tidbits of knowledge about security, the server-side of the web and similar topics. My view on spammers is: An obstinate spammer will spam regardless of the measures you adopt to prevent it. Fortunately, most spammers aren't obsessed with a single target, they weight the costs and benefits just like anyone else, and they tend to choose the low-hanging fruit.
 
Manual spam isn't really common nowadays, it's too costly for spammers (time). Clever bots are on the rise. They're costly, but ultimately pay off the investment spammers do because they're terribly efficient. Spammers automate a ridiculous number of tasks nowadays.
 
Bots & registration forms
 
A spambot can break classic image captchas easily. A captcha alone might stop more humans than bots now, hah. Those bots are also well-equipped to break Q&A captchas, having a database of common answers; they can do math, and no answer which can clearly be answered with a Google search is safe enough. They certainly are able to activate accounts all by themselves too. The best counter-measure is using a Q&A requiring human creativity/perception to be answered.
 
Original word puzzles like the following are fairly effective to stop bots: "Mary is reading a book on physics. She has 3 books on her shelves. Scattered on her table are a volume on geography, two tomes about history, one on English, and five assorted pencils. How many books does Mary have?". (notice the use of synonyms and mixed numerals)
 
Asking about the contents of an image also works. It should be an original image (so reverse image search gives no hints). E.g.:
Q: Name the item(s) in the scene below (in alphabetical order).

 
It's good practice to ask about subjects unrelated to the forum content. In the past, forum owners used Q&A like "what is the most popular section of the forum?", "Who created this forum?" and such, making bots evolve and learn how search for these easy answers.
 
Bot types
 
Classic spammer: Those bots which create a high number of topics with advertising. The majority is blatant spam, but some clever spammers setup their bots to be more discreet. They can be programmed to rotate between a predefined number of topic titles and content.
 
Link spammer: A pest clever enough to know how spam legit topics to increase the post count and bypass those forums which block new users topics until they reach X posts. A variation of these are the bots who post seemingly innocent answers but filled their website profile fields and signatures with links for the fishy sites they're advertising. They're fed a number of random answers that can look related to the topic/forum content at first glance, e.g.: "Hello, I'm new here, but I'd really like to thank you for the helpful post!", "Hi, I like X. Can you recommend me good X?", "Interesting post. I don't know.".
 
2015 bot activity
 
I'm a member of another forum which suffered vicious attacks, starting at the end of July. I ended helping to put an end to the attack just by ban-hammering the already existent fake profiles and implementing a good Q&A captcha in the registration form. I suspect a famous bot software was updated by the end of May because that's when the earliest attacks occurred. At first they were sparse, as if testing the terrain, once the spammers detected the registration form was bypassable they grew until we saw more than 100 new spam topics a day.
 
I don't know the nature of the attacks here, since I'm new, but I can say that those attacks we saw in the other forum weren't done by more than 3 different bots. It was easy to see the pattern of the usernames, timezone, activation delay (or lack of), spam style and content. The lowest point happened when we almost managed to stop the bots. They usually created a number of new profiles every day, then made 5 posts per profile. Once we got a decent Q&A, the one or two profiles which bypassed it went berserk mode and created ~45 posts each; and they also activated older, dormant profiles. After we patched the holes it became peaceful, not a single spam in ~2 months, and no added inconvenience for the real users.
 
That forum is relevant in its niche but isn't big, and the staff is really small; so constant moderation and manual user activation was out of question. The 17th Shard is a juicy target due its size, but I believe that it wouldn't make spammers persistent enough to keep track of a good rotating registration Q&A or spam manually once you manage to stop the bots. It's like a siege, survive long enough and they'll give up and move on to another target.

Edited September 20, 2015 by Sera

[Chaos he/him]

Sera, on 17 Sept 2015 - 2:02 PM, said:

[september 20 edit]: Now I've seen the posts here I'm pretty sure it's not manual spam, and thanks to the spam content, naming patterns and other details I suspect there's a single botmaster behind the current attacks. He added this forum registration Q&A answer to his bots DBs or has these updated bots I mentioned. In any case, changing the Q&A to a stronger one should completely stop the attacks. (:

Risking to repeat something already said in this topic because I'm a lazyass who skimmed over parts of this topic.

I'm a hobbyist programmer who has random tidbits of knowledge about security, the server-side of the web and similar topics. My view on spammers is: An obstinate spammer will spam regardless of the measures you adopt to prevent it. Fortunately, most spammers aren't obsessed with a single target, they weight the costs and benefits just like anyone else, and they tend to choose the low-hanging fruit.

Manual spam isn't really common nowadays, it's too costly for spammers (time). Clever bots are on the rise. They're costly, but ultimately pay off the investment spammers do because they're terribly efficient. Spammers automate a ridiculous number of tasks nowadays.

Bots & registration forms

A spambot can break classic image captchas easily. A captcha alone might stop more humans than bots now, hah. Those bots are also well-equipped to break Q&A captchas, having a database of common answers; they can do math, and no answer which can clearly be answered with a Google search is safe enough. They certainly are able to activate accounts all by themselves too. The best counter-measure is using a Q&A requiring human creativity/perception to be answered.

Original word puzzles like the following are fairly effective to stop bots: "Mary is reading a book on physics. She has 3 books on her shelves. Scattered on her table are a volume on geography, two tomes about history, one on English, and five assorted pencils. How many books does Mary have?". (notice the use of synonyms and mixed numerals)

Asking about the contents of an image also works. It should be an original image (so reverse image search gives no hints). E.g.:

Q: Name the item(s) in the scene below (in alphabetical order).

It's good practice to ask about subjects unrelated to the forum content. In the past, forum owners used Q&A like "what is the most popular section of the forum?", "Who created this forum?" and such, making bots evolve and learn how search for these easy answers.

Bot types

Classic spammer: Those bots which create a high number of topics with advertising. The majority is blatant spam, but some clever spammers setup their bots to be more discreet. They can be programmed to rotate between a predefined number of topic titles and content.

Link spammer: A pest clever enough to know how spam legit topics to increase the post count and bypass those forums which block new users topics until they reach X posts. A variation of these are the bots who post seemingly innocent answers but filled their website profile fields and signatures with links for the fishy sites they're advertising. They're fed a number of random answers that can look related to the topic/forum content at first glance, e.g.: "Hello, I'm new here, but I'd really like to thank you for the helpful post!", "Hi, I like X. Can you recommend me good X?", "Interesting post. I don't know.".

2015 bot activity

I'm a member of another forum which suffered vicious attacks, starting at the end of July. I ended helping to put an end to the attack just by ban-hammering the already existent fake profiles and implementing a good Q&A captcha in the registration form. I suspect a famous bot software was updated by the end of May because that's when the earliest attacks occurred. At first they were sparse, as if testing the terrain, once the spammers detected the registration form was bypassable they grew until we saw more than 100 new spam topics a day.

I don't know the nature of the attacks here, since I'm new, but I can say that those attacks we saw in the other forum weren't done by more than 3 different bots. It was easy to see the pattern of the usernames, timezone, activation delay (or lack of), spam style and content. The lowest point happened when we almost managed to stop the bots. They usually created a number of new profiles every day, then made 5 posts per profile. Once we got a decent Q&A, the one or two profiles which bypassed it went berserk mode and created ~45 posts each; and they also activated older, dormant profiles. After we patched the holes it became peaceful, not a single spam in ~2 months, and no added inconvenience for the real users.

That forum is relevant in its niche but isn't big, and the staff is really small; so constant moderation and manual user activation was out of question. The 17th Shard is a juicy target due its size, but I believe that it wouldn't make spammers persistent enough to keep track of a good rotating registration Q&A or spam manually once you manage to stop the bots. It's like a siege, survive long enough and they'll give up and move on to another target.

Hey Sera,

Thanks for the awesome analysis. We've created more strict, comprehension-based Q&A's that should be resistant to botting. There are a number of permutations, too, so the answers are different. I looked at the old Q&As, and yeah, they sucked, especially considering how bots can just Google things. We'll see how many spammers we get (and how many report emails I get :/) but it should be good. Check them out sometime. (Yes, I kind of stole your general format, haha.)

[Sera]

Hey Sera,

Thanks for the awesome analysis. We've created more strict, comprehension-based Q&A's that should be resistant to botting. There are a number of permutations, too, so the answers are different. I looked at the old Q&As, and yeah, they sucked, especially considering how bots can just Google things. We'll see how many spammers we get (and how many report emails I get :/) but it should be good. Check them out sometime. (Yes, I kind of stole your general format, haha.)

 

Hey, I "stole" the format idea from an official spam prevention guide, so that's fine! =P

 

The question looks good! Let me know if you need any more help.

 

Something to take into consideration: Sometimes the question is almost enough. Not as many bots manage to crack it, but one or another will and it will still spam. In those cases, don't panic! It's likely to be a more clever bot (rework the question a little) or it just randomly guessed it right (leave it as it is, or ask for answers with numbers written in full, using lowercase, to see if that helps).

 

P.s.: I see they're waking up the dormant profiles. E.g. this thread by something registered in May. That's a wonderful sign!

Edited September 28, 2015 by Sera