Forum instability
- fluffy
- Eruption
- Posts: 11087
- Joined: Sat Sep 25, 2004 10:56 am
- Instruments: sometimes
- Recording Method: Logic Pro X
- Submitting as: Sockpuppet
- Pronouns: she/they
- Location: Seattle-ish
- Contact:
Forum instability
Lately it seems that the forum is generating Internal Server Errors a lot, and I suspect this is simply Dreamhost having overallocated our server, as they tend to do. I'll see if there's anything we can do on our end but most likely we're going to need to get Dreamhost to migrate our site to a different server or something.
- fluffy
- Eruption
- Posts: 11087
- Joined: Sat Sep 25, 2004 10:56 am
- Instruments: sometimes
- Recording Method: Logic Pro X
- Submitting as: Sockpuppet
- Pronouns: she/they
- Location: Seattle-ish
- Contact:
Re: Forum instability
Okay, it looks like we were being DoSed by some sort of botnet from China for some reason (like, we were getting thousands of hits a second from a single IP block all from spoofed mobile user agents with no referrers). I've blocked them from accessing the forum, hopefully that helps.
- Lunkhead
- You're No Good
- Posts: 8156
- Joined: Sat Sep 25, 2004 12:14 pm
- Instruments: many
- Recording Method: cubase/mac/tascam4x4
- Submitting as: Berkeley Social Scene, Merisan, Tiny Robots
- Pronouns: he/him
- Location: Berkeley, CA
- Contact:
Re: Forum instability
Ugh, that stinks. Thanks for handling that. Where would I go to see that kind of info? Is that in the boards admin/mods area somewhere or is that in the Dreamhost account admin?fluffy wrote:Okay, it looks like we were being DoSed by some sort of botnet from China for some reason (like, we were getting thousands of hits a second from a single IP block all from spoofed mobile user agents with no referrers). I've blocked them from accessing the forum, hopefully that helps.
- fluffy
- Eruption
- Posts: 11087
- Joined: Sat Sep 25, 2004 10:56 am
- Instruments: sometimes
- Recording Method: Logic Pro X
- Submitting as: Sockpuppet
- Pronouns: she/they
- Location: Seattle-ish
- Contact:
Re: Forum instability
Neither, I went into the server logs:
And I blocked them by adding them to the songfight.net/.htaccess file.
Anyway the forum's already feeling WAY more responsive, go figure.
Code: Select all
[scott]$ cd logs/songfight.net/https
[scott]$ cut -f1 -d\ access.log | sort | cut -f1-3 -d. | uniq -c | sort -n | tail -n 20
121 162.233.200
127 109.70.100
131 78.46.176
156 40.77.167
156 54.36.148
160 76.121.51
181 199.249.230
223 176.31.101
233 167.114.1
298 144.76.38
301 178.63.87
356 144.76.14
763 52.83.219
831 157.55.39
928 91.242.162
993 188.138.188
2275 207.46.13
4241 46.229.168
115257 220.243.136
115335 220.243.135
Anyway the forum's already feeling WAY more responsive, go figure.
- Lunkhead
- You're No Good
- Posts: 8156
- Joined: Sat Sep 25, 2004 12:14 pm
- Instruments: many
- Recording Method: cubase/mac/tascam4x4
- Submitting as: Berkeley Social Scene, Merisan, Tiny Robots
- Pronouns: he/him
- Location: Berkeley, CA
- Contact:
Re: Forum instability
Awesome, thanks again. I always wondered if there were access logs somewhere. I don't suppose we're running a log analyzer are we?
- Lunkhead
- You're No Good
- Posts: 8156
- Joined: Sat Sep 25, 2004 12:14 pm
- Instruments: many
- Recording Method: cubase/mac/tascam4x4
- Submitting as: Berkeley Social Scene, Merisan, Tiny Robots
- Pronouns: he/him
- Location: Berkeley, CA
- Contact:
Re: Forum instability
Looks like there is an analyzer running, cool.
- fluffy
- Eruption
- Posts: 11087
- Joined: Sat Sep 25, 2004 10:56 am
- Instruments: sometimes
- Recording Method: Logic Pro X
- Submitting as: Sockpuppet
- Pronouns: she/they
- Location: Seattle-ish
- Contact:
Re: Forum instability
Yeah, although it doesn't really do much useful (it's a crusty old instance of awstats that only does the same analysis that's trivial to do from the command line) and I don't know what the password to the dashboard is anyway.
- fluffy
- Eruption
- Posts: 11087
- Joined: Sat Sep 25, 2004 10:56 am
- Instruments: sometimes
- Recording Method: Logic Pro X
- Submitting as: Sockpuppet
- Pronouns: she/they
- Location: Seattle-ish
- Contact:
Re: Forum instability
They mostly happened when searching or posting. Ken first reported it a couple days ago and it seemed transitory, then I started getting them a LOT. I could dig into the logs to see when this Chinese bot swarm started; the logs show there were hundreds of machines constantly requesting every forum URL at a rapid rate, purpose unknown.