Rob, what the hell happened!


#1

Maybe you noticed- TAO has been down the past week. But good news! We’re back (almost).

The short version is:
[ol]
[li]Our server was shut down.[/li][li]I was lax and didn’t have a backup, which caused a long delay.[/li][li]We decided to switch hosts to a Windows server.[/li][li]I’m taking over sys admin duties going forward.[/li][li]Everything else (wiki, theme) should be back to normal in a couple days.[/li][li]If you want to be a moderator or administrator, or have some cool idea for something with TAO, tell me![/li][/ol]

The long version is:

Last week, our host shut down our server because spammers were using it to send email, by exploiting a vulnerability in the VB version we were using. This was unfortunate, because it was something we easily could have dealt with. It was also unfortunate, because we did not have an up to date backup. So our server was shut down, and we had to wait four days until we had an up to date backup.

Between the awful customer service to a long-term customer (did you know our FOURTH anniversary is coming up?), and my utter inexperience with Linux, we decided to switch to a Windows host. It took an additional two days to get the database ported over, all the software and host set up, and the necessary upgrades performed, mostly because I and Bjorn, our current server admin, had never done Windows administration.

The good news is, we’re in the clear. The scary parts are over- we just need to get the Wiki stood up, a new theme (the provider of our previous theme seems to be down or out of business and our theme was only VB3 compatible), and clean up some other style issues, and we’ll be as good as new. Better actually. The primary reason we moved to Windows is so I could take care of the server administration myself and stop burdening Bjorn, who does this completely out of the goodness of his heart and has spent more time administering the site in the past year than I have. It also allows us to use the server as a staging point for much more cool stuff, since I can actually facilitate it. An example is the planet RSS feed, which took so long to deploy because I actually got an AWS Windows server (which cost me almost as much per month as our TAO server) to run the script on. We can start to use the server to do some cool stuff.

The only persistent piece of bad news is, the hg repository for community code we had is gone. It will not be missed. Go use GitHub or BitBucket.

For me, the most important thing that has come out of this is: the buck stops here. I was the administrator of a forum I didn’t technically control, which was just a situation that should not have been. We lucked out- imagine if the server had been destroyed, and our host couldn’t get a backup. Poof, four years gone. You would have never seen me at another GDC and I’d probably have to leave the games industry. I knew I was playing with fire, but there was only so much that could be done without committing to it myself. Now I can, and I ask for your forgiveness and understanding, and vow to not let it happen again. I say that with the caveat that I can’t guarantee the forum won’t be hacked, so if you do have experience with Windows server administration, please tell me so we can audit the server to make sure I didn’t overlook anything when setting it up.

In the same vein, we can use some additional moderators, so if you’re an active member, just PM me and I’ll see about making you a moderator. TBH moderators just delete spam, I’ve had to edit maybe 8 posts in our 4 years, so don’t expect much glory. If you want to help administrate, please PM me as well. If you have some cool ideas for things we can do on the forum, or something you’d like to use the server for, or even if you want a host for your tech-art related website or project, please PM me. The switch to a Windows server has given me some new life and I hope to push some interesting projects out over the next year.

Thanks,
Rob


#2

Hey Rob glade to hear that every thing is back the way it should be. I like this ver of the site more then the last.
You are right it could have been really bad if you lost every thing. However you did not and every thing is the way it should be. talk to you soon
Ryan


#3

A dark theme is back - yes! My eyes thank you already.

Glad to see we’re back up and running - I’ve felt lost the past few days heh.


#4

Hehe, nice to hear the website is back up. I was feeling empty without my TAO fix. Might want to edit the template so I can see what I type into the login box though. Took me 3 times to figure out the text was white and not that my browser had crashed. Some aspiring TA I am. :wink:


#5

Yay, it’s back! :smiley: Was getting all panicky when TAO stopped loading, so good to see it up and running again.

Wuffles, odd that you’re getting white-on-white; I’m on Chrome, and I’ve definitely got black-on-white in the quick-login fields. What browser are you rocking?


#6

Great it’s back! My morning routine was really boring without TAO to read… good that’s solved now


#7

Good to see the site back up and running and sad to hear that you guys had to go through that much bs. If I would have known that there were so many issues with you guys using Linux I would have gladly offered my experience/expertise since I came to this industry from web dev.


#8

I was starting to worry TAO would be gone forever. I’m getting black-on-white in the login fields too and I’m on Safari 5.1.5 on Mac OSX, so it may indeed be a browser specific thing going on (the fun world of CSS hiccups!)


#9

I’m glad that you’re back up again! Good work sorting it all out :slight_smile:


#10

Excellent work! Good to see it back up.


#11

Ah, this is a good start of my day.


#12

Woo,

Glad to hear you’ve got complete control now, myself and a few friends were concerned when TAO dropped off the planet!

~ Joe


#13

Anndddd we’re back! Oh yeah!


#14

@LoTekk: I am on Firefox 12.0 - but it seems to be black-on-white now. Weird. :slight_smile: Either way - it’s great to see TAO online again.


#15

Thanks for getting this back up and on line. Whoo!


#16

Ahh, what a great piece of news to wake up to. :slight_smile: Thanks Rob!


#17

I concur! : )


#18

Now I can finally round out my day of forum reading :slight_smile:


#19

Oh, good! It’s back! Hi everybody!

FYI, the Windows Phone browser just shows a blank white page.

I checked on iPad as well and that works (and so I assume it works on iPhone, also).


#20

Hurah!

Keir

(Also I don’t mind helping out with moderator or administrator duties)