TechDonkey.com

Our server fainted some time this morning. I became aware of it at 10am, and have been in contact with the folks who manage the hardware. They’re tightening bolts, greasing cogs, and heaving wrenches around as I type. No, wait, that greasing cogs thing was a what machine techs did a century ago. They’re, instead, checking hardware and running diagnostics. Watch this space.

I’ve just discovered the outage, and am taking steps to get it kickstarted. Watch this space.

Edit, 1:41am: The rebooting is proceeding. Hold your breath!

Edit, 1:44am: Hooray! We’re up and running again.

Edit, 1:49am: Okay, we’ve got HTTP and POP3 and all, but the database server is still hiccuping. Watch this space…

Edit: 2:05am: Everything seems operational!

Please watch this space for updates, or contact me through this site if you need to!

-Elaine

Edit:  Dec 16, evening

The hard drive on the server failed. A new drive has been installed, and backups reinstated. Everything should be up and running now — but please let me know if anything is still awry for you.

The MySQL server’s down for anyone on Abaris server (that’s most folks). I’m on it and will keep you posted here.

edit: I just found out that SMTP (mail SENDing from yourdomain.com) seems to be affected as well, which is odd as they’re not in the least connected. I’ll look into that as well. POP3 (Mail receiving) is working fine.

edit: 3:46pm PST: MySQL is going up and down like a yo-yo. I’m still on it. Bugger.

edit: 6:34 pm: I’m getting a huge number of very different error messages from MySQL — but a few seem to tell the tale that we may be seeing a DOS attack, either accidental (a script gone wrong, querying the database over and over and over and over and over and over) or on purpose (a script written to go wrong, querying the database over and over and over and over and over and over). Watch this space for updates.

edit: 11:19pm: The database has settled somewhat, but is still blanking at intervals. We’re still working on it!

Resolved: The rogue script has been found and stomped upon. If you have any database troubles now, contact me, and I’ll investigate!

Right about now, the “signal” from my main hosting server is interrupted someplace between my ISP and the server farm, and it’s being dropped instead of re-routing. In layman’s terms, the phone line is frayed between here and there — and sadly, it’s not frayed in the place that I control.
I can’t FTP, collect email or access the sites I run without hideous proxy contortions. I have people on it, but if you’ve tried to reach me, that’s what’s up.

Watch this space.

-Elaine

DEC 3, morning — update

Everything seems to be  righted. A firewall at the server end had gone a bit rogue. It’s been soundly thumped, and should behave slightly better hereafter.

7:59 PM. PST

We’re experiencing the same troubles as before.

I’m communicating with the server techs again to try to straighten it out. I’m also gnawing on the ears of the folks who manage the server farm about settling the server thing once and for all.

If you send using your domain’s SMTP, swap temporarily to your ISP’s (eg: Telus or Shaw). The SMTP is unreliable in routing.

More info as it happens.

1:26am September 23

Okay, we discovered the cause.

One of the accounts on the server was so bombed by incoming spam that the entire mail queue choked.

Things should be back to normal for you. If not, please get right back to me via the support interface (link to your right —>  )

I hate spammers. But you knew that, right?

Today, since about 3pm PST, we’ve all been having some email issues.

Both SMTP and POP, plus mailman mailing lists: all are affected.

I’ve been working with the on-site server tech for the last hour to resolve this. More info as it happens.

If you need to reach me, elaine *at* techdonkey *dot* com is the way to go.

11pm update:

After a great deal of harassing my hapless server cowboys, things are coming back online. Some sent mail and some rec’d mail is intact, but I’m not yet getting an accurate count of what’s working correctly.

noon Sept 15th update:

Everything seems to be working correctly, according to our tests.

CPanel has been upgraded, and this apparently may cause some changes in the optimum email account setup. I’m researching it, and will send out an announcement if it affects you.

If you have lingering email issues, please open a Support Ticket here, and I’ll get under the hood again.

Sorry for the technological inconveniences!

-Elaine

Somewhere around 12:19pm PST (give or take a millisecond), the database server (MySQL) for all our sites fainted.

I’m on it. Watch this space.

Update: 12:30pm

It’s up and running. I’d like to state that I did something extraodrinary, but I’d just started kicking the tires when it revived and bounced back to work without a murmur of explanantion. Okay, MySQL server, if that’s the way you’d like to handle it. I see how you are.

Every DNS transfer has resolved, as far as I know.

If you own a site that has not resolved (eg it has a blue page that says “there is no website configured at this address”) contact me and we’ll troubleshoot your DNS troubles.

I’m working on re-upping the sites I directly geek for (eg: I built it and run it for you). If you need it faster, let me know, because in the absense of other info, I’m going alphabetically.

Again, if you have a recent backup, let me know.

more soon.

-Elaine

I have limited access to the old server, for those few folks who are having a hard time getting the DNS records changed. It’s web only, no cpanel, no frigging email.
If you need to be temporarily re-upped on the old server, let me know.

Note that this won’t bring our old sites back. It’s just moving back into an empty house that’s about to be torn down, just in case you need to be found during the transfer.