Tuesday, January 13, 2004

I forgot to blog this day, so I'm doing it now (Wednesday).

I also forgot what happened today I think.

I got rung up at about 10.15am. Damn, I slept through my alarm, this getting up is really hard, I want to go back to sleeping all day.

"Are you in Gosford putting the server back yet?", "uh, no, I'm still in bed", "well we can't get to the other server", "great, I'll look at it when I get there".

Maybe it's been stolen or moved.

I ate some breakfast, got dressed, loaded the server into cardboard box, and strapped it to my bike.

I very carefully rode into Gosford, dragged the server inside, (noticed it's about 10:45am) put it back in the rack where it belongs, plugged some cables into it that should work, and booted it up.

it booted up, but apache failed to start, because dns was broken, on both the machine itself (hmm, is the network broken too?) and on a backup server (dns is probably down on it, it usually is).

I pulled the console out, and plugged it into the backup server. Yep, dns was down, ok, I started it up manually. Ah, I see the reason, when I updated named/bind on here, I put a debian init.d script on, which won't work on Red Hat. I must fix this (yeah, haha).

I was able to log into the missing server, apache did not start though, and named/bind did not start. No worries, I'll just restart bind and then apache. Uh, no sudo. Uh, I don't know the root password.

Ok, no biggie, maybe I can get in with webmin, since I might know the root password for webmin, and then change root's password. Uh, webmin did not start. Ugh.

Ok, time to break in. Reboot, lilo= linux init=/bin/sh

This should have worked. Dumb Red Hat. libraries scattered everywhere, required partitions not mounted. ls does not work, vi works, but everything is read only. mount does not work. Hmm.

Here's a Tom's root boot disk. I booted the machine with it, I can't mount anything. The kernel on this disk claims to not support e2fs, so I can't mount any of the partitions. wtf.

Ok, I decided to go back to that backup server, go download a new version of this boot disk, recreate it, and try again. I went back over, found the home page, started downloading the file, I'm only getting 1k/sec, this can't be right. I just waited.

I went back to the other server, booted it up again, this time a different website failed to resolve while apache was trying to start, maybe we're getting somewhere. I went back to other server again, the download was finished, I saved it.

I checked apache on the missing server, it was up, and serving. I checked dns, bind seemed to be working, webmin was also up. Ok, so the box is back, and operating normally, now we just have the issue of the unknown root password.

I was able to use a windows machine in the ajoining rack, since webmin is impossible to use in lynx, logged in, went to the "other" tab, went to commands thing, where you can enter single commands to execute.

I was sneaky, I put in "echo 'toor:x:0:0:root:/root:/bin/bash' >> /etc/passwd" and then "echo 'root::11622:0:99999:7:::' >> /etc/shadow"

Now I was able to log in as "toor" instead of root, with no password, and what do you know, I'm in /root as root. passwd root, to something I know, and then vi /etc/passwd and /etc/shadow and delete "toor".

OK. I'm all finished here. It's only like 1.30pm. That should not have taken 3 hours.

While I was trying to do the above, I had several people calling me, some guy wanting his computer fixed, the guy who had the missing server, and both the guy I go to dinner with, and the guy who fiddles around on the servers (several times calling me) trying to find out where I was up to. This probably did not help the amount of time it took.

I was just about to leave, I went to tell them I was finished, some chick goes "I can't get on my F drive and my email doesn't work". ("Uh, do I care?"). "Well, I was working on my equipment, I had nothing to do with your network or your servers" "Well it only happened since you've been here, so you must have done it" "yeah, whatever, let me look". I was hoping to find something dumb like her typing something in wrong or windows shitting itself and needing a reboot, so I could be smart that it was her problem.

No such luck, after a reboot, it claimed the domain server was unavailable. I then ran ipconfig "Cable Disconnected" ah, have a look, maybe the stupid woman has kicked it out. Damn, nope, still plugged in to the network card, and the wall, no link light though.

I went back to the bloody rack I'd just sat in for 3 hours, all the workstation cable runs come into a patch panel in the top of this rack.

I rebooted both switches all these runs are plugged in to, no change.

I started pulling them all in and out of the switches, no luck.

I lost the plot here, I was hot and bothered, very sore from my sunburn, I did not want to be crawling around under desks and stretching up to cables coming out of the roof. Here I was, in behind the rack, swearing as loud as I could under my breath while angrily pulling cables around. This wasn't even my fault. If this server had not have been moved, I wouldn't have even been here, and when her network run broke, it would have been their problem. This is not productive. I have to get out of here, I'm supposed to help my mate move equipment out of the storeroom in the shopping centre.

I gave up, I found a longer network cable, I plugged it in to her network card, and ran it around the corner, and plugged it into the switch that was hanging there, to make up for the fact there are not enough ports on the wall. Bang, link light up, IP issued, home drive accessible. I'm outta here.

I told the guy I was finished, and about the busted network link I fixed, that was not my fault, he was appreciative at least.

He tells me again about the fact they are moving shortly. This is a hassle, I don't think we can, I don't even know what services terminate in here.

I left, I jammed the box back on my bike, and fanged it up the freeway to the shop.

I had forgotten everything I intended to do today, all wiped out by screwing around putting that server back.

I fiddled around on all the cafe pcs, cleaning up virii, and removing spyware.

The solicitors came again, we looked through the bills, tried to sort out payment of some of them, had a chat about things, and ended up there until about 5:30.

I went from there to my mates place, sat around there for a while chatting about what had gone on with the server, and I got a bit carried away, I confessed that I was worried about the fact I'm running out of money, and I should have worked a proper paid day on Monday, but instead I came into sort things out.

We decided that something would be done about getting paid properly.

We went to dinner, I wasn't particularly hungry, still feeling a bit sick being sunburnt.

Went back from there, I usually sit round for ages, but I was too sore.

I came home and got in the bath, tried the stuff my parents told me about yesterday, I was no bloody good either, my skin is all stuffed.

Trying to sleep is a nightmare.