bosconet.org: chocolately writings

I finally took a few minutes this afternoon to get Apache back up and running.

It failed to come back after the server restarted a 2 weeks ago today. Why did the server restart you might ask. Because the collocation facility where it is house mucked up something with the back-up power. The story is they while doing a weekly test of the generator the switch back to main power failed and the UPS batteries failed before they could get the mains back online or the generators running again.

While I got the most critical functions of the server back as soon as I could (and it was non trivial thanks to an issue with my RAID 1 mirrored config and a corrupted boot file) and restarted critical functions (my mail server and a 2nd mailserver that is relied on for a business). The web server and blog could wait. And wait they did all the way until today.

Now it is back and I am happy.

Oh and two other things the power outage claimed 1 server immediately and I hold it responsible for a firewall failing last week. And I have to say I am no fan of software RAID 1 mirroring on this machine. It made recovering about 10x as hard b/c I had to unmirror the root partition.

Paul Customer Service, Random, Technical

This morning after being frustrated with getting a USB drive to function properly on linux machine I turned my attention to breaking into an APC Power Switch I had acquired but didn’t have the opportunity to do a reset and recover before I had to install it. This would be simple if I could reset the switch during a reboot, unfortunately in doing this I would reboot several servers that needed to stay up and available.

So I went searching a bit and found that this APC Power Switch is vulnerable to a local attack that would allow you to find what the current administrative user account is defined as as well as the current password associated with that account. The vulnerability was in a different, more recent, version of the software currently being run by the device. I was hopeful that my older version would also be vulnerable.

The first step is to connect to the serial port on the device using a standard null modem cable running at 2400baud 8-N-1. Once you get a User Name : prompt enter apc and the password: TENmanUFactOryPOWER

This will drop you to the Factory Menu that will look like this:

User Name : apc
Password  : *******************

Factory Menu
<ctrl -A> to exit

1AP9606
2WA0033002610
310
408/08/2000
500 C0 B7 63 FA F1
6v2.5.3
7A
8A
9198.180.62.87
A255.255.255.240
B198.180.62.81
C
D
E
F
G

The instruction I found indicated at this point to enter 13 which will drop you to this prompt:

Selection> 13

Enter byte address in Hex(XXXX):

The instructions further indicated that the address to view taht would contain the information was: 1d10. However in my version of the software this is what I saw:

Enter byte address in Hex(XXXX): 1d10

1D10   FF FF FF FF FF FF FF FF  ........
1D18   FF FF FF FF FF FF FF FF  ........
1D20   FF FF FF FF FF FF FF FF  ........
1D28   FF FF FF FF FF FF FF FF  ........
1D30   FF FF FF FF FF FF FF FF  ........
1D38   FF FF FF FF FF FF FF FF  ........
1D40   FF FF FF FF FF FF FF FF  ........
1D48   FF FF FF FF FF FF FF FF  ........
1D50   FF FF FF FF FF FF FF FF  ........
1D58   FF FF FF FF FF FF FF FF  ........
1D60   FF FF FF FF FF FF FF FF  ........
1D68   FF FF FF FF FF FF FF FF  ........
1D70   FF FF FF FF FF FF FF FF  ........
1D78   FF FF FF FF FF FF FF FF  ........
1D80   FF FF FF FF FF FF FF FF  ........
1D88   FF FF FF FF FF FF FF FF  ........

<sp>nxt,b-bck,p-pch,other-exit

Obviously there is no useful information there. So I started to systematically tr other address locations to see if the information I was seeking was someplace else. After about 5 minutes of trial and error I found the information I was seeking at 1c2:

Enter byte address in Hex(XXXX): 1c2

01C2   FF FF FF FF FF FF FF FF  ........
01CA   FF FF FF FF FF FF FF 50  .......P
01D2   46 61 64 6D 69 6E 00 64  Fadmin.d
01DA   6D 69 6E 00 68 69 68 61  min.hiha
01E2   74 31 31 00 00 00 FF 64  t11....d
01EA   65 76 69 63 65 00 FF FF  evice...
01F2   FF FF 61 70 63 00 FF FF  ..apc...
01FA   FF FF FF FF FF 72 6F 62  .....rob
0202   68 61 72 72 69 73 68 61  harrisha
020A   73 61 62 61 62 79 00 FF  sababy..
0212   FF FF FF FF FF FF FF FF  ........
021A   FF FF FF FF FF FF 64 65  ......de
0222   76 69 63 65 20 75 73 65  vice use
022A   72 20 70 68 72 61 73 65  r phrase
0232   00 FF FF FF FF FF FF FF  ........
023A   FF FF FF FF FF FF FF 00  ........

<sp>nxt,b-bck,p-pch,other-exit

In case you don’t see it the userid is admin and the password is hihat11

After gaining that information I was able to easily login, and reset the password to something I will easily remember as well as configuring the network interface to enable web management of the device.

Paul Technical

I was looking at the DNS servers I admin last week and noticed, among other things, the following error message showing up frequently in the system logs:

Feb 23 10:23:08 baltimore named[14105]: [ID 873579 daemon.info] \
edns-disabled: info: too many timeouts resolving \
'171.221.32.207.sbl.spamhaus.org/TXT' \
(in 'sbl.spamhaus.org'?): disabling EDNS

This looks like a problem that should be fixed. I googled this error message and 9 out of 10 replies to people asking how to deal with this error was (and I’m paraphrasing)

“Oh just ignore it, and here’s how to configure logging to not log those errors”

WTF? How about some helpful information about why this is happening and how to fix it? Or more information why it is happening so a person can be informed about what is happening and the root causes.

here is a posting from Mark Williamson the bind-users mailing list that does provide some useful information that can be used to made an informed decision if you want to disable logging these events:

"disabling EDNS" is issued when named experiences too many
timeouts to EDNS queries and named decides to give up on
EDNS and revert to plain old DNS.   Now timeouts can be the
result of many things.  Broken nameservers that don't respond
to EDNS queries.  Firewalls that block EDNS queries.
Firewalls that block fragmented responses.  Firewalls/NATs
that don't handle out of order fragments.

Timeouts can also be due to other network problems including
unreachable servers.

If you are getting lots of these then you do have network /
firewall problems.  They may however *not* be caused by EDNS.

The message has the symptom "too many timeouts", what it
was trying to do "resolving 'ns.cmmail.com/AAAA' (in
'cmmail.com'?)" and what named doing "disabling EDNS" to
try to rectify the problem.

based on that information I think I will be disabling these messages.

Paul Opinion, Technical, Uncategorized

I finally bothered to upgrade to Wordpress 2.7 tonight. It took a little longer than I would have liked but that is because of the various plug-ins I have installed, several needed upgrading which took some additional time. But now I’m up to date.

Administrator Technical

For the past few days at work I’ve been banging my head against my desk because I can’t see to get VMWare Server 2.0 to remember virtual machines after a reboot.

I tried many things including creating new directories and setting the permissions on them to 777 (accessible to all). None of those things helped. Then it dawned on me….maybe it was something else….

The what else was the file system I am using is a software RAID 5 array mounted to /export. This file system isn’t mounted until last in the boot process, after vmware started. Perhaps that was the problem. And after a restart of the vmware process it did indeed turn out to be the problem.

My quick and dirty solution is to restart the vmware service immediately after I mount /export.

This also indicates VMware does not dynamically read a data store but only once on boot and then just adds to it if you create a new Virtual Machine. Given my experience this doesn’t seem to be an ideal way to do this.

Paul Technical

I mentioned recently that IMAPS is being shut off for my work email. Well they just sent out a 2nd notice with some justifications. The one that JUMPS out at me is that (and I’m paraphrasing): “in order to provide POP and IMAP services we use approximately 50 IPs on our DMZ. “

Can someone with some insight into the inner workings of exchange explain why so many IPs would be required to provide these services? Thanks.

Paul Uncategorized

I work for a large consulting firm and my employer has decided that instead of empowering their employers they will handcuff them.

Specifically my employer has decided to limit its consultants to only access our email via internal access to an Exchange server, or external access via either Outlook Web Access (a web mail client for Exchange) or via our corporate Blackberry BES (Blackberry Enterprise Server). The later is only an option for connectivity if you are either senior enough and get a company issued Blackberry or have management sign off on allowing you access the company owned BES.

This means for the average consultant at my company we will no longer have easy access to their email outside of the office. Will this impact the rank and file’s ability to be responsive to both internal demands and our customers? In my ever so humble opinion, yes. Will this be noticed by the people who made this decision? Probably not because they are disconnected from the the people who actually generate revenue for our firm.

The sad part is that in the name of security they migh end up encouraging insecure practices. For instance some people might start using external email addresses for business purposes which exposes potentially sensitive corporate data to outside parties. This didn’t have to happen either it is entirely possible despite this statement “IMAPS - it turns out the “S” is pretty darned weak” [1] to allow for a publicly available email solution.

Sadder still is that unless this impacts someone senior enough, which it won’t because they all have company issued blackberries, it will happen no matter how much the rank and file might complain, which they won’t anyway.

[1] I asked for clarification on this statement on Tuesday and to date have not received a response.

Paul Opinion, Uncategorized

Here in Charm City we are experiencing the FRIGID temperature’s and the president elect is stopping to speak in the city this afternoon. Because of the security insanity involved with any presidential appearance if you are heading down you need to follow all sorts of rules about what you can bring inside the security zone. Here is what is prohibited:

• Weapons
• explosives
• aerosols
• laser pointers
• Packages
• coolers
• thermal or glass containers
• backpacks
• structures
• bicycles

The weapons and explosives make sense. Aerosols, maybe, but this is Baltimore what’s a good Hon supposed to do? Leaver her Aqua Net at home? What if she get’s a chance to meet the president to be? She might need to fresher up her do and look her best…..sadly fashion will have to take a back seat to ’security’. Laser Pointers make a little sense but unless you do a FULL inspection of everything a person is carrying you can’t hope to catch one as they are too small and easy to conceal. Heck I have one that is on the end of normal looking pen. Good look finding that unless your screeners are REALLY looking at everything. Now packages? well that just seems impractical to bring to an event such as this. Coolers? Come on it is colder out than in a cooler…..again impractical for today. Now thermal containers? Today when the expected high is only going expected to be 23? That seems foolish. Even more foolish when you learn that there will be no concessions near by when you could hope to purchase a hot beverage to help warm up. Backpacks? Come on, inspect them but as long as they don’t contain any contraband let them through. Structures? What do they mean by that?!? An Bicycles make some sense since with the crowds they would just get in the way.

I predict numerous problems with the cold from today’s event, partly from people being dumb, partly from poor preparedness by local emergency services, and partly from security restrictions preventing people from brings say….warm beverages with them…..

Hopefully any problems encountered today will be learned from and applied to the inauguration on Tuesday.

[update: I just learned on WBAL TV that Donna's will be inside the security perimeter service food and HOT beverage. Smart move.]

Paul Baltimore, Opinion

Danielle bought me this device for my birthday. I had been hesitant to do so for myself for a couple reasons. The primary one was some of the reviews I had seen mentioned issues with video quality and streaming issues. The secondary one was I didn’t really want yet another A/V component and remote around. But I am glad Danielle didn’t consider any of this and just bought it because it is a wonderful device for my wants/needs.

The quality and streaming issues I have yet to see any problems with. For the former it is probably related to the fact that I’m still using an older standard definition CRT TV. So unless something is very noticeably pixelated or the like I don’t notice. As for any streaming issues I credit my ISP () with providing me a good quality connection.

For me the roku added to my Tivo’s with Amazon Unboxed makes me happy and have zero desire to get an Apple TV even with something like Boxee added to it (via a hack).

Paul Opinion, Technical

Since I had a gift card and a want for a phone with a better web interface I bought a (ancient) unlocked Blackberry 7130. It cost me $2 out of pocket so I felt it was a justified impulsive purchase.

It arrive today, and tonight I set it up. I was frustrated for a long while because despite copying the exact network setting I had on my old phone (a Sony Ericsson K750i). Every time I would turn the wireless on the message “Data Connection Refused”. This was very frustrating, and I was beginning to think I might need to spend more and upgrade my T-Mobile service to one of the Blackberry options.

Thankfully I just ignored it and tried Opera Mini and it successfully loaded a web page.

Now I can go to bed happy. Tomorrow I need to figure out how to set up a personal email account which doesn’t seem to be an option at the moment.

Paul Uncategorized