How To Convert Large XML Files to CSV

[ad]

I usually struggle with converting very large XML files to other formats just because they are in a dynamic format and most programs you find run out of memory before properly parsing them.¬†Well, I’m happy to say, I found a FAST and EASY solution. Of course this will work for small files as well as big files.

You’ll want to grab a copy of the msxsl command line utility from Microsoft.

After you’ve got that, you’ll need to setup a XSL file to tell the program how to format your file. If you’re unfamiliar with XSL, you can familiarize yourself here.

After you’ve got your XSL file created, it’s a simple command line entry:

msxsl xml_file.xml xsl_file.xsl -o output_file.csv

The following is a sample XML and XSL file that I used.

XML File:

	
		772500
		Tue, 20 Jan 2009 16:28:08 CST
		Tue, 20 Jan 2009 16:51:01 CST

			61951
			The Hills Season 1

		773000
		Tue, 20 Jan 2009 16:28:08 CST
		Tue, 20 Jan 2009 16:53:54 CST

			61926
			Hogan Knows Best Season 2

		775500
		Thu, 22 Jan 2009 14:49:12 CST
		Thu, 22 Jan 2009 14:51:35 CST

			62068
			Carlos Mencia 2007

			1402
			Comedy Central

XSL File (Creates tab-delimited file)








	

How to mount Amazon EC2 EBS volume on an instance’s file system

[ad]
The following assumes you have created a volume in AWS Management Console and attached it to your instance as /dev/sdh.
 
A brand new volume is unformatted block storage.  It shows up as another device, but doesn’t contain a filesystem.  (An EBS volume that previously had a filesystem created on it won’t be mounted either by the attachment process).
 
You can verify that your instance can see the EBS volume by running the following command on the instance:
 
cat /proc/partitions
 
You’ll see an entry for sdh.  At this point, you should create a filesystem and mount it manually.  You could create an EXT3 filesystem on the device and then mount it on ‘/mnt/ebs’ with the following commands, (run as root):
 
mke2fs -F -j /dev/sdh
mkdir /mnt/ebs

mount /dev/sdh /mnt/ebs 

Android 1.6 Version Update (Donut) adds new Power Control Widget

I’m a developer and have made a few apps for android, so I updated to the newest SDK today, Android 1.6, named "Donut". It’s supposed to roll out in October.

One of the first new things I noticed was a nifty power control widget, which gives you the ability to quickly toggle wifi, bluetooth, gps, data syncing, and brightness. Not a particularly big deal, because I’m currently using worthy widgets downloaded from the market, but it’s nice to have it natively for sure. Still waiting on a 3g toggle though…

[ad]
 

PHP File-System Cache

[ad]

I’m just sharing a couple of functions I created a while back to cache some resource-intensive processed dada for quick and easy access. This is pretty ideal for large amounts of data and it’s very simple to set up. The beauty of this is that you can store just about any data type – it doesn’t have to be a string.

Just setup the cache directory to have proper read/write permissions (preferably in a non-accessible from web directory). Then use the two functions.

Example usage:

1
2
3
4
5
6
7
8
9
// The 2nd argument, $hours is how long to retain data before getting new
$contents = get_cache('TEST_KEY', 24);
if ($contents === false) {
	// This is where you'd get data from an API, DB or whatever
	$contents = 'Just some example contents';
	set_cache('TEST_KEY', $contents);
}
echo $contents;
}

Code:

1
2
3
4
5
6
7
8
9
10
11
12
13
define('CACHE_DIR', 'cache/'); // Include trailing slash
 
function get_cache($key, $hours) {
	$file = CACHE_DIR . md5($key) . '.cache';
	if (!file_exists($file) || filemtime($file) < time() - $hours * 3600)
		return false;
	return unserialize(file_get_contents($file));
}
 
function set_cache($key, $value) {
	$file = CACHE_DIR . md5($key) . '.cache';
	file_put_contents($file, serialize($value));
}