Python: How to parse XML/RSS feeds with namespaces using lxml.etree

Alright, this one had me stumped for a good hour or two.

Take for example a Flickr RSS feed.

image

Those namespaces are a pain, but it's not too bad if you can sort them out before you use them.

# Some basic setup
from urllib2 import urlopen
from lxml import etree

# Namespaces copied straight out of the feed source
namespaces = {
'media': "http://search.yahoo.com/mrss/",
'dc': "http://purl.org/dc/elements/1.1/",
'creativeCommons': "http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html",
}

# Untested fetching code, just for understanding
file = urlopen(feed_url)
xml = etree.parse(file)
item = xml.get_root().find('channel')[0]

Now here's the basic structure of an RSS feed item.

<item>
<title>For the lazy</title>
<link>http://www.flickr.com/photos/handles/7429952158/</link>
<description>blah blah blah</description>
<pubDate>Sat, 23 Jun 2012 21:59:07 -0700</pubDate>
<dc:date.Taken>2012-06-24T14:58:54-08:00</dc:date.Taken>
<author flickr:profile="http://www.flickr.com/people/handles/">Handles</author>
<guid isPermaLink="false">tag:flickr.com,2004:/photo/7429952158</guid>
<media:content url="http://farm6.staticflickr.com/5347/7429952158_962a849b30_b.jpg" type="image/jpeg" height="1024" width="768"/>
<media:title>For the lazy</media:title>
<media:thumbnail url="http://farm6.staticflickr.com/5347/7429952158_962a849b30_s.jpg" height="75" width="75" />
<media:credit role="photographer">Handles</media:credit>
</item>

To get information off those elements, you'll need some slightly different syntax.

# Now to fetch the data from the namespaced elements
media_title = item.find("{%s}title" % namespaces['media']).text

media_thumbnail = media_title = item.find("{%s}thumbnail" % namespaces['media'])
thumbnail = {
'url': media_thumbnail.get('url'),
'width': media_thumbnail.get('width'),
'height': media_thumbnail.get('height'),
}

taTtM
Problem solved, like a boss!

Source

Diablo III: Extracting graphics from .tex files from MPQ and converting them to PNG (or whatever you want)

As always, once I'm bored with a Blizzard game I usually snoop around to see what's in the MPQ files. Diablo 3 is no exception.

Ingredients

Recipe

OK, I'll be honest. I'm going to assume you have a fair level of computer browsing competency. Some of these tools are still in their infancy so they're not the most user friendly.

If you're gonna need help figuring out how to extract things, navigating folders on your computer or editing batch files, you might want to get someone to help you.

  • Start up MPQ Editor
  • Open "Diablo III\Data_D3\PC\MPQs\Texture.mpq"
  • Browse to the "Textures" folder and extract the files you want to "C:\d3textures\" (or whatever folder you want)
  • Feel free to close MPQ Editor if you want.
  • Also extract "xvi32.exe" and "D3Texconv.exe" into that folder.

Now for a little explanation to the stuff you're about to do.

If you go ahead and attempt to convert the .tex files, you're gonna get an error saying "Invalid Texture or Surface (SNO Header Mismatch)". I'm not going to pretend I know what that means.

The whole reason why we have a hex editor is because we can tweak the files to be correct. But of course, we're lazy and we don't want to do it by hand to all 17,000+ TEX files.

  • Now create "textures.xsc", edit it in notepad and paste the following:

ADR 0
REPLACE EF BE AD DE 2D BY EF BE AD DE 2B

  • Save and exit.
  • Create another file called "texfix.bat", open up for editing in notepad. Paste the following (and change your paths if required)

@for /f "tokens=*" %%a in ('dir /b *.tex') do ( "C:\d3textures\xvi32.exe" %%a /S="C:\d3textures\textures.xsc" )

  • Save and exit.
  • Create one last file called "convert.bat", edit and paste:

@for /f "tokens=*" %%a in ('dir /b *.tex') do D3TexConv.exe %%a

  • Save and exit.

Now we're ready to roll!

  • To patch up all the TEX file headers so D3TexConv can read it, run "texfix.bat".
  • Once that's done, run "convert.bat" to extract the DDS data from the TEX files.
  • You'll now get a whole stack of TEX, DDS and TXT files. The only ones you care about are the DDS ones.
  • Install IrfanView and the plugins pack.
  • Now you should be able to view the DDS files.
  • Select the images you want and batch convert them into PNG (or whatever other format you need)

This should take a good while, but you'll be messing with the graphic files in no time!

image
Browsing the converted PNG files using ACDSee

Sources

Big thanks all round to all the crazy ass reverse engineers who've worked on this. I really don't know who they are, but only through their hard work has this been possible.

I've done my best to source my information so I hope you guys eventually get the credit you deserve!

Windows 7: Automatically connect to wireless network

I was a little perplexed to why the network wasn't reconnecting after a reboot.

I've set it to automatically connect even if SSID isn't even detected.

To fix it, you'll need to have administrator access.

  • Open up Windows Explorer
  • Paste in "C:\ProgramData\Microsoft\Wlansvc\"
  • Right click the folder "Profiles"
  • Properties
  • Security
  • Click "Edit"
  • Select "Everyone"
  • Under "Allow", click tick "Full control"
  • Click OK to save

Most people recommend deleting the "Interfaces" folder under "Profiles". Try my method first, since you won't lose any data.

Secondly if you delete the folder, you'll have to re-enter all the wireless password and settings back in.

tumblr_lfkbc8HspD1qzu6nx 
Now get back to what you're doing before you lose any more time aging!

Sources

Dell: Replace all driver "download" buttons with direct links (Australian)

I'm not sure if this works on the US site, but this worked a treat for me on the Australian site.

The main issue was I was getting annoyed with the Javascript download question asking me to use the download manager.

I just wanted to queue up files into my own download manager and batch download the files during my off-peak period.

When you're on the driver download page, open up Firebug and paste this into the console.

var rex = /javascript:DownloadFile\('\d+','\d+','.+','(.+)','\w+','HTTP'\);/;

var x = $('a[href*="javascript:DownloadFile"]').each(function(index, item) {
var obj = $(item);
var href = obj.attr('href');
var match = rex.exec(href);

if (match != null) {
var url = unescape(match[1])
obj.attr('href', url);
}
});

This will change all the "download file" buttons so they point to the direct file download link.

I'm sure someone can easily convert this into a GreaseMonkey script.

BOXOd

Now, back to the horrors of formatting!

PostgreSQL: ERROR: item pointer (16895,17) already exists

The numbers in the error are specific to you, but if you see something like this, be sure to check if your indexes are somehow corrupted.

  • In the console, type "\d tablename" to bring up the details.
  • Under the table, you'll see a list of indexes.
  • For each index, type "reindex INDEX indexname"

Try performing your query again.

barrel-roll
Do a barrel roll!

Android: Upgrading Samsung i9000 to Gingerbread 2.3.6 (build XWJW6, baseband XWJW1) with root

<rant>Alright, I'll be honest. This whole Android flashing business is way more complicated than it should be.

WTF is with all these stupid build "numbers"!? They're not very comprehensible for the average person.</rant>

12-monkeys-crazy-2-o 
My reaction when reading a typical Android-hacker post from XDA

Anyways, I've tried to simplify this process down as much as possible.

Preliminaries

First off, some checking. You need to have the 3-button download/recovery mode fix working in case shit hits fan. It did for me and it'll definitely save your ass if it happens to you. If you decide to proceed without fixing this first, you're either very brave or very stupid.

To test it, turn your phone off. Hold Volume Down + Home + Power to turn it on. If you enter download mode, then you're good to go.

Make sure you've backed up what you need!

Installing the firmware

What you'll need is:

This is probably the easiest part of the process.

  • Extract the firmware
  • Run Odin (without the phone plugged in yet)
  • Select the PDA/Phone/CSC files accordingly
  • Put the phone into download mode
  • Connect via USB
  • When Odin's label turns yellow and assigns the phone a COM port, click Start.
  • Wait a few minutes while it performs the update.
  • It should restart by itself.

If Odin3 gets stuck on "SetupConnection", just make sure you plug the phone in quicker after starting up Odin. It's a bit fussy.

Rooting your phone

Initially I was going to write about it here, but it was getting too lengthy already and not everybody will need it.

I tried a few different root techniques but not all were stable or easy. I eventually found one that's super easy and stable, so if you're keen on rooting your phone, see this post.

Fix your APN's

Remember, the downside to flashing your phone is that you will lose some settings. This also includes stuff like settings for internet and MMS.

If you're an Aussie, you can get the settings from Ausdroid APN.

Sources

Android: Samsung Galaxy S i9000 3-button download/recovery mode fix

Man, I've been putting this one off for years. Not sure why it only affect some models, but a lot of phones left the factory with a faulty secondary bootloader (SBL).

Because of this, it won't recognise the Volume Down + Home + Power combo.

I had to filter through a LOT of jargon and version hunting for these steps.

Prerequisites

  • Removed SIM and SD cards.
  • Phone in USB debugging mode.
  • Without charging or USB connection, type *#0228# in the phone dialer and ensure that the "Voltage" is at least 3800 (mV).
  • Phone detected by computer.
  • Hex codes on your BML1 block (I won't lie, I have no idea what this means. Just check below for instructions)

Just a word of advice, DO THE CHECKS before applying this bootloader. Otherwise you'll brick it, no questions asked.

This is copy pasted from the thread, so please make sure your phone is compatible before fixing the SBL.

HOW-TO check if your mobile is compatible (Windows):
Step 1: Download the XVI32 Hex Editor
Step 2: Dump your BML1 block (rooted phone needed)

 

Open a new command prompt and type:
adb shell
su (a superuser request will be displayed on the phone screen, accept it).
dd if=/dev/block/bml1 of=/sdcard/bml1.dump

The dump will be copied to the root of the internal SD card.

Step 3: Open the bml1.dump file with XVI32.
Step 4: Search (in ASCII Mode) for OFNI.

If the block reads "!@" just follow the tutorial below starting with Step 1.
If it reads "x0" then STOP as this FiX will BRICK your phone if applied !!

Downloads

I really wanted to use Heimdall for this, but it just wouldn't let me.

  • Odin v1.7
  • I already had adb.exe from the Android SDK package. There are plenty of places to get this.
  • Modified SBL

Procedure

  • Open up a command prompt and navigate to adb.exe
  • Type "adb reboot download" to enter download mode
  • Run Odin
  • Wait for the "COM###" label to turn yellow
  • Click on the PDA button and select "P-SBL.tar.md5.tar"
  • Tick "Phone BootLoader update"
  • Tick "Autoreboot"
  • Tick "F.Reset Time"
  • Ensure the other options are NOT ticked.
  • Click start whenever you're ready

The whole process will only take a few seconds. You may need to reboot your phone manually.

Turn the phone off and test if download mode works (Volume Down + Home + Power).

Ensure that your phone still works! For me, it got stuck on the loading screen and refused to start up. Take out the battery and try it again.

If it doesn't work, you'll have to reflash your firmware. This doesn't break the bootloader though, so feel free to reflash.

Sources

Android: Gingerbread 2.3.6 root for Samsung Galaxy S i9000

SuperOneClick. That's all there is to it. No jibberish jargon, checking of build versions, phone models, etc. SO EASY!

There's more information about it here on XDA, but this program worked a charm on my phone where the other methods weren't so "clean".

Preparation

Make sure you've got:

  • Microsoft .NET Framework 2.0+ or Mono v1.2.6+
  • One of the supported operating systems (Vista+, Ubuntu Hardy+, Debian Lenny+)
  • Phone on USB Debugging mode
  • Computer detects your phone (may require USB drivers for your phone to be installed)
  • Download "Mount /system (rw/ro)" app

Procedure

  • Download the package, extract. I tried this on v2.3.3.
  • Run SuperOneClick.
  • Plug in the phone
  • Click root
  • Wait for it to finish
  • Unplug and reboot the phone manually

It couldn't be simpler!

*edit 25/6/2012* (Added following bits)

Now the last part.

You still won't be able to edit system files because the /system folder is mounted as read only. To change that to read/write, you'll need the "Mount /system (rw/ro)" app.

Start it up, enable write and away you go!

Sources

 
Copyright © Twig's Tech Tips
Theme by BloggerThemes & TopWPThemes Sponsored by iBlogtoBlog