Useless Lua Code

Got a hex dump of some firmware which was in kinda an odd format (offsets, data, checksum(maybe?) in hexadecimal ascii text). Wrote a small script to dump the printable characters. Not really interesting.

dump-text.lua

local function readsection(f)
    local colon = f:read(1)
    if colon == nil then return nil end
    if colon ~= ':' then error('section did not start with colon') end
    local data = ''
    while true do
        local char = f:read(1)
        if(char:byte() == 13) then
            f:read(1) -- eat cr
            return data
        end
        data = data .. char
    end
end


local function decodecharacter(c)
    if c:len() ~= 1 then error('must be single character') end
    local map = {['0']=0, ['1']=1, ['2']=2, ['3']=3, ['4']=4, ['5']=5, ['6']=6,
                 ['7']=7, ['8']=8, ['9']=9, A=10, B=11, C=12, D=13, E=14, F=15}
    return map[c]
end


local function decodepair(s)
    if s:len() ~= 2 then error('must be two characters in length') end
    return (decodecharacter(s:sub(1,1)) * 16) + decodecharacter(s:sub(2,2))
end


local function decodesection(s)
    local offset = {}
    local data = {}

    -- todo: last two chars = checksum(offset+data)?
    for i=1, s:len(), 2 do
        local pair = s:sub(i, i+1)
        local value = decodepair(pair)

        if i <= 8 then
            table.insert(offset, value)
        else
            table.insert(data, value)
        end
    end
    return offset, data
end


local function safeprintdata(data)
    local chars = {}
    for index, byte in ipairs(data) do
        if byte >= 32 and byte <= 126 then
            table.insert(chars, string.char(byte))
        else
            table.insert(chars, '.')
        end
    end
    return table.concat(chars)
end


local f = io.open('P2008-V-02.08.02.hex', 'rb')

while true do
    line = readsection(f)
    if line == nil then break end
    local offset, data = decodesection(line)
    print(string.format('%-42s  %s', line, safeprintdata(data)))
end

Output

102170005B204D317878205D2020000000000000B9  [ M1xx ]  .......
102180005B204D313133205D202000000000000035  [ M113 ]  ......5
10219000562D30322E30382E3032202000000000F4  V-02.08.02  .....
1021A00028632920616C6973746172207379737478  (c) alistar systx
1021B000656D732C20323030312D32303132202099  ems, 2001-2012  .

Posted on

Crimsonland Pak Format Again

Have been spending a little bit of time playing with Lua and an early project was to rewrite the Python script for reading a Crimsonland Pak file. So far I’m enjoying Lua and its language design, even with the very limited standard libraries.

clpak.lua

BinaryFile = {}

BinaryFile.new = function(file)
    return setmetatable({file=file}, {__index=BinaryFile})
end

BinaryFile.readint32 = function(self)
    local data = self.file:read(4)
    return (data:byte(4) * (256^3)) +
           (data:byte(3) * (256^2)) +
           (data:byte(2) * (256^1)) +
           (data:byte(1) * (256^0))
end

BinaryFile.readnulstring = function(self)
    local buf = ''
    while true do
        local ch = self.file:read(1)
        if ch:byte() == 0 then break end
        buf = buf .. ch
    end
    return buf
end

BinaryFile.read = function(self, ...)
    return self.file:read(...)
end

BinaryFile.skip = BinaryFile.read

BinaryFile.seek = function(self, ...)
    return self.file:seek(...)
end


local readindex = function(filename)
    local f  = io.open(filename, 'rb')
    local bf = BinaryFile.new(f)

    -- Header
    assert(bf:readnulstring() == 'PAK')
    assert(bf:readnulstring() == 'V11')

    -- index offsets
    local indexstartoffset = bf:readint32()
    local indexendoffset   = bf:readint32()

    -- just to index
    bf:seek('set', indexstartoffset)

    -- number of entries
    local indexsize = bf:readint32()

    local index = {}

    for i=1, indexsize do
        table.insert(index, {
            name   = bf:readnulstring(),
            offset = bf:readint32(),
            length = bf:readint32()
        })

        -- Unknown (junk?)
        bf:skip(8)
    end

    f:close()

    return index
end


local index = readindex('D:\\Steam\\steamapps\\common\\Crimsonland\\data.pak')

for _,v in ipairs(index) do
    print(string.format('name:%s offset:%d length:%d', v.name, v.offset, v.length))
end

Posted on

CloudFlare FlexSSL

Thanks to CloudFlare this website now sortof has a SSL cert. Lots of posts needed to be cleaned up with links and embedded content that forced http (rather than being agnostic). Was interesting looking to look back at the old posts (first was in Jan 2006!). So much awful code; but I’m happy that I recorded it somewhere.

Posted on

Who Hung up an Asterisk Call

Little snippet to work out which party hung up on a call in Asterisk.

extensions.conf

; Detect which party hungup an Asterisk call
; If we (the caller) hangup after Dial() we jump directly to h
; Because of the 'g' option if the other party hangs up we continue on
exten => s,1,Set(whoHungUp=CALLER)
exten => s,n,Dial(IAX2/provider/${number},,g)
exten => s,n,Set(whoHungUp=OTHERPARTY)
exten => s,n,Hangup
exten => h,1,NoOp(whoHungUp ${whoHungUp})

Posted on

Crimsonland (Steam) Pak Format

A remake of Crimsonland was recently released on Steam (and soon PS4/Vita) and I thought I’d take a look at the pak data file. Didn’t turn out to be anything really interesting, just art assets, sounds, music, and some frontend lua. But was good fun (even if it was a simple format).

clpak.py

import os
import struct


"""
Crimsonland (Steam) pak file format
Header format:
50 40 4B 00 56 31 31 00   ("PAK" NUL "V11" NUL)
int32 index offset     (eg: 28679361)
int32 end index offset (eg: 28885444)
Index Format:
int32 number of indexes?
Index_File format:
null terminated string
int32 absolute offset of file
int32 file length
unknown maybe always (?) equal to: FF 26 E2 50 20 00 00 00
"""


class ClPak(object):
    def __init__(self, filename):
        self.file = open(filename, 'rb')

    def read_header(self):
        self.file.seek(0)
        bytes = self.file.read(16)

        (magic_1, magic_2, index_start_offset, index_end_offset) = 
            struct.unpack('3sx3sxii', bytes)

        if magic_1 != 'PAK' or magic_2 != 'V11':
            raise Exception('Unknown file format')

        return index_start_offset, index_end_offset


    def read_index(self):
        start_offset, end_offset = self.read_header()

        self.file.seek(start_offset)
        bytes = self.file.read(end_offset - start_offset)

        # index_size = struct.unpack('i', bytes[:4])

        return self.read_index_file_details(bytes[4:])


    def read_index_file_details(self, bytes):

        details = {'name': '', 'offset': 0, 'length': 0, 'unknown': ''}

        sequence = iter(bytes)

        for char in sequence:
            if char != chr(0):
                details['name'] += char
            else:
                offset_bytes = sequence.next() + sequence.next() +
                               sequence.next() + sequence.next()
                length_bytes = sequence.next() + sequence.next() +
                               sequence.next() + sequence.next()

                (details['offset'],) = struct.unpack('i', offset_bytes)
                (details['length'],) = struct.unpack('i', length_bytes)

                for x in range(0, 8):
                    details['unknown'] += sequence.next()

                yield details

                details = {'name': '', 'offset': 0, 'length': 0, 'unknown': ''}


    def dump_file(self, details, base_directory):
        dest_filename = base_directory + '/' + details['name']

        if not os.path.exists(os.path.dirname(dest_filename)):
            os.makedirs(os.path.dirname(dest_filename))

        with open(dest_filename, 'wb') as dest:
            self.file.seek(details['offset'])
            dest.write(self.file.read(details['length']))


if __name__ == '__main__':
    cl_pak = ClPak(r'D:\Steam\steamapps\common\Crimsonland\data.pak')
    for file_details in cl_pak.read_index():
        print file_details
        cl_pak.dump_file(file_details, 'd:/clpak_files')

Posted on

Global Hot Keys in Python for Windows

I used to think I had a basic understanding of Python. I’m not so sure anymore…

globalhotkeys.py

import ctypes
import ctypes.wintypes
import win32con


class GlobalHotKeys(object):
    """
    Register a key using the register() method, or using the @register decorator
    Use listen() to start the message pump
    Example:
    from globalhotkeys import GlobalHotKeys
    @GlobalHotKeys.register(GlobalHotKeys.VK_F1)
    def hello_world():
        print 'Hello World'
    GlobalHotKeys.listen()
    """

    key_mapping = []
    user32 = ctypes.windll.user32

    MOD_ALT = win32con.MOD_ALT
    MOD_CTRL = win32con.MOD_CONTROL
    MOD_CONTROL = win32con.MOD_CONTROL
    MOD_SHIFT = win32con.MOD_SHIFT
    MOD_WIN = win32con.MOD_WIN

    @classmethod
    def register(cls, vk, modifier=0, func=None):
        """
        vk is a windows virtual key code
         - can use ord('X') for A-Z, and 0-1 (note uppercase letter only)
         - or win32con.VK_* constants
         - for full list of VKs see: http://msdn.microsoft.com/en-us/library/dd375731.aspx
        modifier is a win32con.MOD_* constant
        func is the function to run.  If False then break out of the message loop
        """

        # Called as a decorator?
        if func is None:
            def register_decorator(f):
                cls.register(vk, modifier, f)
                return f
            return register_decorator
        else:
            cls.key_mapping.append((vk, modifier, func))


    @classmethod
    def listen(cls):
        """
        Start the message pump
        """

        for index, (vk, modifiers, func) in enumerate(cls.key_mapping):
            if not cls.user32.RegisterHotKey(None, index, modifiers, vk):
                raise Exception('Unable to register hot key: ' + str(vk))

        try:
            msg = ctypes.wintypes.MSG()
            while cls.user32.GetMessageA(ctypes.byref(msg), None, 0, 0) != 0:
                if msg.message == win32con.WM_HOTKEY:
                    (vk, modifiers, func) = cls.key_mapping[msg.wParam]
                    if not func:
                        break
                    func()

                cls.user32.TranslateMessage(ctypes.byref(msg))
                cls.user32.DispatchMessageA(ctypes.byref(msg))

        finally:
            for index, (vk, modifiers, func) in enumerate(cls.key_mapping):
                cls.user32.UnregisterHotKey(None, index)


    @classmethod
    def _include_defined_vks(cls):
        for item in win32con.__dict__:
            item = str(item)
            if item[:3] == 'VK_':
                setattr(cls, item, win32con.__dict__[item])


    @classmethod
    def _include_alpha_numeric_vks(cls):
        for key_code in (range(ord('A'), ord('Z')) + range(ord('0'), ord('9'))):
            setattr(cls, 'VK_' + chr(key_code), key_code)


# Not sure if this is really a good idea or not?
#
# It makes decorators look a little nicer, and the user doesn't have to explicitly use win32con (and we add missing VKs
# for A-Z, 0-9
#
# But there no auto-complete (as it's done at run time), and lint'ers hate it
GlobalHotKeys._include_defined_vks()
GlobalHotKeys._include_alpha_numeric_vks()

globalhotkeys_test.py

from globalhotkeys import GlobalHotKeys


@GlobalHotKeys.register(GlobalHotKeys.VK_F1, GlobalHotKeys.MOD_SHIFT)
def hello_world():
    print "Hello World!"


@GlobalHotKeys.register(GlobalHotKeys.VK_F2)
def hello_world_2():
    print "Hello World again?"


# Q and ctrl will stop message loop
GlobalHotKeys.register(GlobalHotKeys.VK_Q, 0, False)
GlobalHotKeys.register(GlobalHotKeys.VK_C, GlobalHotKeys.MOD_CTRL, False)

# start main loop
GlobalHotKeys.listen()

Posted on

Phone Fun (Snom300)

"""
Finds all Snom brand phones in a /24 and tells them to key in 6405 turn on
the hands free speaker.  (6405 is an extension that plays the stock monkies
sound from Asterisk)
"""

import urllib2
import socket
import sys

def enumerate_snom_ips(base):
    original_timeout = socket.getdefaulttimeout()
    socket.setdefaulttimeout(0.2)

    ips_to_check = [base + '.' + str(i) for i in range(2, 255)]
    snom_ips = []

    for ip in ips_to_check:
        try:
            data = urllib2.urlopen('http://' + ip + '/index.htm').read()
            if '<TITLE>snom 300</TITLE>' in data:
                snom_ips.append(ip)
        except KeyboardInterrupt:
            sys.exit()
        except:
            pass

    socket.setdefaulttimeout(original_timeout)

    return snom_ips


def send_command(ip, key):
    urllib2.urlopen('http://' + ip + '/command.htm?key=' + key).read()

def monkies(ip):
    for key in ['6', '4', '0', '5', 'SPEAKER']:
        send_command(ip, key)

print 'monkies.py - Finding all Snom phones...'
for ip in enumerate_snom_ips('10.10.12'):
    print ip
    monkies(ip)

Posted on

Python + matplotlib

After two hours solid of banging my head against a wall trying to get matplotlib do what I wanted, I was able to go from this LibreOffice graph:

To something a little nicer:

It’s still ugly, and confusing, but it’s a lot closer to what I wanted. The IPython environment is a interesting way of working with data too. I will have to play around with it more in the future.

Awful, awful Python code

(Really. Everything below is probably wrong)

%matplotlib inline
import matplotlib.pyplot as plt
from matplotlib.dates import strpdate2num
import matplotlib.dates as mdates
import numpy
import csv

filename = 'c:\\Users\Matthew\\adsl.csv'

rows = ("\t".join(i) for i in csv.reader(open(filename, 'r'), quotechar='"'))
converters = {0: strpdate2num('%Y/%m/%d %H:%M:%S')}
(x, sync_up, sync_down, snr_up, snr_down, attune_up, attune_down) = numpy.genfromtxt(rows, delimiter="\t", skip_header=1, converters=converters, unpack=True)

fig = plt.figure()
fig.set_size_inches(16,8)

ax = fig.add_subplot(111)

# Configure x-ticks
# ax.set_xticks(x) # Tickmark + label at every plotted point
ax.xaxis.set_major_formatter(mdates.DateFormatter('%d/%m/%Y %H:%M'))

ax.plot_date(x, sync_up, '--', c='r', label='Sync(up)') # ls='-', marker='o')
ax.plot_date(x, sync_down, '--', c='g', label='Sync(down)') # ls='-', marker='o')
ax.set_title('ADSL2+ Line Quality')
ax.set_ylabel('Speed (kbps)')
ax.grid(True)

ax.legend(bbox_to_anchor=(1.05, 1), loc=2)

highlight_start = x[212]
highlight_end = x[220]

ax.axvspan(highlight_start, highlight_end, facecolor='red', alpha=0.1)


ax2 = ax.twinx()
ax2.plot_date(x, snr_up,'-', c='r', label='SNR(up)')
ax2.plot_date(x, snr_down, '-', c='g', label='SNR(down)')
ax2.plot_date(x, attune_up, '-', c='c', label='Attenuation(up)')
ax2.plot_date(x, attune_down, '-', c='m', label='Attenuation(down)')
ax2.set_ylabel('Decibel (db)')

ax2.legend(bbox_to_anchor=(1.05, 0.80), loc=2)

fig.autofmt_xdate(rotation=45)

fig.show()

Posted on

Tyrian Music

Tyrian is a vertical shooter by World Tree Games released in 1995 and published by Epic games. I have fond memories of the music (and game) but was having trouble finding a copy of the soundtrack as MP3s (etc).

Luckily this site has a copy of the original LDS (Loudness Sound System) files and using Foobar2k + AdPlug input component I was able to get them to play (note: the component expects the extension to be be ldsa). While I was at it, I also made a copy of them as MP3 files.

You can listen to the music below, or download a zip file of all the MP3s: tyrianlds-mp3.zip (40MB)

Posted on

Website downtime

My website went down for around 1:30 to 2 hours today, which was a little odd. After it came back up I noticed the 15min load was > 30. Turns out someone tried to log into WordPress which isn’t that odd. It happens all the time, so much so that I run Limit Login Attempts plugin to apply a 20min timeout a IP address after 4 incorrect attempts (and then a 24 hour timeout after 16 attempts). This time though, it was a kind of a large attack.

In 118 minutes, 2337 different IP addresses tried to log into WordPress. Most were timed-out after 4 attempts, but due to oom-killer sometimes the timeout details weren’t added to the database and more than 4 attempts were made (Two ips made > 26 attempts each, though only 8 IPs made 7 or more attempts). Using a Geo IP database we can quickly get a breakdown of where the IPs came from:

Country Attempts
Russia 1786
Ukraine 1225
Vietnam 909
Thailand 817
Taiwan 613
Romania 380
Turkey 379
Bulgaria 342
Iran 278
Belarus 276
India 241
Poland 164
Serbia 159
Egypt 146
Hungary 137
Brazil 122
United States of America 121
Canada 115
South Africa 115
Other (73 other unique countries) 1762

The user agent for every attempt was “Mozilla/5.0 (Windows NT 6.1; rv:19.0) Gecko/20100101 Firefox/19.0″ which is a valid user agent for FF19 running Windows 7. I’m guessing it was someone’s small botnet. Though the distribution of the countries seems a bit off? Maybe it’s a compromised FF addon? If it continues I may need to switch to a whitelist for logins, or move the admin login page elsewhere as current login limit plugin isn’t really suited to an attack from such a large number of IP addresses.

Code

__author__ = 'Matthew'

import pygeoip

ACCESS_FILENAME = 'access.log'

GEOIP = pygeoip.Database('GeoIP.dat')

ip_breakdown = {}
country_breakdown = {}

for line in open(ACCESS_FILENAME):
    if 'wp-login' not in line:
        continue

    ip = line[0:line.find(' ')]

    if not ip_breakdown.has_key(ip):
        ip_breakdown[ip] = 0

    ip_breakdown[ip] += 1

    ########

    info = GEOIP.lookup(ip)
    if not info.country:
        country = 'unknown'
    else:
        country = info.country

    if not country_breakdown.has_key(country):
        country_breakdown[country] = 0

    country_breakdown[country] += 1


print 'Number ips:', len(ip_breakdown.keys())
print 'Number countries:', len(country_breakdown.keys())

for country in sorted(country_breakdown, key=country_breakdown.get, reverse=True):
    print country, country_breakdown[country]

Output

Number ips: 2337
Number countries: 92
RU 1786
UA 1225
VN 909
TH 817
TW 613
RO 380
TR 379
BG 342
IR 278
BY 276
IN 241
PL 164
RS 159
EG 146
HU 137
BR 122
US 121
CN 115
SA 115
PK 93
HK 91
CZ 90
ID 82
KZ 78
GE 74
GB 73
SK 63
MD 59
DE 57
AE 56
GR 53
AZ 50
IQ 45
ES 44
BD 43
IL 43
CA 39
AR 36
NL 34
CO 34
CL 32
LV 32
MY 30
AT 29
IT 28
KG 22
BE 21
DK 19
JP 18
QA 17
ZA 15
LK 15
AU 15
LT 14
MX 13
MO 12
KH 12
PH 11
NO 11
PS 10
SE 10
MA 9
MN 9
CY 9
AM 9
BA 8
PY 8
FR 8
GH 7
EE 7
YE 7
DZ 7
PT 6
A2 6
OM 4
HR 4
EC 4
NZ 4
LB 4
LA 3
JO 2
PE 2
MK 2
MZ 2
BH 1
AO 1
ET 1
UZ 1
NP 1
CD 1
SY 1
SD 1

Posted on