Things are due for some major change, as I got my L1 work visa for the United States today. My family also got their L2 visa’s so now we can transfer to the US office and begin our adventure there.

This also explains why things have been quiet around here for the last few months as I have been busy doing the paperwork involved with the application, and getting things ready around home.

Lots of DIY to complete before we leave, which should be around January next year (2010).

I’m very excited, for both myself and my family, this will be a big change for us all, but should be very rewarding/mind expanding.

This is the second post in a serries on reverse engineering Pools of Darkness.

The first is here, and I’ll add links to the later posts as we go.

Also note this post, the previous post and next posts can be skipped by using unp (unprotect)

Now loading dump.bin into IDA

This process is very messy…

Points to remember from the dumping, the memory base address was 0xba1, the first chunk of code moved the base segment to 0xbb1, after the descrambler the code jumps to 0xbb1:2

Open IDA, and start a new project, select dump.bin, at the loading screen set the loading segment to 0xba1. Press ok.

When asked if you want 32bit or 16bit choose 16bit.

Edit the first segment (Edit -> Segments -> Edit Segments) and change the End address to 0xbb10 and unselect ‘Move adjacent segments’

Create a new segment (Edit -> Segments -> Create Segments) seg001, Start address 0xbb10, End Address 0x3ba10, Base 0xbb1, 16bit, OK

I’m using the free version of IDA Pro and it has a create segment bug, where the segment is 32bit even though marked it 16bit. So edit like we did for seg000

Now we can jump to seg001:0002 and press C to say this is code, as this is the Pascal __SystemInit function. We can see that it makes lots of calls to address that are not correctly decoded.

Long story short, these are each segment’s init function, and they run from seg001:0002 to seg001:0101

These actually provide mapping of the segments of the game, so we that seg001 must end at 0xBE1 as that’s the next segment, which ends at 0xBEE, so we could hand create these segments or we could write a IDA script (.idc) to do the work for us.

Select the text area (seg001:0002 to seg001:0101), copy the contents to Visual Studio run this Replace Regex

^seg001\\::h:h:h:h:b+call:b+(far:bptr:b)*{:h+}h\\:.*$

to:

SegCreate(0x\\10, 0x\\10, 0x\\1, 0, 1, 2);\\nSegRename(0x\\10, "seg000");\\nSegClass(0x\\10, "CODE");

To build the idc code to build the segment tables.

There is a bug in IDA Freeware Version 4.9 in which the first segment you create ignores the 16bit/32bit setting you chose, in our script above this can be fixed by adding

SegAddrng(0x13D60,0);

IDA gives you the ability to export your work as a IDC file so we will do this, via (File -> Produce File -> Dump database to IDC file)

Giving us darkness-0.idc, we can now merge our segment creating code from Visual Studio, to give us darkness-1.idc

Now we can run this script to back to our current state. As we progress it’s a good idea to ether hand edit your idc file and rerun that to check you have all of your edits, and if you do major work via the UI, export a new IDC file, very much like save games for RPGs you can never have too many saves. Heck even version control might be a good idea…

Now when we run this script on our loaded dump.bin, we get some of the initial segments setup.

The stack and the data segment are not correct, if we turn seg001:02 to code, we see those initial setup functions.

If we follow the first call (sub_13D60) we get to the seg053 init code.

The first instruction sets DS to 0x1598, which we will see elsewhere. If we look in the segment window, the last segment seg053 starts at 0x13D6 so a long story short, this is the start address of the data segment. A very useful piece of information

So we update our script. Adding a new segment called data at 0x1598h

SegCreate(0x15980, 0x3BA20, 0x1598, 0, 1, 2);
SegRename(0x15980, "Data");
SegClass(0x15980, "DATA");
SegAddrng(0x15980,0);

And to avoid having to press C (code) at seg001:0002 again we can add that command like this:

MakeCode(0XBB12);

thus giving darkness-2.idc

At this point you can explore the code, and start to get an idea of how it works. You can name functions (n) or make bits of assembly into functions (p) jump around the code, but we’ll not do that just get, because as you will discover most the code is missing.

To show this go back to the seg001 init call block (seg001:48)

and follow the call to sub_d310

The int 13h calls the overlay handler, which loads the actual code block into an different location in memory, then it rewrites this segment (seg039) so each function location (stub) jumps to the overlay code. Any calls to seg039 functions are then redirected with no extra over head. If the current loaded overlay changes (due to call into another segment) then seg039 gets rewritten back to how it was, and the new segment get mapped.

The actual code that sets up the overlay manager is one of the first segments called in the init call block, and I may point out later, but for now we have to load all missing code by hand, and rewrite the existing code to we can follow what’s happening, and that will make a exciting next post…

I thought I would reverse engineer Pools of Darkness to capture the process in a serries of blog posts.

This is the first, and I’ll add links to the later posts as we go. Part 2

Prerequisite:

• Pools of Darkness for DOS
• IDA Pro (full or demo)
• Windows OS

Open game.exe in IDA Pro, jump to the start function (Ctrl+E). Here it is:

In summary: it moves the executable to a new location in memory, then “jumps to the next function” via pushing the new memory address (seg001:0038) onto that stack and the returning to it (which is just a pop and jump).

The next blob of code seg001:0038 - seg001:00FE is a fancy pants scrambler, which by the way is exactly the same as Curse of the Azure Bonds had.

Line seg001:00FE is the interesting line:

seg001:00FE     jmp     dword ptr cs:[bx]

This is where the descrambler jumps to the actual game code.  The code in IDA is the scrambled version, so we want to get access to the descrambled memory layout.  So quit IDA Pro, and don’t keep this database.

For Curse I used the DOS debugger to get the descrambled memory, so lets do it again.

Here is the generic script to dump the memory:

g AAAA:37
p
g BBBB:fe
d 0 ffff
a BBBB:fe
mov dx,CCCC
mov ds,dx

p
p
d 0 ffff
a 1ab1:103
mov dx,DDDD
mov ds,dx

p
p
d 0 ffff
q

To get the values of AAAA, BBBB, CCCC and DDDD we start from a command prompt:

enter R to get the register dump

C:\\games\\DARKNESS>debug game.exe
-r
AX=0000  BX=0000  CX=F8CB  DX=0000  SP=0080  BP=0000  SI=0000  DI=0000
DS=0BA1  ES=0BA1  SS=1DC7  CS=1AED  IP=0012   NV UP EI PL NZ NA PO NC
1AED:0012 8CC0          MOV     AX,ES
-

DS is the beginning of the memory, and CCCC is DS + 0x1000 and DDDD is DS + 0x2000, AAAA is CS, and BBBB is DS + 10 + word_1F3CC, which happens to be 0x11C5, but I just step (p) six times, and read the value from AX

now we can run the debugger a couple more time to double check the base address are the same each run…. which they are.

now we can run the script as input to debug.exe to dump the 192Kb of game ram, via this command

debug game.exe < run.txt > out.txt

I found a few times I had to kill the process, as the quit (q) command at the end was not working…

But you now have a file looking like this

-g 1aed:0037
AX=0038 BX=0000 CX=0000 DX=0001 SP=007C BP=0000 SI=FFFF DI=FFFF
DS=1AED ES=1D76 SS=1DC7 CS=1AED IP=0037 NV DN EI PL NZ NA PE NC
1AED:0037 CB RETF
-p
AX=0038 BX=0000 CX=0000 DX=0001 SP=0080 BP=0000 SI=FFFF DI=FFFF
DS=1AED ES=1D76 SS=1DC7 CS=1D76 IP=0038 NV DN EI PL NZ NA PE NC
1D76:0038 06 PUSH ES
-g 1d76:fe
AX=0BA1 BX=0000 CX=0000 DX=F000 SP=4000 BP=0000 SI=2576 DI=4000
DS=0BA1 ES=0BA1 SS=2576 CS=1D76 IP=00FE NV UP EI PL NZ NA PO NC
1D76:00FE 2E CS:
1D76:00FF FF2F JMP FAR [BX] CS:0000=0002
-d 0 ffff
0BA1:0000 CD 20 FF 9F 00 9A F0 FE-1D F0 4F 03 95 05 8A 03 . ........O.....
0BA1:0010 95 05 17 03 95 05 84 05-03 04 01 00 02 FF FF FF ................
0BA1:0020 FF FF FF FF FF FF FF FF-FF FF FF FF 42 0B F1 49 ............B..I
0BA1:0030 95 05 14 00 18 00 A1 0B-FF FF FF FF 00 00 00 00 ................
0BA1:0040 05 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0BA1:0050 CD 21 CB 00 00 00 00 00-00 00 00 00 00 20 20 20 .!...........

Next we need to turn that into a binary file, the code I wrote for Curse required only the hex dump lines so we trim the extra lines from out.txt file first.

#include <iostream>
#include <stdio.h>

using namespace std;

unsigned char decode(char h, char l)
{
  unsigned char hv, lv;
  if ( h >= 'A' && h <= 'F' )
    hv = h - 'A' +10;
  else
    hv = h - '0';

  if ( l >= 'A' && l <= 'F' )
    lv = l - 'A' +10;
  else
    lv = l - '0';

  return (lv+(hv*16));
}

void main()
{
  const int lineSize = 1024;
  char line[lineSize];

  FILE *out = fopen( "dump.bin", "wb" );

  unsigned char byte;

  while(cin.good())
  {
    cin.getline( line, lineSize );

    int offset=11;
    for(int i=0; i<8; i++)
    {
      char h,l;
      h = line[offset+(i*3)];
      l = line[offset+(i*3)+1];

      byte = decode(h, l);

      fwrite(&byte,1,1,out);
    }

    offset=35;
    for(int i=0; i<8; i++)
    {
      char h,l;
      h = line[offset+(i*3)];
      l = line[offset+(i*3)+1];

      byte = decode(h, l);

      fwrite(&byte,1,1,out);
    }
  }
  fclose(out);
}

Now using this like so:

dumpparse.exe < out2.txt

We get file dump.bin

Which we will load into IDA Pro in Part 2…

I read Why Software Sucks by David Platt over a month ago, and have been wanting to blog about it since then.

Firstly the book is so true,  as an engineer I love control, doesn’t everybody?

No is the answer apparently, and once I got this, I have noticed myself asking questions like, what if I’m wrong? or more to the point, what if less is more?

There where some great quotes in the book, I read a few out to workmates, and they all nodded their heads (testers/managers). I’ve returned the book to the library, so I cannot share them with you.

But the biggest win was a few weeks ago, when discussion a feature in a meeting, one of the testers proposed a different and really stupid ugly way of presenting a concept to the user. I started on my usual, pros/cons discussion. But now taking a less control freak perspective, I slowly found more winning points on his design, and in fact mine had discovery problems (the biggest problem actually), which as a in-the-know person all made sense, but how would we teach the users this feature, or allow them to discover it and learn it.

After a 10 minutes of talking, I said “you win” and he had not said a thing. I’m now ok with it, because I concivnced myself his was the overall better option.

So there you go, a great read, well presented look into software, the mindsets that developers have (me me me) when building software, and how they are not the same people that actually use that software.

Listened to Stack Overflow podcast #67 last week, and then yesterday noticed Seth posting (If Craigslist cost $1)  the guts of the Craigslist discussion.

Fair enough for him to promote the idea, as he has a larger/different audience, but it still seems odd to think of Seth listing to the Stack Overflow podcast…

Conor (condor) put me onto Monopoly City Streets, which is very cool idea, albeit the server is running slow as a wet rag.

All was going well until Conor bulldozed one of my building.

Check it out, I’m playing as the very imaginative ‘simeonpilgrim’

Woot!, lucky number combo time.

It doesn’t even matter if mix your day/month up like the US or not.

We bought a new camera on Tuesday, the Sony DSC-HX1. We have been wanting a camera with more zoom for a while now.

There have been too many school events where we can see the kids fine, but due to lack of zoom, our camera has failed to capture the scene, or the shot has been ruined by the super contrast of the stage lights making bright spots.

We were also considering the Cannon Powershot SX1 IS, but the deal closers for me were:

1. The single lithium battery. We had so many issues with rechargeable AA’s that I was not keen to deal with the 4xAA needs of the SX1
2. The sweeping panorama shooting mode is really cool. I love panorama’s and having the picture auto-magic stitched is very cool
3. The HX1 was noticeably lighter than the SX1
4. There was only a $9 price difference between them, due to Sony lowering the list price by $200NZD  the day before. Which once you take into account having to buy batteries for the SX1 made the the HX1 cheaper

I miss the idea of not having a raw mode, and the ultra swivel LCD on the Cannon would be useful in more situations, but really, the batteries was a major issue.

Part of me was wanting to step-up to the bigger boys 450/1000 Cannons, but price was a consideration. If I use this camera enough and find myself limited by it, then a full DSLR may have to be the next step.

For now I’ll continue playing with it a lot, trying out different situations, and learning how to use it.

Having not researched the topic at all before buying, I was pleasantly pleased with the DSC-HX1 review at dpreview.com, with there not being any worst-camera-in-the-world type surprises.

We recently upgraded our subversion server at work because it was having performance problems, yet the new server also performed poorly, so our dev-svn-admin guy did a dump reload of all the repositories, to gain the benefits of the new Subversion file layouts.

There were a couple of hung transactions on some repositories, and two repositories (one was ours) didn’t have the UUID set, thus wouldn’t reload. No problem, he just set a UUID, and ta-da it loaded, but then our local working copies would not work, giving this error:

Repository UUID 'new UUID' doesn't match expected UUID '????????-????-????-????-???????????'

The first team just re-checked-out all the working copies the manually merged their local changes.

I pulled out Visual Studio and did a file based Find-and-Replace,

Whereas one of the my other team members came up with the following Cygwin command:

find . -name "entries" -exec sed -i 's\????????-????-????-????-????????????\1923097a-7eed-ce49-a323-f810e19527ea\' {} \;

Build 1.0.20 has been released. Fixed in this version:

• Issue 32, Fixed manual targeting
• Issue 33, Fixed Magic-Users spell selection on levelling to show new spells, not current
• Issue 34, Fixed the combat Aiming crash at the edges of the screen
• Issue 35, Fixed Magic-Users spell selection on levelling to show level 5 spells
• Issue 38, Loss of excess Exp when level, was in original DOS game, but I’ve removed it
• Add new cheat to sort treasure (most valuable at top)
• Screen Capture now save to “My Pictures” as “Curse - xxxx.png”
• Settings are now kept across upgrades

I have some more issues to track down, but I’m hoping this build will solve the worst of them.

Enjoy.