Getting Started with SoftICE

Before I start, I just want to issue a quick disclaimer that I used to be in the Driver Group at NuMega and was on the SoftICE team. Obviously, I am quite partial to SoftICE and I just feel that you should be aware of that fact up front.

While it might seem a little weird to admit, but I have always dreams of being omnipotent. I’m willing to settle for only partially omnipotent if I could just have a chance to stop the world every once in a while so I could get a peek under the covers so I could really see what’s going on. Up until now, I haven’t figured out how to break the laws of physics, but I do know one way to control all life on my computer: Compuware/NuMega’s SoftICE! While many people have heard of this magical software called SoftICE, not that many people have seen it. If you have seen it, you might have run out of the room screaming because with all the power of SoftICE, the tradeoff is that it’s really hard to use. In this column, I want to introduce what SoftICE is and how to get started using it to help flatten a tiny bit the almost vertical learning curve associated with it.

Example of Code Debugging Tools

What is the SoftIce CodeDebugger for C# and .Net?

Anyone who uses SoftICE on a regular basis will tell you that it’s the most powerful debugger on the planet. SoftICE is a kernel debugger that sits in between the operating system and the CPU so when SoftICE is active, the entire operating system stops dead. The fact that SoftICE can gain control of the operating system like that means that it can hunt anything it wants to down in memory and tell you about it. The real power of SoftICE is that its informational commands make it a piece of cake to find out what just about anything is in the system. For example, you can look at any event in the system and see what its name is and what its state is. The following list shows you just a taste at the Windows internals SoftICE can tell you about:

  • Complete thread and process information
  • All events, mutex, and semaphore information
  • All object handle information
  • Complete process and driver memory information
  • All Win32 heap information
  • Structured Exception Handling (SEH) frames
  • DLL exports

SoftICE is also a regular debugger as well that offers full source and symbolic debugging just like any GUI debugger. The vast majority of SoftICE users are using it to debug device drivers. However, because of SoftICE’s extreme power, I find that SoftICE is the only tool that helps in three key scenarios for us user mode developers. The first is in heavily multithreaded applications. Since graphical debuggers using system resources along with the debuggee, they can interfere with the debuggee and mask subtle timing problems. The second area where SoftICE really shines is when debugging larger multi-process applications and NT services such as cross process COM systems, MTS, and IIS applications. Some of the big web sites that people are developing might have five to fifteen different processes associated with them. If you have 4GB of RAM on the machine, you might be able to get all processes running under different copies of the always memory hungry VC debugger. With SoftICE, you just load the symbols for all the processes and set breakpoints in the processes where you want to stop. If you want to single step from a client application to a server application going through the RPC channel, it’s a piece of cake with SoftICE. The final scenario where SoftICE is invaluable is reverse engineering. At times, you need to see how something works or is implemented and with SoftICE’s ability to break on anything anywhere and with all those informational commands, the whole operating system is much easier to figure out.

While there are many positive things about SoftICE, it’s only fair if I mention some of the problems associated with it. The first is that SoftICE is not cheap if you are only interested in user mode debugging. The device driver developer is SoftICE’s main audience so it’s only sold as part of NuMega’s DriverStudio or the SoftICE Driver Suite. There are wonderful tools, such as BoundsChecker, TrueTime, and TrueCoverage for drivers, in those packages but if you are only looking for SoftICE, you have to purchase some tools you might not use. The biggest problem with SoftICE is that it is just hard to use. On one hand, we should commend NuMega that it has kept a consistent SoftICE user interface from the original DOS version through to today’s Windows 2000 version. On the other hand, the fact that that user interface is character based with none of the ease of use features we are used to in Windows applications means you must make a solid time investment to get up speed with SoftICE because the majority of the interaction with the tool is through typing commands. Additionally, you must also make the time investment to understand what SoftICE is trying to tell you from an informational standpoint. The SoftICE programmers assume that you already know what the items they display are so you need to know what’s going on from a system-programming standpoint. Even with all these issues, the pros certainly outweigh the cons and if you need SoftICE, you need it bad.

SoftIce Pre-requisites

Now that you have an idea of the gestalt of SoftICE, it’s time to turn to using it and smoothing that almost vertical learning curve. There are a few things that you need to do before you first load SoftICE and press the magical “CTRL+D” sequence to pop into that beautiful character-based interface. The first thing is to always check the NuMega web site for updated versions of SoftICE. Because SoftICE is so low level and has to have such intimate knowledge of the operating system, it’s not uncommon for SoftICE to break between operating system service pack levels. Consequently, if you update your OS, check with NuMega to see if SoftICE will break. NuMega is excellent about getting updated versions out as soon as beta versions of the operating systems are available.

Once you have the latest and greatest version, it’s time to check your video card. SoftICE video support used to be the biggest pain in the neck for all SoftICE users. Fortunately, with the advent of DirectX, SoftICE can now grab the video memory in a much cleaner manner so each video card doesn’t need a specific SoftICE video driver. During SoftICE installation, you will have to pick the video mode. In the Display Adapter setup, check the “Universal Video Driver” check box, set the card selection to Standard VGA, and press the “Test” button. If your video card supports the Universal Video Driver, you should be in good shape. Even if the Display Adapter setup appears to work, there are odd cases where SoftICE still might not properly display. If that’s the case, check with NuMega Technical Support and try using a different video card.

Finally, before you ever use SoftICE, take an afternoon and read the manuals! The SoftICE Technical Writers have done an excellent job and put in all sorts of excellent advice on how to get the most out of this very complicated product. An excellent chapter in the Using SoftICE manual is Chapter 11, Exploring Windows NT. That chapter will introduce you to using SoftICE to reverse engineer a few things in Windows NT. For the rest of the column, I will assume that you’ve read the manuals and have at least debugged “Hello World!” with SoftICE.

Getting Up To Speed with SoftIce Code Debugger

Once you’ve got everything set up and you can pop into and out of SoftICE at will, you’re ready to make the time investment you need to start learning SoftICE. Before you try to go off and completely reverse engineer the operating system loader, you should just try to use SoftICE as a regular user mode debugger for a few days. That will give you an idea of the basic operations and how to manipulate things in source, data, and the command window. Don’t worry about getting to fancy, just get used to the basic commands needed to do general debugging. Have the Command Reference manual handy because you will be referring to it quite a bit.

Before I talk about various tips and tricks, I want to briefly discuss a few screen positioning commands, a small bit about the mouse, and what to do when the mouse goes nuts. The default screen position for SoftICE is on the upper left corner of the screen, which is not the best place since you mostly look at the center of the screen. The magic key command of “CTRL+ALT+C” will automatically center the SoftICE window on the screen. If you want to move the SoftICE window around, your can use “CTRL+ALT+” to move in any direction you want. SoftICE also has three different fonts, numbered one, two, and three, for your viewing pleasure. Font 1 is the smallest and font 3 is the largest. The default is font 1 and unless you are running at 800×600, it will probably be too small to read. You can use the “set font #” command to switch between the fonts. I personally find font 2 to be the most readable on all the computers I use. Two SoftICE macros I have found very useful are as follows:

MACRO BIG="set font 2;lines 52"
MACRO SMALL="set font 1;lines 94"

I add these two to WINICE.DAT, which you can add either manually or through LOADER32.EXE’s SoftICE Initialization Settings dialog Macro tab, so I can always use them. The BIG macro forces the font to the medium size version and increases the lines shown in the window. Depending on your screen resolution, you might need more or less. The SMALL macro, as you can guess, shrinks the font and increases the number of lines you can see.

One thing you will want to get familiar with very quickly is SoftICE’s mouse support as it is very helpful to speeding up your debugging. First off, double clicking on a source line or an address will set a breakpoint on that line. Single clicking on a register value in the Register window or a data location in the Data window allows you to edit the values in place. Right clicking will bring up the context menu. If you right click on a valid address or symbol, right clicking will also highlight that value. One thing that’s a little different about the SoftICE context menu is that it includes the “Copy & Paste” command. When you are looking to add parameters to normal commands, the “Copy & Paste” command can make your usage much faster. For example, if you want to set a breakpoint on a function in the code window disassembly, type “BPX ” (notice the space!), right click on the function and select “Copy & Paste” from the right click context menu. Another really cool command on the right click menu is the “What” command. If you ever wanted to know what any value in a register, the disassembly, or in the Data window was, right click and select “What.” SoftICE is quite good at figuring out most items so just ask away!

Once of the biggest frustrations with SoftICE is when the mouse goes berserk. If you happen to be moving the mouse just right when SoftICE pops up because of an exception or a breakpoint, SoftICE interrupts the mouse packets in the middle so the mouse can be out of synchronization. The trick is to carefully tap the mouse so that the right click menu appears. When the menu appears, press “CTRL+M” a few times to get the mouse back in the groove.

Helpful Tricks and Commands for SoftIce Code Debugging

Once you get comfortable debugging simple user mode applications under SoftICE, you’ll want to start digging a little deeper into what SoftICE can do. In this section, I’ll discuss commands and techniques to help you get past some of the confusing parts.

Getting Back to the Current Instruction Pointer

If you debug as I do, you will be looking at various things and moving all over the place in the process. Once you get down a few layers, it can get a little confusing to figure out where you started. You can use either the “.” command or right clicking on EIP in the register window and selecting “Unassemble.”

Setting the Address Context

Sometimes, half of using SoftICE is getting into the right process before doing specific operations. Since SoftICE is global, you sometimes need to force SoftICE into shifting it’s internal pointers so that any information command you’ll want to execute apply to a specified process. The “ADDR” command allows you to see the running processes as well as to force SoftICE into a specific address context. You can see what address context is current by looking at the lower right status bar. If you type “ADDR” by itself, you will see a listing of all processes with the current address context denoted by an asterisk.

Breakpoints

As with any modern debugger, SoftICE supports some incredible breakpoint options. I strongly recommend that you take the time to read the Breakpoints and Expressions chapters in the manual because they will show you quite a few very cool breakpoints examples. The most important type of breakpoint SoftICE supports is the memory breakpoint with the “BPM” command. SoftICE is the only debugger capable of debugging user mode programs and stopping on any access to a particular piece of memory. When I say, any access, I mean any read, write, or attempted execution. I don’t know how many times I have used the BPM command to hunt down wild writes to uninitialized memory. Another breakpoint command very helpful in certain situations is the “BSTAT” command, which reports all sorts of breakpoint statistics. Finally, one breakpoint trick I use is “BPX @ESP” which will break on the caller of the current function. That way I can get back to who called the current function in case I want to set a breakpoint at the caller.

Unassemble Command

While you probably realize that the “U” command will unassemble addresses, the unassemble command will also do where you have breakpoints set just by passing “BP#” (where # is the breakpoint number) as the command. The “U” command will also jump to a specific line in a source file if “.###” (where ### is the source line) as the parameter as well.

Minimizing Extraneous Noise

At times, SoftICE can report way too much information in the Command window when you have the default verbose processing turned on. In general, I turn off the verbose processing with “SET VERBOSE OFF” so the only output I see in the Command window is my calls to OutputDebugString.

Getting Out of Infinite Loops

SoftICE will pop up and report any unhandled exceptions on the computer. Occasionally, SoftICE will seem to get in an infinite loop where it never passes the exception on to the operating system. When that happens, use “FAULTS OFF” to tell SoftICE to turn its fault trapping off. Once you get past the problem area, you can pop back into SoftICE and do a “FAULTS ON” to restore the fault trapping.

Wrap Up

I hope I’ve given you some tips and tricks to make the pain of learning SoftICE a little easier. If you have other SoftICE tricks that you’d like to share, I’d love to hear about them!

More by Author

Get the Free Newsletter!

Subscribe to Developer Insider for top news, trends & analysis

Must Read