원문 : symhttp.doc


HTTP Symbol Stores and the Symbol Server Proxy


Configuring an HTTP Symbol Store

Requirements:  A computer running Windows Server.  Internet Information Services 6.0 is required, so in the examples below we use Windows Server 2003. 


Configuring the Directories

You need to create or select a directory on the computer to act as the symbol store.  For the sake of example, we will call this c:\symstore. We will also presume that the server computer is named \SymMachineName on the network.  Permissions must be set so that users can access the site.  Add the security groups that will need to access the symbol content over the network.  The amount of security to enable will vary from environment to environment.  For some installations, this group will be “Everyone”.

Setup the permissions for the directory

  1. Open the “Windows Explorer”
  2. Expand “My Computer”
  3. Expand the “C” drive
  4. Right-click “c:\symstore” and choose “Sharing and Security”
  5. Check the “Share this folder” button
  6. Click the “Permissions” button
  7. Make sure that the desired security groups have “read” access by adding them under “Group or user names” and checking the appropriate box
  8. Click “OK” to exit “Permissions”
  9. Click “OK” to exit “Symstore Properties”

Normally files are placed in a symbol store with a single tier directory structure in which a single subdirectory exists to store all versions of a certain filename.  Such a tree may look like this 

      c:\symstore\ntdll.dll\

      c:\symstore\ntdll.pdb\

      c:\symstore\kernel32.dll\

      c:\symstore\kernel32.pdb\

However if you are going to store a massive amount of filenames, you may prefer to use the two-tier structure.  To do this, place a file called index2.txt in the root of c:\symstore.  The contents of the file are of no importance.  This would result in a tree that looks like this

      c:\symstore\nt\ntdll.dll\

      c:\symstore\nt\ntdll.pdb\

      c:\symstore\ke\kernel32.dll\

      c:\symstore\ke\kernel32.pdb\

The added layer of directories allows you to use DFS or directory junctions to manage and split up your files.  

Make certain you can view the contents of this directory from other computers on your network.

If you are going to use the Symbol Server Proxy, you can skip the following steps and go to the Configuring IIS section.

  1. Use symstore.exe to populate the contents of c:\symstore.  Instructions for this are available in the debugger documentation.  
  1. Make sure the store works by attempting to debug with it.  Do this by debugging from another computer with a symbol path of srv*\\SymMachineName\symstore.


Configuring IIS

Now you need to configure IIS to serve out the symbols.


Create a Virtual Directory

  1. From “Administrative Tools” open “Internet Information Services (IIS) Manager”.
  2. Expand “Web Sites”
  3. Right-click “Default Web Site” (or other desired site)
  4. Select “New Virtual Directory…”
  5. Click “Next” on the “Welcome” screen.
  6. For the “Alias” enter “Symbols” and click “Next”
  7. For the “Path” enter “c:\SymStore” and click “Next”
  8. Uncheck “Run scripts” (recommended)
  9. Put a check in “Browse” (optional)
  10. Click “Next” then “Finish” to complete the wizard.

Configure MIME Types

  1. Right-click the “Symbols” virtual directory and choose “Properties”
  2. Click the “HTTP Headers” tab.
  3. Click the “MIME Types” button.
  4. Click “New”
  5. For “Extension” enter “*”
  6. For “Mime type” enter “application/octet-stream”
  7. Click “OK” to exit the “MIME Types” dialog.
  8. Click “OK” to exit the “Symbols Properties”

If you want to use Anonymous authentication, then IIS is now ready to serve up files from http://SymMachineName/Symbols.  Test it by restarting the IISAdmin service.  You can do this by running “iisreset.exe” from a command prompt.  Then configure a debugger to get files with the symbol path set to srv**http://SymMachineName/Symbols.  Otherwise, complete the rest of these steps.

It is possible to configure IIS to use “Integrated Windows Authentication” so that clients (windbg.exe for example) can automatically authenticate against IIS without prompting the end-user for credentials. SymSrv.dll on the client, however, does not currently work correctly using Kerberos authentication when connecting to IIS. For this reason it is necessary to remove Kerberos as an option.

Configure the Authentication method

  1. From “Administrative Tools” open “Internet Information Services (IIS) Manager”.
  2. Right-click the “Symbols” virtual directory that was added in the previous steps then choose “Properties”.
  3. Click the “Directory Security” tab.
  4. Under “Authentication and access control” click “Edit”
  5. Ensure that only “Integrated Windows Authentication” is checked.
  6. Make sure “Enable anonymous access” is unchecked
  7. Click “OK” to exit the “Authentication Methods” dialog.
  8. Click “OK” to exit “Symbols Properties”

The following command includes the web site “Identifier”. If you are not using the “Default Web Site” (which is site “1”), you will need to determine the correct identifier by highlighting the “Web Sites” folder and looking at the identifier listed in the right-hand pane for the applicable web site. Replace “1” in the following command with the correct identifier number.

Force NTLM authentication (remove Kerberos authentication)


  1. Open a Command Prompt window.
  2. Change directory to c:\inetpub\AdminScripts
  3. Type the following including the quotes:
    cscript adsutil.vbs set W3SVC/1/root/Symbols/NtAuthenticationProviders "NTLM"


IIS is now ready to serve up files from http://SymMachineName/Symbols.  Test it out by restarting the IISAdmin service and configuring a debugger to get files with the symbol path set to srv**http://SymMachineName/Symbols.


Configuring an HTTP Symbol Server Proxy

You can configure your HTTP-based symbol store to act as a proxy between client computers and other symbol stores.  The implementation is through an ISAPI filter called SymProxy.dll.  The SymProxy server can be used as a gateway computer to the Internet or other sources within your company network. 

The Symbol Server Proxy can be a valuable setup for many situations. The following are some examples. 

  • You are debugging many systems within a lab environment in which the computers are not attached to the company network, but the symbols are stored in the network and must be accessed using Integrated Windows Authentication. 

  • Your corporate computing environment includes a firewall that prevents access to the internet from computers that are debugging and you need to get symbols from an internet web site such as http://msdl.microsoft.com/download/symbols. 

  • You wish to present a single symbol path for all users in your company so that they don’t need to be aware of where symbols come from and so that you can add new symbol stores without user-intervention. 

  • You have a remote site that is physically far from the rest of your company resources and network access is slow.  This system can be used to acquire symbols and cache them to the remote site. 

Installation of SymProxy requires the manual location of files, a way to authenticate your SymProxy to remote symbol stores, and reconfiguration of IIS.  You are also required to go through the “Configuring an HTTP Symbol Store” section of this document.

 

Installing the Filter 

Begin by installing the filter on the server.  Copy symproxy.dll and symsrv.dll to %WINDIR%\system32\inetsrv.   

Create a file called %WINDIR%\system32\inetsrv\symsrv.yes.  The contents of this file are of no consequence.  This prevents problems accessing the Microsoft Symbol Store at http://msdl.microsoft.com/downloads/symbols.

 

Configuring the Registry 

SymProxy stores its settings in the following registry key 

HKLM/Software/Microsoft/Symbol Server Proxy 

From this location, SymProxy learns what logging level to use as well the location from which to grab symbols for storage in the web site. 

You can create this key by running the symproxy.reg tool.  This tool is provided with the product.  Type “symproxy.reg” on the command line or simply double-click it from “Windows Explorer”.


Web Directories 

Within this key, a subkey named “Web Directories” exists.  It is in this key that the source paths from obtaining symbols is stored.  You need to create a separate key below here that matches the name of every virtual directory you generated in IIS to run a symbol store from.  For example, if you named this directory “symbols”, then create a registry key under “Web Directories” with that name. 

Then in each of the keys you have created, create a new “String Value” (REG_SZ) or “Expandable String Value” (REG_EXPAND_SZ) called “SymbolPath”.  Edit the contents of this value to contain all the symbol stores that will be used to feed the SymProxy symbol store.  If there is more than one, then separate them with semicolons.  A maximum of 10 stores is supported.  HTTP paths need to include the “http://” prefix and UNC paths need to include the “\\” prefix.

For example:

                \\symbols\symbols;http://msdl.microsoft.com/download/symbols

In this example, symbols will first be looked for in \\symbols\symbols.  If the files are not found there, then the Microsoft Symbol Store will be used. 

If the “SymbolPath” value is not properly set, the filter will not know where to obtain symbols from and it will not work.

It is also possible to define a global source for all symbol files by generating a “SymbolPath” value directly in the “Web Directories” key.  However this is highly discouraged, because it will cause malformed symbol requests from debugging clients to generate bogus directories and files in the root of your web site.


Logging

SymProxy reports its activity to the Application Event Viewer.  The level of reporting is set by the value stored in “LogLevel”.  This is a REG_DWORD located in the “Symbol Server Proxy” key.  Normally it is set to zero.  You can change this to any of the following values

0 - quiet 

1 - error

2 - success

3 – info

4 – warning

5 - debug

If you are having trouble moving files into the symbol store, you can adjust these values to see what is happening.  Set the “LogLevel” to 5 and all activity will be put in the Event Viewer.  If you want to see which files come from where and be able to examine the initialization, set the “LogLevel” to 3.

In order to complete the configuration of the logging, you need to register SymProxy with the Application Event Viewer.  Do this by adding the following key to the registry…

                HKLM\SYSTEM\CurrentControlSet\Services\EventLog\Application\SymProxy

Within this key, create a new “Expandable String Value” (REG_EXPAND_SZ) called “EventMessageFile”.  Set the contents of this to read as so…

“%WINDIR%\system32\inetsrv\symproxy.dll”. 

This will add symproxy.dll as a message source to the registry and cause application logging to work. 


HTTP Proxies

You may also need to set up HTTP proxy settings so that the service can access outside network resources.  You can do this using the proxycfg program.  Type “proxycfg.exe -?” for instructions.  The default behavior of SymProxy is to use whatever HTTP proxy is designated by this program.  If no HTTP proxy is configured, then SymProxy will use a dummy proxy.  The reason for this is to allow for access to secure HTTP sites within your intranet.  As a side effect, this prevents SymProxy from working with direct connections to the external internet.  You can disable this through the registry.  Create a REG_DWORD called “NoInternetProxy” located in the “Symbol Server Proxy” key.  If this value is set to 1 and there is no HTTP proxy indicated by proxycfg.exe, then SymProxy will operate with a direct connection to the internet.


Choosing Network Security Credentials

The web service needs to run from a security context that has the appropriate privileges to access the symbol stores that you intend to grab symbols from.  If you will be sourcing from an external web store such as http://msdl.microsoft.com/download/symbols, then this account will need to be able to access the web from outside any firewalls.  If you will be picking up any files from other computers on your network, then the account will need to have appropriate privileges to read files from those shares.  Some options available are to set the SymProxy to authenticate as the “Network Service” account, or to create a user account that is managed within Active Directory along with other user accounts, such as your own.

Authenticate as “Network Service”

The “Network Service” account is built-in to Windows, so there is no extra step of creating a new account.  For this example, we will presume the machine where the Symbol Proxy is being configured is SymMachineName on a domain called corp.

Note that external symbol stores (or internet proxy) will have to be configured to allow this machine’s “Network Service” account (Machine Account) to authenticate successfully.   There are 2 general methods of ensuring access:

  1. Allow access to the “Authenticated Users” group on the external store (or internet proxy).
  2. Allow access to the Machine Account, “corp\SymMachineName$”  This option is more secure because it limits access to just the Symbol Proxy machine’s “Network Service” account.


Authenticate as a Domain User

For this example, we will presume the user account is named SymProxyUser on a domain called corp

It is good practice that this account should only have privileges necessary to read files and copy them to c:\symstore.  This is to prevent the system from being corrupted in some manner by clients accessing your HTTP store.  

Add user account to the IIS_WPG group

  1. From “Administrative Tools” open “Computer Management”
  2. Expand “Local Users and Groups”
  3. Click “Groups”
  4. Double-click “IIS_WPG” in the right pane
  5. Click the “Add” button
  6. Enter “corp\SymProxyUser” in the pane labeled “Enter the object name to select”
  7. Click “OK” to exit “Select Users, Computers, or Groups”
  8. Click “OK” to exit “IIS_WPG Properties”
  9. Close the “Computer Management” console

Note that external symbol stores (or internet proxy) will have to be configured to allow this user to authenticate successfully.  


Configuring IIS for SymProxy

IIS needs to be configured to use the symproxy.dll as an ISAPI filter.  Furthermore, permissions need to be set so that IIS can grab symbols.

Configure the Application Pool

  1. From “Administrative Tools” open “Internet Information Services (IIS) Manager”
  2. Right-click “Application Pools” and choose “New” à “Application Pool”
  3. For the “Application ID” enter “SymSrv Pool”
  4. Click “OK” to exit “Add New Application Pool”
  5. Right-click the “SymSrv Pool” application pool and choose “Properties”
  6. Click the “Identity” tab.
    1. If you chose to authenticate as “Network Service”

                                                               i.      Choose “Predefined” for the “Application pool Identity”

                                                             ii.      Select “Network Service”

    1. If you chose to authenticate as a Domain User

                                                               i.      Choose “Configurable” for the “Application pool identity”

                                                             ii.      Enter the credentials of the account that has permissions to access the remote symbol server store(s) (“corp\SymProxyUser”) , then click “OK”.

  1. Click “OK” to exit “SymSrv Pool Properties” 


Set up the Virtual Directory

  1. Expand “Web Sites”
  2.  Expand “Default Web Site”
  3. Right-click the “Symbols” virtual directory and choose “Properties”
  4. While on the “Virtual Directory” tab click the “Create” button.
  5. For the “Application Pool” drop-down choose “SymSrv” Pool”
  6. Click “OK” to exit the “Symbols Properties”.


Configure the ISAPI Filter

  1. Right-click the “Default Web Site” and choose “Properties” (or properties of applicable web site).
  2. Click the “ISAPI Filters” tab.
  3. Click “Add”
  4. For “Filter Name” type “SymProxy” or some other meaningful name.
  5. For “Executable” enter “c:\windows\system32\inetsrv\symproxy.dll”
  6. Click “OK” to exit the “Filter Properties” dialog.
  7. Click “OK” to exit the “Default Web Site Properties” 


Exclusions

In some environments, you may find yourself debugging systems for which there is a large quantity of modules loaded for which you cannot obtain symbols.  This is often the case if you have code that is called by another third-party vendor.  This can result in a lot of failed attempts to grab symbols.  This can be time-consuming and clog up network resources.  To alleviate this situation, you can use an Exclusion List to specify symbols that you don’t want to search for.  This feature exists in the client debugger. 

Alternately, you can configure the SymProxy filter to utilize its own the Exclusion List and prevent such network activity where it is most likely to take up resources.

The list is made up of the names of the files for which you want to prevent processing.  The file names can contain wildcards.  For example

      dbghelp.pdb

      symsrv.*

      mso*

The list can be implemented in two ways.  The first is an ini file, %WINDIR%\system32\inetsrv\symsrv.ini.  A section called “exclusions” should contain the list.

      [exclusions]

      dbghelp.pdb

      symsrv.*

      mso*

Alternately you can store the exclusions in the registry.  Create a key called HKLM\ Software\Microsoft\Symbol Server\Exclusions.  Store the file name list as string values (REG_SZ) within this key.  The name of the string value acts as the file name to exclude.  The contents of the string value can be used as a comment describing why the file is being excluded.

SymProxy reads from the Exclusion List every half-hour so that you don’t need to restart the web service to see changes take effect.  If the ini file exists, it will be used.  Otherwise the registry is checked.  SymProxy does not support the use of both symsrv.ini and the registry.  Simply add files to the list in the registry or ini file and wait a short period for the exclusions to be used.


Timeouts

If one of the symbol server stores that Symbol Server is configured to grab files from is down or otherwise unavailable, the result can be long waits from the client for every file request.  You can avoid most of these waits by setting up Symbol Server to stop trying to access the store in question.  When this feature is engaged, Symbol Server will stop trying to use the store for a set period of time after it experiences a specified number timeouts from the same store in another set period of time.  All three of these values can be controlled by an ini file or from the registry.

  • trigger indicates the amount of minutes to look for the indicated amount of timeouts.
  • count is the amount of timeouts to look for within the trigger period.
  • blackout is the amount of minutes to disable the store once the threshold is reached.

In %WINDIR%\system32\inetsrv\symsrv.ini, create a section called “timeouts” and add the following values…

      [timeouts]

      trigger=10

      count=5

      blackout=15

You can set the “trigger” and blackout values to any amount of minutes you think is appropriate.  In this recommended example configuration, the store access will be turned off if 5 timeouts are experienced in a 10 minute period.  At the completion of a 15 minute blackout, the store will be reactivated. 

You can also use the registry in the same way.  Create a key called HKLM\ Software\Microsoft\Symbol Server\Timeouts.  Then add a REG_DWORD values called trigger, count, and blackout within this key.  Set the values as described previously.

Whether using the registry or an ini file, if the any of the trigger, count, or blackout values are set to 0 or if all such keys and values do not exist, this functionality will be disabled. 

This feature of Symbol Servers is currently only available when running as a service.  This means that the only practical application of this feature is when it is called from SymProxy.


Status

It is possible to see where symproxy is configured to acquire symbols from by using any browser software.  Just bring up status to get the information.  So if the symbols web site is http://server/symbols, then go to http://server/symbols/status.  You can also use this to cause symproxy to re-read its configuration information after making a change to the registry.  This is handy for changing the paths without having to restart the IIS service.


File Pointers

A UNC symbol store supports placing the actual files to be served in a separate location, with the client code learning of their whereabouts through file pointers.  These pointers are generated in the symbol store using symstore.exe with the “/p” option.  This scenario is supported with vanilla HTTP-based symbol stores only if the file pointers point to a UNC location that is directly accessible by the client.  When SymProxy is loaded into the web server, this functionality is enhanced.  The client no longer needs to be able to directly access the target files, because SymProxy will serve them through the HTTP interface.  This feature runs exclusive from the main proxy component discussed earlier and requires no preparation of the “Symbol Server Proxy” key in the registry.

Since this feature is automatically applied, an option exists to turn it off.  This is in case you want to use the proxy for serving some files and regular file pointer implementation for others.  To do this, create a REG_DWORD called NoFilePointerHandler in HKLM\Software\Microsoft\Symbol Server.  Set this value to 1 (or anything other than 0) to turn off the internal file pointer handler in symproxy. 


Double vs. Single File Caching

Normally, SymProxy caches the files it acquires into the directory designated within IIS as the virtual root for the associated web site.  Then IIS makes the file available to the client debugger.  Since the debugger cannot open a file directly from HTTP, it copies the file to a local cache, specified by the symbol path.

                srv*c:\localcache*http://server/symbols

In this example, the client debugger copies the file to c:\localcache.  In a situation such as this, the file is copied twice – once by SymProxy to the virtual root of the web site, and again by the debugger to its local cache.

It is possible to avoid the second copy operation and speed up processing.  To do this, you first need to share the virtual root of the web site as a UNC path that can be accessed by the debuggers.  For sake of example, we will call this \\server\symbols.

Then, remove the IIS configuration for mime types…

  1. From “Administrative Tools” open “Internet Information Services (IIS) Manager”.
  2. Expand “Web Sites”
  3. Right-click “Default Web Site” (or other desired site)
  4. Right-click the “Symbols” virtual directory and choose “Properties”
  5. Click the “HTTP Headers” tab.
  6. Click the “MIME Types” button.
  7. Select all types in the list box labeled “Registered Mime Types”.
  8. Click the “Remove” button.
  9. Click “OK” to exit the “MIME Types” dialog.
  10. Click “OK” to exit the “Symbols Properties”


 This will cause IIS to return “file not found” to the debugging client for all transactions on the web site.  However it will not prevent SymProxy from populating the virtual root with the file.

Lastly configure the debugger clients to look for symbols first in the HTTP store, and then in the share that maps to the virtual root of the store.

                srv**http://server/symbols;srv*\\server\symbols


In the above example, the first element of the symbol path (srv**http://server/symbols) says to get files from the HTTP store and copy them to the default symbol store as a local cache.  The specified cache is of no importance because no file will ever be received from the HTTP store.  After this failure, it will attempt to grab the file from the actual location of the virtual root of the store (srv*\\server\symbols).  This attempt should succeed because the file was copied to that location as a side effect of the previous path processing.


Try it out

The system should now be ready to acquire and serve up files.  Test it out by restarting the IISAdmin service and configuring a debugger to get files with the symbol path set to srv**http://server/symbols.

728x90

원문 : srcsrv.doc



Microsoft Source Server


General Concepts and Usage


Usage and Source Indexing


The source server enables a client to retrieve the exact version of the source files that were used to build an application. Because the source code for a module can change between versions and over the course of years, it is important to look at the source code as it existed when the version of the module in question was built.


The source server retrieves the appropriate files from source control. To use the source server, the application must have been source indexed.


Using Source Server with a Debugger


To use the source server with WinDbg, KD, NTSD, or CDB, ensure that you have installed a recent version of the Debugging Tools for Windows package (version 6.3 or later). Then, include srv* in the .srcpath command as follows:


.srcpath srv*;c:\mysource


Note that this example also includes a traditional source path. If the debugger cannot retrieve the file from the source server, it will search the specified path.


If a source file is retrieved by the source server, it will remain on your hard drive after the debugging session is over. Source files are stored locally in the src subdirectory of the Debugging Tools for Windows installation directory.


If you experience any trouble extracting the source files from the debugger, start the debugger with the –n command line parameter to view the actual source extraction commands along with the output of those commands.  !sym noisy will do the same thing, but you may have already missed important information from previous extraction attempts.  This is because the debugger will give up trying to access source from version control repositories that appear to be unreachable.


Retrieving the Source File


To facilitate the use of Source Server from tools other than the debuggers listed above, the DbgHelp API provides access to source server functionality through the SymGetSourceFile function. To retrieve the name of the source file to be retrieved, call the SymEnumSourceFiles or SymGetLineFromAddr64 function.


Source Indexing


The source indexing system is a collection of executable files and Perl scripts. The Perl scripts require Perl 5.6 or greater (not included). They are installed with the debugger package in a subdirectory sdk\srcsrv.  To work with these tools, they should all be installed in the path.


Generally, binaries are source indexed during the build process after the application has been built. The information needed by Source Server is stored in the PDB files. Source Server currently supports the following source-control systems…


Perforce
Microsoft Visual SourceSafe
Microsoft Team Foundation Server
CVS (Concurrent Versions System)


You can also create a custom script to index your code for a different source-control system.  One such module for Subversion is included in this package. 


While every attempt has been made to make these above-mentioned modules as bulletproof and robust as possible, the simple fact is, there is no way to account for all of the variance found in version control enlistments found throughout the world.  It is quite possible that you will not see success on the first attempt to use these tools and it is also possible that you will have to do a little debugging and even a little Perl coding to customize the scripts to your needs.  Here are a couple of hints.


Get familiar with srctool.exe and all of its command line options.  It is a valuable tool to see what is going on.


If it comes down to Perl debugging and programming, focus on the module that corresponds with your specific version control system.  These files end with an extension of .pm.  It is somewhat unlikely that your problem is in the master script, (ssindex.cmd).


Feel free to ask questions on our newsgroup, microsoft.public.windbg.


The following table lists the Source Server tools.


Tool

Description

SrcSrv.ini

This file is the master list of all source control servers.  Each entry has the following format:

MYSERVER=serverinfo

When using Perforce, the server info consists of the full network path to the server, followed by a colon, followed by the port number it uses.  For example:

MYSERVER=machine.corp.company.com:1666

Srcsrv.ini is a required file when actually source indexing a build using the modules shipped with this package.  This entry creates an alias that will be used to describe the server info.  The value should be unique for every server you support.

This file can also be installed on the computer running the debugger.  When Source Server starts, it looks at srcsrv.ini for values; these values will override the information contained in the PDB file.  This enables users to configure a debugger to use an alternate source control server at debug time.  However if you manage your servers well and don’t rename them, there should be no need to include this file with your client debugger installations.

This file also serves other purposes on the client side.  For more information, see the sample srcsrv.ini installed with the Source Server tools.

SSIndex.cmd

This script builds the list of files checked into source control along with the version information of each file.  It stores a subset of this information in the PDB files generated when you built the application.  The script uses one of the following Perl modules to interface with source control…

p4.pm (Perforce)
vss.pm (Visual SourceSafe)
tfs.pm (Team Foundation Server)
cvs.pm (Concurrent Versions System)
svn.pm (Subversion)

For more information, run the script with the “-?” or “-??” (verbose help) option or examine the script.

SrcTool.exe

This utility lists all files indexed within the PDB file.  For each file, it lists the full path, source control server, and version number of the file.  You can use this information for reference.

You can also use srctool.exe to list the raw source file information from the pdb.  To do this, use the “-s” switch on the command line. 

SrcTool.exe has other options as well.  Use the “-?” switch to see them.  Of most interest, is that srctool.exe can get used to actually extract all of the source files from version control.  This is done with the “-x” switch.

Note:  Previous versions of this program used to create a directory called “src” below the current one, when extracting files.  This is no longer the case.  If you want this directory used, you need to create it yourself and run out of there.

PdbStr.exe

This utility is used by the indexing scripts to insert the version control information into the “srcsrv” alternate stream of the target PDB file.  It can also read any stream from a PDB file.  You can use this information to verify that the indexing scripts are working properly.

For more information, run the utility with the “-?” option.

VSSDump.exe

This utility is used by the indexing script for Visual SourceSafe.  It gathers the list of source files to be indexed.  This program is also a valuable command line utility that you can use to examine which files may be processed by the indexing script.


 


To prepare for source indexing, edit srcsrv.ini so that it contains an entry for your version control server.  This is an operation that you will do only once.  Details are listed in the sample version of this file.  You can use an environment variable or switch to the indexing script to denote the location of this file.   However, it is best to put it in the same directory as the script, because the contents of this file are intended to be global across all projects in your business or development system.  This file serves to uniquely identify different version control servers.  Note that you can provide a different version of this file to debuggers so that they look at a replicated copy of the version control server.  This can be useful if you want to reduce traffic on the server.


To source index a particular build, make certain that no source files are checked out on the build computer.  If any files are checked out and edited, then those changes will not be reflected in the final source indexed PDB files.  Furthermore, if your build process includes a pre-compilation pass that generates source files from other files, then you need to check those generated files into version control as part of the pre-compilation.


Upon completion of the build, set the current working directory to be the root of all source and generated PDB files.  Then run the indexing script ssindex.cmd.  You will need to specify what version control system you are using as a parameter.  For example:


ssindex.cmd –server=vss


Ssindex.cmd accepts parameters that allow you to run the script from anywhere and to specify the location of the source files and pdb separately.  This is most useful if the source is kept in another location from the output pdb files.  Example:


ssindex.cmd –server=vss –source=c:\source –symbols=c:\outputdir


These directories can also be specified with environment variables.  Use the -? or -?? command line options for more information...


C:\ >ssindex.cmd -system=vss -?
--------------------------------------------------------------------------------
SSIndex.cmd [/option=<value> [...]] [ModuleOptions] [/Debug]


General source server settings:


     NAME              SWITCH      ENV. VAR        Default
  -----------------------------------------------------------------------------
  1) srcsrv.ini        Ini         SRCSRV_INI      .\srcsrv.ini
  2) Source root       Source      SRCSRV_SOURCE   .
  3) Symbols root      Symbols     SRCSRV_SYMBOLS  .
  4) Control system    System      SRCSRV_SYSTEM   <N/A>
  5) Save File (opt.)  Save        SRCSRV_SAVE     <N/A>
  6) Load File (opt.)  Load        SRCSRV_LOAD     <N/A>


Visual Source Safe specific settings:


     NAME            SWITCH      ENV. VAR        Default
  -----------------------------------------------------------------------------
  A) VSS Server      Server      SSDIR            <N/A>
  B) VSS Client      Client      SSROOT           <Current directory>
  C) VSS Project     Project     SSPROJECT        <N/A>
  D) VSS Label       Label       SSLABEL          <N/A>


Precedence is: Default, environment, cmdline switch. (ie. env overrides default,
switch overrides env).


Using '/debug' will turn on verbose output.


Use "SSIndex.cmd -??" for verbose help information.


See the Source Server documentation for more information.
--------------------------------------------------------------------------------


You can also use one of the provided wrapper scripts (vssindex.cmd) to avoid specifying the version control system.  The script will source index all PDB files found in the current directory and below, with version control information to locate all source files found in the current directory and below.  You can specify different locations for these files by using environment variables and command line switches.


Upon completion of the source indexing, you can test the output by running srctool.exe on the PDB files.  This program will display whether the PDB is source indexed or not.  It will also display the specific information for every source file.  Lastly, it displays the number of indexed sources and the number of sources that no indexing information was found for.  It sets an %ERRORLEVEL% of -1 if the file is not source indexed.  Otherwise it sets %ERRORLEVEL% to the number of indexed source files.


Creating a Source Control Provider Module


The source server includes provider modules for Perforce (p4.pm) and Visual Source Safe (vss.pm).   It is possible to generate your own modules to support other version control systems.  Included in this package is svn.pm, which was written by Shahar Talmi of Safeend, Ltd.  This shows how you can modify one of the existing scripts to support the Subversion Version Control system.


To create your own provider module, you must implement the following set of interfaces.


$module::SimpleUsage()


Purpose: Displays simple module usage information to STDOUT.


Parameters: None


Return value: None


$module::VerboseUsage()


Purpose: Displays in-depth module usage information to STDOUT.


Parameters: None


Return value: None


$objref = $module::new(@CommandArguments)


Purpose: Initializes an instance of the provider module.


Parameters: All @ARGV arguments that weren’t recognized by ssIndex.cmd as being general arguments.


Return value: A reference that can be used in later operations.


$objref->GatherFileInformation($SourcePath, $ServerHashReference)


Purpose: Enables the module to gather the required source indexing information for the directory specified by the $SourcePath parameter. The module should not assume that this entry will be called only once for each object instance, as ssIndex.cmd may call it multiple times for different paths.


Parameters:  (1) The local directory containing the source to be indexed.  (2)  A reference to a hash containing all of the entries from the specified srcsrv.ini file.


Return value: None


($VariableHashReference, $FileEntry) = $objref->GetFileInfo($LocalFile)


Purpose: Provides the necessary information to extract a single, specific file from the source control system.


Parameters: A fully-qualified file name


Return value:  (1)  A hash reference of the variables necessary to interpret the returned $FileEntry.  SSIndex.cmd caches these variables for every source file used by a single debug file to reduce the amount of information written to the source index stream.  (2)  The file entry to be written to the source index stream to allow SrcSrv.dll to extract this file from source control. The exact format of this line is specific to the source control system.


$TextString = $objref->LongName()


Purpose: Provides a descriptive string to identify the source control provider to the end user.


Parameters: None


Return value: The descriptive name of the source control system.


@StreamVariableLines = $objref->SourceStreamVariables()


Purpose: Enables the source control provider to add source control specific variables to the source stream for each debug file. The sample modules use this method for writing the required EXTRACT_CMD and EXTRACT_TARGET variables.


Parameters: None


Return value: The list of entries for the source stream variables.


Language Specification Version 1


Source Server Data Blocks


Source server relies on two blocks of data within the PDB file.


  • Source file list.  Building a module automatically creates a list of fully-qualified paths to the source files used to build the module.
  • Data block.  Indexing the source as described previously adds an alternate stream to the PDB file named “srcsrv”. The script that inserts this data is dependent on the specific build process and source control system in use.


The data block is divided into three sections: ini, variables, and source files. It has the following syntax.


SRCSRV: ini ------------------------------------------------
VERSION=1
VERCTRL=<source_control_str>
DATETIME=<date_time_str>
SRCSRV: variables ------------------------------------------
SRCSRVTRG=%sdtrg%
SRCSRVCMD=%sdcmd%
SRCSRVENV=var1=string1\bvar2=string2
DEPOT=//depot
SDCMD=sd.exe -p %fnvar%(%var2%) print -o %srcsrvtrg% -q %depot%/%var3%#%var4%
SDTRG=%targ%\%var2%\%fnbksl%(%var3%)\%var4%\%fnfile%(%var1%)
WIN_SDKTOOLS= sserver.microsoft.com:4444
SRCSRV: source files ---------------------------------------
<path1>*<var2>*<var3>*<var4>
<path2>*<var2>*<var3>*<var4>
<path3>*<var2>*<var3>*<var4>
<path4>*<var2>*<var3>*<var4>
SRCSRV: end ------------------------------------------------


All text is interpreted literally, except for text enclosed in percent signs (%). Text enclosed in percent signs is treated as a variable name to be resolved recursively, unless it is one of the following functions:


%fnvar%()


The parameter text should be enclosed in percent signs and treated as a variable to be expanded.


%fnbksl%()


All forward slashes (/) in the parameter text should be replaced with backward slashes (\).


%fnfile%()


All path information in the parameter text should be stripped out, leaving only the file name.


The ini section contains variables that describe the requirements. The indexing script can add any number of variables to this section. The following are examples:


VERSION


The language specification version. This variable is required.   Someone developing a script based on this language specification should set this value to 1.  The Source Server client code will not attempt to execute any script that has a value greater than it.  Current versions of srcsrv.dll use the value of 2. 


VERCTL


A string that describes the source control product. This variable is optional.


DATETIME


A string that indicates the date and time the PDB file was processed. This variable is optional.


The variables section contains variables that describe how to extract a file from source control. It can also be used to define commonly used text as variables to reduce the size of the data block.


SRCSRVTRG


Describes how to build the target path for the extracted file. This is a required variable.


SRCSRVCMD


Describes how to build the command to extract the file from source control. This includes the name of the executable file and its command-line parameters. This is required if any extraction command needs to be executed.


SRCSRVENV


A string that lists environment variables to be created during the file extraction. Separate multiple entries with a backspace character (\b). This is an optional variable.


SRCSRVVERCTRL


Specifies the version control system in use.  For Perforce, this is “perforce”.  For Visual SourceSafe, this is “vss”.  For Team Foundation Server, this is “tfs”.  This variable is used to persist server errors. This is an optional variable.


SRCSRVERRDESC


This is a text snippet that matches what is emitted by the version control client when it is unable to contact the server that contains the source files to extract.  Source Server uses this value to check for connection problems. This is an optional variable.


SRCSRVERRVAR


This indicates which variable in a file entry corresponds to a version control server.  It is used by Source Server to identify commands that won’t work, based on previous failures.  The format of the text is “varX” where X is the number of the variable being indicated.  This is an optional variable.


The source files section contains an entry for each source file that has been indexed. The contents of each line are interpreted as variables with the names VAR1, VAR2, VAR3, and so on until VAR10. The variables are separated by asterisks. VAR1 must specify the fully-qualified path to the source file as listed elsewhere in the PDB file. For example, the following line:


c:\proj\src\file.cpp*TOOLS_PRJ*tools/mytool/src/file.cpp*3


is interpreted as follows:


VAR1=c:\proj\src\file.cpp
VAR2=TOOLS_PRJ
VAR3=tools/mytool/src/file.cpp
VAR4=3


In this example, VAR4 is a revision number. However, most source control systems support labeling files in such a way that the source state for a given build can be restored. Therefore, you could alternately use the label for the build. The sample data block could be modified to contain a variable such as the following:


LABEL=BUILD47


Then, presuming the source control system uses the at sign (@) to indicate a label, you could modify the SRCSRVCMD variable as follows:


sd.exe -p %fnvar%(%var2%) print -o %srcsrvtrg% -q %depot%/%var3%@%label%


How Source Server Works


The source server client is implemented in symsrv.dll. The client does not extract information directly from the PDB file; it uses a symbol handler such as the one implemented in dbghelp.dll. It is essentially a recursive variable substitution engine that creates a command line that can be used to extract the proper source file from the source control system. Your code should not call Symsrv.dll directly. To integrate its functionality into your application, use the SymGetSourceFile function.


The first version of source server works as follows.  (This behavior may change in future versions.)


First, the client calls SrcSrvInit with the target path to be used as a base for all source file extractions.  This path is stored in the special variable TARG.


When dbghelp loads a module’s PDB file, it extracts the SrcSrv stream from the PDB and passes this data block to SrcSrv by calling SrcSrvLoadModule.


Then when dbghelp needs to grab a source file, it calls SrcSrvGetFile to retrieve a specified source file from version control.


Srcsrv looks through all the source file entries in the data block for an entry that matches exactly the requested source spec.  This match is to be found in VAR1.


Once srcsrv finds the entry, it fills in the special variables (VAR1, VAR2, etc.) with the contents of this source file entry.


Now the SRCSRVTRG variable is resolved using these special variables.  Beginning with the source line used in the previous example, the following shows how these variables are resolved.  Each line shows the resolution of one more special variable.  The resolved variables are shown in red.


SRCSRVTRG=%sdtrg%


SDTRG=%targ%\%var2%\%fnbksl%(%var3%)\%var4%\%fnfile%(%var1%)


c:\src\%var2%\%fnbksl%(%var3%)\%var4%\%fnfile%(%var1%)


c:\src\WIN_SDKTOOLS\%fnbksl%(%var3%)\%var4%\%fnfile%(%var1%)


c:\src\WIN_SDKTOOLS\%fnbksl%(sdktools/debuggers/srcsrv/shell.cpp)\%var4%\%fnfile%(%var1%)


c:\src\WIN_SDKTOOLS\sdktools\debuggers\srcsrv\shell.cpp\%var4%\%fnfile%(%var1%)


c:\src\WIN_SDKTOOLS\sdktools\debuggers\srcsrv\shell.cpp\3\%fnfile%(%var1%)


c:\src\WIN_SDKTOOLS\sdktools\debuggers\srcsrv\shell.cpp\3\%fnfile%(c:\db\srcsrv\shell.cpp)


c:\src\WIN_SDKTOOLS\sdktools\debuggers\srcsrv\shell.cpp\3\shell.cpp


Notice how this generated target path is unique and will not allow two versions of the same file to be extracted to the same location.


Srcsrv now looks to see if the file is already there.  If it is, srcsrv returns the location to the caller.  Otherwise, srcsrv builds the execution command to extract the file by resolving SRCSRVCMD.


In the following example, each line shows the resolution of one more special variable.  The resolved variables are shown in red.


DEPOT=//depot


WIN_SDKTOOLS= sserver.microsoft.com:4444


SRCSRVCMD=%sdcmd%


SDCMD=sd.exe -p %fnvar%(%var2%) print -o %srcsrvtrg% -q %depot%/%var3%#%var4%


sd.exe -p %fnvar%(WIN_SDKTOOLS) print -o %srcsrvtrg% -q %depot%/%var3%#%var4%


sd.exe -p sserver.microsoft.com:4444  print -o %srcsrvtrg% -q %depot%/%var3%#%var4%


sd.exe -p sserver.microsoft.com:4444  print -o c:\src\WIN_SDKTOOLS\sdktools\debuggers\srcsrv\shell.cpp\3\shell.cpp -q %depot%/%var3%#%var4%


sd.exe -p sserver.microsoft.com:4444  print -o c:\src\WIN_SDKTOOLS\sdktools\debuggers\srcsrv\shell.cpp\3\shell.cpp -q //depot/%var3%#%var4%


sd.exe -p sserver.microsoft.com:4444  print -o c:\src\WIN_SDKTOOLS\sdktools\debuggers\srcsrv\shell.cpp\3\shell.cpp -q //depot/ sdktools/debuggers/srcsrv/shell.cpp#%var4%


sd.exe -p sserver.microsoft.com:4444  print -o c:\src\WIN_SDKTOOLS\sdktools\debuggers\srcsrv\shell.cpp\3\shell.cpp -q //depot/ sdktools/debuggers/srcsrv/shell.cpp#3


Now srcsrv executes this command line.  If the result of this command line is a file in the expected location, then this path is returned to the caller.


Note that if a variable cannot be resolved, an attempt is made to look it up as an OS environment variable.  If that fails, the variable name is deleted from the text being processed.


Two consecutive percent sign characters are interpreted as a single percent sign.


Handling Server Errors


Sometimes a client will be unable to extract any files at all from a single version control server.  This can be because the server is down and off the network or because the user does not have appropriate privileges to access the source.  However the time consumed attempting to get this source can slow things down significantly.  In this situation, it is desirable to disable any attempt to extract from a source that has been proven to be unavailable.


Whenever Source Server fails to extract a file, it examines the output text produced by the command.  If any part of this command contains an exact match for the contents of the SRCSRVERRDESC, then all future commands to the same version control server will be skipped.  Note that you can define multiple error strings by adding numbers or arbitrary text to the end of the SRCSRVERRDESC variable name.  Examples:


SRCSRVERRDESC=foo: server not found
SRCSRVERRDESC_2=bar: network error


The identity of the server is acquired from SRCSRVERRVAR. So if SRCSRVERRVAR contains “var2” and the file entry in the pdb looks like this…


c:\proj\src\file.cpp*TOOLS_PRJ*tools/mytool/src/file.cpp*3


Then all future attempts to grab source using a file entry that contains “TOOLS_PRJ” in variable 2 will be bypassed.               


You can also add error indicators on the debugger client by editing srcsrv.ini.  Instructions are provided in the sample file provided.


API and Usage


This document allows you to understand how to prepare instrumentation scripts so that this source server technology can be integrated into your build or version control system.


This document exposes many of the specifics of how srcsrv.dll is called by symbol handler code.  However this information is not static and will not become so in the future.  No one should attempt to write code that calls srcsrv.dll directly.  This is reserved for dbghelp.dll or the Visual Studio products.  Third party vendors wishing to integrate such functionality in their products should use the SymGetSourceFile function.  This function, along with others in the DbgHelp API, is described in the dbghelp.chm documentation.


Language Specification Version 2


SRCSRVCMD


This package ships with a srcsrv.dll that is able to operate without SRCSRVCMD being defined in the srcsrv data stream of a PDB.  While such capability is of no use for normal file extraction from source control systems, it is useful for direct access of files from a UNC share or HTTP site.  Such usage is described later in this document.


Notes on Visual SourceSafe


The Database


The default Visual SourceSafe database can be set with the SSDIR environment variable.  If it is not set there, the Source Server indexing script can usually determine this from the registry.  If it can’t, the script will prompt you to set SSDIR.  Regardless, this database value must be listed in your srcsrv.ini file along with a simple alias to identify the database.  More instructions can be found in the comments in srcsrv.ini.  You should add the entry in the [variables] section of the file as so…


   MYDATABASE=\\server\VssShare


Current Project


Visual SourceSafe works with the concept of a “current project”.  The Source Server indexing script respects this value and limits itself to process only source files that are part of the current project.  You can see what the current project is with the “ss.exe project” command.  If you want to change the current project, you can do so with the “ss.exe cp” command.  For example, if you want all projects to be processed you would do so by setting the current project to the root, as so…


   ss.exe cp $/


Working Folder


Visual SourceSafe projects are associated with “working folders”.  A working folder is the location on the client computer which corresponds to the root of a project.  However it is possible for such a computer to obtain and build source without a working folder being specified, normally through “ss.exe get –R” or when creating projects through the Visual Studio interface.  This mode of operation is incompatible with Source Server indexing.  Source Server requires that a working folder be specified.  If you have not set a working folder, you can do so with the following command…


   ss.exe workfold <project> <folder>


Labels


Visual SourceSafe is not able to determine what version of a file exists within the working directory of a build machine.  Consequently, you must use labels to stamp the project with an identifier that will be used to extract source files on the debugger client.  So before indexing, you need to make sure that all changes are checked into the database and then apply a label to it with the “ss.exe label” command.  In the following example, the label “VERSION_3” is applied to a project called “$/sdktools”…


E:\nt\user>ss.exe label $/sdktools
Label for $/sdktools: VERSION_3
Comment for $/sdktools:
This is a comment.


After the label is applied, you can specify this label to the Source Server indexing script by setting the environment variable, SSLABEL, or by passing it as a command line parameter…


   vssindex.cmd –label=VERSION_3


Again, this label is required for Source Server indexing to work.  Note that if you pass a label that doesn’t exist in the project, it might take a long time for the script to complete and the results will be of no use.


Credentials


If your Visual SourceSafe database requires a user and optional password in order to access it, then these values must be set through the SSUSER and SSPWD environment variables.  This applies not only at the time the build is indexed, but also on the debugger client. 


Platform


Visual SourceSafe is an X86 application and the source indexing scripts and tools for it are installed only with the X86 package of Debugging Tools for Windows.


Debugging


As described previously, SSUSER and SSPWD may need to be set in your debugging environment.  Furthermore, please note that the version of srcsrv.dll that ships with Visual Studio 2005 is not able to detect if Visual SourceSafe (ss.exe) is prompting for credentials.  This will result in a hang in the application.  You can upgrade with the srcsrv.dll that ships with “Debugging Tools for Windows” to prevent this.  Regardless, the credentials must be set for source extraction to work, if your Visual SourceSafe database requires them.


If Visual SourceSafe is not set in the path of your debugging computer, you can get around this by adding an entry to the srcsrv.ini file that works with your debugger.  When using a standard installation of Visual Studio, this file should be located in the following location…


%PROGRAMFILES%\Microsoft Visual Studio 8\Common7\IDE\srcsrv.ini


In the [trusted commands] section described previously, add an entry like the following, making sure that the path points to the correct location of ss.exe.


ss.exe=”C:\Program Files\Microsoft Visual SourceSafe\ss.exe”


VSSDump


VSSDump.exe is a program that is used by the indexing scripts to locate and resolve source files to the database.  You can also use this program standalone so as to diagnose issues when source indexing.  Calling the program with “-?” will list the command line parameters…


vssdump: enumerates files in the current VSS project.
      -a                ignores the current project and lists all projects.
      -p:<projectname>  specifies a VSS project to use.
                        not to be used -with '-a'
      -d:<directory>    specifies a root directory
                        otherwise the current directory is used.
      -l:<label>        specifies a version label
      -t                don't test version to match a passed label
      -v:<sharepath>    specifies location of the VSS database - overrides SSDIR
      -r                recurse subdirectories
      -f                don't list files - just directories
      -i                ignore current directory and list entire project
                        not to be used with '-r'
      -s                format output for scripting
      -c                display only the VSS configuration info


Notes on CVS (Concurrent Versions System)


Disclaimer


The cvs module for Source Server was developed using Concurrent Versions System (CVS) 1.11.17 (client).  It has not been tested with any other versions of cvs.  Furthermore it has not been tested extensively.  We have noticed that the great variance in user installations has exposed a lot of opportunities for the scripts to fail.  So anyone using this script should consider that there is a chance he or she will have to do a little Perl debugging of the scripts to make they work out.  Those finding they need to do this should please let us know by posting to the microsoft.public.windbg newsgroup so that we can help you and enhance the scripts.


CVSROOT


On the machine that you source index the build, CVSROOT cannot contain password and user information.  You should use cvs.exe to set your credentialing information.


To prepare srcsrv.ini for cvs indexing you need to enter an alias for your repository that will uniquely identify it from any others in your network.  This repository needs to match the value of CVSROOT in your environment.  There is no need to set this value in the srcsrv.ini that you keep with your debugger clients since the alias will be defined in the source indexed pdb.


 Client Computer


The client computer that will be extracting files during debugging does not need a cvs sandbox or CVSROOT set.  It does need cvs binaries in the path and if the repository is locked, you will need to set the username and password with cvs.exe.


Revision Tags


Cvs is unable to extract a file by its version number.  Instead it must be done using what is known as a “tag”.  So when indexing a cvs-based system you need to make sure that all changes are checked into the repository and then apply a tag with the “cvs tag” command.  Then when indexing the file, make certain you use the “label” command line parameter to specify the tag that you want to associate with the build you are indexing.   You can achieve the same result by setting CVS_LABEL in the environment.  Other values can be set from the environment or the command line.  Don’t forget to use the -? command line parameter to examine your choices and to make sure that all was configured correctly.


ssindex.cmd –system=cvs -??


Notes on Team Foundation Server


The script provided for processing using TFS is incapable of working with labels that have spaces in them.  Code as been added to the script to handle spaces in the labels, however it is commented out for stability.  If you wish to source index with TFS using labels and you want to allow spaces in the labels, please examine the code at line 453 of tfs.pm.


HTTP Sites and UNC Shares


It is possible to set up a web site that provides version-specific source to the windbg.exe debugger through the srcsrv module.  Such a mechanism does not provide dynamic extraction of the source files from version control; however it is a valuable feature because it allows you to set the source path of windbg.exe to a single unified path that will provide source from many versions of many modules, instead of having to set separate paths for each debugging scenario.  This is not of interest to debugging clients that have direct access to the actual version control systems, but can be of assistance to those wanting to provide secure http-based access to source from remote locations.  The web sites in question can be secured through HTTPS and smartcards, if desired.  This same technique can be used to provide source files through a simple UNC share. 


Set up the web site


Set up a web site to share the source files from and note the root directory of the site.  You will probably also want to make the pdbs available on the web site, using Symbol Server technology.  So you might have symbols and source made available from locations such as this…


   https://diehelkthidoeodlby.com/symbols
   https://diehelkthidoeodlby.com/source


While it is outside the scope of this document to describe how to set up a web site, there are some step-by-step instructions for doing this in symhttp.doc that ships with the “Debugging Tools For Windows” package.  Please note that much of these instructions apply specifically to symbol servers.


Extract and locate the source files


Use srctool.exe –x to extract all the source files from all the modules that you want to provide source for.  You need to run this tool on pdb files that have already been source indexed for your version control system and you will need to run the tool on a computer that has version control access.  This will put all the source files into a common directory tree.  Copy the contents of this tree to the root of the web site.  You can do this as often as you wish on any new products or modules that you wish to add.  There is no worry about files overwriting each other, since the directory tree structure keep dissimilar files separated and uniquely accessible.


A script file called walk.cmd is provided in this package to make it easier to run srctool.exe and other scripts on all matching files within a directory tree.  walk.cmd walks recursively through a directory tree and executes any specified command on any file that matches a passed file mask.  You can pass a simple file mask if you want it to start in the current working directory, or you can pass the starting directory with the file mask.  Here is an example…


   walk.cmd c:\symbols\*.pdb srctool.exe -x


This will run the srctool file extraction on all pdb files in c:\symbols or directories below it.


Modify the pdb source indexing streams


pdb:       specifies the name of the pdb to modify. For the debugger clients to use the Source Server web site, the pdbs must be modified to point to it.  Normally you would make a copy of all the pdbs, change them, and make them available from a separate location, usually the web site, itself. 


Three files are provided to assist you in reconfiguring the pdbs.  cv2http.cmd and cv2http.pl extract the Source Server stream, modify it through a perl script, and put the altered stream back in the pdb.


   cv2http.cmd: pdb  alias  url
                you must specify a pdb file,
                a logical http site alias
                and the URL of the site


The script takes three required parameters…


pdb:             specifies the name of the pdb to modify.
alias:            specifies the logical name to apply to your web site.
url:                specifies the full URL of the site.


The alias parameter is especially interesting; as it will be stored in the pdb as a variable name that can be overridden on the debugger client in srcsrv.ini should you ever move the location of the web site.


This script requires that all the standard Source Server tools be available in the path, since it calls both srctool.exe and pdbstr.exe.  Don’t forget that cv2http.pl is a Perl script and can be modified as you see fit to meet your needs.


You can use walk.cmd to modify an entire set of pdb files.


   walk.cmd *.pdb cv2http.cmd HTTP_ALIAS https://diehelkthidoeodlby.com/source


The above command will call cv2http.cmd on every pdb file in a tree, passing the following parameters…


the name of each pdb
HTTP_ALIAS for the alias
https://diehelkthidoeodlby.com/source for the url.


After running this command on a tree of pdb files, they will be ready for installation into the web site (or whatever location you want to put them.  You may want to refer to the Symbol Server documentation for more details on this.  Remember that you can use srctool.exe and pdbstr.exe to examine the changes to the pdbs.


UNC Shares


There is nothing preventing you from using these same scripts to provide source files from a simple UNC share.  To do this, just eliminate all the previously described steps for setting up a web site.  When calling cv2http.cmd, replace the url parameter with the root of the UNC share that you extracted the source files to with srctool.exe, like so…


walk.cmd *.pdb cv2http.cmd MY_SOURCE_ROOT \\server\share


Supporting HTTP/UNC along with regular version control with the same pdb


Imagine the scenario in which you want to support your developers using the standard Source Server functionality that extracts files from version control, however you also want to make source files available through a web site or UNC share.  This may make sense if you have set up a test lab that does not have access to version control.  It is possible to support both users using the same set of pdbs.


First, extract the source files using srctool.exe as described previously and make the share available as either a web site or UNC share.  Then ignore all the steps that convert the pdbs using cv2http.cmd. 


Now on the computers that will use the HTTP/UNC shares, edit the srcsrv.ini that is in the debugger directory.  In the [variables] section of the file, add the following three statements…


MY_SOURCE_ROOT=\\server\share
SRCSRVCMD=
SRCSRVTRG=%MY_SOURCE_ROOT%\%var2%\%var3%\%var4%\%fnfile%(%var1%)


You should replace “\\server\share” with the root of  the UNC share you are providing or the URL of the web site that contains the source files.  You can also change “MY_SOURCE_ROOT” to be any alias you want that describes this location.  With these exceptions, everything else should be entered exactly as described.


All debuggers set up in this fashion will ignore the standard version control extraction instructions, and will, instead, access the source files from the location specified.  Meanwhile, all debuggers without these items included in srcsrv.ini will use the normal version control mechanism to extract source files.


728x90

원문:  kernel_debugging_tutorial.doc

Kernel Debugging with WinDbg

Getting Set Up

Fundamentals

Selected Techniques

Getting More Information

The debuggers in Debugging Tools for Windows are powerful, but they have a steep learning curve. This is particularly true for WinDbg and KD, the kernel debuggers used by driver developers (CDB and NTSD are user-space debuggers). The aim of this tutorial is to give a developer experienced with other debuggers enough information to get launched into kernel debugging and to use the Debugging Tools for Windows help file as a reference. The developer is assumed to be familiar with the general concepts of the Windows operating system and the build process.

The focus will be mainly on WinDbg, a kernel-mode and user-mode debugger with a graphical interface. KD is more useful for scripts and automated debugging and enjoys the reputation of being the tool of choice of the most serious programmers, but this tutorial will focus on WinDbg and will merely allude to KD from time to time.

The target operating system is Windows 2000 or later. Much of what is described here works for Windows NT 4.0, too. Furthermore, the target computer's processor uses x86 architecture. While much here will work for 64-bit targets, no specific attention will be given them.

In overview, the tutorial begins with a brief description of setting up for debugging. The bulk of the tutorial is two sections, fundamentals and selected techniques. Fundamentals are the basic and most-often used debugger commands. Selected techniques are those other commands and investigative approaches that will be useful in many situations. This latter section is not, however, an exploration of substantive techniques that might be used to investigate things like deadlocks, memory corruption or resource leaks. On first reading of the tutorial, you may want to skip the selected techniques. The tutorial concludes by pointing to the Microsoft debugger discussion group and the debugger feedback e-mail address for further questions.

Getting set up

Get the latest!

Obtain the latest debugger, and update it regularly. The value of having the latest release can hardly be overstated, because the debugger enjoys frequent improvements and fixes. The debugger can be downloaded at http://www.microsoft.com/whdc/devtools/debugging/default.mspx.

Host and target connection

The debugging setup will be a two computer arrangement, connected by a null-modem cable or by a 1394 cable. This tutorial does not examine “local” debugging on a single system (where the debugger examines the computer it is running on). Three computer debugging (target, debugging server and debugging client) will be briefly discussed.

A debugging session is a cooperative process between the host-side debugging application (WinDbg or KD) and the target operating system; each party must do something. More specifically, WinDbg is not a “hypervisor operating system” that runs the target as a guest and is a real operating system in its own right. WinDbg is a debugging application in partnership with a target operating system that is aware of its role in the debugging process. In that partnership, the target sends information to and receives information from WinDbg. The communication mechanism must be good, and it must be efficient.

A serial protocol is the tried-and-true mechanism for communication between the debugger application and the target system. You connect the host and target machines with a null-modem cable at a COM port at each end. An alternative communication mechanism is 1394. The Debugging Tools for Windows help file describes each of these and how to configure the target system in the topic “Configuring Software on the Target Computer.”

Your first session

Your host computer is assumed to be running Windows 2000 or later. The host operating system can be a different version of Windows than the target operating system. The host computer may be where you do your usual development, maintenance or troubleshooting. It should be connected to the network if you want access to symbol and source servers (see symbols and source).

From a Command Prompt window, change the current directory to the installation directory of Debugging Tools for Windows. This is the directory in which windbg.exe and kd.exe are located. Type windbg and press Enter. You should see this:

Windowing

At this point you can arrange your windows. The following example involves floating windows. Start with the combined window that is the first above. Click on the bar labeled “Command” and drag that bar and its window away from the main frame. Then shrink the main frame, since you can use keystrokes instead of directly using the menu or the buttons.

Then use FileàKernel Debug to get the protocol popup, and choose 1394 with channel 1. At this point, your desktop looks like this:


Then click OK in the Kernel Debugging window.

Making the connection active

Now you are ready to make a connection between host and target. Go to the target machine and boot Windows from one of the debugging entries. Immediately go back to the host system, touch the WinDbg command window with the cursor to make it active, and press CTRL+BREAK. In a few seconds you should see this:


Don't worry for now about the messages concerning symbols. You have a WinDbg connection to a Windows Server 2003 system. You're in business!

A small but crucial thing to understand: The area at the very bottom of the command window shows the “kd>” prompt. That means that WinDbg is ready to accept commands there. If no prompt is displayed, WinDbg cannot process commands at this moment, although any commands you type will be stored in a buffer and executed as soon as possible. You must wait for “kd>” to appear to be sure WinDbg is ready to respond. It is a fact of WinDbg’s life that sometimes it is busy doing something you cannot see (such as getting information from the target, and the information can be voluminous). The absence of “kd>” is your only clue that WinDbg is busy. One possibility is that WinDbg is working to resolve a symbol and taking longer than you might expect. Unfortunately, there is also the occasional situation where WinDbg is waiting for something such as connection with a target that will never respond (maybe boot.ini was configured badly, or the wrong option was chosen). You will have to decide when enough time has passed to resort to drastic measures like pressing CTRL+BREAK or, conceivably, stopping WinDbg and starting a new instance.

Finding symbols and source

By now you are probably eager to start debugging, but there are a few more things you should do, since they will improve your debugging experience enormously.

The first thing is to ensure that WinDbg can find the symbols for a module of interest. Symbols indicate to what statement a binary instruction corresponds and what are the variables in force. Symbols map, in other words. You have available the symbols and the source files for your own modules, if they are in the same place as they were at build time. But what if you need to step through some other code that may have been built long before now? Or, for that matter, what if your code isn’t in the same location where it was built?

To explicitly set a location for symbols, use the .sympath command. Break (CTRL-BREAK) in the command window and type

.sympath SRV*<DownstreamStore>*http://msdl.microsoft.com/download/symbols

to tell WinDbg to look for symbols on the Microsoft public symbols server. To get WinDbg to use that server and to keep a copy of downloaded symbols in a local store, for example, in D:\DebugSymbols, you would do:

.sympath SRV*d:\DebugSymbols*http://msdl.microsoft.com/download/symbols

Occasionally you may have trouble in getting symbols from the symbols server. In such a case, begin with the !sym noisy command to get more information about what WinDbg is trying to do to obtain symbols. Next, use !lmi to see what WinDbg knows about the one essential Windows module, ntoskrnl. Then try to get symbols for ntoskrnl, using .reload /f. Thus:

kd> !sym noisy

noisy mode - symbol prompts on

 

kd> !lmi nt

Loaded Module Info: [nt]

         Module: ntoskrnl

   Base Address: 80a02000

     Image Name: ntoskrnl.exe

   Machine Type: 332 (I386)

     Time Stamp: 3e80048b Mon Mar 24 23:26:03 2003

           Size: 4d8000

       CheckSum: 3f6f03

Characteristics: 10e 

Debug Data Dirs: Type  Size     VA  Pointer

             CODEVIEW    25,  ee00,    e600 RSDS - GUID: (0xec9b7590, 0xd1bb, 0x47a6, 0xa6, 0xd5, 0x38, 0x35, 0x38, 0xc2, 0xb3, 0x1a)

               Age: 1, Pdb: ntoskrnl.pdb

     Image Type: MEMORY   - Image read successfully from loaded memory.

    Symbol Type: EXPORT   - PDB not found

    Load Report: export symbols

The syntax of the commands used here is described in the Debugging Tools for Windows help file.

Exported symbols are usually pretty meager. Debugging Tools for Windows includes a symbol server that can connect to a public symbol store on Microsoft's internet site.  Add this to your symbol path, and then load symbols:

kd> .sympath SRV*d:\DebugSymbols*http://msdl.microsoft.com/download/symbols

Symbol search path is: SRV*d:\ DebugSymbols *http://msdl.microsoft.com/download/symbols

kd> .reload /f nt

SYMSRV:  \\symbols\symbols\ntoskrnl.pdb\EC9B7590D1BB47A6A6D5383538C2B31A1\file.ptr

SYMSRV:  ntoskrnl.pdb from \\symbols\symbols: 9620480 bytes copied        

DBGHELP: nt - public symbols

         d:\DebugSymbols\ntoskrnl.pdb\EC9B7590D1BB47A6A6D5383538C2B31A1\ntoskrnl.pdb

kd> !lmi nt

Loaded Module Info: [nt]

         Module: ntoskrnl

   Base Address: 80a02000

     Image Name: ntoskrnl.exe

   Machine Type: 332 (I386)

     Time Stamp: 3e80048b Mon Mar 24 23:26:03 2003

           Size: 4d8000

       CheckSum: 3f6f03

Characteristics: 10e 

Debug Data Dirs: Type  Size     VA  Pointer

             CODEVIEW    25,  ee00,    e600 RSDS - GUID: (0xec9b7590, 0xd1bb, 0x47a6, 0xa6, 0xd5, 0x38, 0x35, 0x38, 0xc2, 0xb3, 0x1a)

               Age: 1, Pdb: ntoskrnl.pdb

     Image Type: MEMORY   - Image read successfully from loaded memory.

    Symbol Type: PDB      - Symbols loaded successfully from symbol server.

                 d:\DebugSymbols\ntoskrnl.pdb\EC9B7590D1BB47A6A6D5383538C2B31A1\ntoskrnl.pdb

       Compiler: C - front end [13.10 bld 2179] - back end [13.10 bld 2190]

    Load Report: public symbols

                 d:\DebugSymbols\ntoskrnl.pdb\EC9B7590D1BB47A6A6D5383538C2B31A1\ntoskrnl.pdb

While symbols will give you some information, they do not provide source code. In the simplest case, source files will be found in the same place they were at build time (the location will be in the binary and symbol files). But in many cases, they cannot be found there (they may have been moved), and you must specify where to look. For that you need a source path, for example,

.srcpath e:\Win2003SP1

That means: For source files, look in the e:\Win2003SP1 directory.

Another solution is to name a source server, if you have one:

.srcpath \\MySrcServer

If you experience trouble in getting source files, you should do .srcnoisy 1 to get more information about what the debugger is doing to find them.

Workspaces

You’ve not begun actual debugging, yet you have done a good deal of typing already. A lot of the settings can be kept in a workspace. So you might use FileàSave to save the settings in a workspace, perhaps one you name kernel1394Win2003. After that, you could start WinDbg with that workspace:

windbg -W kernel1394Win2003 -k 1394:channel=1

where –W specifies a workspace and –k gives the communication protocol (refer to the Debugging Tools for Windows help file under “WinDbg Command-Line Options”). Note: You should be careful to preserve lower and upper case in the command-line options you give to WinDbg or KD.

To make things easier, you can put a shortcut on your desktop to start WinDbg with the desired workspace, for example, with a 1394 connection:


The contents of the file above are:

cd /d "d:\Program Files\Debugging Tools for Windows"

start windbg.exe -y SRV*d:\DebugSymbols*http://msdl.microsoft.com/download/symbols -W kernel1394Win2003

The first line makes the Debugging Tools for Windows installation directory the current directory, to ensure that debugger modules will be found. The second line starts WinDbg, specifying a symbol path (-y) and workspace (-W).

A sample driver

It will be helpful to exercise WinDbg with the sample driver IoCtl. This can be found in the Windows Device Driver Kit (DDK) and its successor, the Windows Driver Kit (WDK).  Install this kit and look in the src\general\Ioctl subdirectory. The advantages of IoCtl are that it is simple and that it is a “legacy” driver, that is, one loaded by the Service Control Manager (SCM) and not a part of Plug-and-Play (the in’s and out’s of PnP are not of interest here). You should build both the user-space executable (ioctlapp.exe) and the kernel-space driver (sioctl.sys), since the former will load the latter.

Here is something fairly important to understand. The build process is quite smart about optimizing code, and optimization can result in code movement (logic is of course preserved) and in keeping variable values solely in registers. To ensure a straightforward debugging experience, you should produce a checked build with this compiler directive given in the build window or in the appropriate sources file:

MSC_OPTIMIZATION=/Od

(That is “Oh d” and not “zero d.”)

Sometimes the above will result in a build problem with intrinsic functions like memcmp. If you run into that problem, try:

MSC_OPTIMIZATION=/Odi

Please understand that preventing optimization is not a good choice for a production build. With the above directive, you would not be creating and would not be testing a production-type build. It is nonetheless a good practice to start testing with non-optimized builds and, once you are familiar with the code and have eliminated the simpler errors, to advance to production builds. When you have to deal with optimized code, you will find some assistance in dealing with optimized code.

Starting to debug the sample driver

Set a breakpoint in IoCtl at DriverEntry. Before starting the driver, break into the WinDbg command window and type this:

bu sioctl!DriverEntry

The bu (“Breakpoint Unresolved”) command defers the actual setting of the breakpoint until the module is loaded; at that point, WinDbg will look in it for “DriverEntry.” Since there is nothing more to do now, hit F5 (or you could type g, “Go”).

Next, copy ioctlapp.exe and sioctl.sys to a place on the target system such as C:\Temp\IOCTL, log in with Administrator privileges on that system, and make C:\Temp\IOCTL the current directory in a command window there. (You need not put this path into a symbol path or source path in WinDbg.) In that same command window, type ioctlapp and press Enter; in WinDbg you will see:

In the above, after execution stopped at the breakpoint, the !lmi command showed that WinDbg did pick up the symbols from a DDK build. The timestamp is about what you would expect, and the location of the symbol file is exactly that expected.

Depending on your docking arrangement, it may not be obvious, since windows can be hidden by other windows, but you will have a source code window somewhere (the key sequence ‘alt-Keypad *’ without single quotation marks ― will bring that window to the front):


The line marked in pink (the Debugging Tools for Windows help file calls it purple) is where the breakpoint was set and is where execution stopped. Have execution proceed down to IoCreateDevice (controlling execution says how this might be accomplished):


Here you see again the original breakpoint (highlighted in red, since now control is stopped there), and you see the current statement marked in dark blue.

Fundamentals

That was a “test drive” in a debugging session. Here are the debugger’s basic operations.

Commands, extensions, etc.

Commands come in several families: Plain (unadorned), ones beginning with a period (“.”) and ones beginning with an exclamation point (“!”). The Debugging Tools for Windows help file describes these families as commands, meta-commands and extension commands, respectively. For present purposes, the families are pretty much the same.

Breakpoints

Causing an interruption in execution is one of the basic capabilities of the debugger. Here are some ways that can be done.

·         Breakin at operating system startup.

To get a break at the earliest point in operating system startup, be sure WinDbg is properly connected, and do the key sequence CTRL-ALT-K repeatedly until you see:


At the next boot, the operating system will halt very shortly after ntoskrnl has started but before any drivers have been loaded, and WinDbg will get control. This is when you may wish to define breakpoints for drivers to be started at boot time.

·         Ordinary breakpoints

The simplest breakpoints are those set via the bp (“Breakpoint”) command. For example:

bp MyDriver!xyz

bp f89adeaa

In the first, the breakpoint is at a name in a module (<module>!<name>); in the second, it is at the given address. When execution reaches one of those points, the operating system halts and gives control to WinDbg. (You will see how to get the address for the second command in finding a name.)

Note: The syntax of the first command assumes that the operating system has loaded the module and that enough information is available through a symbol file or through external names to identify xyz. If xyz cannot be found in the module, the debugger will say so.

·         Deferred breakpoints.

Speaking of drivers that haven’t loaded, your very first breakpoint, set by bu (see starting to debug the sample driver above), was a “deferred” breakpoint. The bu command takes as a parameter the module and a name in it, for example:

bu sioctl!SioctlDeviceControl

where SioctlDeviceControl is an entry point or other name in the module sioctl.sys. This form assumes that when the module is loaded, enough information will be available to identify SioctlDeviceControl so that the breakpoint can be set. (If the module is already loaded and the name is found, the breakpoint is set immediately.) If the operating system cannot find SioctlDeviceControl, the debugger will say so, and there will be no halt at SioctlDeviceControl.

A useful characteristic of deferred breakpoints is that they operate on modules!names, whilst ordinary breakpoints operate on addresses or on modules!names immediately resolved into addresses. Another feature of deferred breakpoints is that they are remembered across boots (that wouldn’t make sense for address-expressed breakpoints). Yet another characteristic is that a deferred breakpoint is remembered if the associated module is unloaded, whilst an ordinary breakpoint is removed when the module is unloaded.

·         Another way to set an ordinary breakpoint is via a source window. Return to sioctl.sys. When you broke at DriverEntry, you could have scrolled the window down to some place where you wished to stop, placed the cursor on the desired line and hit F9:


The line in red is the breakpoint set via F9.

·         To see all the breakpoints you’ve defined, use the bl (“Breakpoint List”) command:

kd> bl

 0 e [d:\winddk\3790\src\general\ioctl\sys\sioctl.c @ 123]    0001 (0001) SIoctl!DriverEntry

 1 e [d:\winddk\3790\src\general\ioctl\sys\sioctl.c @ 338]    0001 (0001) Sioctl!SioctlDeviceControl+0x103

Notice a couple of things: Each breakpoint has a number, and breakpoint status is shown, “e” for “enabled” or “d” for “disabled.”

·         Now suppose you wish to take a breakpoint temporarily out of operation. bd (“Disable Breakpoint”) would do the trick. You have to specify just the breakpoint number:

kd> bd 1

kd> bl

 0 e [d:\winddk\3790\src\general\ioctl\sys\sioctl.c @ 123]    0001 (0001) SIoctl!DriverEntry

 1 d [d:\winddk\3790\src\general\ioctl\sys\sioctl.c @ 338]    0001 (0001) SIoctl!SioctlDeviceControl+0x103

·         In a similar way, to remove breakpoint number 1 permanently, use bc 1 (“Clear Breakpoint”). Now that breakpoint will be gone from the breakpoint list.

·         Well and good. Sometimes, however, a breakpoint is set at a very busy place in the operating system or a driver, and you may want to apply some conditions or conditional actions so that the breakpoint stops only under those circumstances. Here is the basic form:

bp SIoctl!SioctlDeviceControl+0x103 "j (@@(Irp)=0xffb5c4f8) ''; 'g'"

The meaning of this is:  Break only if the pointer Irp equals the address 0xFFB5C4F8; if that condition is not met, proceed.

To go deeper into the above, the breakpoint is not itself conditional. Rather, the breakpoint has an action clause (in double quotation marks); in that clause, the command j (“Execute IF/ELSE”) is a conditional action. j's function is to execute its TRUE|FALSE clauses (in single quotation marks). In the above, the TRUE clause (first) is empty, so that when the breakpoint fires and the TRUE condition is met, WinDbg would do nothing more than the default of halting. If the FALSE clause (second) is met, execution would proceed, by executing g. One action or the other would be done, depending on the condition in force.

 Consider next this slightly more elaborate form of the above:

bp SIoctl!SioctlDeviceControl+0x103 "j (@@(Irp)=0xffb5c4f8) '.echo Found the interesting IRP' ; '.echo Skipping an IRP of no interest; g' "

Here the TRUE clause puts out a message and stops. The FALSE clause puts out a message and continues (a message is helpful because WinDbg would evaluate the condition to FALSE and would otherwise go on silently).

Something to note: The following breakpoint, where register Eax is tested (you will find registers treated more thoroughly in registers), won’t work as you might expect:

bp SIoctl!SioctlDeviceControl+0x103 "j (@eax=0xffb5c4f8) '.echo Here!' ; '.echo Skipping; g' "

The reason is that evaluation may “sign-extend” the register’s value to 64 bits, i.e., to 0xFFFFFFFF`FFB5C4F8, and that would not match 0x00000000`FFB5C4F8. This will be a issue only if the highest bit of the 32-bit value is 1 and if some other conditions (for example, a 32-bit register) apply. “Sign Extension” in the Debugging Tools for Windows help file gives details (see also “Setting a Conditional Breakpoint” there).

A breakpoint may have conditions, with or without conditional actions. One condition is “one-shot” firing: The breakpoint is to fire only once (it is cleared after being hit). This is handy for a very high-traffic piece of code where you are interested in the first hit only.

bp /1 SIoctl!SioctlDeviceControl+0x103

Another useful condition is a test for a process or thread:

bp /p 0x81234000 SIoctl!SioctlDeviceControl+0x103

bp /t 0xff234000 SIoctl!SioctlDeviceControl+0x103

which mean, respectively, Stop at the indicated point only if the process block (EPROCESS) is at 0x81234000, and Stop at the indicated point only if the thread block (ETHREAD) is at 0xFF234000.

Conditions can be combined:

bp /1 /C 4 /p 0x81234000 SIoctl!SioctlDeviceControl+0x103

The meaning here is, Break once but only when the call stack depth is greater than four (here capitalization of “C” is significant, because “c” means “less than”) and the process block is at 0x81234000.

·         A different kind of breakpoint is one that is specified for access. For example,

ba w4 0xffb5c4f8+0x18+0x4

The address was taken from the IRP whose address you saw above, and at the offset 0x18+0x4 into the IRP is the IoStatus.Information member. So the breakpoint will fire when something attempts an update of the four bytes constituting IoStatus.Information in that IRP.  Such breakpoints are called data breakpoints (because they are triggered by data access) or processor breakpoints (because they are implemented by the processor, not the debugger itself).

Expressions:  MASM versus C++

You will probably agree that it is handy to use a variable in a driver program to provide a parameter value like process address. Doing that, however, demands that you understand something of debugger expressions.

The debugger has two ways of evaluating expressions, referred to “MASM” (Microsoft Macro Assembler) and “C++.” To quote the Debugging Tools for Windows help file under “MASM Expressions vs. C++ Expressions”:

In a MASM expression, the numerical value of any symbol is its memory address. In a C++ expression, the numerical value of a variable is its actual value, not its address.

Reading and re-reading that section will repay your time well.

An expression will be evaluated by MASM rules, C++ rules or a combination. In brief,

1.    The default expression type is MASM.

2.    The default type can be changed by .expr (see the Debugging Tools for Windows help file).

3.    Certain commands always use C++ evaluation.

4.    Evaluation of a specific expression (or part of an expression) can be changed to the form opposite to the usual expression type by prefixing the expression with “@@.”

Even this summary is fairly involved, and you should refer to “Evaluating Expressions” in the Debugging Tools for Windows help file for the details. For now, here are a few examples to give a sense of how evaluation works.

Earlier you were stopped at Sioctl!SioctlDeviceControl+0x103, so use dv to look at a variable known there (see the dv command for more information):

kd> dv Irp

            Irp = 0xff70fbc0

The response means, The variable Irp contains 0xFF70FBC0. Further, dv is interpreting its parameter in C++ terms, in that the response is based on the variable’s content and not on its address. You can confirm that so:

kd> ?? Irp

struct _IRP * 0xff70fbc0

 

since ?? always functions on the basis of C++ (see the ?? command). For the MASM type of evaluation, try ? (see the ? command):

kd> ? Irp

Evaluate expression: -141181880 = f795bc48

which means, The variable Irp is located at 0XF795BC48. You can confirm that the variable at that location truly does contain the value 0xFF70FBC0 by displaying memory via dd (see the dd command):

kd> dd f795bc48 l1

f795bc48  ff70fbc0

And the storage pointed to is:

kd> dd 0xff70fbc0

ff70fbc0  00940006 00000000 00000070 ff660c30

ff70fbd0  ff70fbd0 ff70fbd0 00000000 00000000

ff70fbe0  01010001 04000000 0006fdc0 00000000

ff70fbf0  00000000 00000000 00000000 04008f20

ff70fc00  00000000 00000000 00000000 00000000

ff70fc10  ff73f4d8 00000000 00000000 00000000

ff70fc20  ff70fc30 ffb05b90 00000000 00000000

ff70fc30  0005000e 00000064 0000003c 9c402408

This indeed looks like an IRP, as dt shows (see the dt command), since the Type and Size members have plausible values:

kd> dt Irp

Local var @ 0xf795bc48 Type _IRP*

0xff70fbc0

   +0x000 Type             : 6

   +0x002 Size             : 0x94

   +0x004 MdlAddress       : (null)

   +0x008 Flags            : 0x70

   +0x00c AssociatedIrp    : __unnamed

   +0x010 ThreadListEntry  : _LIST_ENTRY [ 0xff70fbd0 - 0xff70fbd0 ]

   +0x018 IoStatus         : _IO_STATUS_BLOCK

   +0x020 RequestorMode    : 1 ''

   +0x021 PendingReturned  : 0 ''

   +0x022 StackCount       : 1 ''

   +0x023 CurrentLocation  : 1 ''

   +0x024 Cancel           : 0 ''

   +0x025 CancelIrql       : 0 ''

   +0x026 ApcEnvironment   : 0 ''

   +0x027 AllocationFlags  : 0x4 ''

   +0x028 UserIosb         : 0x0006fdc0

   +0x02c UserEvent        : (null)

   +0x030 Overlay          : __unnamed

   +0x038 CancelRoutine    : (null)

   +0x03c UserBuffer       : 0x04008f20

         +0x040 Tail             : __unnamed

Sometimes you will want to employ C++ evaluation inside a MASM expression. The “@@” prefix achieves that. Since extensions always use parameters as MASM expressions, you can see the effect of @@ when it is employed with the extension!irp (see IRPs):

kd> !irp @@(Irp)

Irp is active with 1 stacks 1 is current (= 0xff70fc30)

 No Mdl System buffer = ff660c30 Thread ff73f4d8:  Irp stack trace. 

     cmd  flg cl Device   File     Completion-Context

>[  e, 0]   5  0 82361348 ffb05b90 00000000-00000000   

             \Driver\SIoctl

                  Args: 00000064 0000003c 9c402408 00000000

To repeat, without the @@ prefix to the variable Irp above, !irp would have used the address of the variable rather than the value of the variable. To make the point concrete, if the variable were located at 0xF795BC48 and contained the value 0xFF70FBC0, doing !irp Irp instead of !irp @@(Irp) would ask WinDbg to format the IRP stack for an IRP supposedly located at 0xF795BC48.

A further thing to understand: The @@ prefix is rather versatile, for its true meaning is, Use the evaluation method other than that currently being used in the surrounding expression. If the overall evaluation is MASM, @@ means C++, and if it is C++, @@ means MASM.

A final bit of advice: If you cannot get an expression to work as you expect, consider whether you’re asking the debugger to understand it in MASM or C++ terms.

Displaying and setting memory, variables, registers and so forth

There are quite a few ways to display and change things.

  • To display the variables in the current routine (the current “scope”), use dv (“Display Variables”). For example, if stopped at Sioctl!SioctlDeviceControl+0x103:

kd> dv

   DeviceObject = 0x82361348

            Irp = 0xff70fbc0

   outBufLength = 0x64

         buffer = 0x00000000 ""

          irpSp = 0xff70fc30

           data = 0xf886b0c0 "This String is from Device Driver !!!"

       ntStatus = 0

            mdl = 0x00000000

    inBufLength = 0x3c

        datalen = 0x26

         outBuf = 0x00000030 ""

          inBuf = 0xff660c30 "This String is from User Application; using METHOD_BUFFERED"

This is a list of parameter variables and local variables known at the breakpoint. “Known” is an important qualifier. For example, if a variable is optimized into a register, it will not be displayed, although one can dip into disassembly (View(Disassembly brings up the disassembly window) and examine registers.

If only a single variable is of interest, you can do:

kd> dv outBufLength

   outBufLength = 0x64

  • Another useful command is dt (“Display Type”). For example, continuing to use the breakpoint at Sioctl!SioctlDeviceControl+0x103:

kd> dt Irp

Local var @ 0xf795bc48 Type _IRP*

0xff70fbc0

   +0x000 Type             : 6

   +0x002 Size             : 0x94

   +0x004 MdlAddress       : (null)

   +0x008 Flags            : 0x70

   +0x00c AssociatedIrp    : __unnamed

   +0x010 ThreadListEntry  : _LIST_ENTRY [ 0xff70fbd0 - 0xff70fbd0 ]

   +0x018 IoStatus         : _IO_STATUS_BLOCK

   +0x020 RequestorMode    : 1 ''

   +0x021 PendingReturned  : 0 ''

   +0x022 StackCount       : 1 ''

   +0x023 CurrentLocation  : 1 ''

   +0x024 Cancel           : 0 ''

   +0x025 CancelIrql       : 0 ''

   +0x026 ApcEnvironment   : 0 ''

   +0x027 AllocationFlags  : 0x4 ''

   +0x028 UserIosb         : 0x0006fdc0

   +0x02c UserEvent        : (null)

   +0x030 Overlay          : __unnamed

   +0x038 CancelRoutine    : (null)

   +0x03c UserBuffer       : 0x04008f20

   +0x040 Tail             : __unnamed

The above says that the variable Irp is at 0xF795BC48 and that it contains 0xFF70FBC0; since dt knows the variable to be an IRP pointer (“Type _IRP*”), the area at 0xFF70FBC0 is formatted as an IRP.

To expand the structure one level:

kd> dt -r1 Irp

Local var @ 0xf795bc48 Type _IRP*

0xff70fbc0

   +0x000 Type             : 6

   +0x002 Size             : 0x94

   +0x004 MdlAddress       : (null)

   +0x008 Flags            : 0x70

   +0x00c AssociatedIrp    : __unnamed

      +0x000 MasterIrp        : 0xff660c30

      +0x000 IrpCount         : -10089424

      +0x000 SystemBuffer     : 0xff660c30

   +0x010 ThreadListEntry  : _LIST_ENTRY [ 0xff70fbd0 - 0xff70fbd0 ]

      +0x000 Flink            : 0xff70fbd0  [ 0xff70fbd0 - 0xff70fbd0 ]

      +0x004 Blink            : 0xff70fbd0  [ 0xff70fbd0 - 0xff70fbd0 ]

   +0x018 IoStatus         : _IO_STATUS_BLOCK

      +0x000 Status           : 0

      +0x000 Pointer          : (null)

      +0x004 Information      : 0

   +0x020 RequestorMode    : 1 ''

   +0x021 PendingReturned  : 0 ''

   +0x022 StackCount       : 1 ''

   +0x023 CurrentLocation  : 1 ''

   +0x024 Cancel           : 0 ''

   +0x025 CancelIrql       : 0 ''

   +0x026 ApcEnvironment   : 0 ''

   +0x027 AllocationFlags  : 0x4 ''

   +0x028 UserIosb         : 0x0006fdc0

      +0x000 Status           : 67142040

      +0x000 Pointer          : 0x04008198

      +0x004 Information      : 0x2a

   +0x02c UserEvent        : (null)

   +0x030 Overlay          : __unnamed

      +0x000 AsynchronousParameters : __unnamed

      +0x000 AllocationSize   : _LARGE_INTEGER 0x0

   +0x038 CancelRoutine    : (null)

   +0x03c UserBuffer       : 0x04008f20

   +0x040 Tail             : __unnamed

      +0x000 Overlay          : __unnamed

      +0x000 Apc              : _KAPC

      +0x000 CompletionKey    : (null)

It is possible to display some structures even when they aren’t in scope (assuming, that is, that the memory in question has not been reused for some other purpose):

kd> dt nt!_IRP 0xff70fbc0

   +0x000 Type             : 6

   +0x002 Size             : 0x94

   +0x004 MdlAddress       : (null)

   +0x008 Flags            : 0x70

   +0x00c AssociatedIrp    : __unnamed

   +0x010 ThreadListEntry  : _LIST_ENTRY [ 0xff70fbd0 - 0xff70fbd0 ]

   +0x018 IoStatus         : _IO_STATUS_BLOCK

   +0x020 RequestorMode    : 1 ''

   +0x021 PendingReturned  : 0 ''

   +0x022 StackCount       : 1 ''

   +0x023 CurrentLocation  : 1 ''

   +0x024 Cancel           : 0 ''

   +0x025 CancelIrql       : 0 ''

   +0x026 ApcEnvironment   : 0 ''

   +0x027 AllocationFlags  : 0x4 ''

   +0x028 UserIosb         : 0x0006fdc0

   +0x02c UserEvent        : (null)

   +0x030 Overlay          : __unnamed

   +0x038 CancelRoutine    : (null)

   +0x03c UserBuffer       : 0x04008f20

   +0x040 Tail             : __unnamed

The command above exploits your knowledge that there is an IRP at 0xFF70FBC0 and the fact that there is a mapping of the IRP structure in ntoskrnl.

  • What if you’re interested in a single field of a structure with many member fields? Take the member Size, for example:

kd> dt nt!_IRP Size 0xff70fbc0

unsigned short 0x94

A rather more intuitive way is the ?? (“Evaluate C++ Expression”) command:

kd> ?? Irp->Size

unsigned short 0x94

That is, ?? understands that its parameter is a pointer to a member of the appropriate structure.

  • To display memory without the formatting above, commands like dd, dw and db (“Display Memory”) are available:

kd> dd 0xff70fbc0 l0x10

ff70fbc0  00940006 00000000 00000070 ff660c30

ff70fbd0  ff70fbd0 ff70fbd0 00000000 00000000

ff70fbe0  01010001 04000000 0006fdc0 00000000

ff70fbf0  00000000 00000000 00000000 04008f20

 

kd> dw 0xff70fbc0 l0x20

ff70fbc0  0006 0094 0000 0000 0070 0000 0c30 ff66

ff70fbd0  fbd0 ff70 fbd0 ff70 0000 0000 0000 0000

ff70fbe0  0001 0101 0000 0400 fdc0 0006 0000 0000

ff70fbf0  0000 0000 0000 0000 0000 0000 8f20 0400

 

kd> db 0xff70fbc0 l0x40

ff70fbc0  06 00 94 00 00 00 00 00-70 00 00 00 30 0c 66 ff  ........p...0.f.

ff70fbd0  d0 fb 70 ff d0 fb 70 ff-00 00 00 00 00 00 00 00  ..p...p.........

ff70fbe0  01 00 01 01 00 00 00 04-c0 fd 06 00 00 00 00 00  ................

ff70fbf0  00 00 00 00 00 00 00 00-00 00 00 00 20 8f 00 04  ............ ...

(Note: The second parameter in each of the three commands above is a length, given by l (the letter “l”) immediately followed by a value such as 0x10.)

The first displays 16 doublewords (four bytes each, or 64 bytes total) as double words. The second displays the same as words. The third, the same as bytes.

  • What about changing a variable? Continuing at Sioctl!SioctlDeviceControl+0x103, you would see that the intuitive form

kd> outBufLength = 00

     ^ Syntax error in 'outBufLength = 00'

doesn't work. But ?? does the job:

kd> ?? outBufLength = 0

unsigned long 0

Now go back to the IRP you used with dt above:

kd> dt Irp

Local var @ 0xf795bc48 Type _IRP*

0xff70fbc0

   +0x000 Type             : 6

   +0x002 Size             : 0x94

   +0x004 MdlAddress       : (null)

   +0x008 Flags            : 0x70

   +0x00c AssociatedIrp    : __unnamed

   +0x010 ThreadListEntry  : _LIST_ENTRY [ 0xff70fbd0 - 0xff70fbd0 ]

   +0x018 IoStatus         : _IO_STATUS_BLOCK

   +0x020 RequestorMode    : 1 ''

   +0x021 PendingReturned  : 0 ''

   +0x022 StackCount       : 1 ''

   +0x023 CurrentLocation  : 1 ''

   +0x024 Cancel           : 0 ''

   +0x025 CancelIrql       : 0 ''

   +0x026 ApcEnvironment   : 0 ''

   +0x027 AllocationFlags  : 0x4 ''

   +0x028 UserIosb         : 0x0006fdc0

   +0x02c UserEvent        : (null)

   +0x030 Overlay          : __unnamed

   +0x038 CancelRoutine    : (null)

   +0x03c UserBuffer       : 0x04008f20

   +0x040 Tail             : __unnamed

To change the first word (two bytes, that is) via ew (“Enter Values):

kd> ew 0xff70fbc0 3

 

kd> dt Irp

Local var @ 0xf795bc48 Type _IRP*

0xff70fbc0

   +0x000 Type             : 3

   +0x002 Size             : 0x94

   +0x004 MdlAddress       : (null)

   +0x008 Flags            : 0x70

   +0x00c AssociatedIrp    : __unnamed

   +0x010 ThreadListEntry  : _LIST_ENTRY [ 0xff70fbd0 - 0xff70fbd0 ]

   +0x018 IoStatus         : _IO_STATUS_BLOCK

   +0x020 RequestorMode    : 1 ''

   +0x021 PendingReturned  : 0 ''

   +0x022 StackCount       : 1 ''

   +0x023 CurrentLocation  : 1 ''

   +0x024 Cancel           : 0 ''

   +0x025 CancelIrql       : 0 ''

   +0x026 ApcEnvironment   : 0 ''

   +0x027 AllocationFlags  : 0x4 ''

   +0x028 UserIosb         : 0x0006fdc0

   +0x02c UserEvent        : (null)

   +0x030 Overlay          : __unnamed

   +0x038 CancelRoutine    : (null)

   +0x03c UserBuffer       : 0x04008f20

   +0x040 Tail             : __unnamed

Of course, the following is probably more natural than ew:

kd> ?? irp->type = 3

Type does not have given member error at 'type = 3'

kd> ?? irp->Type = 3

short 3

 

kd> dt irp

ioctlapp!Irp

Local var @ 0xf795bc48 Type _IRP*

0xff70fbc0

   +0x000 Type             : 3

   +0x002 Size             : 0x94

   +0x004 MdlAddress       : (null)

   +0x008 Flags            : 0x70

   +0x00c AssociatedIrp    : __unnamed

   +0x010 ThreadListEntry  : _LIST_ENTRY [ 0xff70fbd0 - 0xff70fbd0 ]

   +0x018 IoStatus         : _IO_STATUS_BLOCK

   +0x020 RequestorMode    : 1 ''

   +0x021 PendingReturned  : 0 ''

   +0x022 StackCount       : 1 ''

   +0x023 CurrentLocation  : 1 ''

   +0x024 Cancel           : 0 ''

   +0x025 CancelIrql       : 0 ''

   +0x026 ApcEnvironment   : 0 ''

   +0x027 AllocationFlags  : 0x4 ''

   +0x028 UserIosb         : 0x0006fdc0

   +0x02c UserEvent        : (null)

   +0x030 Overlay          : __unnamed

   +0x038 CancelRoutine    : (null)

   +0x03c UserBuffer       : 0x04008f20

   +0x040 Tail             : __unnamed

There are a couple of things to notice in the above. First, the case of a member of a structure is significant, as shown by WinDbg’s claim that there was no “type” member of Irp. Second, dt irp was ambiguous, but WinDbg helpfully displayed the instances it thought were likely fits, one in ioctlapp.exe and one in sioctl.sys. Since case can be significant, you should employ it whenever you know it.

In addition to ew, there are other “Enter Values” commands: eb for a byte, ed for a doubleword, eq for quadword (8 bytes) and so forth. Refer to the Debugging Tools for Windows help file under “Enter Values.”

  • The Locals window may be easier to use for things like a structure with imbedded pointers to structures:

You can, not so incidentally, overtype values in the Locals window to change values.

  • Registers (as well as segment pointers and flags) can be displayed and altered. For example:

kd> r

eax=81478f68 ebx=00000000 ecx=814243a8 edx=0000003c esi=81778ea0 edi=81478f68

eip=f8803553 esp=f7813bb4 ebp=f7813c3c iopl=0         nv up ei ng nz ac pe nc

cs=0008  ss=0010  ds=0023  es=0023  fs=0030  gs=0000             efl=00000292

Or just:

kd> r eax

eax=81478f68

Sometimes you will want to alter a register. Eax, for instance, is often used at exit from a routine to carry the return value, so, just before exit from a routine:

r eax = 0xc0000001

which happens to denote the status value STATUS_UNSUCCESSFUL.

Here are some other examples:

r eip = poi(@esp)

r esp = @esp + 0xc

which mean, respectively, Set Eip (the instruction pointer) to the value found at offset 0x0 from the stack pointer, and Add 0xC to Esp (the stack pointer), effectively unwinding the stack pointer. The Debugging Tools for Windows help file, under “Register Syntax,” explains poi and why register names have to be prefixed with a single “@” in some places.

You may be asking yourself how the above register-setting commands could be useful. Consider the case where there is a “bad” driver whose DriverEntry will cause a bug check ("blue screen") — due to an access violation, perhaps. You could deal with the problem by setting a deferred breakpoint when ntoskrnl loads. The following command must be typed on a single line:

bu sioctl!DriverEntry "r eip = poi(@esp); r eax = 0xc0000001; r esp = @esp + 0xc; .echo sioctl!DriverEntry entered; g"

The meaning is: At sioctl.sys’s DriverEntry, 1) set the instruction pointer (Eip) thus; 2) set the return code (Eax) thus; 3) set the stack pointer (Esp) thus; 4) announce that DriverEntry has been entered; and 5) proceed. (Of course, this technique merely removes the possibility of DriverEntry causing a blowup such as an access violation. If the operating system expects the driver to be supplying function, that function is not going to be available, and down the road there may be other problems.)

In case you’re wondering whether it is possible to use a register to set a variable, it is. For example, returning yet again to the IoCtl dispatch routine:

kd> r

eax=00000000 ebx=00000000 ecx=81a88f18 edx=81a88ef4 esi=ff9e18a8 edi=ff981e7e

eip=f87a40fe esp=f88fac78 ebp=f88fac90 iopl=0         nv up ei pl zr na po nc

cs=0008  ss=0010  ds=0023  es=0023  fs=0030  gs=0000             efl=00000246

 

kd> ?? ntStatus = @ecx

long -2119659752

kd> dd &ntStatus l1

f88fac78  81a88f18

In this case, the form @ecx should be employed, to ensure WinDbg knows you’re referring to a register.

There are more registers than those shown by default. To see the full complement, use the rM command (“M” must be uppercase; this is actually the r command with the parameter M, with no space allowed between command and parameter):

kd> rM 0xff

eax=00000001 ebx=0050e2a3 ecx=80571780 edx=000003f8 esi=000000c0 edi=d87a75a8

eip=804df1c0 esp=8056f564 ebp=8056f574 iopl=0         nv up ei pl nz na pe nc

cs=0008  ss=0010  ds=0023  es=0023  fs=0030  gs=0000             efl=00000202

fpcw=0000: rn 24 ------  fpsw=0000: top=0 cc=0000 --------  fptw=0000

fopcode=6745  fpip=2301:a0020000  fpdp=dcfe:efcdab89

st0= 5.143591243081972142170e-4932  st1= 0.001025530551233493990e-4933

st2= 0.000000002357022271740e-4932  st3= 2.471625214254630491460e-4906

st4= 3.370207406893238285120e-4932  st5=-7.461339669368745455450e+4855

st6= 6.698191557136036873700e-4932  st7=-2.455410815115332972380e-4906

mm0=c3d2e1f010325476  mm1=0000ffdff1200000

mm2=000000018168d902  mm3=f33cffdff1200000

mm4=804efc868056f170  mm5=7430804efb880000

mm6=ff02740200000000  mm7=f1a48056f1020000

xmm0=0 9.11671e-041 3.10647e+035 -1.154e-034

xmm1=-7.98492e-039 -2.83455e+038 -2.91106e+038 5.85182e-042

xmm2=1.77965e-043 -1.17906e-010 -4.44585e-038 -7.98511e-039

xmm3=-7.98511e-039 0 0 -7.98504e-039

xmm4=-7.98503e-039 1.20545e-040 -1.47202e-037 -1.47202e-037

xmm5=-2.05476e+018 -452.247 -1.42468e-037 -8.60834e+033

xmm6=2.8026e-044 -1.47202e-037 -452.247 0

xmm7=8.40779e-045 -7.98503e-039 0 -7.98511e-039

cr0=8001003b cr2=d93db000 cr3=00039000

dr0=00000000 dr1=00000000 dr2=00000000

dr3=00000000 dr6=ffff0ff0 dr7=00000400 cr4=000006d9

  • If you don’t like to use commands to change things, you can bring up a memory window (ViewàMemory), variable window (ViewàLocals) or register window (ViewàRegisters) and overtype values as you like. For example,

In the above, you could overtype the hexadecimal values.

Controlling execution

Earlier (see IoCreateDevice) you were asked to let execution proceed from one point to the next, without being told how. There are several ways to control execution. All the below, except the first, assume that execution is currently halted.

  • Break in (CTRL-BREAK ) — This sequence will always interrupt a system, so long as that system is running and is in communication with WinDbg (the sequence in KD is CTRL-C).
  • Step over (F10) — This causes execution to proceed one statement (if C or C++ and WinDbg is in “source mode,” toggled by DebugàSource Mode) or one instruction at a time, with the provision that if a call is encountered, execution passes over the call without entering the called routine.
  • Step in (F11) — This is like step-over, except that execution of a call does go into the called routine.
  • Step out (SHIFT-F11) — This causes execution to run to an exit from the current routine (current place in the call stack). Useful if you’ve seen enough of the routine.
  • Run to cursor (F7 or CRTL-F10) — Place the cursor in a source or disassembly window where you want execution to break, then hit F7; execution will run to that point. A word of caution, however: If the flow of execution were not to reach that point (e.g., an IF statement isn’t executed), WinDbg would not break, because execution did not come to the indicated point!
  • Run (F5) — Run until a breakpoint is encountered or an event like a bug check occurs. You may think of Run as normal execution mode.
  • Set instruction to current line (CTRL-SHIFT-I) — In a source window, you can put the cursor on a line, do that keystroke sequence, and execution will start from that point as soon as you let it proceed (e.g., F5 or F10). This is handy if you want to retry a sequence. But it requires some care. For example, registers and variables are not set to what they would be if execution had reached that line naturally.
  • Direct setting of Eip — You can put a value into the Eip register, and as soon as you hit F5 (or F10 or whatever), execution commences from that address. As should be obvious, this is like setting instruction to the cursor-designated current line, except that you specify the address of an Assembly instruction.

The call stack

At almost any point in execution, there is an area of storage that is used as the stack; the stack is where local state, including parameters and return addresses, is saved. There is a kernel stack, if execution is in kernel space, and a user stack for execution in user space. When you hit a breakpoint, it is likely there will be several routines on the current stack, and there can be quite a few. For example, if instruction has stopped because of a breakpoint in the routine PrintIrpInfo in sioctl.sys, use k (“Stack Backtrace”):

kd> k

ChildEBP RetAddr 

f7428ba8 f889b54a SIoctl!PrintIrpInfo+0x6 [d:\winddk\3790.1824\src\general\ioctl\sys\sioctl.c @ 708]

f7428c3c 804e0e0d SIoctl!SioctlDeviceControl+0xfa [d:\winddk\3790.1824\src\general\ioctl\sys\sioctl.c @ 337]

WARNING: Stack unwind information not available. Following frames may be wrong.

f7428c60 80580e2a nt!IofCallDriver+0x33

f7428d00 805876c2 nt!CcFastCopyRead+0x3c3

f7428d34 804e7a8c nt!NtDeviceIoControlFile+0x28

f7428d64 00000000 nt!ZwYieldExecution+0xaa9

The topmost line (newest) is the stack frame where control stopped. You can see earlier callers, too, but if you don’t have symbols, they may not be represented correctly. Since you enjoy access to the symbols for sioctl.sys, you are presented with file and line number information for each frame involving the driver.

You can readily switch to the source window for IoCtl’s IRP handler, but suppose you want to see an earlier routine? You bring up the calls window (ViewàCall stack), so:


You can double-click on an entry and be taken to the source file, if that source file can be located.

If you are interested only in variables pertaining to a routine on the stack, you can make the routine current by double-clicking on its line above, or you can do kn (a sibling of k) and then .frame. For example, to get information about the dispatch routine that called PrintIrpInfo:

kd> kn

 # ChildEBP RetAddr 

00 f7428ba8 f889b54a SIoctl!PrintIrpInfo+0x6 [d:\winddk\3790.1824\src\general\ioctl\sys\sioctl.c @ 708]

01 f7428c3c 804e0e0d SIoctl!SioctlDeviceControl+0xfa [d:\winddk\3790.1824\src\general\ioctl\sys\sioctl.c @ 337]

WARNING: Stack unwind information not available. Following frames may be wrong.

02 f7428c60 80580e2a nt!IofCallDriver+0x33

03 f7428d00 805876c2 nt!CcFastCopyRead+0x3c3

04 f7428d34 804e7a8c nt!NtDeviceIoControlFile+0x28

05 f7428d64 00000000 nt!ZwYieldExecution+0xaa9

kd> .frame 1

01 f7428c3c 804e0e0d SIoctl!SioctlDeviceControl+0xfa [d:\winddk\3790.1824\src\general\ioctl\sys\sioctl.c @ 337]

Having set the frame number, you’re able to display (or change, if you wish) variables known in that frame and registers belonging to that frame:

kd> dv

   DeviceObject = 0x80f895e8

            Irp = 0x820572a8

   outBufLength = 0x64

         buffer = 0x00000000 ""

          irpSp = 0x82057318

           data = 0xf889b0c0 "This String is from Device Driver !!!"

       ntStatus = 0

            mdl = 0x00000000

    inBufLength = 0x3c

        datalen = 0x26

         outBuf = 0x82096b20 "This String is from User Application; using METHOD_BUFFERED"

          inBuf = 0x82096b20 "This String is from User Application; using METHOD_BUFFERED"

kd> r

eax=00000000 ebx=00000000 ecx=80506be8 edx=820572a8 esi=81fabda0 edi=820572a8

eip=f889bcf6 esp=f7428ba4 ebp=f7428ba8 iopl=0         nv up ei ng nz ac pe nc

cs=0008  ss=0010  ds=0023  es=0023  fs=0030  gs=0000             efl=00000292

SIoctl!PrintIrpInfo+0x6:

f889bcf6 8b4508           mov     eax,[ebp+0x8]     ss:0010:f7428bb0=820572a8

Finding a name in a module

The x (“Examine Symbols”) command locates symbols in modules. For example, if you want to put a breakpoint at the Ioctl routine to handle DeviceIoControl IRPs but don’t quite remember the routine’s name, you might do:

kd> x sioctl!*ioctl*

f8883080 SIoctl!SioctlUnloadDriver (struct _DRIVER_OBJECT *)

f8883010 SIoctl!SioctlCreateClose (struct _DEVICE_OBJECT *, struct _IRP *)

f8883450 SIoctl!SioctlDeviceControl (struct _DEVICE_OBJECT *, struct _IRP *)

This means, Tell me all the symbols in the module sioctl that contain “ioctl.”

That may seem trivial. Consider, however, this message seen in the debugger in an actual support case:

PopPolicyWorkerAction: action request 2 failed c000009a

On a guess that the routine PopPolicyWorkerAction is in ntoskrnl, you might look there:

kd> x nt!PopPolicy*

805146c0 nt!PopPolicyWorkerThread = <no type information>

8064e389 nt!PopPolicySystemIdle = <no type information>

805b328d nt!PopPolicyWorkerNotify = <no type information>

8056e620 nt!PopPolicyLock = <no type information>

8064d5f8 nt!PopPolicyWorkerActionPromote = <no type information>

805c7d10 nt!PopPolicyWorkerMain = <no type information>

8064d51b nt!PopPolicyWorkerAction = <no type information>

80561c70 nt!PopPolicy = <no type information>

8056e878 nt!PopPolicyIrpQueue = <no type information>

80561a98 nt!PopPolicyLockThread = <no type information>

8064e74a nt!PopPolicyTimeChange = <no type information>

8056e8b0 nt!PopPolicyWorker = <no type information>

With that information, you might put a breakpoint in the routine shown in red.

Dealing with optimized code

If an executable was built in a free build and/or with some optimization, it may be difficult to follow execution in a source window, and local variables may not be available or may be shown with incorrect values. With x86 code you might try following execution in the source window and also in the disassembly window (it helps to place those windows side by side). You don’t have to understand very much about x86 Assembly to follow the flow of control; look primarily for test (e.g., test or cmp ) and branch (e.g., jnz) instructions in order to follow flow.

Selected techniques

That covers fundamental operations. Although the overarching focus here is not on how to investigate particular areas, there are nonetheless a number of substantive debugger commands — technically, they are extensions and are written as DLLs — as well as techniques that deserve mention because they are used over and over in many areas.

Processes and threads

To see the current process (at a stopped point):

kd> !process

PROCESS 816fc3c0  SessionId: 1  Cid: 08f8    Peb: 7ffdf000  ParentCid: 0d8c

    DirBase: 10503000  ObjectTable: e1afeaa8  HandleCount:  19.

    Image: ioctlapp.exe

    VadRoot 825145e0 Vads 22 Clone 0 Private 38. Modified 0. Locked 0.

    DeviceMap e10d0198

    Token                             e1c8e030

    ElapsedTime                       00:00:00.518

    UserTime                          00:00:00.000

    KernelTime                        00:00:00.109

    QuotaPoolUsage[PagedPool]         9096

    QuotaPoolUsage[NonPagedPool]      992

    Working Set Sizes (now,min,max)  (263, 50, 345) (1052KB, 200KB, 1380KB)

    PeakWorkingSetSize                263

    VirtualSize                       6 Mb

    PeakVirtualSize                   6 Mb

    PageFaultCount                    259

    MemoryPriority                    BACKGROUND

    BasePriority                      8

    CommitCharge                      48

 

        THREAD 825d2020  Cid 08f8.0708  Teb: 7ffde000 Win32Thread: 00000000 RUNNING on processor 0

The addresses of the process block (EPROCESS) and thread block (ETHREAD) have been marked in red. You could use these in a conditional breakpoint.

To see all the processes in summary form:

kd> !process 0 0

**** NT ACTIVE PROCESS DUMP ****

PROCESS 826af478  SessionId: none  Cid: 0004    Peb: 00000000  ParentCid: 0000

    DirBase: 02c20000  ObjectTable: e1001e60  HandleCount: 363.

    Image: System

 

PROCESS 82407d88  SessionId: none  Cid: 0158    Peb: 7ffdf000  ParentCid: 0004

    DirBase: 1fbe8000  ObjectTable: e13ff740  HandleCount:  24.

    Image: smss.exe

 

PROCESS 82461d88  SessionId: 0  Cid: 0188    Peb: 7ffdf000  ParentCid: 0158

    DirBase: 1f14d000  ObjectTable: e15e8958  HandleCount: 408.

    Image: csrss.exe

 

...

To see a particular process with a thread summary, give the process block’s address and ask for detail via the second parameter (see the Debugging Tools for Windows help file for information about the detail parameter):

kd> !process 826af478 3

PROCESS 826af478  SessionId: none  Cid: 0004    Peb: 00000000  ParentCid: 0000

    DirBase: 02c20000  ObjectTable: e1001e60  HandleCount: 362.

    Image: System

    VadRoot 81a43840 Vads 4 Clone 0 Private 3. Modified 18884. Locked 0.

    DeviceMap e1002868

    Token                             e1002ae0

    ElapsedTime                       07:19:11.250

    UserTime                          00:00:00.000

    KernelTime                        00:00:11.328

    QuotaPoolUsage[PagedPool]         0

    QuotaPoolUsage[NonPagedPool]      0

    Working Set Sizes (now,min,max)  (54, 0, 345) (216KB, 0KB, 1380KB)

    PeakWorkingSetSize                497

    VirtualSize                       1 Mb

    PeakVirtualSize                   2 Mb

    PageFaultCount                    4179

    MemoryPriority                    BACKGROUND

    BasePriority                      8

    CommitCharge                      7

 

        THREAD 826af1f8  Cid 0004.0008  Teb: 00000000 Win32Thread: 00000000 WAIT: (WrFreePage) KernelMode Non-Alertable

            80580040  SynchronizationEvent

            80581140  NotificationTimer

 

        THREAD 826aea98  Cid 0004.0010  Teb: 00000000 Win32Thread: 00000000 WAIT: (WrQueue) KernelMode Non-Alertable

            80582d80  QueueObject

 

        THREAD 826ae818  Cid 0004.0014  Teb: 00000000 Win32Thread: 00000000 WAIT: (WrQueue) KernelMode Non-Alertable

            80582d80  QueueObject

 

...

To see a particular thread in maximum detail, use !thread with 0xFF as the detail parameter:

kd> !thread 826af1f8 0xff

THREAD 826af1f8  Cid 0004.0008  Teb: 00000000 Win32Thread: 00000000 WAIT: (WrFreePage) KernelMode Non-Alertable

    80580040  SynchronizationEvent

    80581140  NotificationTimer

Not impersonating

DeviceMap                 e1002868

Owning Process            826af478       Image:         System

Wait Start TickCount      1688197        Ticks: 153 (0:00:00:02.390)

Context Switch Count      9133            

UserTime                  00:00:00.0000

KernelTime                00:00:03.0406

Start Address nt!Phase1Initialization (0x806fb790)

Stack Init f88b3000 Current f88b2780 Base f88b3000 Limit f88b0000 Call 0

Priority 0 BasePriority 0 PriorityDecrement 0

ChildEBP RetAddr 

f88b2798 804edb2b nt!KiSwapContext+0x26 (FPO: [EBP 0xf88b27c0] [0,0,4])

f88b27c0 804f0e7a nt!KiSwapThread+0x280 (FPO: [Non-Fpo]) (CONV: fastcall)

f88b27f4 80502fc2 nt!KeWaitForMultipleObjects+0x324 (FPO: [Non-Fpo]) (CONV: stdcall)

Driver and device objects

If you are writing a driver, you will often be looking at device stacks. You might begin by finding devices belonging to a certain driver and then examine the stack a device is in. Suppose you are interested in the ScsiPort miniport driver aic78xx.sys. Start with !drvobj:

kd> !drvobj aic78xx

Driver object (82627250) is for:

 \Driver\aic78xx

Driver Extension List: (id , addr)

(f8386480 8267da38) 

Device Object list:

82666030  8267b030  8263c030  8267ca40

There are four device objects here. Use To look at the first, use !devobj to get some information about the device and !devstack to show the device-object stack to which the device object belongs:

kd> !devobj 82666030 

Device object (82666030) is for:

 aic78xx1Port2Path0Target1Lun0 \Driver\aic78xx DriverObject 82627250

Current Irp 00000000 RefCount 0 Type 00000007 Flags 00001050

Dacl e13bb39c DevExt 826660e8 DevObjExt 82666d10 Dope 8267a9d8 DevNode 8263cdc8

ExtensionFlags (0000000000) 

AttachedDevice (Upper) 826bb030 \Driver\Disk

Device queue is not busy.

kd> !devstack 82666030 

  !DevObj   !DrvObj            !DevExt   ObjectName

  826bbe00  \Driver\PartMgr    826bbeb8 

  826bb030  \Driver\Disk       826bb0e8  DR2

> 82666030  \Driver\aic78xx    826660e8  aic78xx1Port2Path0Target1Lun0

!DevNode 8263cdc8 :

  DeviceInst is "SCSI\Disk&Ven_QUANTUM&Prod_VIKING_II_4.5WLS&Rev_5520\5&375eb691&1&010"

  ServiceName is "disk"

IRPs

The most common method of communication amongst drivers is the I/O Request Packet, or IRP. To see an IRP’s I/O completion stack, as for example at Sioctl!SioctlDeviceControl+0x103:

kd> !irp @@(Irp)

Irp is active with 1 stacks 1 is current (= 0xff70fc30)

 No Mdl System buffer = ff660c30 Thread ff73f4d8:  Irp stack trace. 

     cmd  flg cl Device   File     Completion-Context

>[  e, 0]   5  0 82361348 ffb05b90 00000000-00000000   

             \Driver\SIoctl

                  Args: 00000064 0000003c 9c402408 00000000

To get the full IRP plus its stack, ask for detail:

kd> !irp @@(Irp) 1

Irp is active with 1 stacks 1 is current (= 0xff70fc30)

 No Mdl System buffer = ff660c30 Thread ff73f4d8:  Irp stack trace. 

Flags = 00000070

ThreadListEntry.Flink = ff70fbd0

ThreadListEntry.Blink = ff70fbd0

IoStatus.Status = 00000000

IoStatus.Information = 00000000

RequestorMode = 00000001

Cancel = 00

CancelIrql = 0

ApcEnvironment = 00

UserIosb = 0006fdc0

UserEvent = 00000000

Overlay.AsynchronousParameters.UserApcRoutine = 00000000

Overlay.AsynchronousParameters.UserApcContext = 00000000

Overlay.AllocationSize = 00000000 - 00000000

CancelRoutine = 00000000

UserBuffer = 04008f20

&Tail.Overlay.DeviceQueueEntry = ff70fc00

Tail.Overlay.Thread = ff73f4d8

Tail.Overlay.AuxiliaryBuffer = 00000000

Tail.Overlay.ListEntry.Flink = 00000000

Tail.Overlay.ListEntry.Blink = 00000000

Tail.Overlay.CurrentStackLocation = ff70fc30

Tail.Overlay.OriginalFileObject = ffb05b90

Tail.Apc = 00000000

Tail.CompletionKey = 00000000

     cmd  flg cl Device   File     Completion-Context

>[  e, 0]   5  0 82361348 ffb05b90 00000000-00000000   

             \Driver\SIoctl

                  Args: 00000064 0000003c 9c402408 00000000

To get only the IRP proper, with its first-level members:

kd> dt nt!_IRP @@(Irp)

   +0x000 Type             : 6

   +0x002 Size             : 0x94

   +0x004 MdlAddress       : (null)

   +0x008 Flags            : 0x70

   +0x00c AssociatedIrp    : __unnamed

   +0x010 ThreadListEntry  : _LIST_ENTRY [ 0xff70fbd0 - 0xff70fbd0 ]

   +0x018 IoStatus         : _IO_STATUS_BLOCK

   +0x020 RequestorMode    : 1 ''

   +0x021 PendingReturned  : 0 ''

   +0x022 StackCount       : 1 ''

   +0x023 CurrentLocation  : 1 ''

   +0x024 Cancel           : 0 ''

   +0x025 CancelIrql       : 0 ''

   +0x026 ApcEnvironment   : 0 ''

   +0x027 AllocationFlags  : 0x4 ''

   +0x028 UserIosb         : 0x0006fdc0

   +0x02c UserEvent        : (null)

   +0x030 Overlay          : __unnamed

   +0x038 CancelRoutine    : (null)

   +0x03c UserBuffer       : 0x04008f20

   +0x040 Tail             : __unnamed

IRQL

A command (available for targets that are Windows Server 2003 or later) for occasional use is !irql, because it shows the current IRQL on the concerned processor. Breaking at Sioctl!SioctlDeviceControl+0x0:

kd> !irql

Debugger saved IRQL for processor 0x0 -- 0 (LOW_LEVEL)

For an example of a higher IRQL, suppose you added the following code (in curly braces) to Sioctl!SioctlDeviceControl just before the break statement at the end of the IOCTL_SIOCTL_METHOD_BUFFERED clause:

Irp->IoStatus.Information = (outBufLength<datalen?outBufLength:datalen);

{ /* Begin added code */                                          

 KIRQL saveIrql;                           

 ULONG i = 0;                               

                                           

 KeRaiseIrql(DISPATCH_LEVEL, &saveIrql);   

 i++;                                      

 KeLowerIrql(saveIrql);                    

} /* End added code */

break;                                           

Now, if you set a breakpoint at the statement after KeRaiseIrql and hit that breakpoint:

kd> !irql

Debugger saved IRQL for processor 0x0 -- 2 (DISPATCH_LEVEL)

Note, by the way, that the !pcr command will not ordinarily show the IRQL you’re interested in, namely, the IRQL in force at the point a breakpoint caused a halt.

Dump Files

There is little to say about dump files that is peculiar to them. A few things are worth mentioning.

  • There are three kinds of kernel dump files. A full-memory dump is best, but the somewhat less voluminous kernel dump suffices for most purposes. There is also the small-memory dump, which is 64KB in size (and so will be generated more quickly than the two other types). Since a small-memory dump does not have full information about executables, it may be necessary to employ the .exepath command to point to executable images if investigating them is required.  Windows can be configured to create one of these dump files if a crash (bug check) occurs.
  • To investigate a dump file, start WinDbg but do not specify a protocol for a target system. When in WinDbg, open the dump file with FileàOpen Crash Dump. It will help if the symbol path and possibly the source path are already set.
  • Now, in the WinDbg command window, do !analyze –v, to get a summary. The command will probably suggest an execution context (.cxr); setting that context will give you access to the call stack at the time the bug check was issued (which, hopefully, will be close to the actual error). You may need to proceed to processes and threads (!process and !thread), to look at the list of kernel modules (lmnt), from that list to look at selected driver objects (!drvobj) and possibly to look at device nodes (!devnode), device objects (!devobj) and device stacks (!devstack). But beyond !analyze –v, there are no simple prescriptions in working with a dump file.

If a kernel-mode dump file has been created after a bug check has occurred, debugging this file is similar to debugging a bug check that occurs when a debugger is attached.  The following section shows an example of live debugging, but it is similar to the analysis of a dump file.

Debugging a Bug Check

Here is how to begin analyzing a bug check.  In this example, the kernel debugger is attached when the crash occurs, but the procedure when analyzing a kernel-mode dump file is similar.

In this example, the sample driver Sioctl.sys is loaded, and a breakpoint is set at Sioctl!DriverEntry.  When the debugger stops at this breakpoint, set Eip to 0.  This is never a valid value, since the instruction pointer cannot be zero.  Then let execution proceed via F5.  A kernel error will occur, and a bug check will be issued.  You can then use the !analyze extension command to investigate:

kd> !analyze -v

*******************************************************************************

*                                                                             *

*                        Bugcheck Analysis                                    *

*                                                                             *

*******************************************************************************

 

SYSTEM_THREAD_EXCEPTION_NOT_HANDLED (7e)

This is a very common bugcheck.  Usually the exception address pinpoints

the driver/function that caused the problem.  Always note this address

as well as the link date of the driver/image that contains this address.

Arguments:

Arg1: c0000005, The exception code that was not handled

Arg2: 00000000, The address that the exception occurred at

Arg3: f88f2bd8, Exception Record Address

Arg4: f88f2828, Context Record Address

 

Debugging Details:

------------------

 

EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at "0x%08lx" referenced memory at "0x%08lx". The memory could not be "%s".

 

FAULTING_IP:

+0

00000000 ??               ???

 

EXCEPTION_RECORD:  f88f2bd8 -- (.exr fffffffff88f2bd8)

ExceptionAddress: 00000000

   ExceptionCode: c0000005 (Access violation)

  ExceptionFlags: 00000000

NumberParameters: 2

   Parameter[0]: 00000000

   Parameter[1]: 00000000

Attempt to read from address 00000000

 

CONTEXT:  f88f2828 -- (.cxr fffffffff88f2828)

eax=ffff99ea ebx=00000000 ecx=0000bb40 edx=8055f7a4 esi=e190049e edi=81e826e8

eip=00000000 esp=f88f2ca0 ebp=f88f2cf0 iopl=0         nv up ei pl nz na pe nc

cs=0008  ss=0010  ds=0023  es=0023  fs=0030  gs=0000             efl=00010202

00000000 ??               ???

Resetting default scope

 

DEFAULT_BUCKET_ID:  DRIVER_FAULT

 

CURRENT_IRQL:  0

 

ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at "0x%08lx" referenced memory at "0x%08lx". The memory could not be "%s".

 

READ_ADDRESS:  00000000

 

BUGCHECK_STR:  0x7E

 

LAST_CONTROL_TRANSFER:  from 805b9cbb to 00000000

 

STACK_TEXT: 

WARNING: Frame IP not in any known module. Following frames may be wrong.

f88f2c9c 805b9cbb 81e826e8 8123a000 00000000 0x0

f88f2d58 805b9ee5 80000234 8123a000 81e826e8 nt!IopLoadDriver+0x5e1

f88f2d80 804ec5c8 80000234 00000000 822aeda0 nt!IopLoadUnloadDriver+0x43

f88f2dac 805f1828 f7718cf4 00000000 00000000 nt!ExpWorkerThread+0xe9

f88f2ddc 8050058e 804ec50d 00000001 00000000 nt!PspSystemThreadStartup+0x2e

00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

 

 

FAILED_INSTRUCTION_ADDRESS:

+0

00000000 ??               ???

 

FOLLOWUP_IP:

nt!IopLoadDriver+5e1

805b9cbb 3bc3             cmp     eax,ebx

 

SYMBOL_STACK_INDEX:  1

 

SYMBOL_NAME:  nt!IopLoadDriver+5e1

 

MODULE_NAME:  nt

 

IMAGE_NAME:  ntoskrnl.exe

 

DEBUG_FLR_IMAGE_TIMESTAMP:  3e800a79

 

STACK_COMMAND:  .cxr fffffffff88f2828 ; kb

 

FAILURE_BUCKET_ID:  0x7E_NULL_IP_nt!IopLoadDriver+5e1

 

BUCKET_ID:  0x7E_NULL_IP_nt!IopLoadDriver+5e1

kd> .cxr fffffffff88f2828

eax=ffff99ea ebx=00000000 ecx=0000bb40 edx=8055f7a4 esi=e190049e edi=81e826e8

eip=00000000 esp=f88f2ca0 ebp=f88f2cf0 iopl=0         nv up ei pl nz na pe nc

cs=0008  ss=0010  ds=0023  es=0023  fs=0030  gs=0000             efl=00010202

00000000 ??               ???

kd> kb

  *** Stack trace for last set context - .thread/.cxr resets it

ChildEBP RetAddr  Args to Child             

WARNING: Frame IP not in any known module. Following frames may be wrong.

f88f2c9c 805b9cbb 81e826e8 8123a000 00000000 0x0

f88f2d58 805b9ee5 80000234 8123a000 81e826e8 nt!IopLoadDriver+0x5e1

f88f2d80 804ec5c8 80000234 00000000 822aeda0 nt!IopLoadUnloadDriver+0x43

f88f2dac 805f1828 f7718cf4 00000000 00000000 nt!ExpWorkerThread+0xe9

f88f2ddc 8050058e 804ec50d 00000001 00000000 nt!PspSystemThreadStartup+0x2e

00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

The topmost entry in the above stack looks wrong. That is something you might encounter in a dump file. How would you work with that if you didn’t know how the bug check had been produced?

1.    Do .frame 1, to get to the frame for the caller of the mystery routine, nt!IopLoadDriver.

2.    Go to the disassembly window, where the call by nt!IopLoadDriver is conveniently displayed, in instructions:

8062da9e ff572c           call    dword ptr [edi+0x2c]

8062daa1 3bc3             cmp     eax,ebx

3.    The call was to the address in a doubleword pointed to by the Edi register’s value plus 0x2C. That is the address you need. So, display the Edi register:

kd> r edi

Last set context:

edi=81a2bb18

4.    A little arithmetic:

kd> ? 81a2bb18+0x2c

Evaluate expression: -2120041660 = 81a2bb44

5.    So the address is in storage at 0x81A2BB44:

kd> dd 81a2bb44 l1

81a2bb44  f87941a3

6.    What’s at that address?

kd> dt f87941a3

GsDriverEntry

 SIoctl!GsDriverEntry+0(

                  _DRIVER_OBJECT*,

                  _UNICODE_STRING*)

Thus you would have determined the real routine at the top of the stack.

Pseudo-registers

Pseudo-registers are variables that you can employ for various purposes. A number of pseudo-registers are predefined in meaning: $ra for the return point in the current entry on the call stack, $ip for the instruction pointer, $scopeip for the address of the current scope (local context, which makes available local variables in the current routine), $proc that points to the current EPROCESS, and so forth. These might be useful in conditional statements.

There are also pseudo-registers with operator-defined meaning, $t0 through $t19. These can be turned to various purposes, like counting breakpoints. A pseudo-register was put to that use in a real-life case where there were many updates to a piece of storage:

ba w4 81b404d8-18 "r$t0=@$t0+1;as /x ${/v:$$t0} @$t0;.block {.echo hit # $$t0};ad ${/v:$$t0};dd 81b404d8-18 l1;k;!thread -1 0;!process -1 0"

The approximate meaning of the above is, When the doubleword at 0x81B404D8 is updated, update the pseudo-register $t0 as a hit counter, say what is the number of the hit and show the value at 0x81B404D8, the call stack, the current process and the current thread. (To understand details in the above, refer to aliasing below.)

Another illustrative use is from a support case where it was necessary to track the activity of Atapi.sys’s DPC routine (Atapi.sys is a standard operating system driver). This routine was entered extremely often, but the investigating engineer knew that at a specific point there would be an interesting IRP about to be completed, and the variable irp pointed to that IRP. In another routine in Tape.sys (another standard operating system driver), there was a variable named Irp that pointed to the same IRP. The engineer wanted to stop in Tape.sys at just the right time, so he started by setting a one-time breakpoint in the Atapi.sys DPC:

bp /1 Atapi!IdeProcessCompletedRequest+0x3bd "dv irp; r$t0=@@(irp)"

That breakpoint’s action was to set the pseudo-register $t0 to the value of the variable irp, which was the address of the IRP of interest. (It also displayed the value of irp.)

When that one-time breakpoint was hit, the engineer did this:

bp Tape!TapeIoCompleteAssociated+0x1c6 "j (@@(Irp)=$t0) '.echo stopped at TAPE!TapeIoCompleteAssociated+0x1c6; dv Irp' ; 'g'"

This means: When this second breakpoint in Tape.sys is hit, if the local variable Irp matches $t0, put out some eye-catching information and display the value of Irp. If, on the other hand, Irp does not match $t0, just go. When this second breakpoint caused execution to stop, control had halted where the engineer desired.

Aliasing

It may be convenient to replace a set of characters with another set in a command string. One use is to define a short string for a long set of commands. For example,

kd> as Demo r; !process -1 0; k; !thread -1 0

kd> al

  Alias            Value 

 -------          -------

 Demo             r; !process -1 0; k; !thread -1 0

kd> demo

Couldn't resolve error at 'emo'

kd> Demo

eax=00000001 ebx=001a6987 ecx=80571780 edx=ffd11118 esi=0000003e edi=f8bcc776

eip=804df1c0 esp=8056f564 ebp=8056f574 iopl=0         nv up ei pl nz na pe nc

cs=0008  ss=0010  ds=0023  es=0023  fs=0030  gs=0000             efl=00000202

nt!RtlpBreakWithStatusInstruction:

804df1c0 cc               int     3

PROCESS 80579f60  SessionId: none  Cid: 0000    Peb: 00000000  ParentCid: 0000

    DirBase: 00039000  ObjectTable: e1000e78  HandleCount: 234.

    Image: Idle

 

ChildEBP RetAddr 

8056f560 804e8682 nt!RtlpBreakWithStatusInstruction

8056f560 804e61ce nt!KeUpdateSystemTime+0x132

80579f60 00000000 nt!KiIdleLoop+0xe

THREAD 80579d00  Cid 0000.0000  Teb: 00000000 Win32Thread: 00000000 RUNNING on processor 0

Note that the alias and what replaces it are case-sensitive.

If you look back at pseudo-registers, you will now recognize that alias commands were used in the first example there. To begin, suppose that this was the actual command:

bp SioctlDeviceControl "r$t0=@$t0+1;as /x ${/v:$$t0} @$t0;.block {.echo hit # $$t0};ad ${/v:$$t0};k ;!thread -1 0;!process -1 0;g"

 

Here’s what would be put out in the WinDbg command window, with the effect of aliasing shown in red:

hit # 0x1

ChildEBP RetAddr 

f747dc20 80a2675c SIoctl!SioctlDeviceControl

f747dc3c 80c70bed nt!IofCallDriver+0x62

f747dc54 80c71b0d nt!IopSynchronousServiceTail+0x159

f747dcf4 80c673aa nt!IopXxxControlFile+0x665

f747dd28 80afbbf2 nt!NtDeviceIoControlFile+0x28

f747dd28 7ffe0304 nt!_KiSystemService+0x13f

0006fdc8 04003bcb SharedUserData!SystemCallStub+0x4

0006fde8 04002314 ioctlapp!_ftbuf+0x1b

0006ff78 04002e02 ioctlapp!main+0x1e4

0006ffc0 77e4f38c ioctlapp!mainCRTStartup+0x14d

WARNING: Frame IP not in any known module. Following frames may be wrong.

0006fff0 00000000 0x77e4f38c

THREAD feca2b88  Cid 0714.0e2c  Teb: 7ffde000 Win32Thread: 00000000 RUNNING on processor 0

PROCESS ff877b50  SessionId: 1  Cid: 0714    Peb: 7ffdf000  ParentCid: 0d04

    DirBase: 048f0000  ObjectTable: e2342440  HandleCount:  19.

    Image: ioctlapp.exe

...

hit # 0x2

...

hit # 0x3

...

The basic trick above is to define the command block associated with hitting the breakpoint in such a way that the imbedded alias definition isn’t evaluated immediately but only when a breakpoint is hit. That is achieved by using the ${} (“Alias Interpreter”) token with the /v option to specify an alias not evaluated at specification (in the bp command, that is) and the .block (“Block”) token to cause alias evaluation at the time of hitting the breakpoint and executing the associated commands. Finally, the /x option of the as command ensures 64-bit values are used, and the ad ensures the latest alias is cleaned up.

Script files and other ways to reduce work

It is possible to run a script file to execute a number of WinDbg commands. Consider as an example a dump file from an x64 system. Here the focus is on SCSI Request Blocks (SRBs) from a device driver called xStor.sys:

  1. Use !irpfind (see the Debugging Tools for Windows help file) to find IRPs in the non-paged pool. You will get lines like this:

fffffadfe5df1010 [fffffadfe5ee6760] irpStack: ( 4, 0)  fffffadfe78cc060 [ \Driver\dmio] 0xfffffadfe6919470

The value in red is the address of that specific IRP.

  1. Copy those lines into a file.
  2. In the file, select all lines containing xStor and put those lines into another file, debugtst1.txt. That yields lines like this one:

fffffadfe5e4c9d0 [00000000] irpStack: ( f, 0)  fffffadfe783d050 [ \Driver\xStor] 0xfffffadfe6919470

  1. Editing debugtst1.txt, change each line:

!irp fffffadfe5e4c9d0 1

The !irp extension displays the IRP at the given address, including the IRP’s major fields and its stack. Save debugtst1.txt.

  1. Now, in WinDbg, give the command $$<c:\temp\debugtst1.txt. You will get a lot of output, starting with:

1: kd> $$<c:\temp\debugtst1.txt

1: kd> !irp fffffadfe5e4c9d0 1

Irp is active with 2 stacks 2 is current (= 0xfffffadfe5e4cae8)

 Mdl = fffffadfe600f5e0 Thread 00000000:  Irp stack trace.

Flags = 00000000

ThreadListEntry.Flink = fffffadfe5e4c9f0

ThreadListEntry.Blink = fffffadfe5e4c9f0

IoStatus.Status = c00000bb

IoStatus.Information = 00000000

 

...

 

Tail.Apc = 0326cc00

Tail.CompletionKey = 0326cc00

     cmd  flg cl Device   File     Completion-Context

 [  0, 0]   0  0 00000000 00000000 00000000-00000000

 

                        Args: 00000000 00000000 00000000 00000000

>[  f, 0]   0 e1 fffffadfe783d050 00000000 fffffadfe3ee46d0-fffffadfe6869010 Success Error Cancel pending

               \Driver\xStor   CLASSPNP!TransferPktComplete

                        Args: fffffadfe6869130 00000000 00000000 00000000

The value in red is, under these conditions, the SRB address for that IRP.

  1. To get the SRBs formatted, create debugtst2.txt by finding and copying all lines in the above output that contain ‘Args: ffff’. Then change each line to be like this:

dt nt!SCSI_REQUEST_BLOCK fffffadfe6869130

Note: Since the Microsoft symbol store contains only "public symbols", so nt!SCSI_REQUEST_BLOCK may not be available. For present purposes, think of it simply as a structure defined in a driver for which you have complete symbols.

  1. You would save debugtst2.txt. Then in WinDbg you would do $$<c:\temp\debugtst2.txt. The output would be like this:

1: kd> dt nt!SCSI_REQUEST_BLOCK fffffadfe6869130

   +0x000 Length           : 0x58

   +0x002 Function         : 0 ''

   +0x003 SrbStatus        : 0 ''

   +0x004 ScsiStatus       : 0 ''

   +0x005 PathId           : 0 ''

   +0x006 TargetId         : 0xff ''

   +0x007 Lun              : 0 ''

 

...

 

   +0x038 SrbExtension     : (null)

   +0x040 InternalStatus   : 0x21044d0

   +0x040 QueueSortKey     : 0x21044d0

   +0x044 Reserved         : 0

   +0x048 Cdb              : [16]  "*"

Thus, with a few minutes’ work, you would have found and displayed all the SRBs of interest. It is possible to write a debugger extension to do the same thing, but for a one-off investigation, a simple script may be the better approach. You can take a script a step further by packaging commands with control logic to form a command program. The control is exercised through flow-operation tokens like .if and .for. For details about scripts and command programs, see “Run Script File” and “Using Debugger Command Programs” in the Debugging Tools for Windows help file.

A debugger extension is even more powerful, but it takes more time to write. An extension, written in C or C++ and built as a DLL, has available the full power of the debugger and its “engine.” Many of commonly used commands such as !process are in fact extensions. The specifics of writing an extension are beyond the present scope. Refer to “Debugger Extensions” in the Debugging Tools for Windows help file.

Remote debugging

WinDbg (and KD) can be connected to a target to act as a server for a debugger instance acting as client, via TCP/IP or other protocol. A test system is connected via COMx or 1394 to a debugger, and the debugger is made into a server; then a developer can investigate problems or exercise function at a distance. This setup is very valuable in automated testing, allowing a person to investigate lab problems from his or her desk.

To get this capability, you can start the debugger with a command-line option to indicate its role:

windbg -server tcp:port=5555

or you can issue a command in WinDbg after it has started:

.server tcp:port=5005

Either method causes WinDbg to act as a debugging server, listening at TCP/IP port 5005.

In a different instance of WinDbg, do this to connect as a client:

.tcp:server=myserver,port=5005

To start a WinDbg client from scratch:

windbg -remote tcp:server=myserver,port=5005

COMx, named-pipe and SSL are other available protocols.

A few things to realize about remote debugging:

  • If there is a firewall between the host system in one local or corporate network and the target system in another network, remote debugging is more complicated.  See the Debugging Tools for Windows help file for details.
  • Access to symbols and to source are based on the permissions of the person logged in at the remote server, not the permissions of the person that is running the client.
  • The client sets source file locations by the .lsrcpath (“local source path”) command, not .srcpath.

“Short” call stacks

WinDbg does its best to figure out the call stack, but sometimes it is defeated. Retrieving such a situation is one of the most difficult tasks facing the person who is debugging, since she or he must use her/his own knowledge to supplement WinDbg’s expert knowledge. Be warned, then! Tough slogging ahead.

Consider this example from a dump file where there was a double-fault bug check (unexpected kernel-mode trap with first argument 0x00000008):

kd> k

ChildEBP RetAddr 

be80cff8 00000000 hal!HalpClockInterruptPn+0x84

It would seem there was only one routine on the stack, an operating system clock-interrupt routine. It is a bit suspicious that that should have failed. So, begin by looking at the current thread:

kd> !thread

THREAD 88f108a0  Cid 8.54c  Teb: 00000000  Win32Thread: 00000000 RUNNING

 

...

 

Start Address rpcxdr!SunRpcGetRxStats (0xf7352702)

Stack Init be810000 Current be80fd34 Base be810000 Limit be80d000 Call 0

The stack starts at 0xBE810000 and goes down to 0xBE80D000 (three pages, which is normal). The stack base (ChildEBP) for the apparently failing clock routine is 0xBE80CFF8, which is beyond (below) the stack’s end. Is it likely that a clock routine would have used more than the standard stack?

The detective work begins with looking for addresses in the stack that may indicate other routines. For this the usual tool is dds (“Display Words and Symbols”), to look for stored addresses (dqs and dps can be used, too; note that all three of these commands are case-sensitive). For present purposes ignore the clock-interrupt routine and instead focus on the stack up to but not including the routine interrupted by the clock routine. But don’t ignore the clock routine entirely: You start with the fact that its base stack pointer (ChildEBP above) is 0xBE80CFF8.

Next look at 0xBE80CFF8 and see if anything interesting shows up (annotations below are shown with C-style comment delimiters):

2: kd> dds be80cff8 be80cff8+0x100

be80cff8  ????????           /* Invalid storage address. */

be80cffc  ????????           /* Invalid storage address. */

be80d000  00000000

 

...

 

be80d034  00000000

be80d038  00000020

 

...

 

be80d058  be80d084

be80d05c  00000000

be80d060  bfee03e0 zzznds+0x103e0

be80d064  00000008

be80d068  00000246

be80d06c  8a61c004

be80d070  bfbb7858

be80d074  88c1da28

be80d078  00000000

be80d07c  00000000

be80d080  00000000

be80d084  be80d0d8           /* Saved Ebp of zzznds+0xBED7, as explained below. */

be80d088  bfedbed7 zzznds+0xbed7

 

...

 

Assume that the line with the identification “zzzndx+0x103E0” is the driver routine interrupted by the clock routine. You will notice the earlier line (higher stack address) with the identification “zzznds+0xBED7.”

Now look at disassembled instructions a little before zzznds+0xBED7 (the point of call):

zzznds+0xbed0:

bfedbed0 53               push    ebx

bfedbed1 57               push    edi

bfedbed2 e8e7420000       call    zzznds+0x101be (bfee01be)

Note that the call is to zzznds+0x101BE, which is fairly close to the first identified line. Thus, the disassembly could well be the call to that.

Now disassemble zzznds+0x101BE to see how that call worked:

bfee01be 55               push    ebp          /* Save the caller's EBP.         */

bfee01bf 8bec             mov     ebp,esp      /* Make the current ESP our EBP.  */

bfee01c1 83ec0c           sub     esp,0xc      /* Adjust ESP by subtracting 0xC. */

bfee01c4 53               push    ebx

Looking back to the output from dds above, you can see at 0xBE80D088 the caller’s saved Ebp. But the operations of pushing down that Ebp (push ebp at 0xBFEE01BE) and of saving it at 0xBE80D088 mean that Esp after the pushing is 0xBE80D084, and since Esp becomes the current Ebp (instruction at 0xBFEE01BF), and since 0xC is subtracted from Esp at 0xBFEE01C1, the resulting Esp value at the instruction at 0XBFEE01C4 has to be 0xBE80D078.

Now you have determined the Ebp, Esp and Eip values for what was called by zzznds+0xBED7, namely, 0xBE80D084, 0xBE80D078 and 0xBFEE01C4, so you supply them to the k command for it to use them rather than try to discover values:

2: kd> k = 0xBE80D084 0xBE80D078 0xBFEE01C4

ChildEBP RetAddr 

WARNING: Stack unwind information not available. Following frames may be wrong.

be80d084 bfedbed7 zzznds+0x101c4

be80d0d8 bff6030f zzznds+0xbed7

be80d0fc 8046d778 SCSIPORT!SpStartIoSynchronized+0x139

be80d114 bff60e4f nt!KeSynchronizeExecution+0x28

be80d148 8006627b SCSIPORT!SpBuildScatterGather+0x249

be80d174 8041d30e hal!HalAllocateAdapterChannel+0x11b

be80d18c bff5f8c8 nt!IoAllocateAdapterChannel+0x28

be80d1bc 8041f73f SCSIPORT!ScsiPortStartIo+0x2ea

be80d1e0 bff5f4ec nt!IoStartPacket+0x6f

be80d214 bff601d0 SCSIPORT!ScsiPortFdoDispatch+0x26c

be80d22c bff622f7 SCSIPORT!SpDispatchRequest+0x70

be80d248 bff5e390 SCSIPORT!ScsiPortPdoScsi+0xef

be80d258 8041deb1 SCSIPORT!ScsiPortGlobalDispatch+0x1a

 

...

 

That is only part, the latest part, of the stack (there was more). But you should have the general idea.

To arrive at the parameters given to k immediately above, a good deal of detective work was necessary, involving a search of the stack and looking at code to see how the stack is built at a certain point. The work will vary from case to case. The lesson here is that if WinDbg’s stack backtrace looks short, see if the kernel stack allotted to the failing thread is plausibly accounted for. If not, begin to dig in.

Unexpected change of thread context during stepping

If you spend an extended period of time in stepping through kernel code (with F10 or F11, for example), you may notice that control suddenly jumps to a point you didn’t expect. This is the more likely if the code is running at an IRQL less than DISPATCH_LEVEL and you employ step-over (F10). If you knew that you were following control running under a specific thread and now checked the running thread, you would know for certain that the thread changed.

This behavior is normal, if disconcerting. The debugger accomplishes stepping by putting a debugging instruction (e.g., int 3 on x86) at the next instruction or the next statement (the debugging instruction isn’t ordinarily visible to the person debugging). If the thread’s quantum expires in moving from the current instruction/statement to the next, the operating system may dispatch a different thread, and that thread might encounter the debugging instruction, whereupon the debugger would get control. The debugger does not check that the thread is the same as that for the last stepping point but simply stops execution. In such a situation you may observe a jump. This scenario becomes more likely if the stepped-over code involves a great deal of processing, as might happen in stepping over an API that calls an API that calls an API and so forth.

There is no simple way of dealing with this behavior, which, to repeat, is expected. What can work for you is to be on the watch for unexpected jumps in stepping, and to check the current thread if you are suspicious. If you find the thread has switched, you should look back to find the last good point, restart the testing from square 1, set a one-time thread-qualified breakpoint at that last good point, and let things run until that breakpoint is reached. Then you can proceed: You are still exposed to switches, but you are that much further along the path of interest.

Getting more information

This brief essay did not plumb all the debugger’s possibilities. For more information, consult the Debugging Tools for Windows help file.  If you still have unanswered questions, bring them up in the public newsgroup microsoft.public.windbg (hosted by msnews.microsoft.com), or send email to windbgfb@microsoft.com.


728x90

+ Recent posts

728x90