Blog Moved

This blog has been moved to http://imperfect.work.  Existing content has been migrated over, new content will only show up on the new site.  Thanks for reading!

Advertisements

Effective Code Search on Windows

No one remembers everything. In a large code base, we usually only know the details of a tiny part of it. Often times we have to search the code to know how things are defined or used across large number of files. Here I’d like to share a few tips to search source code.

Many software including Visual Studio, notepad++, Sublime Text provide a multiple-file search option, such as “Find in Files”. I don’t use them too often, my alternative is “findstr /snipc:STRING files…” for literal string search. If the amount of files isn’t too huge, these non-indexing options are often acceptable. Note that “regex” in findstr is very limited. If you do need regex search, PowerShell Select-String cmdlet works much better.

For pure source code in Git and you only need to search one repo, “git grep” is faster than brute-force search. If you don’t know this command, try it now and you will be surprised.

For huge code base, indexing is the only effective option that I know of. Fortunately it is super simple. Here is what I do:

  1. Create a separate directory for all source code you may use. Store all repos/enlistments in this directory.
  2. Open “Indexing Options” in Control Panel, click “Modify” button and select the directory above. It will be listed in “Index these locations”.
  3. Make sure the filter description for the interested File Type is “Index Properties and File Contents”. I haven’t found any file types are set incorrectly by default, but your situation might be different
  4. Indexing will take a while. Wait for a day or two.
  5. All above steps only need to be performed once. Later you can update the code as usual, and search the code in File Explorer.

My source code stored in a spinning disk is about 50 GB with 240k text and binaries. Searching is instant (lower than sub-second), which is amazing for so many files.

Git Performance Tuning

The vanilla Git installed using Chocolatey or git-scm (same source) is slow.  On my company dev machine it is even slower than my home machine. If you compare the performance with Linux counterpart, it is noticeably slower. In fact if you open Git Bash, you may notice every command, including ls, has significant slow response. There are fundamental issues that we cannot fix, for instance extensive using of fork() which is cheap on Linux but expensive on Windows, but with some tuning we can make it faster.

Exclude source location in virus scanning

This has nothing to do with Git. By excluding the source location Git, the build process, etc. effectively bypasses the FS filter driver. You should do this even if you are not using Git for development.

Modify bash profile

Git is built on top of mingw, and it often involves bash which in turns load user profile. In Git bash, launch of shell will load /etc/profile and then $HOME/.bashrc. In msysgit, the shell prompt is set to display various repo information, and this requires running Git commands. This overhead can be simply eliminated by setting the following in $HOME/.bashrc:

#!/bin/bash
export PS1='\w $ '

Caching in Git

The following configuration will turn on file system operations in parallel and cache:

git config --global core.preloadindex true
git config --global core.fscache true

Code Signing

Msysgit, repo.exe (distributed by Azure build), and Chocolatey are all unsigned (Authenticode signing). Thanks to awesome AppLocker (enforced by IT department) Windows will try to validate the source of publisher and load all catalogue files under %windir%\System32\catroot\{F750E6C3-38EE-11D1-85E5-00C04FC295EE}. On my machine, that’s over 5500 files to filter and causes over one half second. And this is done for each process launched every time (i.e. no caching). To eliminate this, using your own code signing certificate to sign *.exe in the following locations:

  • C:\Chocolatey\bin
  • C:\Program Files (x86)\Git\bin
  • C:\Program Files (x86)\Git\cmd

To check whether you have code signing cert or not, run the following command in PowerShell:

dir Cert:\CurrentUser\my -CodeSigningCert

The simplest way to sign all executables is (run this as Administrator):

for %f in (*.exe) do "C:\Program Files (x86)\Windows Kits\8.1\bin\x86\signtool.exe" sign /a /uw %f

This will look for all valid certs and select the one that is valid the longest. It doesn’t matter what cert it is, as long as the cert chain is valid.

With these adjustment, “git status” in a workspace with about 32k files takes about 0.4 seconds, also the Git SH is much more useful. I hope this tip is useful to you.

Poor man’s Start Menu in Windows 8

Start Menu was introduced in Windows 95, it has been in use by many versions of Windows for many years. Now it’s gone in Windows 8, at least in Release Preview.  As a keyboard guy, I am used to directly type the app name partially since Vista, so it’s not a big deal to say bye bye to Start Menu.  But sometimes I do want to see the hierarchical list of apps instead of sorting them alphabetically, particularly when there are duplicated names.  Here is the quick way to have a poor man’s start menu without using any app (in fact it’s a folder toolbar).

1. Right click the task bar, and choose Toolbars à New toolbar…

clip_image002

2. Browse to C:\ProgramData\Microsoft\Windows\Start Menu, and click “Select Folder” button.

image

Note that C:\ProgramData is hidden directory, so you have to manually enter the location.

3. Enjoy the brand new poor man’s Start Menu!

image

Copy app.config in the project reference

This is a note of how to ensure the app.config file is copied along with assemblies for the project reference in managed project.

Problem: suppose we have the two C# projects A and B, project A is referenced in B.

  • Project A
    • app.config
    • bin
      • A.dll
      • A.dll.config
  • Project B
    • app.config
    • bin
      • B.dll
      • B.dll.config
      • A.dll

When the project B is built, project A assembly will be copied to the output path but the app.config will not follow.

Solution: build the project with MSBuild with options “/fl /v:d”, read MSBuild.log and analyze when the output of project A is copied to the output path of project B.  We can see that it is the following parameters and target in file %windir%\Microsoft.NET\Framework\v4.0.30319\Microsoft.Common.targets that control what to be copied and how:

        <!--
        These are the extensions that reference resolution will consider when looking for files related
        to resolved references.  Add new extensions here if you want to add new file types to consider.
        -->
        <AllowedReferenceRelatedFileExtensions Condition=" '$(AllowedReferenceRelatedFileExtensions)' == '' ">
            .pdb;
            .xml
        </AllowedReferenceRelatedFileExtensions>

...

        <ResolveAssemblyReference
            Assemblies="@(Reference)"
            AssemblyFiles="@(_ResolvedProjectReferencePaths);@(_ExplicitReference)"
            TargetFrameworkDirectories="@(_ReferenceInstalledAssemblyDirectory)"
            InstalledAssemblyTables="@(InstalledAssemblyTables);@(RedistList)"
            IgnoreDefaultInstalledAssemblyTables="$(IgnoreDefaultInstalledAssemblyTables)"
            IgnoreDefaultInstalledAssemblySubsetTables="$(IgnoreInstalledAssemblySubsetTables)"
            CandidateAssemblyFiles="@(Content);@(None)"
            SearchPaths="$(AssemblySearchPaths)"
            AllowedAssemblyExtensions="$(AllowedReferenceAssemblyFileExtensions)"
            AllowedRelatedFileExtensions="$(AllowedReferenceRelatedFileExtensions)"
...
            &lt;Output TaskParameter="CopyLocalFiles" ItemName="ReferenceCopyLocalPaths"/>
...
        </ResolveAssemblyReference>

Therefore the solution is to put the following lines in csproj file in project B at the beginning of <PropertyGroup>:

...
  <PropertyGroup>
    <AllowedReferenceRelatedFileExtensions>
        .pdb;
        .xml;
        .exe.config;
        .dll.config
    </AllowedReferenceRelatedFileExtensions>
...

With the change, app.config will follow the assemblies in both the build and unit test.

Autologon and run GUI programs remotely in desktop session

This is a note of how to automate the remote execution of a GUI program on a test machine.

Note: you should not use this against a non-test machine because the autologon may impose a security risk.

Scenario: there is a test machine “ZY” with Windows Server 2008R2 installed, and I want to automate the process of running a GUI program on it, the program itself is fully automated but cannot run in session 0.

The basic idea is to use scheduled task to run the program in the login session.  PSEXEC will not work because the process started from session 0 by NT services will be running in session 0, thus no GUI will show up.  In order to ensure the program can be started after scheduled task is created, the given user must be logged on to the computer.  This can be done by automatic logon.

Step 1: Turn on automatic logon

Run the following commands to change the registry values on ZY to turn on autologon:

reg add \\zy\HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon /v DefaultUserName /d [MyUserName] /f 
reg add \\zy\HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon /v DefaultPassword /d [MyPassword] /f 
reg add \\zy\HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon /v DefaultDomainName /d [MyDomain] /f
reg add \\zy\HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon /v AutoAdminLogon /d 1 /f

For more information please read http://support.microsoft.com/kb/315231

Step 2: Create scheduled task

Suppose the program we want to run is C:\Windows\System32\notepad.exe.  The credential (must be identical with step 1) is: CONTOSO\MyUser, password is MyPassword.  The the following command to create a task:

schtasks /Create /S zy /U CONTOSO\MyUser /P MyPassword /RU CONTOSO\MyUser /RP MyPassword /SC once /TN "notepad" /TR C:\Windows\System32\Notepad.exe /ST 14:50 /IT /F

Note that:

  • /ST must specify a future time.  It is not important how exact the start time is, since we will use “schtasks /run” to run the command.
  • If the current user is the same as the remote computer where we want to run the command, options /U /P can be ignored.
  • It is important to use /IT in order to run the command interactively when the user is logged in.

Step 3: Reboot the machine

There are several ways to do this, for instance

shutdown /r /m \\zy /t 0

Step 4: Check if the computer is rebooted and user has logged in

This can be done by check if there is process running on Console (session 1) or user sessions (session > 1) with given user name:

tasklist /s zy /u CONTOSO\MyUser /p MyPassword /fi "session gt 0" /fi "username eq MyUser"

If explorer.exe and dwm.exe are listed, the login has completed.

Step 5: Run the task

We don’t have to wait until the task is started, instead we may start it immediately by:

schtasks /run /S zy /U CONTOSO\MyUser /P MyPassword /I /TN "notepad"

Note that the task name in /TN should be the same as the name in step 2.

Step 6: Delete the task

Run the following command to delete the scheduled task:

schtasks /delete /S zy /U CONTOSO\MyUser /P MyPassword /TN "notepad" /F

Step 7: Disable the autologon

Refer to commands in step 1 and replace with “reg delete” to delete DefaultPassword and change AutoAdminLogon to 0.

Bypass access restrictions in managed code

For some reason I need to access a private field in a class in the system library.  .NET provides a way to do it.  I think it will be cool to share it although it is not a good coding practice that we will be using it on a regular basis.

Hypothetically suppose we want to construct an instance of System.ConfigNode.  This class is an internal class in mscorlib, and the constructor is internal too.  So we cannot just use it as usual.

image

Fortunately we have System.Reflection.  Firstly we load the assembly and find the type:

var assembly = Assembly.Load("mscorlib, Version=2.0.0.0"); 
var configNodeType = assembly.GetType("System.ConfigNode");

The we can construct an instance:

var configNode = configNodeType.InvokeMember( 
                "", 
                BindingFlags.CreateInstance | BindingFlags.NonPublic | BindingFlags.Instance, 
                null, 
                null, 
                new object[] { "SampleConfigNode", null });

or

var configNode = Activator.CreateInstance( 
    configNodeType, 
    BindingFlags.Instance | BindingFlags.NonPublic, 
    null, 
    new object[] { "SampleConfigNode", null }, 
    null);

Now we can change a private field:

configNodeType.InvokeMember( 
    "m_value", 
    BindingFlags.NonPublic | BindingFlags.SetField | BindingFlags.Instance, 
    null, 
    configNode, 
    new object[] { "new value" });

or call a private method:

configNodeType.InvokeMember( 
    "AddAttribute", 
    BindingFlags.NonPublic | BindingFlags.InvokeMethod | BindingFlags.Instance, 
    null, 
    configNode, 
    new object[] { "some key", "some value" }); 

BTW, the format for the code is horrible.  I need to find a better way to handle this.