Using SQL Server profiler can give us the answers that we are looking for. Check out this blog post for the details. Cheers.

It turns out that this is easy to do. There is a method on the ConfigurationManager called OpenMappedMachineConfiguration that will let you define another config file to use.

I have a little method for pulling in the configuration from a given file:

// MSCrmConfigurationSection is my custom config section type
MSCrmConfigurationSection GetConfig( string in_filename ) {
	ConfigurationFileMap fileMap = new ConfigurationFileMap( in_filename );
	Configuration configuration = System.Configuration.ConfigurationManager.OpenMappedMachineConfiguration( fileMap );
	MSCrmConfigurationSection config = ( MSCrmConfigurationSection )configuration.GetSection( "MSCrm" );
	return config;
}

A really nice feature that we get for free is that we can mix and match files at will, pulling configuration sections from different files. And, as a bonus, the app.config file still takes effect, so any assembly redirects or publisher policies that are defined there still work as usual. Pretty awesome.

Briefly, by default, custom configuration uses XML attributes, unlike the AppSettings section, which uses a special configuration case of a key/value dictionary. I think that this is confusing and is not well treated in MSDN.

If we want a custom appSettings-like config like the following:

<myAppSettings>
      <add key="NewKey0" value="Monday, March 30, 
           2009 1:36:33 PM" />
      <add key="NewKey1" value="Monday, March 30, 
           2009 1:36:40 PM" />
  </myAppSettings>

We need to use a DictionarySectionHandler instead of the usual ConfigurationSection.

Using a ConfigurationSection, our XML markup would look like this:

<myAppSettings
     NewKey0="Monday, March 30, 2009 1:36:33 PM"
     NewKey1="Monday, March 30, 2009 1:36:40 PM"
/>

It is possible to create elements as children with some additional coding using ConfigurationElement.

There is a long CodeProject article on this stuff that uses C++ here that I found useful and there is an extensive treatment on configuration here, but I’ve abridged things greatly here for my own reference. Hopefully this helps you too.

Although the API supports most things that we’d want to do, there are a few tricky things that aren’t that obvious from the documentation. I’m going to go into this in more detail in a future post, but there are a few key takeaways that I want to make sure I remember, so I’m posting them here now.

1) Creating an entity doesn’t create any attributes other than the primary attribute.
Even if we give CRM a fully-populated entity metadata graph, only the primary attribute will be created on the CreateEntityRequest call.

2) An entity must have a primary attribute specified.
Creating the entity will fail if one string attribute is not provided and the primary attribute field is set to this entity. Both of these conditions must be met.

3) Lookup attributes cannot be created via the normal attribute metadata service call.
Entity references are created via a completely separate call.

4) Creating lookup attributes will fail if the referenced entity does not exist.

5) Entity names must adhere to the CRM naming prefix convention.
Entity names and field names must have a prefix and an underscore in order to be considered valid schema names.

6) Cryptic “Generic SQL” errors are often caused by missing labels
I’m still compiling a checklist of things that cause this frustrating error. Many times a call will fail with a SQL error, and it is very difficult to figure out what went wrong since there is no detail given.

7) Adding entities and fields don’t require that we publish, but adding new picklist options does require a separate publish action before the items can be used.

Boolean fields must be specially treated and given labels for the true and false textual values.
If this is not done, the field will silently fail to be created.

That’s a brief list for now. Expanding on all of these things will take a whole series of posts I’m sure. This stuff is way under-documented and is hard-won knowledge, so I’ll be sure to give it a proper treatment here on the blog.

Microsoft provides a lot of options out of the box with the .NET framework, and even more through the Configuration Management Application Block. Most configuration concerns are the same whether it is a Windows app or a Web app, but since there are a lot of different options, how do we pick what is best for our application?

Before we get into any .NET-specific items, here are some general guidelines that I have used in the past:

Don’t rely on implicit configuration – allow the app to be explicitly configured programmatically

In the case of a library, we’ll want to be able to override configuration as needed for testing and special use cases. For example, I have a library for accessing CRM that has a configuration section. In some cases I want to test different configurations against different CRM servers in my tests, so I need a way to override the configuration file. Yes, you can do this in a convoluted way using the .NET framework, but it is much better to bake this into the library design. Another scenario is doing data export/import between multiple servers. I have a few tools that use the library I mentioned to take data from one CRM installation and move it to another one. If the library only supported one configuration section we’d have to build this functionality into the library, which is a concern that shouldn’t be part of the underlying, but of the application.

However, a very nice feature of a library is to have a default configuration, as well as a default configuration location. This means that if we provide a config specifically it will override everything else, but if not, we can make it easy on ourselves by providing some sensible defaults or by using the app.config file if it exists. This kind of fallthrough can be a little tricky, so it is worth thinking about up-front so that things go smoothly and predictably as development progresses. Configuration behavior is one of those things that is tough to change once it gets into production.

Use separate classes to hold configuration data

This should go without saying, but I’ll say it anyway – don’t put configuration variables into any other classes. Keep these concerns separate. If a class needs the configuration data, use composition to make the configuration object a private member. Another thing to watch out for is not using a configuration data in their own objects at all. I’ve seen a lot of project where configuration data was scattered all over the place. Don’t do this!

I’m going to follow up with some code examples soon so stay tuned.

This works out pretty well most of the time. However, recently I wanted to use my SSH backhaul trick to grab some data from a client’s site back through their firewall. In order to get this to work, I had to do some extra experimentation with CopSSH user accounts and my VirtualBox settings.

First off, let me recap exactly what we are trying to do. It might be worthwhile to look at my SSH backhaul article first, but what we are doing is running a secure shell server locally on the virtual machine and connecting to it from the remote server using Putty. This lets us access things like Microsoft CRM services on the remote machine for doing things like data dumps and schema upgrades.

I’m using VirtualBox as my virtualization environment. I happen to be using NAT (network address translation) instead of bridged network connection. This means that there is one extra step that I didn’t cover in my previous article, which I will outline here. The complete end-to-end scenario becomes:

Putty on remote server -> firewall on my local network -> VirtualBox NAT on my laptop -> VirtualBox VM -> CopSSH daemon

So I covered everything in the previous article except for setting up VirtualBox NAT. Fortunately it is very simple. We need to set up port forwarding across the NAT. To do this go into the network settings of the running virtual machine and look for the button that says “port forwarding”. This lets you set up the host and guest port. I had to set the IP addresses rather than leave them blank, but what I did was set both to 0.0.0.0, which means “all addresses”. Here is a screenshot:

I’m mapping the SSH port 22 to port 2222 to avoid conflicts with the native sshd daemon that is running on my Ubuntu laptop.

Once we have this set up, we can test the connection locally by using Putty to connect to localhost on port 2222. We should get a login prompt from CopSSH.

Once we know that CopSSH is working and reachable via the port forwarded over the VirtualBox NAT, we need to authorize the user accounts that can log in via SSH. This is where we have to pay close attention. The thing that caused me a lot of pain was that the domain and account names are case sensitive. When adding the user account put the domain name in all capitals and pay attention to the case. Check out the screenshots:

One last thing: the user accounts will need the right to log on locally. Double check using the Local Security Policy tool (look in Administrative Tools):

Test logging in with the domain account using domain\user:

If login works, set up the tunnel the same way as in my previous article and rock on!

Since Visual Studio project files are just custom msbuild definitions supplied by Microsoft, we can build them with msbuild on the commandline. I didn’t want to do any custom tweaking of the files themselves, since I’ve had issues with losing my changes when someone decided to mess with the project file. What I really want out of the project file is the list of the files that need to be built. In the future I’m thinking about devising a way to create a wrapper msbuild project that looks at the parts of the Visual Studio file for the source file list and dependencies and defines its own build targets, but I’m too busy to try to get that working for now.

I’ve had all sorts of issues with trying to tweak msbuild files. Microsoft really came up with a powerful build system, but unfortunately it is hamstrung when trying to work with Visual studio. Some things I wanted to do in the past included getting a version number from the Assemblyinfo.cs file. This turned into a fiasco because of a silly limitation in the scope of property variables within the build targets.

After all of this fiddling I have come to the conclusion that, at least for now, the best way to get msbuild to build a Visual Studio project in a particular way is to manhandle the property settings on the commandline when calling msbuild.

Here is a sample batch file that I use to call msbuild. I want to set the .NET framework version and project configuration (debug/release) along with assertion of some conditional compilation flags used to generate different targeted versions of the code (CRM 4/CRM 2011):

set msbuild=C:\Windows\Microsoft.NET\Framework64\v4.0.30319\MSBuild.exe
set PointVersion=1.0.0
set outpathprop=/property:OutputPath=..\release\%PointVersion%\3.5

%msbuild% ^
    /property:Configuration=Debug ^
    /property:TargetFrameworkVersion=v3.5 ^
    %outpathprop%\CRM4\Debug ^
    /p:DefineConstants="CRM4" ^
    CrmQuery\CrmQuery.csproj

You can see that I set the output path to include the release version number and the framework version. I could parameterize this even further, but that’s an exercise I’ll leave to the reader or until I refine this technique next time around. Notice that I define a constant called ‘CRM4′ which is responsible for triggering some #ifdefs in the code that include CRM4-specific code. The beauty of this script is that now I can just add a section for each configuration that I need to build, run it once and I’ve got all the debug and release configurations of both CRM4 and 2011 versions of my project.

The file layout that I’ve chosen is:

\release\<version>\<crmversion>\<configuration\

Effectively, this build script controls the versioning of the build. I got tired of trying to do this by reading Assemblyinfo.cs, but we’ll have to deal with this if we want to build a signed assembly.

The advantage of having a consistent folder structure is that we can now put the built artifacts into an assembly cache and use a dependency management script to copy them to a local lib\ folder under the project directory. We can manage multiple versions of the same assembly and not get completely confused as to which version we are referencing in the project. Let me know about your quick-and-dirty build techniques in the comments!

For those of you that don’t know, the DynamicEntity type gives us a way to write code against an arbitrary CRM4 installation without having to generate a custom service proxy. We don’t get the advantages of having all of the field names specified (early binding) but we don’t have to recompile for different client installations in order to get access to new fields.

However, despite of (or perhaps due to) the flexibility that DynamicEntities afford, there are some idiosyncratic things about how data property values are set and retrieved.

The hallmark of CRM 4 DyanmicEntities is the use of property classes for the data fields of the entity. These classes are all of the form XXXProperty, where XXX is something like String, CrmMoney, etc. CRM 2011 has largely done away with these special-purpose field wrappers, but in CRM 4 we are stuck with them. This wouldn’t be so bad, but there are some inconsistencies in their use in the Microsoft CRM API. I’ll go over a few cases that have bitten me in the past here.

1. Iteration

foreach( Property prop in m_entity.Properties ) {
    /// do something
}

When we use a foreach construct, the IEnumerator implementation returns a Property type. Why is this inconsistent? Well it isn’t from an API standpoint, but it was very confusing to me when I took a look at how the data was actually stored at runtime. Hint: there are no Property objects to be found in the backing store. They are created dynamically in the IEnumerator implementation. Take a look at the disassembly of IGetEnumerator():

private IEnumerator<Property> InternalGetEnumerator()
{
    List<Property> list = new List<Property>();
    foreach (KeyValuePair<string, object> pair in this._nameToPropertyValue)
    {
        list.Add(PropertyFactory.Instance.CreateInstance(pair.Key, pair.Value));
    }
    return list.GetEnumerator();
}

We can see that the Property types are actually being created and returned on-the-fly from the internal data store _nameToPropertyValue.

2. Assignment

We can do assignment in one of two ways. In each case the value that we assign will be slightly different.

DyanmicEntity entity = new DynamicEntity();
entity.Properties.Add( new StringProperty( key, value ) );

or

entity[ key ] = value;

This example also applies to retrieval of the property value, that is, if we want to get the plain string without the StringProperty wrapper we would write:

string val = entity[ key ];

I’ve noticed that when we make an assignment using a Property, the SDK library actually throws it away and only stores the inner value! This makes the property simply an ephemeral container that is effectively used only for conveying the field name. Check out disassembly for PropertyCollection.Add():

public void Add(Property property)
{
    this._nameToPropertyValue[property.Name] = property.GetValue();
}

So when iterating and using Add() we are dealing with Property types. When using [] indexer notation we are dealing with the underlying value data type. I find this to be quite inconsistent and tedious, but as long as it is kept in mind, things work out ok.

Hopefully this clears up some confusion about the finer points of DynamicEntities for you!

Fortunately CRM 2011 gets this right, so we no longer have to worry about that. In fact, MS has shown us how to implement our own IOrganizationService right in the sample code. Just look under samplecode\cs\client\soaplogger\soaplogger in the SDK. In the example, the SOAP requests are logged from the intermediate IOrganizationService implementation. However I have found using custom services to provide dummy data very useful for testing.

Eventually I fleshed out the mock CRM service and released it as FakeCRM.

I’ll be blogging more about this topic in the future, stay tuned!

Without further ado, the list is here.

I’m including it inline here for quick reference: