COM Security Primer, Part I

Next to threading and apartments, security is the part of COM that seems to cause developers the most pain. Nearly everyone’s first experience with DCOM is to attempt to launch a remote COM server, only to be met with an E_ACCEESSDENIED error. E_ACCESSDENIED may be the most hated HRESULT in all of COM, because it tells you that the security subsystem prevented you from doing what you wanted to do, but it doesn’t tell you why you were prevented from doing it.

I get regular e-mail messages from developers who have written COM applications that work on one machine, but fail miserably when they’re deployed to work across machines. In almost every case, the problem is security. Security isn’t a big deal when COM clients and COM servers reside on the same machine, but it’s a huge deal when activation requests and method calls start flying between machines. COM can’t simply allow someone to walk up to a machine, log in, and begin launching remote server processes on other machines; to do so would constitute a hole in security. If you want a COM client to launch a COM server on another machine, you must configure the security subsystem to allow it. And that’s not all. There’s also access control, remote server process identity, and authentication to think about. If you get a distributed COM application to work without factoring all these aspects of the COM security model into your design, you got lucky. And even though your application runs fine right now, something as simple as a user logging out from a remote server might cause it to fail.

The story of COM security is a story that needs to be told. In this article, the first in a two-part series, I’ll discuss two important aspects of the COM security model: activation security and access security. Two articles can’t hope to convey everything there is to know about COM security, but they can help you build the fundamental understanding necessary to have the security subsystem work for you instead of against you.

Activation Security

Let’s say Bob walks up to a machine running Windows NT or Windows 2000, logs in, and runs an application that attempts to create a COM object on another machine on the network. By default, that attempt will fail. If Bob is to launch remote server processes on other machines, Bob-and anyone else who wishes to launch a server process remotely-must be granted permission to do so. COM refers to such permissions as launch permissions.

Launch permissions can be applied at two levels. First, the COM server that’s to be launched remotely can be assigned launch permissions that apply only to it. Such launch permissions are called server-specific launch permissions. Second, the machine on which the remote COM server is installed can have default launch permissions defined. Server-specific launch permissions, if present, take precedence over default launch permissions. When a remote activation request arrives at its doorstep, COM first checks to see if server-specific launch permissions have been defined for the COM server targeted by the request. If the answer is yes, COM uses those launch permissions to determine whether to succeed or fail the activation attempt. If it finds no server-specific launch permissions, COM then falls back to the machine’s default launch permissions.

Both types of launch permissions are recorded by writing access control lists (ACLs) to the registry of the machine on which the COM server is installed. An ACL is a standard NT security construct used to identify users and groups of users. Each entry in a launch ACL specifically grants or denies launch permission to a user or group of users. Server-specific launch permissons are applied by adding a value named LaunchPermission to the registry’s HKCR\AppID\{} key and assigning to that value a binary ACL, as shown in Figure 1. (HKCR stands for HKEY_CLASSES_ROOT; {}is the COM server’s AppID.) Default launch permissions are applied by adding a DefaultLaunchPermission value to the registry’s HKLM\Software\Microsoft\Ole key and assigning it an ACL, as shown in Figure 2. (HKLM is short for HKEY_LOCAL_MACHINE.) Because it’s no fun to encode ACLs by hand, launch permissions are normally added to the registry by running DCOMCNFG, a configuration tool that comes with COM.


Figure 1: Server-specific launch and access permissions


Figure 2: Default launch and access permissions

The upshot of all this is that if Bob runs a COM client on machine A that launches a COM server on machine B, Bob must be granted launch permission on machine B. that permission can come from a LaunchPermission entry or a DefaultLaunchPermission entry, but it must come from one of the two. Bob must also be authenticatable on machine B. If Bob logged in using a domain account and both machine A and machine B belong to that domain, then Bob can be authenticated on both machines. However, if Bob logged onto machine A using a local account, then machine B must have an identical local account (same user name and password), or else the activation request will fail.

Note that you can effectively disable activation security by granting launch permission to Everyone. Granting launch permission to Everyone is the only way to allow anonymous (unauthenticated) users to launch remote COM servers.

As an aside, I often receive e-mail from developers asking whether it’s possible for a COM client to launch a remote COM server using an alternate identity. For example, suppose Alice has permission to launch a particular COM server, but Bob doesn’t. Can Bob launch a remote COM server as if he were Alice? The answer is yes-if Bob knows Alice’s user name and password. The trick is to initialize a COAUTHIDENTITY structure with Alice’s credentials, put the address of the COAUTHIDENTITY structure in a COAUTHINFO structure, put the address of the COAUTHINFO structure in a COSERVERINFO structure, and pass the address of the COSERVERINFO structure to a COM activation function such as CoCreateInstanceEx or CoGetClassObject. If you do this, be sure to specify an impersonation level equal to or higher than RPC_C_IMP_LEVEL_IMPERSONATE in the COAUTHINFO structure’s dwImpersonationLevel field. Otherwise, the activation request will fail.

Power COM Programming Tip

Ever been frustrated by CoCreateInstance or CoCreateInstanceEx calls
that take a minute or more to succeed? The delay is probably due to the
fact that an unauthenticatable caller is attempting to launch a remote
server process, which is perfectly legal if you’ve granted launch
permission to Everyone. The problem is that the local COM Service
Control Manager (SCM) tries first to establish an authenticated
connection to its counterpart on the remote machine, and only falls
back to an unauthenticated connection after its earlier attempts
fail-hence the delay.

The solution is to pass CoCreateInstanceEx a COSERVERINFO structure
containing the address of a COAUTHINFO structure whose dwAuthnLevel
field is set to RPC_C_AUTHN_LEVEL_NONE. This tells the local SCM to
go directly to an unauthenticated connection and bypasses the
time-consuming attempt to create an authenticated connection to a
remote SCM.

Access Security

Activation security allows a system administrator-or anyone who has permission to edit the registry-to control who can launch remote server processes. Access security governs who can call into remote server processes once they’re launched. Generally speaking, the same users and groups of users that enjoy launch permission should be granted access permission, too. There are exceptions, however. Occasionally it’s useful to allow someone to call into a running server (or client) process but not grant them permission to launch the process to begin with.

Like launch permissions, access permissions can be applied to individual COM servers and to entire machines. ACLs used for server-specific access permissions are stored in the registry at HKCR\AppID\{}\AccessPermission, where once more {}is the COM server’s AppID (see Figure 1). Default access permissions, which define who can make method calls to a remote server process that isn’t accompanied by server-specific access permissions, are stored at HKLM\Software\Microsoft\Ole\DefaultAccessPermission (see Figure 2). The values assigned to AccessPermission and DefaultAccessPermission are serialized ACLs whose entries explicitly grant or deny access to individual users and groups. As with launch permissions, access permissions recorded in the registry are typically put there with DCOMCNFG.

That’s one way to apply access permissions: write them to the registry. But there’s another way. CoInitializeSecurity is a COM API function that a process can use to create its own programmatic security blanket. It can only be called successfully once per process, so if you call it, you must call it early in the process’s lifetime (before COM beats you to the punch). The first parameter to CoInitializeSecurity defines the process’s access permissions. It can be set to any of the following values:

  • NULL
  • A pointer to a GUID that corresponds to a registered AppId
  • A pointer to an IAccessControl interface
  • A pointer to a security descriptor containing an ACL

Which type of value the first parameter holds is indicated by a flag passed in CoInitializeSecurity’s eighth parameter (dwCapabilities). Setting the first parameter to NULL prevents access checks from being performed, effectively turning access control off for this process. Passing a pointer to an AppID causes COM to set access permissions according to the AccessPermission ACL found in the registry under that AppID. Passing a pointer to a security descriptor assigns access permissions using the ACL in the descriptor. Finally, passing a pointer to an IAccessControl interface permits a COM server to take control of access checking and to succeed or fail individual calls as they arrive based on the caller’s identity-in effect, to implement its own security policy. By implementing IAccessControl, a COM server could, for example, accept calls from users whose user names begin with A through K and reject all others.

Many programmers believe that setting access permissions with CoInitializeSecurity is superior to relying on ACLs in the registry. Why? Because CoInitializeSecurity ensures that your carefully applied access permissions can’t be destroyed by someone with the freedom to edit the registry. It’s unfortunate that launch permissions can’t be applied with CoInitializeSecurity, too, but they can’t, because a process has to be launched before it can call CoInitializeSecurity, and once it’s launched, it’s too late for any API function to prevent the launch from occurring.

Whether you choose to apply access permissions declaratively (with DCOMCNFG) or programmatically (with CoInitializeSecurity), there’s one detail you mustn’t forget. Be sure to give the SYSTEM account-the built-in account under which most NT services run-access permission to your COM servers if you want clients to be able to activate them remotely. During one critical phase of the remote activation process, a part of COM that runs in an NT service must call into the freshly launched server process. If you’ve denied the SYSTEM account access permission, the call will fail and the client’s activation request will fail, too. Due to a bug in Windows NT 4.0, the HRESULT returned to the client might be E_OUTOFMEMORY instead of E_ACCESSDENIED.

What it all means is that if you want Bob to be able to launch a remote server process and then make method calls into that process, you should grant Bob access permission as well as launch permission. Failure to do both will result in the infamous E_ACCESSDENIED errors that COM programmers have come to know all too well.

Power COM Programming Tip

One of the errors that newbies often experience when they first
begin tinkering with DCOM occurs when they fail to give a remote
server process permission to perform callbacks to a client. Suppose
a client on machine A launches a COM server on machine B and receives
an interface pointer in return. Then that client passes an interface
pointer of its own to the server so the server can perform callbacks.

What’s wrong with this picture? Nothing, except for the fact that
callbacks will only be permitted if the server process is granted
access permission to the client process. If the server process is
assigned the identity Mister Server, then Mister Server must be granted
access permission to the client process. One way to grant that access
permission is to have the client process call CoInitializeSecurity.
Another way is to include Mister Server (or Everyone) in the client
machine’s DefaultAccessPermission ACL.

What makes this error especially difficult to diagnose is that if
connection points are involved, the failure typically doesn’t occur
when the server attempts its first callback; it occurs when the client
passes its interface pointer to the server using
IConnectionPoint::Advise. Most implementations of Advise, including
ATL’s, call QueryInterface on the client. But if the server process
lacks access permissions in the client process, QueryInterface will
fail. When Advise sees QueryInterface fail, Advise will fail, too.

The moral: If you’re using connection points to facilitate callbacks from
remote servers and IConnectionPoint::Advise returns E_ACCESSDENIED or
E_OUTOFMEMORY, check the access permissions on the client. Chances are
the security principal whose identity the server process has been
assigned does not have permission to call into the client process.

Next Month…Identity and Authentication

Learning about activation security and access security is a good first step on the road to understanding COM security. But there’s more-much more, as a matter of fact.
In my
next column
, I’ll talk about two more aspects of the COM security model: remote server process identity and authentication. If you found this month’s discussion interesting, I think you’ll find next month’s to be even more so. Stay tuned…

More by Author

Get the Free Newsletter!

Subscribe to Developer Insider for top news, trends & analysis

Must Read