Tue, 28 Aug 2007 16:46:00 GMTThis is a story about leaky abstractions. It’s not a happy story, and I don’t talk about cool things like WPF, AJAX, JSON, POX, or how much I detest the RIAA.Instead, this is the story of a developer and the Process.Start() method. It’s not exciting, and I don’t mention Silverlight anywhere other than this sentence.Still, you should listen.Leaky Abstractions are the technological equivalent of a spring poking through your couch. Until the day that you sat on that rusty, pointy springy-spring, you knew that your couch was comfy. To tell the truth, you really didn’t know, care, or care to know why. Joel Spolsky has his own diatribe on this subject, and it’s worth reading. Of course, he talks about software, and not couch springs, but I’m sure you will still get the point. Some time ago, my team noticed that a core part of our application would fail to respond, and when we restarted it, it would fail to reclaim its remoting TCP/IP port. Specifically, it listened on port 30123, and whenever we restarted the application, we would receive a System.Net.Sockets.SocketException claiming that the port was already in use. From the command line, a netstat –a verified this information. The netstat output also revealed that the “dead” process had the port still locked. We were mystified, as netstat was reporting a Process ID (PID) that was no longer in the task list. After much head scratching, we engaged Microsoft Product Support Services. Unfortunately, Microsoft PSS did about as much head scratching as we did. As this problem was only intermittent, our troubleshooting was particularly difficult. Eventually, we were able to repro the issue in our development environments, and Microsoft escalated the issue to their internal debugging teams. We submitted several crashdumps, and discovered some really, really interesting news: Under certain circumstances, when you launch a process using the Microsoft System.Diagnostics.Process.Start() method, all handles from the parent process are inherited by the child process. This may not be particularly interesting, but consider the following potential chain of events:1. Process A starts.2. Process A opens a TCP port.3. Process A starts Process B using System.Diagnostics.Process.Start4. Process A terminates unexpectedly, and does not properly close its TCP port.5. Process B maintains a copy of the handle to the TCP port.6. As there is an existing open handle, the Operating System does not close the port when cleaning up dead resources for process A.It took several user generated crash dumps, but Microsoft PSS finally confirmed that this was the issue. How does this tie into the concept of Leaky Abstractions? Well, if we examine the Process.Start() method using Reflector, we note the following line of code: flag = NativeMethods.CreateProcess( null, cmdLine, null, null, true, &[...]
Tue, 31 Jul 2007 01:39:00 GMT
Earlier this week, my twelve year old daughter approached me and asked to learn "All that computer stuff that you do." We had chatted previously about my job, but I had taken particular care to not pressure her into learning computers. Such a decision is best left to natural curiousity.
Hanna and I talked about several languages. She had wanted to learn C#, as that is what I spend most of my time with. Instead, I suggested Ruby. I've always wanted to learn the language, and an interpreted environment would make the edit-compile-debug loop a little bit easier.
Ruby has yet to disappoint me as a teaching language. While I haven't found any Ruby books suitable for teaching a pre-teen Ruby programming skills, I have been able to develop simple lesson plans to teach basic concepts. Along the way, I'm also learning the language, which is a big plus for me.
In two days, she has learned console input and output, variables, string comparison, arrays, simple iterators, if / else constructs, and more. It's really been a blast. I'd definitely recommend Ruby as a teaching language.
Next topic: Objects. Wish us luck!
Fri, 29 Dec 2006 19:40:00 GMTI feel like I just got hit by the dodgeball in elementary school gym class. In a unique game of tag, I'm supposed to tell everyone five things that they didn't know about me. Well, here goes:
Mon, 20 Mar 2006 17:07:00 GMT
My Continuous Integration presentation in Dallas, Texas went pretty well. Although I'm not a practiced speaker, it seems that the audience was engaged and interested. I've even been invited to present on the same topic again to another group.
I've been asked to publish the slides. They are oriented around my dev group's experience with CI, so they may not be as interesting to external folks. Also, without me to explain them, it's just not the same :-)
Tue, 14 Mar 2006 04:35:00 GMT
A few loyal readers have complained that the "Occasional Clue" has been a little less occasional than anticipated.
There was a reason, actually. I took an unannounced hiatus from blogging. The reasons are relatively boring, but I certainly didn't stop for lack of writing material.
I've been to several interesting Atlanta C# Meetings, including Paul Wilson's fascinating O/R Mapper talk. I went to an interesting talk by Jeffrey Richter, after which we all went to the local wings joint and downed a few cold brews. I even met the GridViewGirl FormerlyKnownAs DataGridGirl, and didn't even know that I was in the powerful pink presence of perspicacity.
I'm in the process of evaluating several new technologies, which I will post on shortly.
Among other efforts, I found (and fixed) a SQL Server 2000 / dblib bug. This bug potentially would cause my team hundreds of developer-hours. Even after several PSS cases, Microsoft could not give us a solution, so I'm particularly proud of this fix.
I'm also posting from the American Airlines Conference Center in Dallas, Texas, where my company is bringing about a hundred Developers, Managers, and Architects under one roof for a chance to rub elbows, and show our latest "leet skillz." I'm presenting on Continuous Integration, and how implementing CI has improved life on my project team. It's my first major presentation in front of a bunch of people I don't know, so I'll let you "occasional" readers know how it goes.
Thu, 15 Sep 2005 23:31:00 GMT
This is really too funny.
Thu, 15 Sep 2005 20:48:00 GMTOne of the keynote presentations today was on Windows Compute Cluster Solution. Now, I've been working with Windows Clusters in a High Availability environment for some time now, so I've been very interested in what Microsoft's message was going to be in this product space. Microsoft has been getting their lunch handed to them in this area by Linux clusters for a long, long time. While it is currently possible to build High Performance Clusters on Windows without the Compute Cluster Solution, it is certainly not straightforward. There are a number of hurdles that Microsoft faces in this product space: * Linux clusters are a mature solution. Most existing clusters are built on Linux, and programmers/administrators are familiar with this environment. * Linux is more flexible and scriptable from the command line. While third party tools do make Windows relatively scriptable (for example, you may install a Unix-like shell on windows using Cygwin), these tools merely bolt-on pieces that are automatically available in Linux. * Linux clusters are inheritably cheaper. If I am building a 40- or 100-node cluster, per-server operating licensing costs become a major portion of my budget. I'd rather spend this money on additional nodes. * Linux clusters are easier to auto-rollout. While Windows has RIS, and third party imaging tools like ghost, it is very trivial to autoscript installation on Linux. I'm mostly a Windows guy, but I was able to use Intel network cards with PXE to download a bootloader over TFPT, and then auto-image an installation of Linux onto the box. I do understand that all of these things are possible on Windows Server, but the infrastructure overhead is a bit higher. * Linux HPC clusters may be configured for High Availability. Windows Compute Cluster Solution is not designed for High Availability. More on this later. -- So, I was disappointed in this mornings keynote coverage of Microsoft Compute Cluster Solution. As I'm a bit familiar with this product space, I found the demos misleading, at best. A quick rundown of my observations: Bob Muglia (Senior VP, Microsoft Windows Division) was demonstrating job distribution on a Microsoft Compute cluster environment, and seamlessly added a cluster node to an existing cluster. This was actually pretty impressive, though one would expect this functionality in a cluster solution. Everything worked smoothly, and the new cluster node automatically took on existing jobs. To demonstrate that cluster nodes may be removed (fail), Bob removed the network cable to an existing node, and the head node removed that system from the list of available cluster worker nodes. What nobody really noticed was that the failover demo failed. I think a job was stuck, and they quickly hit the kvm switch before anyone could notice. They then went on to demo a new feature of Excel 12, with "Excel Server". The idea is that you may run an Excel spreadsheet on a server. For this demo, they ran a complicated Excel spreadsheet on the Cluster. The cool part of the demo was that you could upload the spreadsheet to the cluster, and the Scheduler would distribute this job to an available cluster node. The node completed the work, and the spreadsheet results were returned to the client. While this demo was cool, it totally misses the purpose of a cluster. Running the Excel spreadsheet on a cluster demonstrates what may be called an "Embarrassingly Serial" problem. The idea of running on a cluster is that you have a problem that may be split into many parallel subtasks. Some problems, such as figuring Prime Numbers, are often referred to as an "Embarrassingly Parallel". This class of problem means that no piece of the problem set is dependant on another piece, and each node may w[...]
Tue, 13 Sep 2005 20:29:00 GMT
Well, everyone else has been posting about the PDC - I thought that I would have written by now, but I haven't really seen anything that is really exciting. So far, most everything seems to be a refinement of things introduced in PDC 2003.
Oh, Office 12 was mentioned. Since use Office as a glorified Wordpad.exe, that's really not that exciting.
I am currently sitting in the session, "High Performance Computing with Windows Server Compute Cluster Solution." It's a slide-show fest, with very little demo or bits to see. This session is a basic review of of Clustering, and is mostly review if you are familiar with these concepts.
There's actually very little about what microsoft is doing in this space, and more about what high performance clustering actually is. I'm still not impressed. Show me, don't slide-show me.
Tue, 23 Aug 2005 12:27:00 GMT
Sometimes things that everyone just knows aren't always true.
For example, when I look at c# code, I know that local variables are stored on the stack. Or, I thought I knew that.
However, Ian Griffiths notes that in .NET 2.0, this isn't neccessarily true. This change was made to support anonymous methods, and mutability of variables in the enclosing scope.
Time for me to 'unlearn' a few things.
Fri, 19 Aug 2005 03:33:00 GMT
So, over the last couple of years, I've been what I consider a 'defender' of MSDN documentation. After all, the docs are miles ahead of what they used to be.
As a user, I've also taken to reporting doc bugs to Microsoft. My first couple of experiences with this was pretty good. In one case, the doc team actually rewrote an entire code example based on my feedback. I was impressed.
Recently, however, my experience has not been as good. Most of the doc bugs I report now get a generic response from a support specialist that is little more than a glorified secretary. It's obvious that these support people have no idea what I am talking about. They merely take the email, and forward it on to the appropriate team. When they get a response, they cut and paste that response in an email. It is frustrating, because these support people have no clue.
The other thing that I'm finding is a resistance to change. I've found a few examples where the docs were wrong, and I could prove it through experimentation. However, the product teams weren't really interested in correcting the issue. For example, although the documentation on debugging windows installer custom actions can be shown to be wrong, I received a 'brush off' response.
So, I'm making a new promise to myself.
Every time I find a doc bug in a Microsoft product, I'm going to post about it. I've tried to correct issues through appropriate channels, and it doesn't seem to work. Instead, I'll let google juice do the work for me.
Sun, 14 Aug 2005 03:23:00 GMT
An interesting meme I've encountered a number of times amongst c#/Java programmers:
Low Level c/_asm programmers are 'smarter' than the rest of us. Kernel hackers are psuedo-gods.
Ok, I've worked with a wide variety of programmers. Embedded programmers, Driver developers, EE/ board designers, Windows Programers, Unix 'hackers', Java devs, .NET developers, etc.
One of the things that I've noticed is that each developer usually considers his technology superior, and others just don't get it.
Actually, I think that they are all correct. Or all incredibly wrong. Take your pick.
Embedded Developers: Can hack 'C' like there's no tomorrow. Typically push memory around like it's ice cream. These developers excel at handling low- and out- of memory conditions. You'll rarely see an Embedded programmer fail to handle a memset failure (or new(), though that command would be rare..!). My experience is that C++ is a major leap for many embedded developers, and Classes are certainly a difficult construct to grasp.
Driver Developers: (See Embedded Developers)
EE / board designers: It always amuses me when EEs think that they can both design the board, and write software for it. Truthfully, you can be successful at one or the other, but not both. In my experience, EEs really don't understand the concept that the hardware actually does fail. Usual shortcomings amongst these developers include a failure to handle error conditions, and a lack of understanding of higher language constructs such as events, classes, and exceptions.
MFC/ATL/Windows developers: These programmers present an intimate knowledge of the arcane structures, return codes, and other esoteric knowledge required to leverage the Win32 API. Unfortunately, these engineers are too stuck in the middle - too high in the food chain to know everything that's going on underneath them, but too low to be able to excuse that lack of understanding.
Java Devs: They really wish that "Write-Once, Run Everwhere" had some actual truth. Oh, and real runtime supported generics would be nice. Hacks need not apply.
C# Developers: Trying to be more respectable than VB.NET, these developers strive to impress everyone else with there knowledge. Eschewing VB.NET's "My" namespace, most of these developers would rather pretend that they code everything on their own. However, the .NET GC is there to pick up the errant memory leacks that c# developers enevitably 'cast' aside.
VB.NET developers: I refuse to touch this flamebate language. I won't even say the obvious. Draw your own conclusions, please.
Unix 'hackers': Major flaw: There really is such a think as 'best tool for the job'. Sometimes that tool is *nix. Sometimes it really is Windows. /. Readers need not apply.
Did I miss your pet language? (LISP, Ruby, Python, PERL, Delphi, VB, PHP,......)
There was a reason for that - no one cares. Least of all, me.
Have a good language war - see you next Armageddon.
Thu, 11 Aug 2005 02:33:00 GMT
My new favorite line of code:
Thu, 11 Aug 2005 00:36:00 GMT
The PDC 'scheduler' is out - visit http://commnet.microsoftpdc.com.
My tentative schedule of breakout sessions:
11:45 AM - 12:30 PM - FUNL02 Lap around the WinFX and Win32 SDKs
1:00 PM - 2:15 PM - DAT301 High Performance Computing with the Windows Server Compute Cluster Solution
2:45 PM - 4:00 PM - DAT303 SQL Server 2005: Building Distributed, Asynchronous Database Applications with the Service Broker
4:15 PM - 5:30 PM -COM304 Networking: Building IPv6, Firewall, and IPsec Aware Applications
11:00 AM - 12:15 PM - COM200 Applications and Communications Roadmap: Platform Presents and Futures
12:30 PM - 1:15 PM - TLNL06 Tips & Tricks: Scrubing Source Code for Common Coding Mistakes (FxCop and PreFast)
1:45 PM - 3:00 PM - PRS309 Windows Presentation Foundation ("Avalon"): Overview of Windows Vista Graphics
3:15 PM - 4:30 PM - DAT411 SQL Server 2005: Extending and Embedding Integration Services (ETL)
5:00 PM - 6:15 PM - COM416 Windows Communications Foundation ("Indigo"): Under the Hood of the Windows Communications Foundation Channel Layer
10:00 AM - 11:15 PM - FUN412 Five Things Every Win32 Developer Should Know
11:45 AM - 12:45 PM - PRS320 ASP.NET: Future Directions for Developing Rich Web Applications with Atlas (Part 2)
1:00 PM - 1:45 PM - PRSL03 Ten Amazing Ways to Speech-Enable Your Application
2:15 PM - 3:30 PM - DAT219 BizTalk Server Future Directions: 2008 and Beyond
3:45 PM - 5:00 PM - COM325 Workflow + Messaging + Services: Developing Distributed Applications with Workflows
5:15 PM - 6:30 PM - FUN034 Windows Vista & "Longhorn" Server: Improving Reliability with the New System.Transactions Classes, File System and Registry Transactions
8:30 AM - 9:45 AM - COM430 Windows Communications Foundation ("Indigo"): A Deep Dive into Extensions for Security and Identity
Sat, 30 Jul 2005 02:32:00 GMT
Well, apparently my PDC letter worked.(image)
I'll be going to the 2005 PDC. See you there!
Tue, 26 Jul 2005 02:24:00 GMT
As my overlord and my unqualified mentor, I am terrified about presenting you with a self-serving opportunity to contribute to my overall geek cache. Whenever Microsoft gets excited they hold the PDC Conference where some of the world's most anxious-to-party developers gather to share and party.
Attending this conference will help me become more informed about upcoming Microsoft bad haircuts, so I can start using them in my job as soon as they're available. And it will help me become more informed about the future of the ubernerd industry. So whether you need my input in deciding what overpriced new hardware to purchase, or what features our next payment for your boat should include, I'll be a more unfireable cube jockey.
As you can ascertain this is a tremendous developer free-for-all that any savvy developer would have to be incarcerated to miss. Thank you very much for your unqualified consideration.
(Autogenerated from http://www.developerpowered.com/letter/form.html)
Fri, 08 Jul 2005 17:09:00 GMT
Credit goes to Michael Trantow for the following:
Undefined behavior getting you down? Now you can refine your searches to find that needle in the proverbial development haystack with the new Google tool, DennaNoogle.
Now in beta, the new search tool was developed and refined by Google in conjunction with longtime Googlist Jerry Dennany. Try the tool today. Better yet, for a limited time, get a free DennaNoogle demo by visiting Jerry in his Cuble. Dennany will also be presenting the seminal PodCast “Google Is Your Friend”, available soon for download on Kazaa, or you can pre-order the DVD at Netflix.
Thu, 07 Jul 2005 02:58:00 GMT
After all of these years, the Win32 API still does not provide a mechanism for updating a System or User Environment Variable.
Wed, 29 Jun 2005 16:50:00 GMT
One more complaint about Windows Installer DLL Custom Actions. If an entry point is not found, then Windows Installer generates a generic error, 1723. This is pretty much the same error for any problem running the DLL. As it's pretty simple for the Windows Installer team to know if the dll entry point wasn't found, why couldn't they generate a more explicit error message, or even log more detailed information in the MSI log?
It's quite obvious that Windows Installer was designed for internal product team by Microsoft, and then later released for general use. Who actually makes their HANDLE a typedef for an unsigned long? Windows Installer's MSIHANDLE, that's who.
Tue, 21 Jun 2005 22:41:00 GMT
This is more of a Windows Installer complaint, but since WiX and Windows Installer are so closely linked, I feel justified making these comments.
The mechanism that Windows Installer uses to call custom actions is convoluted, at best. A DLL embedded into an MSI is extracted at runtime to the %TEMP% directory, and dyamically renamed to a named format, "MSI???.tmp", where the string '???' is psuedo-randomly generated (There may be a pattern, but I haven't spent the time to figure it out.)
Why do I care? Well, I attempted to write an embedded DLL that used Managed Code. Specifically, I used C++ for the Windows Installer entry points, and then generated a c# module. This worked perfectly for my nUnit test suite, but my instalation kept failing when I tried this from Windows Installer. After quite some time debugging, I finally examined the fusion assembly binding logs. I discovered that even though the module is embedded in the same PE file as the original calling assembly, the .NET framework first looks for a '.module' file external to the existing assembly. If it fails to find that '.module', it then examines itself to find the referenced .net module. However, a small 'feature' is that it does this by looking for it's own original assembly name. The problem I experienced is that Windows Installer has renamed my assembly to the MSI???.tmp name, so I would get assembly/module load issues.
So, that went down the drain. As I am relatively C++ averse (especially Managed C++, though C++/CLI is supposed to rescue me from the ugliness that is MC++), I really preferred to write my CAs in C#.
My solution was to write an external assembly that I install into the GAC. Now, my c++ "interop" layer can always find the C# custom actions, as they are globally installed. That does leave me with a chicken and egg problem, though. How do I get my c# custom actions into the GAC before I call them? After all, I must have them available during my UI sequence, and I can't wait until intallation time for my CA's to be present.
I followed the same solution that InstallShield uses for there "InstallSheild Engine", idriver.exe. I wrote a mini-install to place my c# assembly into the GAC, and perform this installation at the beginning of my 'main' installer.
This is a lot of effort to get around the fact that the Windows Installer team hasn't really considered managed custom actions, although the user groups are teaming with such requests. Even though all future versions of the operating system will guarantee a managed framework, the WI team still insists on C++ as the custom code 'story' for windows installations. (Please don't tell me that Windows Installer supports vbscript. We all know this. We also should know that vbscript is very, very bad in installation situations).
So, a long story short - managed CAs are possible, but not easy. I'll post more on my Managed CA adventure soon.
Fri, 17 Jun 2005 01:27:00 GMT
Despite the lack of blog posts, I've really been immersed in WiX. I'm learning to love and hate this project. I really love the XML declarative style of creating MSIs. I really hate the fact that WiX wraps the abomination known as Windows installer.
I was very familiar with Windows Installer before this adventure. I've spent years working with both InstallShield and Wise, and have had to edit raw MSI tables many times in the past. However, these commercial products do a much better job of hiding the complexities of Windows Installer than WiX does. If you decide to adopt WiX, make certain that you have the time to invest in learning the ins and outs of the MSI SDK. Even if you think that you know quite a bit in this area, be prepared to spend some time spelunking through MSI.chm.
Writing Custom Actions using WiX isn't a walk in the park, either. I'll talk a bit about managed CAs a bit in the future. Hopefully people will learn from my trials and tribulations in this area.
Just to make sure that this blog entry has some substance, I'm going to give my WiX UI tip of the day:
To define the default font, place the following in the
<TextStyle Id="Tahoma8" FaceName="Tahoma" Size="8" />