Subscribe: B# .NET Blog
http://blogs.bartdesmet.net/blogs/bart/atom.aspx
Added By: Feedage Forager Feedage Grade B rated
Language: English
Tags:
active directory  blog  call  clr  code  directory  lds  linq  net  powershell  side effects  side  system  windows powershell   
Rate this Feed
Rate this feedRate this feedRate this feedRate this feedRate this feed
Rate this feed 1 starRate this feed 2 starRate this feed 3 starRate this feed 4 starRate this feed 5 star

Comments (0)

Feed Details and Statistics Feed Statistics
Preview: B# .NET Blog

B# .NET Blog



Bart De Smet's on-line blog (0x2B | ~0x2B, that's the question)



Updated: 2010-01-02T08:33:31Z

 



Getting Started with Active Directory Lightweight Directory Services

2010-07-13T01:52:32Z

Introduction In preparation for some upcoming posts related to LINQ (what else?), Windows PowerShell and Rx, I had to set up a local LDAP-capable directory service. (Hint: It will pay off to read till the very end of the post if you’re wondering what I’m up to...) In this post I’ll walk the reader through the installation, configuration and use of Active Directory Lightweight Directory Services (LDS), formerly known as Active Directory Application Mode (ADAM). Having used the technology several years ago, in relation to the LINQ to Active Directory project (which as an extension to this blog series will receive an update), it was a warm and welcome reencounter.   What’s Lightweight Directory Services anyway? Use of hierarchical storage and auxiliary services provided by technologies like Active Directory often has advantages over alternative designs, e.g. using a relational database. For example, user accounts may be stored in a directory service for an application to make use of. While Active Directory seems the natural habitat to store (and replicate, secure, etc.) additional user information, IT admins will likely point you – the poor developer – at the door when asking to extend the schema. That’s one of the places where LDS comes in, offering the ability to take advantage of the programming model of directory services while keeping your hands off “the one and only AD schema”. The LDS website quotes other use cases, which I’ll just copy here verbatim: Active Directory Lightweight Directory Service (AD LDS), formerly known as Active Directory Application Mode, can be used to provide directory services for directory-enabled applications. Instead of using your organization’s AD DS database to store the directory-enabled application data, AD LDS can be used to store the data. AD LDS can be used in conjunction with AD DS so that you can have a central location for security accounts (AD DS) and another location to support the application configuration and directory data (AD LDS). Using AD LDS, you can reduce the overhead associated with Active Directory replication, you do not have to extend the Active Directory schema to support the application, and you can partition the directory structure so that the AD LDS service is only deployed to the servers that need to support the directory-enabled application. Install from Media Generation. The ability to create installation media for AD LDS by using Ntdsutil.exe or Dsdbutil.exe. Auditing. Auditing of changed values within the directory service. Database Mounting Tool. Gives you the ability to view data within snapshots of the database files. Active Directory Sites and Services Support. Gives you the ability to use Active Directory Sites and Services to manage the replication of the AD LDS data changes. Dynamic List of LDIF files. With this feature, you can associate custom LDIF files with the existing default LDIF files used for setup of AD LDS on a server. Recursive Linked-Attribute Queries. LDAP queries can follow nested attribute links to determine additional attribute properties, such as group memberships. Obviously that last bullet point grabs my attention through I will retain myself from digressing here.   Getting started If you’re running Windows 7, the following explanation is the right one for you. For older versions of the operating system, things are pretty similar though different downloads will have to be used. For Windows Server 2008, a server role exists for LDS. So, assuming you’re on Windows 7, start by downloading the installation media over here. After installing this, you should find an entry “Active Directory Lightweight Directory Services Setup Wizard” under the “Administrative Tools” section in “Control Panel”: LDS allows you to install multiple instances of directory services on the same machine, just like SQL Server allows multiple server instances to co-exist. Each instance has a[...]



The Case of The Failed Demo – StackOverflowException on x64

2010-07-07T09:34:57Z

Introduction A while ago I was explaining runtime mechanisms like the stack and the heap to some folks. (As an aside, I’m writing a debugger course on “Advanced .NET Debugging with WinDbg with SOS”, which is an ongoing project. Time will tell when it’s ready to hit the streets.) Since the context was functional programming where recursion is a typical substitute (or fuel if you will) for loops, an obvious topic for discussion is the possibility to hit a stack overflow. Armed with my favorite editor, Notepad.exe, and the C# command-line compiler, I quickly entered the following sample to show “looping with recursion” and how disaster can strike: using System; class Program {     static void Main()     {         Rec(0);     }     static void Rec(int n)     {         if (n % 1024 == 0)             Console.WriteLine(n);         Rec(n + 1);     } } The module-based condition in there is to avoid excessive slowdowns due to Console.WriteLine use, which is rather slow due to the way the Win32 console output system works. To my initial surprise, the overflow didn’t come anywhere in sight and the application kept running happily: I rather expected something along the following lines: So, what’s going on here? Though I realized pretty quickly what the root cause is of this unexpected good behavior, I’ll walk the reader through the thought process used to “debug” the application’s code.   I made a call, didn’t I? The first thing to check is that we really are making a recursive call in our Rec method. Obviously ildasm is the way to go to inspect that kind of stuff, so here’s the output which we did expect. In fact, the statement made above – “which we did expect” – is debatable. Couldn’t the compiler just turn the call into a jump right to the start of the method after messing around a bit with the local argument slot that holds argument value n? That way we wouldn’t have to make a call and the code would still work as expected. Essentially what we’re saying here is that the compiler could have turned the recursive call into a loop construct. And indeed, some compilers do exactly that. For example, consider the following F# sample: #light let rec Rec n =    if n % 1024 = 0 then        printfn "%d" n    Rec (n + 1) Rec 0 Notice the explicit indication of the recursive nature of a function by means of the “rec” keyword. After compiling this piece of code using fsc.exe, the following code is shown in Reflector (decompiling to C# syntax) for the Rec function: The mechanics of the printf call are irrelevant. What matters is the code that’s executed after the n++ statement, which isn’t a recursive call to Rec itself. Instead, the compiler has figured out a loop can be used. Hence, no StackOverflowException will result. Back to the C# sample though. What did protect the code from overflowing the stack? Let’s have some further investigations, but first … some background.   Tail calls One optimization that can be carried out for recursive functions is to spot tail calls and optimize them away into looping – or at a lower level, jumps – constructs. A tail call is basically a call after which the current stack frame is no longer needed upon return from the call. For example, our simple sample can benefit from tail call optimization since the Rec method doesn’t really do anything anymore after returning from the recursive Rec call: static void Rec(int n) {     if (n % 1024 == 0)         Console.WriteLine(n);     R[...]



Hosting Windows PowerShell 2.0 under CLR 4.0

2010-07-06T19:38:51Z

Introduction Recently I’ve been playing with Windows PowerShell 2.0 again, in the context of my day-to-day activities. One hint should suffice for the reader to get an idea of what’s going on: push-based collections. While I’ll follow up on this subject pretty soon, this precursor post explains one of the things I had to work around.   PowerShell: a managed application or not? Being designed around the concept of managed object pipelines, one may expect powershell.exe to be a managed executable. However, it turns out this isn’t the case completely. If you try to run ildasm.exe on the PowerShell executable (which lives in %windir%\system32\WindowsPowerShell\v1.0 despite the 2.0 version number, due to setup complications), you get the following message: So much for the managed executable theory. What else can be going on to give PowerShell the power of managed objects. Well, it could be hosting the CLR. To check this theory, we can use the dumpbin.exe tool, using the /imports flag, checking for mscoree.dll functions being called. And indeed, we encounter the CorBindToRuntimeEx function that’s been the way to host the CLR prior to .NET 4’s in-process side-by-side introduction (a feature I should blog about as well since I wrote a CLR host for in-process side-by-side testing on my prior team here at Microsoft). One of the parameters passed to CorBindToRuntimeEx is the version of the CLR to be loaded. Geeks can use WinDbg or cdb to set a breakpoint on this function and investigate the version parameter passed to it by the PowerShell code: Notice the old code name of PowerShell still being revealed in the third stack frame (from the top). In order to hit this breakpoint on a machine that has .NET 4 installed, I’ve used the mscoreei.dll module rather than mscoree.dll. The latter has become a super-shim in the System32 folder, while the former one is where the CLR shim really lives (“i” stands for “implementation”). This refactoring has been done to aid in servicing the CLR on different version of Windows, where the operating system “owns” the files in the System32 folder. Based on this experiment, it’s crystal clear the CLR is hosted by Windows PowerShell, with hardcoded affinity to v2.0.50727. This is in fact a good thing since automatic roll-forward to whatever the latest version of the CLR is on the machine could cause incompatibilities. One can expect future versions of Windows PowerShell to be based on more recent versions of the CLR, once all required testing has been carried out. (And in that case, one will likely use the new “metahost” CLR hosting APIs.)   Loading .NET v4 code in PowerShell v2.0 The obvious question with regards to some of the stuff I’ve been working on was whether or not we can run .NET v4 code in Windows PowerShell v2.0? It shouldn’t be a surprise this won’t work as-is, since the v2.0 CLR is loaded by the PowerShell host. Even if the hosting APIs weren’t involved and the managed executable were compiled against .NET v2.0, that version’s CLR would take precedence. This is in fact the case for ISE: Trying to load a v4.0 assembly in Windows PowerShell v2.0 pathetically fails – as expected – with the following message: So, what are the options to get this to work? Let’s have a look. Warning:  None of those hacks are officially supported. At this point, Windows PowerShell is a CLR 2.0 application, capable of loading and executing code targeting .NET 2.0 through .NET 3.5 SP1 (all of which run on the second major version of the CLR).   Option 1 – Hacking the parameter passed to CorBindToRuntimeEx If we just need an ad-hoc test of Windows PowerShell v2.0 running on CLR v4.0, we can take advantage of WinDbg once more. Simply break on the CorBindToRuntimeEx and replace the v2.0.50727 string in memory by the v4.0 version, i.e. v4.0.30319. The “eu” command used for this purpose stands for “edit memory Unicode”: If we let go the debugger after th[...]



Where’s Bart’s Blog Been?

2010-07-02T07:19:44Z

A quick update to my readers on a few little subjects. First of all, some people have noticed my blog welcomed readers with a not-so-sweet 404 error message the last few days. Turned out my monthly bandwidth was exceeded which was enough reason for my hosting provider to take the thing offline.

(image)

Since this is quite inconvenient I’ve started some migration of image content to another domain, which is work in progress and should (hopefully) prevent the issue from occurring again. Other measures will be taken to limit the download volumes.

Secondly, many others have noticed it’s been quite silent on my blog lately. As my colleague Wes warned me, once you start enjoying every day of functional programming hacking on Erik’s team, time for blogging steadily decreases. What we call “hacking” has been applied to many projects we’ve been working on over here in the Cloud Programmability Team, some of which are yet undisclosed. The most visible one today is obviously the Reactive Extensions both for .NET and for JavaScript, which I’ve been evangelizing both within and outside the company. Another one which I can only give the name for is dubbed “LINQ to Anything” that’s – as you can imagine – keeping me busy and inspired on a daily and nightly basis. On top of all of this, I’ve got some other writing projects going on that are nearing completion (finally).

Anyway, the big plan is to break the silence and start blogging again about our established technologies, including Rx in all its glory. Subjects will include continuation passing style, duality between IEnumerable and IObservable, parameterization for concurrency, discussion of the plethora of operators available, a good portion of monads for sure, the IQbservable interface (no, I won’t discuss the color of the bikeshed) and one of its applications (LINQ to WMI Events), etc. Stay tuned for a series on those subjects starting in the hopefully very near future.

See you soon!

(image)



Some introductory Rx samples

2010-04-19T07:45:00Z

During my last tour I’ve been collecting quite some fundamental and introductory Rx samples as illustrations with my presentations on the topic. As promised, I’m sharing those out through my blog. More Rx content is to follow in the (hopefully near) future, with an exhaustive discussion of various design principles and choices, the underlying theoretical foundation of Rx and coverage of lots of operators.

In the meantime, download the sample project here. While the project targets Visual Studio 2010 RTM, you can simply take the Program.cs file and build a Visual Studio 2008 project around it, referencing the necessary Rx assemblies (which you can download from DevLabs).

Enjoy!

(image)



New drop of the Reactive Extensions for .NET (Rx) available

2010-03-05T23:02:00Z

It's been a long time I've written epic blog posts over here, but for a good reason. We've been working very hard on getting a new Rx release out the door and I'm proud to announce it's available now through http://msdn.microsoft.com/en-us/devlabs/ee794896.aspx. Notice we got a .NET 4 RC compatible download available as well, so you can play with the latest and greatest of technologies in one big jar :-). More goodness will follow later, so stay tuned!

At some point in the foreseeable future, I'll start a series on how Rx works and what its operators are as well. If you have any particular topics you'd like to see covered, don't hesitate to let me know through my blog. In the meantime, make sure to evaporate all your feedback on the forums at http://social.msdn.microsoft.com/Forums/en-US/rx/threads. We love to hear what you think, what operators you believe are missing, any bugs you find, etc.

Update: We also have published a video on the new release at http://channel9.msdn.com/posts/J.Van.Gogh/Your-RxNET-Prescription-Has-Been-Refilled.

Have fun!
Bart @ Rx

(image)



2010 – A Personal Change: putting my “Head In The Cloud”

2010-01-11T01:26:00Z

Slightly over two years after arriving here in Redmond to work on the WPF team, time has come for me to make a switch and pursue other opportunities within the company. Starting January 13th, I’ll be working on the SQL Cloud Data Programmability Team on various projects related to democratizing the cloud. While we have much more rabbits sitting in our magician hats, Rx is the first big deliverable we’re working on.

(image)

For my blog, there won’t be much change as I’ve always written on topics related to what I’ll be working on: language innovation, data access, LINQ, type systems, lambda fun, etc. I’m planning to stay committed to blogging and other evangelism activities, including speaking engagements from time to time, so feel free to ping me if I’m in your proximity (or if you’re visiting our campus). Next up and confirmed are TechDays “low lands” in Belgium and the Netherlands, end of March.

Needless to say, I’m thrilled to have this opportunity of working together with a relatively small group of smart and passionate people, on the things I’d spend all my free time on anyway. Having this one-to-one alignment between day-to-day professional activities at work and all sorts of free time hacking projects is like a dream coming true. Thanks Danny, Erik, Jeffrey, Mark and Wes for taking me on board.

Expect to see more Rx blogging love over here, and watch out for more goodness to come your way in the foreseeable future. In the meantime, check out the following resources on the matter:

Please keep the feedback on Rx coming: help us, help you!

(image)



More LINQ with System.Interactive – Functional fun and taming side-effects

2010-01-07T11:43:42Z

With the recent release of the Reactive Extensions for .NET (Rx) on DevLabs, you’ll hear quite a bit about reactive programming, based on the IObservable and IObserver interfaces. A great amount of resources is available on Channel 9. In this series, I’ll focus on the dual of the System.Reactive assembly, which is System.Interactive, providing a bunch of extensions to the LINQ Standard Query Operators for IEnumerable. In today’s installment we’ll talk about EnumerableEx’s facilities to tame side-effects in a functionally inspired manner:   To side effect or not to side effect? Being rooted in query comprehensions as seen in various functional programming languages (including (the) pure one(s)), one would expect LINQ to have a very functional basis. Indeed it has, but being hosted in various not functionally pure languages like C# and Visual Basic, odds are off reasoning about side-effects in a meaningful and doable manner. As we’ve seen before, when talking about the Do and Run operators, it’s perfectly possible for a query to exhibit side-effects during iteration. You don’t even have to look that far, since every lambda passed to a query operator is an opportunity of introducing effects. The delayed execution nature of LINQ queries makes that those effects appear at the point of query execution. So far, nothing new. So, the philosophical question ought to be whether or not we should embrace side-effects or go for absolute purity. While the latter would be preferable for various reasons, it’s not enforceable through the hosting languages for LINQ, so maybe we should exploit side-effects if we really want to do so. The flip side of this train of thought is that those side-effects could come and get us if we’re not careful, especially when queries get executed multiple times, potentially as part of a bigger query. In such a case, you’d likely not want effects to be duplicated. Below is a sample of such a problematic query expression: var xrs = EnumerableEx.Generate(new Random(), rnd => EnumerableEx.Return(rnd.Next(100)), /* iterate */ _ => _); xrs.Zip(xrs, (l, r) => l + r).Take(10).Run(Console.WriteLine); Using Generate, we generate a sequence of random numbers. Recall the first argument is the state of the anamorphic Generate operator, which we get passed in the lambdas following it: once to produce an output sequence (just a single random number in our case) and once to iterate (just keeping the same random number generator here). What’s more important is we’re relying on the side-effect of reading the random number generator which, as the name implies, provides random answers to the Next inquiry every time it gets called. In essence, the side-effect can (not) be seen by looking at the signature of Random.Next, which says it returns an int. In .NET this means the method may return the same int every time it gets called, but there are no guarantees whatsoever (as there would be in pure functional programming languages). This side-effect, innocent and intentional as it may seem, comes and gets us if we perform a Zip on the sequence with itself. Since Zip iterates both sides, we’re really triggering separate enumeration (“GetEnumerator”) over the same sequence two times. Though it’s the same sequence object, each of its iterations will produce different results. As a result, the expected invariant of the Zip’s output being only even numbers (based on the assumption l and r would be the same as they’re produced by the same sequence) doesn’t hold: 52 114 112 103 41 135 78 114 59 137 While random number generation is a pretty innocent side-effect, not having it under control properly can lead to unexpected results as shown above. We can visualize this nicely using another side-effect introduce[...]



Top 9 Posts from 2009

2010-01-03T09:08:39Z

select top 9 [Subject] from dbo.cs_Postswhere postlevel = 1 and usertime < '01/01/2010' and usertime >= '01/01/2009'order by TotalViews desc Forgive me for the classic SQL, but here are the results with some short annotations inline: (Mis)using C# 4.0 Dynamic – Type-Free Lambda Calculus, Church Numerals, and more Uses the new C# 4.0 dynamic feature to implement the type-free lambda calculus consisting of an abstraction and application operator. Besides talking about the fundamentals of lambda calculus, this post shows how to implement the SKI combinators and Church Booleans, Church numerals and even recursive functions.   LINQ to Ducks – Bringing Back The Duck-Typed foreach Statement To LINQ Since LINQ to Objects is layered on top of IEnumerable, it doesn’t work against objects that just happen to implement the enumeration pattern consisting of GetEnumerator, MoveNext and Current. Since the foreach statement actually does work against such data sources, we bring back this duck typing to LINQ using AsDuckEnumerable().   Type-Free Lambda Calculus in C#, Pre-4.0 – Defining the Lambda Language Runtime (LLR) We repeat the exercise of the first blog post but now without C# 4.0 dynamic features., encoding application and abstraction operators using none less that exceptions. Those primitives define what I call the Lambda Language Runtime (LLR), which we use subsequently to implement a bunch of samples similar to the ones in the first post.   Taming Your Sequence’s Side-Effects Through IEnumerable.Let Enumerable sequences can exhibit side-effects for various reasons ranging from side-effecting filter predicates to iterators with side-effecting imperative code interwoven in them. The Let operator introduced in this post helps you to keep those side-effects under control when multiple “stable” enumerations over the sequence are needed.   Statement Trees With Less Pain – Follow-Up on System.Linq.Expressions v4.0 The introduction of the DLR in the .NET 4 release brings us not only dynamic typing but also full-fledged statement trees as an upgrade to the existing LINQ expression trees. Here we realize a prime number generator using statement trees and runtime compilation, reusing expression trees emitted by the C# compiler where possible.   LINQ to Z3 – Theorem Solving on Steroids – Part 1 LINQifying Microsoft Research’s Z3 theorem solver has been one of my long-running side-projects. This most recent write-up on the matter illustrates the concept of a LINQ-enabled Theorem and the required visitor implementation to interop with the Z3 libraries. Finally, we show a Sudoku and Kakuro solver expressed in LINQ.   Expression Trees, Take Two – Introducing System.Linq.Expressions v4.0 Just like post 5, we have a look at the .NET 4 expression tree support, now including statement trees. Besides pointing out the new tree node types, we show dynamic compilation and inspect the generated IL code using the SOS debugger’s dumpil command. In post 5, we follow up by showing how to reuse C# 3.0 expression tree support.   Unlambda .NET – With a Big Dose of C# 3.0 Lambdas Esoteric programming languages are good topics for Crazy Sundays posts. In this one we had a look at how to implement the Unlambda language – based on SKI combinators and with “little” complications like Call/CC – using C# 3.0 with lots of lambda expressions. To satisfy our curiosity, we run a Fibonacci sample program.   C# 4.0 Feature Focus – Part 4 – Co- and Contra-Variance for Generic Delegate and Interface Types Generic co- and contra-variance is m[...]



LINQSQO v4.0 and MinLINQ v1.0 Now Available for Download

2010-01-02T16:33:31Z

Introduced in my previous blog post on The Essence of LINQ – MinLINQ, the first release of this project is now available for reference at the LINQSQO CodePlex website at http://linqsqo.codeplex.com. Compared to the write-up over here in my previous post, there are a few small differences and caveats:

  • Only FEnumerable functionality is available currently; the FObservable dual may follow later.
  • Option has been renamed to Maybe, to be CLS compliant and avoid clashes with the VB keyword.
  • Some operators are not provided, in particular GroupBy, GroupJoin and Join. They’re left as an exercise.
  • A few operator implementations are categorized as “cheaters” since they roundtrip through System.Linq.
  • Don’t nag about performance. The MinLINQ code base is by no means optimal and so be it.
  • Very few System.Interactive operators are included since those often require extra foundations (such as concurrency).

A few highlights:

  • FEnumerable.Essentials.cs is where the fun starts. Here the three primitives – Ana, Bind and Cata – form the ABC of LINQ.
  • There’s a Naturals() constructor function generating an infinite sequence of natural numbers, used in operators that use indexes.
  • OrderBy and ThenBy are supported through roundtripping to System.Linq with a handy trick to keep track of IOrderedEnumerable.
  • As a sample, I’ve included Luke Hoban’s LINQified RayTracer with AsFEnumerable and AsEnumerable roundtripping. It works just fine.
  • Creating an architectural diagram in Visual Studio 2010 yields the following result (not meant to zoomed in), where I’ve used the following colors:
    • Green = Ana
    • Blue = Bind
    • Red = Cata

(image)

Obviously, all sorts of warnings apply. People familiar to my blog adventures will know this already, but just in case:

//
// This project is meant as an illustration of how an academically satifying layering
// of a LINQ to Objects implementation can be realized using monadic concepts and only
// three primitives: anamorphism, bind and catamorphism.
//
// The code in this project is not meant to be used in production and no guarantees are
// made about its functionality. Use it for academic stimulation purposes only. To use
// LINQ for real, use System.Linq in .NET 3.5 or higher.
//
// All of the source code may be used in presentations of LINQ or for other educational
// purposes, but references to http://www.codeplex.com/LINQSQO and the blog post referred
// to above - "The Essence of LINQ - MinLINQ" - are required.
//

Either way, if you find LINQ interesting and can stand some “brain pain of the highest quality” (a Twitter quote by dahlbyk), this will likely be something for you.

(image)