itramblings

Ramblings from an IT manager and long time developer.

By

Throttling asynchronous tasks

This is copied from this Stack Overflow Post

I would like to run a bunch of async tasks, with a limit on how many tasks may be pending completion at any given time.

Say you have 1000 URLs, and you only want to have 50 requests open at a time; but as soon as one request completes, you open up a connection to the next URL in the list. That way, there are always exactly 50 connections open at a time, until the URL list is exhausted.

I also want to utilize a given number of threads if possible.

I came up with an extension method, ThrottleTasksAsync that does what I want. Is there a simpler solution already out there? I would assume that this is a common scenario.

Usage:

class Program
{
    static void Main(string[] args)
    {
        Enumerable.Range(1, 10).ThrottleTasksAsync(5, 2, async i => { Console.WriteLine(i); return i; }).Wait();

        Console.WriteLine("Press a key to exit...");
        Console.ReadKey(true);
    }
}

Here is the code:

public static class EnumerableExtensions
{
    public static async Task<Result_T[]> ThrottleTasksAsync<Enumerable_T, Result_T>(this IEnumerable<Enumerable_T> enumerable, int maxConcurrentTasks, int maxDegreeOfParallelism, Func<Enumerable_T, Task<Result_T>> taskToRun)
    {
        var blockingQueue = new BlockingCollection<Enumerable_T>(new ConcurrentBag<Enumerable_T>());

        var semaphore = new SemaphoreSlim(maxConcurrentTasks);

        // Run the throttler on a separate thread.
        var t = Task.Run(() =>
        {
            foreach (var item in enumerable)
            {
                // Wait for the semaphore
                semaphore.Wait();
                blockingQueue.Add(item);
            }

            blockingQueue.CompleteAdding();
        });

        var taskList = new List<Task<Result_T>>();

        Parallel.ForEach(IterateUntilTrue(() => blockingQueue.IsCompleted), new ParallelOptions { MaxDegreeOfParallelism = maxDegreeOfParallelism },
        _ =>
        {
            Enumerable_T item;

            if (blockingQueue.TryTake(out item, 100))
            {
                taskList.Add(
                    // Run the task
                    taskToRun(item)
                    .ContinueWith(tsk =>
                        {
                            // For effect
                            Thread.Sleep(2000);

                            // Release the semaphore
                            semaphore.Release();

                            return tsk.Result;
                        }
                    )
                );
            }
        });

        // Await all the tasks.
        return await Task.WhenAll(taskList);
    }

    static IEnumerable<bool> IterateUntilTrue(Func<bool> condition)
    {
        while (!condition()) yield return true;
    }
}

The method utilizes BlockingCollection and SemaphoreSlim to make it work. The throttler is run on one thread, and all the async tasks are run on the other thread. To achieve parallelism, I added a maxDegreeOfParallelism parameter that’s passed to a Parallel.ForEach loop re-purposed as a while loop.

Bonus: To get around the problem in BlockingCollection where an exception is thrown in Take() when CompleteAdding() is called, I’m using the TryTake overload with a timeout. If I didn’t use the timeout in TryTake, it would defeat the purpose of using a BlockingCollection since TryTake won’t block. Is there a better way? Ideally, there would be a TakeAsync method.

By

Async/Await – Best Practices in Asynchronous Programming

By Stephen Cleary | March 2013 (repost from here)

These days there’s a wealth of information about the new async and await support in the Microsoft .NET Framework 4.5. This article is intended as a “second step” in learning asynchronous programming; I assume that you’ve read at least one introductory article about it. This article presents nothing new, as the same advice can be found online in sources such as Stack Overflow, MSDN forums and the async/await FAQ. This article just highlights a few best practices that can get lost in the avalanche of available documentation.

The best practices in this article are more what you’d call “guidelines” than actual rules. There are exceptions to each of these guidelines. I’ll explain the reasoning behind each guideline so that it’s clear when it does and does not apply. The guidelines are summarized in Figure 1; I’ll discuss each in the following sections.

Figure 1 Summary of Asynchronous Programming Guidelines

Name Description Exceptions
Avoid async void Prefer async Task methods over async void methods Event handlers
Async all the way Don’t mix blocking and async code Console main method
Configure context Use ConfigureAwait(false) when you can Methods that require con­text

Avoid Async Void

There are three possible return types for async methods: Task, Task<T> and void, but the natural return types for async methods are just Task and Task<T>. When converting from synchronous to asynchronous code, any method returning a type T becomes an async method returning Task<T>, and any method returning void becomes an async method returning Task. The following code snippet illustrates a synchronous void-returning method and its asynchronous equivalent:

void MyMethod()
{
  // Do synchronous work.
  Thread.Sleep(1000);
}
async Task MyMethodAsync()
{
  // Do asynchronous work.
  await Task.Delay(1000);
}

Void-returning async methods have a specific purpose: to make asynchronous event handlers possible. It is possible to have an event handler that returns some actual type, but that doesn’t work well with the language; invoking an event handler that returns a type is very awkward, and the notion of an event handler actually returning something doesn’t make much sense. Event handlers naturally return void, so async methods return void so that you can have an asynchronous event handler. However, some semantics of an async void method are subtly different than the semantics of an async Task or async Task<T> method.

Async void methods have different error-handling semantics. When an exception is thrown out of an async Task or async Task<T> method, that exception is captured and placed on the Task object. With async void methods, there is no Task object, so any exceptions thrown out of an async void method will be raised directly on the SynchronizationContext that was active when the async void method started. Figure 2 illustrates that exceptions thrown from async void methods can’t be caught naturally.

Figure 2 Exceptions from an Async Void Method Can’t Be Caught with Catch
private async void ThrowExceptionAsync()
{
  throw new InvalidOperationException();
}
public void AsyncVoidExceptions_CannotBeCaughtByCatch()
{
  try
  {
    ThrowExceptionAsync();
  }
  catch (Exception)
  {
    // The exception is never caught here!
    throw;
  }
}

These exceptions can be observed using AppDomain.UnhandledException or a similar catch-all event for GUI/ASP.NET applications, but using those events for regular exception handling is a recipe for unmaintainability.

Async void methods have different composing semantics. Async methods returning Task or Task<T> can be easily composed using await, Task.WhenAny, Task.WhenAll and so on. Async methods returning void don’t provide an easy way to notify the calling code that they’ve completed. It’s easy to start several async void methods, but it’s not easy to determine when they’ve finished. Async void methods will notify their SynchronizationContext when they start and finish, but a custom SynchronizationContext is a complex solution for regular application code.

Async void methods are difficult to test. Because of the differences in error handling and composing, it’s difficult to write unit tests that call async void methods. The MSTest asynchronous testing support only works for async methods returning Task or Task<T>. It’s possible to install a SynchronizationContext that detects when all async void methods have completed and collects any exceptions, but it’s much easier to just make the async void methods return Task instead.

It’s clear that async void methods have several disadvantages compared to async Task methods, but they’re quite useful in one particular case: asynchronous event handlers. The differences in semantics make sense for asynchronous event handlers. They raise their exceptions directly on the SynchronizationContext, which is similar to how synchronous event handlers behave. Synchronous event handlers are usually private, so they can’t be composed or directly tested. An approach I like to take is to minimize the code in my asynchronous event handler—for example, have it await an async Task method that contains the actual logic. The following code illustrates this approach, using async void methods for event handlers without sacrificing testability:

private async void button1_Click(object sender, EventArgs e)
{
  await Button1ClickAsync();
}
public async Task Button1ClickAsync()
{
  // Do asynchronous work.
  await Task.Delay(1000);
}

Async void methods can wreak havoc if the caller isn’t expecting them to be async. When the return type is Task, the caller knows it’s dealing with a future operation; when the return type is void, the caller might assume the method is complete by the time it returns. This problem can crop up in many unexpected ways. It’s usually wrong to provide an async implementation (or override) of a void-returning method on an interface (or base class). Some events also assume that their handlers are complete when they return. One subtle trap is passing an async lambda to a method taking an Action parameter; in this case, the async lambda returns void and inherits all the problems of async void methods. As a general rule, async lambdas should only be used if they’re converted to a delegate type that returns Task (for example, Func<Task>).

To summarize this first guideline, you should prefer async Task to async void. Async Task methods enable easier error-handling, composability and testability. The exception to this guideline is asynchronous event handlers, which must return void. This exception includes methods that are logically event handlers even if they’re not literally event handlers (for example, ICommand.Execute implementations).

Async All the Way

Asynchronous code reminds me of the story of a fellow who mentioned that the world was suspended in space and was immediately challenged by an elderly lady claiming that the world rested on the back of a giant turtle. When the man enquired what the turtle was standing on, the lady replied, “You’re very clever, young man, but it’s turtles all the way down!” As you convert synchronous code to asynchronous code, you’ll find that it works best if asynchronous code calls and is called by other asynchronous code—all the way down (or “up,” if you prefer). Others have also noticed the spreading behavior of asynchronous programming and have called it “contagious” or compared it to a zombie virus. Whether turtles or zombies, it’s definitely true that asynchronous code tends to drive surrounding code to also be asynchronous. This behavior is inherent in all types of asynchronous programming, not just the new async/await keywords.

“Async all the way” means that you shouldn’t mix synchronous and asynchronous code without carefully considering the consequences. In particular, it’s usually a bad idea to block on async code by calling Task.Wait or Task.Result. This is an especially common problem for programmers who are “dipping their toes” into asynchronous programming, converting just a small part of their application and wrapping it in a synchronous API so the rest of the application is isolated from the changes. Unfortunately, they run into problems with deadlocks. After answering many async-related questions on the MSDN forums, Stack Overflow and e-mail, I can say this is by far the most-asked question by async newcomers once they learn the basics: “Why does my partially async code deadlock?”

Figure 3 shows a simple example where one method blocks on the result of an async method. This code will work just fine in a console application but will deadlock when called from a GUI or ASP.NET context. This behavior can be confusing, especially considering that stepping through the debugger implies that it’s the await that never completes. The actual cause of the deadlock is further up the call stack when Task.Wait is called.

Figure 3 A Common Deadlock Problem When Blocking on Async Code
public static class DeadlockDemo
{
  private static async Task DelayAsync()
  {
    await Task.Delay(1000);
  }
  // This method causes a deadlock when called in a GUI or ASP.NET context.
  public static void Test()
  {
    // Start the delay.
    var delayTask = DelayAsync();
    // Wait for the delay to complete.
    delayTask.Wait();
  }
}

The root cause of this deadlock is due to the way await handles contexts. By default, when an incomplete Task is awaited, the current “context” is captured and used to resume the method when the Task completes. This “context” is the current SynchronizationContext unless it’s null, in which case it’s the current TaskScheduler. GUI and ASP.NET applications have a SynchronizationContext that permits only one chunk of code to run at a time. When the await completes, it attempts to execute the remainder of the async method within the captured context. But that context already has a thread in it, which is (synchronously) waiting for the async method to complete. They’re each waiting for the other, causing a deadlock.

Note that console applications don’t cause this deadlock. They have a thread pool SynchronizationContext instead of a one-chunk-at-a-time SynchronizationContext, so when the await completes, it schedules the remainder of the async method on a thread pool thread. The method is able to complete, which completes its returned task, and there’s no deadlock. This difference in behavior can be confusing when programmers write a test console program, observe the partially async code work as expected, and then move the same code into a GUI or ASP.NET application, where it deadlocks.

The best solution to this problem is to allow async code to grow naturally through the codebase. If you follow this solution, you’ll see async code expand to its entry point, usually an event handler or controller action. Console applications can’t follow this solution fully because the Main method can’t be async. If the Main method were async, it could return before it completed, causing the program to end. Figure 4 demonstrates this exception to the guideline: The Main method for a console application is one of the few situations where code may block on an asynchronous method.

Figure 4 The Main Method May Call Task.Wait or Task.Result
class Program
{
  static void Main()
  {
    MainAsync().Wait();
  }
  static async Task MainAsync()
  {
    try
    {
      // Asynchronous implementation.
      await Task.Delay(1000);
    }
    catch (Exception ex)
    {
      // Handle exceptions.
    }
  }
}

Allowing async to grow through the codebase is the best solution, but this means there’s a lot of initial work for an application to see real benefit from async code. There are a few techniques for incrementally converting a large codebase to async code, but they’re outside the scope of this article. In some cases, using Task.Wait or Task.Result can help with a partial conversion, but you need to be aware of the deadlock problem as well as the error-handling problem. I’ll explain the error-handling problem now and show how to avoid the deadlock problem later in this article.

Every Task will store a list of exceptions. When you await a Task, the first exception is re-thrown, so you can catch the specific exception type (such as InvalidOperationException). However, when you synchronously block on a Task using Task.Wait or Task.Result, all of the exceptions are wrapped in an AggregateException and thrown. Refer again to Figure 4. The try/catch in MainAsync will catch a specific exception type, but if you put the try/catch in Main, then it will always catch an AggregateException. Error handling is much easier to deal with when you don’t have an AggregateException, so I put the “global” try/catch in MainAsync.

So far, I’ve shown two problems with blocking on async code: possible deadlocks and more-complicated error handling. There’s also a problem with using blocking code within an async method. Consider this simple example:

public static class NotFullyAsynchronousDemo
{
  // This method synchronously blocks a thread.
  public static async Task TestNotFullyAsync()
  {
    await Task.Yield();
    Thread.Sleep(5000);
  }
}

This method isn’t fully asynchronous. It will immediately yield, returning an incomplete task, but when it resumes it will synchronously block whatever thread is running. If this method is called from a GUI context, it will block the GUI thread; if it’s called from an ASP.NET request context, it will block the current ASP.NET request thread. Asynchronous code works best if it doesn’t synchronously block. Figure 5 is a cheat sheet of async replacements for synchronous operations.

Figure 5 The “Async Way” of Doing Things

To Do This … Instead of This … Use This
Retrieve the result of a background task Task.Wait or Task.Result await
Wait for any task to complete Task.WaitAny await Task.WhenAny
Retrieve the results of multiple tasks Task.WaitAll await Task.WhenAll
Wait a period of time Thread.Sleep await Task.Delay

To summarize this second guideline, you should avoid mixing async and blocking code. Mixed async and blocking code can cause deadlocks, more-complex error handling and unexpected blocking of context threads. The exception to this guideline is the Main method for console applications, or—if you’re an advanced user—managing a partially asynchronous codebase.

Configure Context

Earlier in this article, I briefly explained how the “context” is captured by default when an incomplete Task is awaited, and that this captured context is used to resume the async method. The example in Figure 3 shows how resuming on the context clashes with synchronous blocking to cause a deadlock. This context behavior can also cause another problem—one of performance. As asynchronous GUI applications grow larger, you might find many small parts of async methods all using the GUI thread as their context. This can cause sluggishness as responsiveness suffers from “thousands of paper cuts.”

To mitigate this, await the result of ConfigureAwait whenever you can. The following code snippet illustrates the default context behavior and the use of ConfigureAwait:

async Task MyMethodAsync()
{
  // Code here runs in the original context.
  await Task.Delay(1000);
  // Code here runs in the original context.
  await Task.Delay(1000).ConfigureAwait(
    continueOnCapturedContext: false);
  // Code here runs without the original
  // context (in this case, on the thread pool).
}

By using ConfigureAwait, you enable a small amount of parallelism: Some asynchronous code can run in parallel with the GUI thread instead of constantly badgering it with bits of work to do.

Aside from performance, ConfigureAwait has another important aspect: It can avoid deadlocks. Consider Figure 3 again; if you add “ConfigureAwait(false)” to the line of code in DelayAsync, then the deadlock is avoided. This time, when the await completes, it attempts to execute the remainder of the async method within the thread pool context. The method is able to complete, which completes its returned task, and there’s no deadlock. This technique is particularly useful if you need to gradually convert an application from synchronous to asynchronous.

If you can use ConfigureAwait at some point within a method, then I recommend you use it for every await in that method after that point. Recall that the context is captured only if an incomplete Task is awaited; if the Task is already complete, then the context isn’t captured. Some tasks might complete faster than expected in different hardware and network situations, and you need to graciously handle a returned task that completes before it’s awaited. Figure 6 shows a modified example.

Figure 6 Handling a Returned Task that Completes Before It’s Awaited
async Task MyMethodAsync()
{
  // Code here runs in the original context.
  await Task.FromResult(1);
  // Code here runs in the original context.
  await Task.FromResult(1).ConfigureAwait(continueOnCapturedContext: false);
  // Code here runs in the original context.
  var random = new Random();
  int delay = random.Next(2); // Delay is either 0 or 1
  await Task.Delay(delay).ConfigureAwait(continueOnCapturedContext: false);
  // Code here might or might not run in the original context.
  // The same is true when you await any Task
  // that might complete very quickly.
}

You should not use ConfigureAwait when you have code after the await in the method that needs the context. For GUI apps, this includes any code that manipulates GUI elements, writes data-bound properties or depends on a GUI-specific type such as Dispatcher/CoreDispatcher. For ASP.NET apps, this includes any code that uses HttpContext.Current or builds an ASP.NET response, including return statements in controller actions. Figure 7demonstrates one common pattern in GUI apps—having an async event handler disable its control at the beginning of the method, perform some awaits and then re-enable its control at the end of the handler; the event handler can’t give up its context because it needs to re-enable its control.

Figure 7 Having an Async Event Handler Disable and Re-Enable Its Control
private async void button1_Click(object sender, EventArgs e)
{
  button1.Enabled = false;
  try
  {
    // Can't use ConfigureAwait here ...
    await Task.Delay(1000);
  }
  finally
  {
    // Because we need the context here.
    button1.Enabled = true;
  }
}

Each async method has its own context, so if one async method calls another async method, their contexts are independent. Figure 8 shows a minor modification of Figure 7.

Figure 8 Each Async Method Has Its Own Context
private async Task HandleClickAsync()
{
  // Can use ConfigureAwait here.
  await Task.Delay(1000).ConfigureAwait(continueOnCapturedContext: false);
}
private async void button1_Click(object sender, EventArgs e)
{
  button1.Enabled = false;
  try
  {
    // Can't use ConfigureAwait here.
    await HandleClickAsync();
  }
  finally
  {
    // We are back on the original context for this method.
    button1.Enabled = true;
  }
}

Context-free code is more reusable. Try to create a barrier in your code between the context-sensitive code and context-free code, and minimize the context-sensitive code. In Figure 8, I recommend putting all the core logic of the event handler within a testable and context-free async Task method, leaving only the minimal code in the context-sensitive event handler. Even if you’re writing an ASP.NET application, if you have a core library that’s potentially shared with desktop applications, consider using ConfigureAwait in the library code.

To summarize this third guideline, you should use Configure­Await when possible. Context-free code has better performance for GUI applications and is a useful technique for avoiding deadlocks when working with a partially async codebase. The exceptions to this guideline are methods that require the context.

Know Your Tools

There’s a lot to learn about async and await, and it’s natural to get a little disoriented. Figure 9 is a quick reference of solutions to common problems.

Figure 9 Solutions to Common Async Problems

Problem Solution
Create a task to execute code Task.Run or TaskFactory.StartNew (not the Task constructor or Task.Start)
Create a task wrapper for an operation or event TaskFactory.FromAsync or TaskCompletionSource<T>
Support cancellation CancellationTokenSource and CancellationToken
Report progress IProgress<T> and Progress<T>
Handle streams of data TPL Dataflow or Reactive Extensions
Synchronize access to a shared resource SemaphoreSlim
Asynchronously initialize a resource AsyncLazy<T>
Async-ready producer/consumer structures TPL Dataflow or AsyncCollection<T>

The first problem is task creation. Obviously, an async method can create a task, and that’s the easiest option. If you need to run code on the thread pool, use Task.Run. If you want to create a task wrapper for an existing asynchronous operation or event, use TaskCompletionSource<T>. The next common problem is how to handle cancellation and progress reporting. The base class library (BCL) includes types specifically intended to solve these issues: CancellationTokenSource/CancellationToken and IProgress<T>/Progress<T>. Asynchronous code should use the Task-based Asynchronous Pattern, or TAP (msdn.microsoft.com/library/hh873175), which explains task creation, cancellation and progress reporting in detail.

Another problem that comes up is how to handle streams of asynchronous data. Tasks are great, but they can only return one object and only complete once. For asynchronous streams, you can use either TPL Dataflow or Reactive Extensions (Rx). TPL Dataflow creates a “mesh” that has an actor-like feel to it. Rx is more powerful and efficient but has a more difficult learning curve. Both TPL Dataflow and Rx have async-ready methods and work well with asynchronous code.

Just because your code is asynchronous doesn’t mean that it’s safe. Shared resources still need to be protected, and this is complicated by the fact that you can’t await from inside a lock. Here’s an example of async code that can corrupt shared state if it executes twice, even if it always runs on the same thread:

int value;

Task<int> GetNextValueAsync(int current);

async Task UpdateValueAsync()

{

  value = await GetNextValueAsync(value);

}

The problem is that the method reads the value and suspends itself at the await, and when the method resumes it assumes the value hasn’t changed. To solve this problem, the SemaphoreSlim class was augmented with the async-ready WaitAsync overloads. Figure 10 demonstrates SemaphoreSlim.WaitAsync.

Figure 10 SemaphoreSlim Permits Asynchronous Synchronization
SemaphoreSlim mutex = new SemaphoreSlim(1);

int value;

Task<int> GetNextValueAsync(int current);

async Task UpdateValueAsync()
{
  await mutex.WaitAsync().ConfigureAwait(false);

  try
  {
    value = await GetNextValueAsync(value);
  }
  finally
  {
    mutex.Release();
  }
}

Asynchronous code is often used to initialize a resource that’s then cached and shared. There isn’t a built-in type for this, but Stephen Toub developed an AsyncLazy<T> that acts like a merge of Task<T> and Lazy<T>. The original type is described on his blog (bit.ly/dEN178), and an updated version is available in my AsyncEx library (nitoasyncex.codeplex.com).

Finally, some async-ready data structures are sometimes needed. TPL Dataflow provides a BufferBlock<T> that acts like an async-ready producer/consumer queue. Alternatively, AsyncEx provides AsyncCollection<T>, which is an async version of BlockingCollection<T>.

I hope the guidelines and pointers in this article have been helpful. Async is a truly awesome language feature, and now is a great time to start using it!

By

Lanuching Visual Studio Android Emulator from the command line

Quick command line to launch the Visual Studio Android Emulator for Lollipop

%ProgramFiles(x86)%Microsoft
XDE10.0.1.0xde.exe /sku Android /displayName "VS Emulator 5"
Lollipop (5.0) XXHDPI Phone" /memSize 2048 /diagonalSize 5 /video
"1080x1920" /vhd
"%LOCALAPPDATA%MicrosoftVisualStudioEmulatorAndroidContai‌
​nersLocalDevicesvhd5_Lollipop_(5.0)_XXHDPI_Phoneimage.vhd" /name
"VS Emulator 5-inch Lollipop (5.0) XXHDPI Phone.%USERNAME%" 

By

How to Profile a SharePoint/Project Server 2013 Workflow using dotTrace

Thought I’d put together a small “How to” for Profiling a SharePoint (or Project Server) 2013 workflow   The steps below are using a 10 day trial of dotTrace from jetbrains – that said, the same “idea” can be used by any profiling tool.

  1. First thing to do is download a copy of dotTrace from Jetbrains from their website (http://www.jetbrains.com/profiler/).  After you install this, you’ll have 10 days to use it without any limitations.
  2. Once it is installed, look for the dotTrace icon  , and start the application up.
  3. One the app is up and running, click on the “Attach to Process” button at the top
  4. Next, Find the process “Microsoft.Workflow.ServiceHost.exe”
  5. At this point you’re “tracing” in enabled.  And you will see a dialog like this
  6. Now you can run your tests that execute the workflow.  One you are done, click the “Get Snapshot” button on the tracing dialog
  7. At which point a snapshot file will open that contains all of the information (summary screen blow)
  8. Click on the “Plain List” icon
  9. At this point you can look through the various calls, the time each call took, etc (sample below)

     

By

Excellent set of JSON samples

Check out this site for a break set of JSON samples.

 

Some information from the site:

 

Over 100 code samples covering Json.NET’s most commonly used functionality.

Samples

  • Serializing JSON – Serializing and deserializing JSON, serializer settings and serialization attributes
  • LINQ to JSON – Parsing, querying, modifying and writing JSON
  • JSON Schema – Loading schemas and validating JSON
  • Converting XML – Converting JSON to XML and XML to JSON
  • BSON – Serializing and deserializing BSON
  • Reading and Writing JSON – Reading JSON with JsonTextReader, writing JSON with JsonTextWriter

By

The SharePoint & jQuery Guide

Repost from here

 

So, you’ve finally succumbed to the hype and decided to use jQuery in SharePoint? Good for you. I’m sure you are fully prepared with the knowledge of the pros and cons using jQuery as well as have all the requisite knowledge.

You should NOT be writing jQuery in SharePoint if…

You don’t? You mean you are going to copy and paste scripts from blogs and then ask the blogger to modify the script to work for your particular circumstance? Oh, that’s nice.

Well, assuming you actually want to understand what you are doing, maybe even learn a thing or two, I’ve decided to link to several of my past jQuery blogs in an order which will hopefully help you learn how to successfully use SharePoint and jQuery together. I’ll even do my best to keep this post updated as I write new blogs on the subject.

Good luck!

Before getting started…

Make sure you have some good background knowledge.

When you start using things like SPServices and the Client Object Model to query SharePoint list data you will need to understand SharePoint’s query language CAML. I always refer to it as the ugly crossbreed of XML and SQL.

If you plan to use SPServices to access SharePoint’s Web Services (which I also reference in several of the blogs below), you need to get that at http://spservices.codeplex.com.

When to Choose SPServices vs. the Client Side Object Model (CSOM)

Also, if you are wondering if you should use SPServices instead of the Client Object Model you can read the helpful post put together by Marc Anderson and myself.

Be sure to keep the jQuery.com site handy as this is by far the most up-to-date site to reference the jQuery api.

The Intro to jQuery and SharePoint blogs

Here are the blogs (in learning order) to help you deploy and get started using jQuery in SharePoint.

A Dummies Guide to SharePoint and jQuery–Getting Started

This blog assumes you know nothing. What is jQuery and how can I deploy it in SharePoint?

Undoubtedly one of the most common tasks you will perform with jQuery is getting and setting SharePoint form fields. This blog walks you through the process with the most common field types. A corresponding blog post to this one is Setting SharePoint Lookup Lists w/ jQuery (+/- 20 items) because unfortunately at some point you will fall victim to this SharePoint quirk.

Creating a SharePoint Parent/Child List Relationship–No SPD Version

So, now that you understand the basics of using jQuery in SharePoint and know how to get/set SharePoint form fields, this blog helps you apply that knowledge to perform the common task of creating an automatic parent/child list relationship. There are MANY ways of accomplishing this functionality, but I actually prefer this method. It may be a little more technical than the other solutions I’ve blogged about for creating a Parent/Child list relationship, but this solution has the least impact on SharePoint and should also upgrade easily.

Okay, at this point you should be ready to start interacting with SharePoint list data using SPServices. This blog post walks you through the basics with a VERY commented script. If you think you have your head wrapped around reading list data with SPServices, you might also check out the following blog posts using SPServices to accomplish some tasks normally achieved using server side code:

Using SPServices & jQuery to Clone a Parent Item and All its Children – Reads items from a SharePoint List and then creates copies of those items.

Using SPServices & jQuery to Find My Stuff from Multi-Select Person/Group Field – Determines the groups a current user belongs to in order to determine if the current user is part of a group or person assigned to a list item.

A Dummies Guide to SharePoint & jQuery – An Easy to Use Content Slider

Now that jQuery and SPServices is old hat, I walk you through the process of integrating SharePoint List Data into a third party jQuery library.

Other SharePoint tips and tricks using jQuery

So, here is a smattering of other blog posts on the subject which you may find helpful, or might give you ideas for your projects. Some of them are just academic, and some you can put into practice immediately.

Using jQuery to Print a Web Part – In this blog I use a very simple third party library to print a specific web part and not the entire page.

Using jQuery to Make a Field Read Only in SharePoint – Another tip you could use to make a SharePoint form field read only.

SharePoint List Views –Quick & Easy jQuery Filter – Using a very simple script, add a filter box to an out of the box list view that filters the rows based upon what the user enters.

Using Google Maps to Populate an Address in a SharePoint New Item Form – Before I started using the Bing Map api, I played around with the Google Maps API.

A Dummies’ Guide to SharePoint and jQuery–A Real World Example – A mostly academic post on the types of things you can do with jQuery to modify a page in the _Layouts directory that you don’t’ have direct access to.

Essential Links for the SharePoint Client Side Developer – This is a list of a lot of the jQuery/JavaScript libraries I use along with some suggestions from others. I actually need to update this list soon.

So, What’s next?

What? Is this not enough? I’d say for the price you paid, it’s a bargain!!

I hope to do a few blog entries on more advanced topics like Callbacks, templates, and design patterns. Also, as we move towards SharePoint vNext I do think it might be a good idea to start to wean myself off of SPServices and start using the Client Object Model more (although nothing leads me to believe that SPServices will not work in vNext). So, look for me to start a series on the Client Object Model as well and provide any tips or tricks I learn along the way.

As always, let me know what YOU want to learn and I’ll see what I can do! Thanks again for stopping by.

By

An extension method to convert a flat collection to a Hierarchical collection

This is a report from here

 

A while ago I posted an article about LINQ to SQL and WPF treeviews. I demonstrated how to use multiple levels in treeviews and how to group data with LINQ. One interesting subject wasn’t covered, namely self referencing tables or the adjacency list model. A WPF TreeView control does not support this. That is why I created a LINQ extension method to convert a flat collection to a hierarchical structure of nested collections which can be assigned to the ItemsSource of a WPF Treeview.

 

 

 

 

Adjacency model / self referencing table

 

Hierarchies like trees, organizational charts, … are sometimes difficult to store in database tables. The most common database pattern to store hierarchical data is known as the adjacency model. It has been introduced by the famous computer scientist Edgar Frank Codd. It’s called the “adjacency” model because a reference to the parent data is stored in the same row as the child data, in an adjacent column. These kind of tables are also called self referencing tables.

 

Let’s crystalize this by displaying the Employees table in the Northwind database.

 

EmployeeID LastName FirstName ReportsTo
1 Davolio Nancy 2
2 Fuller Andrew null
3 Leverling Janet 2
4 Peacock Margaret 2
5 Buchanan Steven 2
6 Suyama Michael 5
7 King Robert 5
8 Callahan Laura 2
9 Dodsworth Anne 5

 

In this table, the child is the same kind of row as the parent. The ReportsTo column is a foreign key referencing the EmpNo column. So each employee has a reference to his/her boss.

 

If you want to use this data in a WPF TreeView, then you have to convert the adjacency list model to a hierarchical model/nested set model. This should be the result :

 

EmployeeID LastName FirstName ReportsTo Depth
2 Fuller Andrew null 1
      1 Davolio Nancy 2 2
      3 Leverling Janet 2 2
      4 Peacock Margaret 2 2
      5 Buchanan Steven 2 2
            6 Suyama Michael 5 3
            7 King Robert 5 3
            9 Dodsworth Anne 5 3
      8 Callahan Laura 2 2

 

In SQL Server 2005 you can use CTE (Common Table Expressions) and a recursive function to query this kind of data. And SQL Server 2008, which will be released one of the next months, will provide a new SQLCLR datatype called HierarchyID which will support manipulating and querying hierarchical data. Besides these database features, I wanted to create a .NET solution which can be used to convert a flat collection to a hierarchical structure of nested collections. Next steps will describe how I implemented the LINQ AsHierarchy extension method.

 

 

 

GetEmployeesHierarchy() method

 

I first started creating a new EmployeeHierarchy class. This class contains an Employee entity (the boss) and a collection of child employees.

public class EmployeeHierarchy
{
  public Employee Employee { get; set; }
  public IEnumerable<EmployeeHierarchy> Employees { get; set; }
}

 

In .NET 3.5 we have LINQ at our’s disposal. So I created a recursive GetEmployeesHierachy function which will start with the director (who does not have to report to anyone, ReportsTo = null) and query all employees which are working for him (ReportsTo == EmployeeId of boss). This will be repeated recursively. Finally this function will return a collection of EmployeeHierarchy objects.

 

public IEnumerable<EmployeeHierarchy> GetEmployeesHierachy(IEnumerable<Employee> allEmployees, Employee parentEmployee)
{
  int? parentEmployeeId = null;

  if (parentEmployee != null)
    parentEmployeeId = parentEmployee.EmployeeID;

  var childEmployees = allEmployees.Where(e => e.ReportsTo == parentEmployeeId);

  Collection<EmployeeHierarchy> hierarchy = new Collection<EmployeeHierarchy>();

  foreach (var emp in childEmployees)
    hierarchy.Add(new EmployeeHierarchy() { Employee = emp, Employees = GetEmployeesHierachy(allEmployees, emp) });

  return hierarchy;
}

 

This function can be called after a LINQ to SQL or LINQ to Entities (Entity Framework) query. Make sure to call the ToList() extension method. It is much more performant to first load all data from the database and then convert it to a hierachical structure. By calling the ToList() method all references to the database will be removed.

 

NorthWindDataContext dc = new NorthWindDataContext();

var employeesHierarchy = GetEmployeesHierachy(dc.Employees.AsEnumerable(), null);

treeViewEmployees.ItemsSource = employeesHierarchy;

 

AsHierarchy() extension method

 

Previous routines did work fine but I wanted a more generic solution. My motto is “do more with less code” so I created a HierarchyNode<T> class and a LINQ AsHierarchy() extension method which uses generics and anonymous functions (=Func delegates). This method will return an IEnumerable collection of generic HierarchyNode<T> objects.

 

This is the source code which does all the magic. Just copy it to one of your own projects.

using System;
using System.Collections.Generic;
using System.Linq;

namespace ScipBe.Common.LinqExtensions
{
  // Stefan Cruysberghs, http://www.scip.be, March 2008

  /// <summary>
  /// Hierarchy node class which contains a nested collection of hierarchy nodes
  /// </summary>
  /// <typeparam name="T">Entity</typeparam>
  public class HierarchyNode<T> where T : class
  {
    public T Entity { get; set; }
    public IEnumerable<HierarchyNode<T>> ChildNodes { get; set; }
    public int Depth { get; set; }
  }

  public static class LinqExtensionMethods
  {
    private static System.Collections.Generic.IEnumerable<HierarchyNode<TEntity>> CreateHierarchy<TEntity, TProperty>
      (IEnumerable<TEntity> allItems, TEntity parentItem,
      Func<TEntity, TProperty> idProperty, Func<TEntity, TProperty> parentIdProperty, int depth) where TEntity : class
    {
      IEnumerable<TEntity> childs;

      if (parentItem == null)
        childs = allItems.Where(i => parentIdProperty(i).Equals(default(TProperty)));
      else
        childs = allItems.Where(i => parentIdProperty(i).Equals(idProperty(parentItem)));

      if (childs.Count() > 0)
      {
        depth++;

        foreach (var item in childs)
          yield return new HierarchyNode<TEntity>() { Entity = item, ChildNodes = CreateHierarchy<TEntity, TProperty>
            (allItems, item, idProperty, parentIdProperty, depth), Depth = depth };
      }
    }

    /// <summary>
    /// LINQ IEnumerable AsHierachy() extension method
    /// </summary>
    /// <typeparam name="TEntity">Entity class</typeparam>
    /// <typeparam name="TProperty">Property of entity class</typeparam>
    /// <param name="allItems">Flat collection of entities</param>
    /// <param name="idProperty">Reference to Id/Key of entity</param>
    /// <param name="parentIdProperty">Reference to parent Id/Key</param>
    /// <returns>Hierarchical structure of entities</returns>
    public static System.Collections.Generic.IEnumerable<HierarchyNode<TEntity>> AsHierarchy<TEntity, TProperty>
      (this IEnumerable<TEntity> allItems, Func<TEntity, TProperty> idProperty, Func<TEntity, TProperty> parentIdProperty)
      where TEntity : class
    {
      return CreateHierarchy(allItems, default(TEntity), idProperty, parentIdProperty, 0);
    }
  }
}

AsHierarchy() examples

Call the AsHierachy() method and pass the Id/Key and the ParentId/Key of the entity as anonymous functions. As you can see, it really is that simple.

NorthWindDataContext dc = new NorthWindDataContext();

var hierachy = dc.Employees.ToList().AsHierarchy(e => e.EmployeeID, e => e.ReportsTo);

treeView1.ItemsSource = hierarchy;

 

Within each level of the hierarchical tree the original order will be preserved. So it is possible to use the LINQ OrderBy and OrderByDescending extension methods.

var hierachy = dc.Employees.OrderByDescending(e => e.LastName).ToList().
  AsHierarchy(e => e.EmployeeID, e => e.ReportsTo);

 

Of course this extension method will also work with anonymous classes.

var hierachy = (from e in dc.Employees
               select new { Id = e.EmployeeID, FullName = e.FirstName + " " + e.LastName, ParentId = e.ReportsTo }).ToList()
               AsHierarchy(e => e.Id, e => e.ParentId);

 

AsHierarchy() in LINQPad

External extension methods can also be used in LINPad. Go to Query >> Advanced Properties and add a reference to the assemby which holds the AsHierachy extension method. Don’t forget to add the name of the namespace at the tabsheet Additional Namespace Imports.

LINQPad will show you a nice representation of the hierarchical structure.

 

 

 

Display hierarchy with WPF TreeView

Binding a collection of generic HierarchyNode<T> objects to a WPF TreeView is quite easy. Just set the ItemsSource of the HierarchicalDataTemplate to ChildNodes. Now the TreeView will recursively display all HierarchyNode<T> objects. Make sure you use the fixed name Entity in the binding of the template.

<TreeView Name="treeView1">
    <TreeView.ItemTemplate>
        <HierarchicalDataTemplate ItemsSource="{Binding Path=ChildNodes}">
            <StackPanel Orientation="Horizontal">
                <TextBlock Text="{Binding Path=Entity.FirstName}"></TextBlock>
                <TextBlock Text=" "></TextBlock>
                <TextBlock Text="{Binding Path=Entity.LastName}"></TextBlock>
            </StackPanel>
        </HierarchicalDataTemplate>
    </TreeView.ItemTemplate>
</TreeView>

 

his is how the TreeView will look like :

Once you have the data in the TreeView it is up to you to give it a nice visual representation. Take a look at the Custom TreeView Layout in WPF article of Josh Smith who demonstrates how to render a TreeView as an organizational chart. I already copied his XAML example and this is how my Employees tree will be displayed :

Let’s summerize this article :

  • Copy the source of the AsHierarchy() extension method to your project
  • Add a TreeView control and define a HierarchicalDataTemplate which uses the ChildNodes and Entity properties in your XAML file
  • Call the AsHierarchy() method with the id/key and parentid/key in a LINQ to Objects, LINQ to SQL or LINQ to Entities query
  • Assign the result query to the ItemsSource of the WPF TreeView

I hope you like this LINQ extension method and that you can take advantage of this free code.

New improvements and a LINQ to SQL (IQueryable) version can be found in a more recent article.

 

By

Dynamically adding JavaScript script tags at runtime

This is a report from here

First Let’s discuss about the issue of loading all the javascript files together at a stroke.

  1. 1. Javascript will not allow parallel download. So the browser will not start any other download until it loads the javascripts.
  2. 2. We use to load javascript files and methods which are unnecessary for that page. For eg: In Login page, we just need some javascripts essential for Login page, but developers will add all the javascripts in common headers.

 

<script language =”javascript” src=”scriptRequiredForUserProfile.js”></script>
<script language =”javascript” src=”scriptRequiredForAddComment.js”></script>
<script language =”javascript” src=”scriptRequiredForReply.js”></script>

<script language =”javascript” src=”scriptRequiredForAddComment.js”></script>

<script language =”javascript” src=”scriptRequiredForReply.js”></script>

How to solve the above 2 issuesjQuery15204168177171158769_1337868541848?

  • Include only one script in the header.

 

<script src="”loadJS.js”"></script>


 

Now we can load the javascript dynamically using 2 ways.

 

Method 1:

 

Generate dynamic // <![CDATA[
tag using DOM

 

Maintain the list of javascripts that you need to download for a respective page.

 

 

var jsFilesArray = new Array();
jsFilesArray['home.html'] = new Array( ‘validation.js’, ‘home.js’ );
jsFilesArray['login.html'] = new Array( ‘validation.js’, ‘error.js’ );

Code to get File Name of the current page

 

 

function getCurrentPageName() {
    var fileName = document.location.href;
    var end = (fileName.indexOf("?") == -1) ? fileName.length : fileName.indexOf("?");
    return fileName.substring(fileName.lastIndexOf("/")+1, end);
}

Code to generate dynamic tag and append to the head tag.

 

 

var head = document.getElementsByTagName("head")[0];

for (var i = 0;i<jsFilesArray[getFileName()].length;i++)
{
  script = document.createElement('script');
  script.id = "id_" + i;
  script.type = 'text/javascript';
  script.src = jsFilesArray[getCurrentPageName ()][i];
  head.appendChild(script);
}

 

 

Click to download the code.

 

Method 2:

 

Use AJAX to load the JS file dynamically whenever required

 

Get the required JS file using xmlHttPRequest and execute the output by passing it into eval() method. The eval() method executes the argument.

 

So guys, load your Javascripts dynamically and experience improved performance of your application.

 

Enjoy Coding :)

By

Working with WebDeployment in MS Project files

Great article: http://stackoverflow.com/questions/2959964/vs2010-web-deploy-how-to-remove-absolute-paths-and-automate-setacl

The displayed path is determined by the property _MSDeployDirPath_FullPath.

This property is setted by this chain of properties:

  • <_MSDeployDirPath_FullPath>@(_MSDeployDirPath->'%(FullPath)')</_MSDeployDirPath_FullPath>
  • <_MSDeployDirPath Include="$(_PackageTempDir)" />
  • <_PackageTempDir>$(PackageTempRootDir)PackageTmp</_PackageTempDir>
  • <PackageTempRootDir>$(IntermediateOutputPath)Package</PackageTempRootDir>
_MSDeployDirPath_FullPath <-- @(_MSDeployDirPath->'%(FullPath)') <-- _PackageTempDir <-- $(PackageTempRootDir)PackageTmp

AS you can see, you can’t have a relative path, because _MSDeployDirPath_FullPath is the fullpath of _MSDeployDirPath.

But you can simplify the displayed path by overriding the property _PackageTempDir with the path you want to be displayed to your customer. (This path will be used as a temporary directory for the package generation)

You could override the property :

  • In command line :

    msbuild.exe projectfile.csproj /t:Package /p:_PackageTempDir=C:Package
  • Or directly in the project file :

    <Import Project="$(MSBuildBinPath)Microsoft.CSharp.targets" />
    <Import Project="$(MSBuildExtensionsPath32)MicrosoftVisualStudiov10.0WebApplicationsMicrosoft.WebApplication.targets" />
    
    
    <!-- Must be after Microsoft.WebApplication.targets import -->
    <PropertyGroup>
      <_PackageTempDir>C:Package</_PackageTempDir>
    </PropertyGroup>

 

By

Looking for a way to list a row count for every table in a database? Look no further…

SELECT OBJECT_NAME(OBJECT_ID) TableName, st.row_count
FROM sys.dm_db_partition_stats st
WHERE index_id < 2
ORDER BY st.row_count DESC
GO