Call Ruby or Python API in C# .Net

Call Ruby or Python API in C# .NET

This is one of the two things that the Dynamic Language Runtime is supposed to do: everybody thinks that the DLR is only for language implementors to make it easier to implement dynamic languages on the CLI. But, it is also for application writers, to make it easier to host dynamic languages in their applications.

Before the DLR, every language had their own hosting API. Now, the DLR has a standardized hosting specification that works the same for every language, and with support for dynamically typed objects in C# 4 and VB.NET 10, it gets easier than ever:

// MethodMissingDemo.cs
using System;
using IronRuby;

class Program
{
    static void Main()
    {
        var rubyEngine = Ruby.CreateEngine();
        rubyEngine.ExecuteFile("method_missing_demo.rb");
        dynamic globals = rubyEngine.Runtime.Globals;

        dynamic methodMissingDemo = globals.MethodMissingDemo.@new();

        Console.WriteLine(methodMissingDemo.HelloDynamicWorld());

        methodMissingDemo.print_all(args);
    }
}

# method_missing_demo.rb
class MethodMissingDemo
  def print_all(args)
    args.map {|arg| puts arg}
  end

  def method_missing(name, *args)
    name.to_s.gsub(/([[:lower:]\d])([[:upper:]])/,'\1 \2')
  end
end

Here you see stuff getting passed around in every possible direction. The C# code is calling a method on the Ruby object which doesn't even exist and the Ruby code is iterating over a .NET array and printing its contents to the console.

Compiling code from a ASP.NET website

You'll need to host all sorts of compilers you'd like to support and use their APIs. If a compiler doesn't expose an API, you'd need to use System.Diagnostics.Process to launch it and grab the output for parsing.

This is far from a trivial task though.

If you're confident you need this, ask yourself several questions.

• Which languages, platforms and languages do you plan to support? Python and Ruby, do you mean .NET versions of them or do you want to use their standard compilers? Which versions?
• Do these compilers have any public APIs? Will you need to run them as executables?
• If the site is going to be used concurrently by many users, how do you plan to distribute the load? Compiling is an intensive process and having a lot of people doing it will overload your server. Also note that compilation always takes time so you'll need to think how to present it to user. Do you want it to start immediately or do you want to put users in a queue?
• What is the user coding scope? Is user input limited to one method contents, or does the user have to write the whole file? Is any code added by system by default? How do you resolve the references e.g. for C# code? Are some libraries referenced by default?

And this is just the tip of the iceberg.

Answering your comment, yes, there is a compiler API for C# and VB .NET which is called CodeDOM.

There's plenty of information about it on the net. You may want to check out this question as well.

SSIS: Execute Ironpython or Ironruby scripts through SSIS

The easiest, at least to my brain, mechanism for using IronPython from the confines of SSIS would be to invoke the external process and dump to a file and then use that as a source for a dataflow.

That said, I was able to host an IronPython app from C# and use the returned data to populate the output buffers and interact with that data in the pipeline. I've only had one machine to perform this on so I'm listing everything I recall doing until the package went green.

Prerequisites

This article set me down the path of how to make this work. Hosting IronPython in a C# 4.0 program I would strongly urge you to create a C#/VB.NET console app and get your IronPython integration working there first as SSIS is going to add an additional layer to everything.

There may be the ability to host older versions of IronPython within C# without requiring the 4.0 framework but that's far beyond the realm of my competency. What I can say is that to use the 4.0 framework, you are looking at SQL Server 2012. A 2008 package can target up to the 3.5 framework (default is 2.0).

Global Assembly Cache, GAC for short. It is a special place in Windows where signed assemblies can live. SSIS may be able to use assemblies that aren't in the GAC, but I've not had luck doing so. This case was no different. My Console app worked fine but when I copied that code into SSIS, it'd tank with Could not load file or assembly 'Microsoft.Scripting... error messages. Blessedly, IronPython-2.7.2.1 (and probably previous versions) are strongly signed dlls. That means you can and must add them into the GAC.

In your Visual Studio directory, look for the Visual Studio Command Prompt (2010).
Assuming your IronPython installation folder is C:\tmp\IronPython-2.7.2.1\IronPython-2.7.2.1 you would type cd C:\tmp\IronPython-2.7.2.1\IronPython-2.7.2.1 Then I registered the following 3 assemblies

C:\tmp\IronPython-2.7.2.1\IronPython-2.7.2.1>gacutil -if Microsoft.Dynamic.dll
Microsoft (R) .NET Global Assembly Cache Utility.  Version 4.0.30319.1
Copyright (c) Microsoft Corporation.  All rights reserved.

Assembly successfully added to the cache

C:\tmp\IronPython-2.7.2.1\IronPython-2.7.2.1>gacutil -if IronPython.dll
Microsoft (R) .NET Global Assembly Cache Utility.  Version 4.0.30319.1
Copyright (c) Microsoft Corporation.  All rights reserved.

Assembly successfully added to the cache

C:\tmp\IronPython-2.7.2.1\IronPython-2.7.2.1>gacutil -if Microsoft.Scripting.dll
Microsoft (R) .NET Global Assembly Cache Utility.  Version 4.0.30319.1
Copyright (c) Microsoft Corporation.  All rights reserved.

Assembly successfully added to the cache

My SSIS project, I had set the Run64bitRuntime to False but in retesting, it does not matter. The default it True and that seems to work fine.

Python script - I don't have enough of a background to make the integration between C# and .NET DLR languages more graceful. It'd have been nice to supply a string or something containing the script I wanted to execute and perhaps that's what a script block is about but I don't have time to investigate. So, this solution requires a script file sitting out somewhere on disk. I had trouble with the imports working from a hosted script (no module named X exceptions). Undoubtedly there's some magic with class paths and all that stuff that needs to provided to the host to make it work well. That's probably a different SO question btw.

Set up

I have a file sitting at C:\ssisdata\simplePy.py

# could not get a simple import to work from hosted
# works fine from "not hosted"
#import os

def GetIPData():
    #os.listdir(r'C:\\')
    return range(0,100)

After adding a script task to the Data Flow, I configured it to have a single column on the output buffer (wstr 1000). I then used this as my source code.

using System;
using System.Collections.Generic;
using System.Data;
using Microsoft.SqlServer.Dts.Pipeline.Wrapper;
using Microsoft.SqlServer.Dts.Runtime.Wrapper;
using IronPython.Hosting;
using Microsoft.Scripting.Hosting;

/// <summary>
/// Attempt to use IP script as a source
/// http://blogs.msdn.com/b/charlie/archive/2009/10/25/hosting-ironpython-in-a-c-4-0-program.aspx
/// </summary>
[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]
public class ScriptMain : UserComponent
{

    /// <summary>
    /// Create data rows and fill those buckets
    /// </summary>
    public override void CreateNewOutputRows()
    {
        foreach (var item in this.GetData())
        {
            Output0Buffer.AddRow();
            Output0Buffer.Content = item;
        }

    }

    /// <summary>
    /// I've written plenty of code, but I'm quite certain this is some of the ugliest.
    /// There certainly must be more graceful means of 
    /// * feeding your source code to the ironpython run-time than a file
    /// * processing the output of the code the method call
    /// * sucking less at life
    /// </summary>
    /// <returns>A list of strings</returns>
    public List<string> GetData()
    {
        List<string> output = null;
        var ipy = Python.CreateRuntime();
        dynamic test = ipy.UseFile(@"C:\ssisdata\simplePy.py");
        output = new List<string>();
        var pythonData = test.GetIPData();
        foreach (var item in pythonData)
        {
            output.Add(item.ToString());
        }

        return output;
    }
}

Quick shot of what my references look like

Click the run button and great success

can ironruby nuget package be installed then used in a vanilla VS2012

I've realized a simple proof of concept at https://github.com/edymtt/csscss-from-ironruby that shows in a console application how to use IronRuby to run the source code of csscss to analyze a CSS loaded from a file. To achieve this result I've started from this SO question -- you can find the additional resources I've used in the comments of the program. I haven't tried this code in a ASP.net MVC site -- anyway this sample should be a good starting point.

This solution is a bit cumbersome to maintain, since you have to manually put the sources for the csscss and its dependent libraries in the solution. An alternative solution is to install Ruby on the machine, install csscss using gem(so it download the dependencies automatically) and to invoke the program from .NET -- I'll also show this approach in the sample. Note that this solution requires that you could install Ruby on the web server.

UPDATE 2013-09-02 18:15 UTC Following the suggestion from Zach Moazeni I've been able to semplify the approach that used IronRuby to run csscss and I've updated accordingly the proof of concept. In a nutshell:

• outside the .NET program I've used bundler to download csscss and json (and dependent gems) to a local folder of the project;
• in the .NET program I've written a function to discover all the paths of the libraries in the gem folder created by bundler (by finding the gems folder and then including for each subfolder the lib folder, this algoritm was inspired by this SO thread);
• I've passed this list of paths to the IronRuby interpreter before launching csscss.

This approach should conjugate the ability to use only .NET to run the program with the ease of the update given by gem and bundler.

How will Python and Ruby applications be affected by .NET?

I can't say anything about IronRuby, but most python implementations (like IronPython, Jython and PyPy) try to be as true to the CPython implementation as possible. IronPython is quickly becoming one of the best in this respect though, and there is a lot of traffic on Planet Python about it.

The main thing that will encourage developers to write code that's different from what they would write in CPython is the lack of C extension modules like NumPy (This is a problem in Jython and PyPy as well).

An interesting project to keep your eye on is IronClad, which will let you call C extension modules from within IronPython. This should eventually mean that you can develop code under CPython, using whatever modules you like, and it will run unmodified on IronPython.

http://www.resolversystems.com/documentation/index.php/Ironclad

So to answer your questions:

It should be easy enough to write IronPython applications that work on CPython as well, but I would probably aim to go the other way around: CPython programs that work on IronPython as well. That way, if it doesn't work then it's more likely to be a known bug with a known work-around.

The advantage of IronPython et al existing is that they provide alternative implementations of the language, which are sometimes useful for spotting bugs in CPython. They also provide alternative methods for deploying your Python applications, if for some reason you find yourself in a situation (like silverlight) where distributing the CPython implementation with your application is not appropriate.

Expression evaluator for C#/Python/Ruby

Check out NCalc. It's .NET and should support your requirements.

Using DLR from Unmanaged Code

Yes. Delphi for Win32 example here: http://interop.managed-vcl.com/

Shows how to use a C# as well as a Delphi.NET assembly from Delphi for Win32.

---

Related Topics