Calling 2 - Managed to Native

Content:

  1. Introduction
  2. Common
  3. Java
    1. JNI
    2. JNA
  4. .NET
    1. P/Invoke or DllImport
    2. C++/CLI mixed mode
  5. Python
    1. ctypes
  6. Windows COM
    1. Concept
    2. Server
    3. Client

Introduction:

Network communication is important today for integration.

But sometimes a simple call is what is needed.

That can create some challenges when the calling language is different than the called language.

This article will cover managed (Java, .NET, Python) to native (C).

Mixing managed and native create some special problems as a variable is either managed or native, so data need to be converted when moving from one to another.

Other articles:

Common:

All the following examples will call this simple C Code.

demolib.h:

#ifndef DEMOLIB_H
#define DEMOLIB_H

/* workaround conflict with *nix dup function */
#if defined(NIX) || defined(VMS)
#define dup mydup
#endif

struct Data1
{
    int iv;
    char sv[50];
};

struct Data2
{
    int iv1;
    char sv1[50];
    int iv2;
    char sv2[50];
};

/* int in arg + int retval */
int add(int a, int b);
/* int array in arg + int retval */
int sum(int n, int *a);
/* string in arg + string retval */
/* caller must call free on retval */
char *dup(char *s);
/* int out arg + string out arg */
/* caller must call free on second arg */
void get_out(int *iv, char **sv);
/* caller must call free on retval */
/* struct retval */
struct Data1 *get_ret();
/* inout struct arg */
void chg(struct Data2 *d);

#endif

demolib.c:

#include <string.h>
#include <stdlib.h>

#include "demolib.h"

/* some very simple functions */

int add(int a, int b)
{
    return a + b;
}

int sum(int n, int *a)
{
    int i, sum;
    sum = 0;
    for(i = 0; i < n; i++)
    {
        sum += a[i];
    }
    return sum;
}

char *dup(char *s)
{
    char *s2;
    s2 = malloc(2 * strlen(s) + 1);
    strcpy(s2, s);
    strcat(s2, s);
    return s2;
}

/* functions simulating real work */

static int IV = 123;
static char *SV = "ABC";

void get_out(int *iv, char **sv)
{
    *iv = IV;
    *sv = malloc(strlen(SV) + 1);
    strcpy(*sv, SV);
}

struct Data1 *get_ret()
{
    struct Data1 *d;
    d = malloc(sizeof(struct Data1));
    d->iv = IV;
    strcpy(d->sv, SV);
    return d;
}

void chg(struct Data2 *d)
{
    d->iv2 = d->iv1 + 1;
    strcpy(d->sv2, d->sv1);
    strcat(d->sv2, "X");
}

The example does not cover all aspects of calling from managed to native, but it does cover some of the more common.

Some of the examples need a DLL/so to call.

For those the following wrapper is used.

stddemolib.c:

#include <stdlib.h>

#include "demolib.h"

#ifdef WIN
#define EXTERNAL(t) __declspec(dllexport) t __stdcall
#else
#ifdef NIX
#define EXTERNAL(t) t
#else
#ifdef VMS
#define EXTERNAL(t) t
#else
#error "Platform not supported or not specified"
#endif
#endif
#endif

EXTERNAL(int) std_add(int a, int b)
{
    return add(a, b);
}

EXTERNAL(int) std_sum(int n, int *a)
{
    return sum(n, a);
}

EXTERNAL(char *) std_dup(char *s)
{
    return dup(s);
}

EXTERNAL(void) std_get_out(int *iv, char **sv)
{
    get_out(iv, sv);
}

EXTERNAL(struct Data1 *) std_get_ret()
{
    return get_ret();
}

EXTERNAL(void) std_chg(struct Data2 *d)
{
    chg(d);
}

EXTERNAL(void) std_free(char *s)
{
    free(s);
}

Build on Windows with GCC:

gcc -Wall -DWIN -shared -Wl,--kill-at stddemolib.c demolib.c -o stddemolib.dll

Build on Linux with GCC:

gcc -Wall -DNIX -shared -fPIC stddemolib.c demolib.c -o stddemolib.so

Build on VMS:

$ cc demolib
$ cc/def="VMS" stddemolib
$ link/share stddemolib+demolib+sys$input/option
SYMBOL_VECTOR=(std_add=PROCEDURE,-
               std_sum=PROCEDURE,-
               std_dup=PROCEDURE,-
               std_get_out=PROCEDURE,-
               std_get_ret=PROCEDURE,-
               std_chg=PROCEDURE)
$
$ define/nolog/super stddemolib sys$disk:[]stddemolib

Java:

JNI:

JNI (Java Native Interface) is the standard for how to call native code from a JVM language.

The API is a bit cumbersome to use and its usage require a C or C++ compiler to build a special DLL/so with a JVM friendly API.

The process is:

  1. A class with native methods is created and compiled normal with javac
  2. The javah utility is used to generate a .h file defining the API the JVM will expect
  3. One or more .c or .cpp file implementing that API is created
  4. A DLL/so is build from that and the real code

Let us get started on the example.

TestJNI.java with native methods:

package m2n;

public class TestJNI {
    // load library
    static {
        System.loadLibrary("TestJNI");
    }
    // define functions
    native public static int add(int a, int b);
    native public static String dup(String s);
    native public static int sum(int[] a);
    native public static void getOut(int[] iv, String[] sv);
    native public static Data1 getRet();
    native public static void chg(Data2 s);
}

Data1.java:

package m2n;

public class Data1 {
    private int iv;
    private String sv;
    public Data1() {
        this(0, null);
    }
    public Data1(int iv, String sv) {
        this.iv = iv;
        this.sv = sv;
    }
    public int getIv() {
        return iv;
    }
    public void setIv(int iv) {
        this.iv = iv;
    }
    public String getSv() {
        return sv;
    }
    public void setSv(String sv) {
        this.sv = sv;
    }
}

Data2.java:

package m2n;

public class Data2 {
    private int iv1;
    private String sv1;
    private int iv2;
    private String sv2;
    public Data2() {
        this(0, null, 0, null);
    }
    public Data2(int iv1, String sv1, int iv2, String sv2) {
        this.iv1 = iv1;
        this.sv1 = sv1;
        this.iv2 = iv2;
        this.sv2 = sv2;
    }
    public int getIv1() {
        return iv1;
    }
    public void setIv1(int iv1) {
        this.iv1 = iv1;
    }
    public String getSv1() {
        return sv1;
    }
    public void setSv1(String sv1) {
        this.sv1 = sv1;
    }
    public int getIv2() {
        return iv2;
    }
    public void setIv2(int iv2) {
        this.iv2 = iv2;
    }
    public String getSv2() {
        return sv2;
    }
    public void setSv2(String sv2) {
        this.sv2 = sv2;
    }
}

javah command:

javah -classpath .. m2n.TestJNI
ren m2n_TestJNI.h TestJNI.h

Generated TestJNI.h (generated by javah not handwritten):

/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class m2n_TestJNI */

#ifndef _Included_m2n_TestJNI
#define _Included_m2n_TestJNI
#ifdef __cplusplus
extern "C" {
#endif
/*
 * Class:     m2n_TestJNI
 * Method:    add
 * Signature: (II)I
 */
JNIEXPORT jint JNICALL Java_m2n_TestJNI_add
  (JNIEnv *, jclass, jint, jint);

/*
 * Class:     m2n_TestJNI
 * Method:    dup
 * Signature: (Ljava/lang/String;)Ljava/lang/String;
 */
JNIEXPORT jstring JNICALL Java_m2n_TestJNI_dup
  (JNIEnv *, jclass, jstring);

/*
 * Class:     m2n_TestJNI
 * Method:    sum
 * Signature: ([I)I
 */
JNIEXPORT jint JNICALL Java_m2n_TestJNI_sum
  (JNIEnv *, jclass, jintArray);

/*
 * Class:     m2n_TestJNI
 * Method:    getOut
 * Signature: ([I[Ljava/lang/String;)V
 */
JNIEXPORT void JNICALL Java_m2n_TestJNI_getOut
  (JNIEnv *, jclass, jintArray, jobjectArray);

/*
 * Class:     m2n_TestJNI
 * Method:    getRet
 * Signature: ()Lm2n/Data1;
 */
JNIEXPORT jobject JNICALL Java_m2n_TestJNI_getRet
  (JNIEnv *, jclass);

/*
 * Class:     m2n_TestJNI
 * Method:    chg
 * Signature: (Lm2n/Data2;)V
 */
JNIEXPORT void JNICALL Java_m2n_TestJNI_chg
  (JNIEnv *, jclass, jobject);

#ifdef __cplusplus
}
#endif
#endif

Note that packagename1.packagename2.Classname.methodname becomes a function called Java_packagename1_packagename2_Classname_methodname.

TestJNI.c implementing the functions from TestJNI.h:

#include <string.h>
#include <stdlib.h>

#include "demolib.h"

#include "TestJNI.h"

// JNI operations:
//      convert Java String to C char* - GetStringUTFChars (must call ReleaseStringUTFChars to free)
//      convert C char* to Java String - NewStringUTF (String will be freed by Java garbage collector)
//      convert Java int[] to C int[] - GetIntArrayElements (must call ReleaseIntArrayElements to free)
//      create Object - FindClass and GetMethodID and NewObject (Object will be freed by Java garbage collector)
//      get int field - FindClass and GetFieldID and GetIntField
//      get String field - FindClass and GetFieldID and SetIntField
//      set int field - FindClass and GetFieldID and GetObjectField
//      set String field - FindClass and GetFieldID and SetObjectField
JNIEXPORT jint JNICALL Java_m2n_TestJNI_add(JNIEnv *ctx, jclass jc, jint a, jint b)
{
    return add(a, b);
}

JNIEXPORT jstring JNICALL Java_m2n_TestJNI_dup(JNIEnv *ctx, jclass jc, jstring s)
{
    const char *ns, *res;
    ns = (*ctx)->GetStringUTFChars(ctx, s, 0);
    res = dup((char *)ns);
    (*ctx)->ReleaseStringUTFChars(ctx, s, ns);
    return (*ctx)->NewStringUTF(ctx, res);
}

JNIEXPORT jint JNICALL Java_m2n_TestJNI_sum(JNIEnv *ctx, jclass jc, jintArray a)
{
    jint *na, res;
    na = (*ctx)->GetIntArrayElements(ctx, a, 0);
    res = sum((*ctx)->GetArrayLength(ctx, a), (int *)na);
    (*ctx)->ReleaseIntArrayElements(ctx, a, na, 0);
    return res;
}

JNIEXPORT void JNICALL Java_m2n_TestJNI_getOut(JNIEnv *ctx, jclass jc, jintArray iv, jobjectArray sv)
{
    int tempiv;
    char *tempsv;
    jint *niv;
    get_out(&tempiv, &tempsv);
    niv = (*ctx)->GetIntArrayElements(ctx, iv, 0);
    niv[0] = tempiv;
    (*ctx)->ReleaseIntArrayElements(ctx, iv, niv, 0);
    (*ctx)->SetObjectArrayElement(ctx, sv, 0, (*ctx)->NewStringUTF(ctx, tempsv));
}

JNIEXPORT jobject JNICALL Java_m2n_TestJNI_getRet(JNIEnv *ctx, jclass jc)
{
    struct Data1 *d;
    jint iv;
    jstring sv;
    jclass data1clz;
    jmethodID data1ctor;
    jobject res;
    jfieldID ivfld, svfld;
    d = get_ret();
    iv = d->iv;
    sv = (*ctx)->NewStringUTF(ctx, d->sv);
    free(d);
    data1clz = (*ctx)->FindClass(ctx, "m2n/Data1");
    data1ctor = (*ctx)->GetMethodID(ctx, data1clz, "<init>", "()V");
    res = (*ctx)->NewObject(ctx, data1clz, data1ctor);
    ivfld = (*ctx)->GetFieldID(ctx, data1clz, "iv", "I");
    (*ctx)->SetIntField(ctx, res, ivfld, iv);
    svfld = (*ctx)->GetFieldID(ctx, data1clz, "sv", "Ljava/lang/String;");
    (*ctx)->SetObjectField(ctx, res, svfld, sv);
    return res;
}

JNIEXPORT void JNICALL Java_m2n_TestJNI_chg(JNIEnv *ctx, jclass jc, jobject d)
{
    jclass data2clz;
    jfieldID iv1fld, sv1fld, iv2fld, sv2fld;
    struct Data2 nd;
    jstring temp;
    const char *ntemp;
    data2clz = (*ctx)->FindClass(ctx, "m2n/Data2");
    iv1fld = (*ctx)->GetFieldID(ctx, data2clz, "iv1", "I");
    nd.iv1 = (*ctx)->GetIntField(ctx, d, iv1fld);
    sv1fld = (*ctx)->GetFieldID(ctx, data2clz, "sv1", "Ljava/lang/String;");
    temp = (*ctx)->GetObjectField(ctx, d, sv1fld);
    ntemp = (*ctx)->GetStringUTFChars(ctx, temp, 0);
    strcpy(nd.sv1, ntemp);
    (*ctx)->ReleaseStringUTFChars(ctx, temp, ntemp);
    chg(&nd);
    iv2fld = (*ctx)->GetFieldID(ctx, data2clz, "iv2", "I");
    (*ctx)->SetIntField(ctx, d, iv2fld, nd.iv2);
    sv2fld = (*ctx)->GetFieldID(ctx, data2clz, "sv2", "Ljava/lang/String;");
    (*ctx)->SetObjectField(ctx, d, sv2fld, (*ctx)->NewStringUTF(ctx, nd.sv2));
}

Note that Java class names are specified like package1/package2/ClassName (not package1.package2.ClassName).

Note that Java method names are specified as methodname(argtypeargtype)returntype where types are I for int, Ljava/lang/String; for String, V for void etc..

Build on Windows with GCC:

gcc -Wall -shared -I%JAVA_HOME%\include -I%JAVA_HOME%\include\win32 -Wl,--export-all,--kill-at TestJNI.c demolib.c -o TestJNI.dll

Build on Linux with GCC:

gcc -DNIX -Wall -shared -I$JAVA_HOME/include -I$JAVA_HOME/include/linux -fPIC TestJNI.c demolib.c -o libTestJNI.so

Build on VMS:

$ cc/def="VMS"/name=as_is/reent=multi/float=ieee/ieee=denorm demolib
$ cc/def="VMS"/name=as_is/reent=multi/float=ieee/ieee=denorm/include=(dka4:[vms$common.java$150.include],dka4:[vms$common.java$150.include.vms]) TestJNI
$ link/share TestJNI+demolib+sys$input/option
CASE_SENSITIVE=YES
SYMBOL_VECTOR=(Java_m2n_TestJNI_add=PROCEDURE,-
               Java_m2n_TestJNI_sum=PROCEDURE,-
               Java_m2n_TestJNI_dup=PROCEDURE,-
               Java_m2n_TestJNI_getOut=PROCEDURE,-
               Java_m2n_TestJNI_getRet=PROCEDURE,-
               Java_m2n_TestJNI_chg=PROCEDURE)
$
$ define/nolog/super TestJNI sys$disk:[]TestJNI.exe

Note that if function names become too long, then /name=(as_is,shortended) should be used and the options should be generated with @scan_globals_for_option.

TestJ.java:

package m2n;

public class TestJ {
    public static void main(String[] args) {
        int a = 123;
        int b = 456;
        int c = TestJNI.add(a, b);
        System.out.println(c);
        //
        int[] six = { 1, 2, 3, 4, 5, 6 };
        int allsix = TestJNI.sum(six);
        System.out.println(allsix);
        //
        String s = "ABC";
        String s2 = TestJNI.dup(s);
        System.out.println(s2);
        //
        int[] iv = new int[1];
        String[] sv = new String[1];
        TestJNI.getOut(iv, sv);
        System.out.println(iv[0] + " " + sv[0]);
        //
        Data1 d1 = TestJNI.getRet();
        System.out.println(d1.getIv() + " " + d1.getSv());
        //
        Data2 d2 = new Data2();
        d2.setIv1(123);
        d2.setSv1("ABC");
        TestJNI.chg(d2);
        System.out.println(d2.getIv2() + " " + d2.getSv2());
    }
}

Output:

579
21
ABCABC
123 ABC
123 ABC
124 ABCX

C++ can also be used instead of C with minimal changes.

TestJNI.cpp:

#include <string.h>
#include <stdlib.h>

extern "C"
{
#include "demolib.h"
}

#include "TestJNI.h"

// JNI operations:
//      convert Java String to C char* - GetStringUTFChars (must call ReleaseStringUTFChars to free)
//      convert C char* to Java String - NewStringUTF (String will be freed by Java garbage collector)
//      convert Java int[] to C int[] - GetIntArrayElements (must call ReleaseIntArrayElements to free)
//      create Object - FindClass and GetMethodID and NewObject (Object will be freed by Java garbage collector)
//      get int field - FindClass and GetFieldID and GetIntField
//      get String field - FindClass and GetFieldID and SetIntField
//      set int field - FindClass and GetFieldID and GetObjectField
//      set String field - FindClass and GetFieldID and SetObjectField
JNIEXPORT jint JNICALL Java_m2n_TestJNI_add(JNIEnv *ctx, jclass jc, jint a, jint b)
{
    return add(a, b);
}

JNIEXPORT jstring JNICALL Java_m2n_TestJNI_dup(JNIEnv *ctx, jclass jc, jstring s)
{
    const char *ns, *res;
    ns = ctx->GetStringUTFChars(s, 0);
    res = dup((char *)ns);
    ctx->ReleaseStringUTFChars(s, ns);
    return ctx->NewStringUTF(res);
}

JNIEXPORT jint JNICALL Java_m2n_TestJNI_sum(JNIEnv *ctx, jclass jc, jintArray a)
{
    jint *na, res;
    na = ctx->GetIntArrayElements(a, 0);
    res = sum(ctx->GetArrayLength(a), (int *)na);
    ctx->ReleaseIntArrayElements(a, na, 0);
    return res;
}

JNIEXPORT void JNICALL Java_m2n_TestJNI_getOut(JNIEnv *ctx, jclass jc, jintArray iv, jobjectArray sv)
{
    int tempiv;
    char *tempsv;
    jint *niv;
    get_out(&tempiv, &tempsv);
    niv = ctx->GetIntArrayElements(iv, 0);
    niv[0] = tempiv;
    ctx->ReleaseIntArrayElements(iv, niv, 0);
    ctx->SetObjectArrayElement(sv, 0, ctx->NewStringUTF(tempsv));
}

JNIEXPORT jobject JNICALL Java_m2n_TestJNI_getRet(JNIEnv *ctx, jclass jc)
{
    struct Data1 *d;
    jint iv;
    jstring sv;
    jclass data1clz;
    jmethodID data1ctor;
    jobject res;
    jfieldID ivfld, svfld;
    d = get_ret();
    iv = d->iv;
    sv = ctx->NewStringUTF(d->sv);
    free(d);
    data1clz = ctx->FindClass("m2n/Data1");
    data1ctor = ctx->GetMethodID(data1clz, "<init>", "()V");
    res = ctx->NewObject(data1clz, data1ctor);
    ivfld = ctx->GetFieldID(data1clz, "iv", "I");
    ctx->SetIntField(res, ivfld, iv);
    svfld = ctx->GetFieldID(data1clz, "sv", "Ljava/lang/String;");
    ctx->SetObjectField(res, svfld, sv);
    return res;
}

JNIEXPORT void JNICALL Java_m2n_TestJNI_chg(JNIEnv *ctx, jclass jc, jobject d)
{
    jclass data2clz;
    jfieldID iv1fld, sv1fld, iv2fld, sv2fld;
    struct Data2 nd;
    jstring temp;
    const char *ntemp;
    data2clz = ctx->FindClass("m2n/Data2");
    iv1fld = ctx->GetFieldID(data2clz, "iv1", "I");
    nd.iv1 = ctx->GetIntField(d, iv1fld);
    sv1fld = ctx->GetFieldID(data2clz, "sv1", "Ljava/lang/String;");
    temp = (jstring)ctx->GetObjectField(d, sv1fld);
    ntemp = ctx->GetStringUTFChars(temp, 0);
    strcpy(nd.sv1, ntemp);
    ctx->ReleaseStringUTFChars(temp, ntemp);
    chg(&nd);
    iv2fld = ctx->GetFieldID(data2clz, "iv2", "I");
    ctx->SetIntField(d, iv2fld, nd.iv2);
    sv2fld = ctx->GetFieldID(data2clz, "sv2", "Ljava/lang/String;");
    ctx->SetObjectField(d, sv2fld, ctx->NewStringUTF(nd.sv2));
}

Build on Windows with GCC:

gcc -c -Wall demolib.c -o demolib.obj
g++ -Wall -shared -I%JAVA_HOME%\include -I%JAVA_HOME%\include\win32 -Wl,--export-all,--kill-at TestJNI.cpp demolib.obj -o TestJNI.dll

I don't like JNI. It is just too cumbersome to work with.

Note that use of JNI is explicitly forbidden in the EJB spec. If JNI is needed in Java EE backend application, then it should be used in a JCA adapter - see here.

JNA:

JNA (Java Native Access) is a library that in the backend has a generic JNI API accessomg a standard DLL/so and in the frontend allow the developer to setup the specific native call in Java.

TestJNA.java:

package m2n;

import com.sun.jna.Library;
import com.sun.jna.win32.StdCallLibrary;
import com.sun.jna.Native;
import com.sun.jna.Pointer;

public interface TestJNA extends Library {
    public TestJNA INSTANCE = Native.load("stddemolib", TestJNA.class);
    public int std_add(int a, int b);
    public Pointer std_dup(String s);
    public int std_sum(int n, int[] a);
    public void std_chg(JNAData2 s);
    public void std_free(Pointer p);
}

TestJ2.java:

package m2n;

import com.sun.jna.Native;
import com.sun.jna.Pointer;
import com.sun.jna.Structure;

public class TestJ2 {
    public static void main(String[] args) {
        int a = 123;
        int b = 456;
        int c = TestJNA.INSTANCE.std_add(a, b);
        System.out.println(c);
        //
        int[] six = { 1, 2, 3, 4, 5, 6 };
        int allsix = TestJNA.INSTANCE.std_sum(six.length, six);
        System.out.println(allsix);
        //
        String s = "ABC";                                                              
        Pointer p = TestJNA.INSTANCE.std_dup(s);
        String s2 = p.getString(0);
        TestJNA.INSTANCE.std_free(p);
        System.out.println(s2);
        //
        int[] iv = new int[1];
        String[] sv = new String[1];
        Pointer[] pa = new Pointer[1];
        TestJNA.INSTANCE.std_get_out(iv, pa);
        sv[0] = pa[0].getString(0);
        TestJNA.INSTANCE.std_free(pa[0]);
        System.out.println(iv[0] + " " + sv[0]);
    }
}

Output:

579
21
ABCABC
123 ABC

I could not get the two last examples (returning pointer to strict and modifying struct to work. It should be possible, but I was not able to make it work.

It is obvious that JNA replace some relative complex C code with some relative simple Java code and that is an obvious benefit for a Java developer.

There are many people that strongly prefer JNA over JNI. I am not so convinced. I will avoid native code if possible (when doing Java). If not possible then I would use JNI. JNI is a lot of work but there are no big unknowns - it will work. JNA seems to work fine for simple examples, but if one encounter a problem then one is stuck.

.NET:

P/Invoke or DllImport:

.NET has a very simple way of calling a C function in a DLL. The feature is called p/Invoke or DllImport.

In simple cases .NET can do the serialization/deserialization automatically. In more complex cases a little bit of custom code is needed.

TestDN.cs:

using System;
using System.Runtime.InteropServices;
using System.Text;

public class TestDN
{
    // structs are defined with explicit layout so they can be serialized/desrialized to/from C stucts 
    [StructLayout(LayoutKind.Sequential,CharSet=CharSet.Ansi,Pack=4)]
    public struct MgData1
    {
        public int iv;
        [MarshalAs(UnmanagedType.ByValTStr,SizeConst=50)]
        public string sv;
    }
    [StructLayout(LayoutKind.Sequential,CharSet=CharSet.Ansi,Pack=4)]
    public struct MgData2
    {
        public int iv1;
        [MarshalAs(UnmanagedType.ByValTStr,SizeConst=50)]
        public string sv1;
        public int iv2;
        [MarshalAs(UnmanagedType.ByValTStr,SizeConst=50)]
        public string sv2;
    }
    // DLL functions are defined, arguments and return values:
    //     in int -> int
    //     out int -> int
    //     return int -> int
    //     in int array -> int[]
    //     in string -> string
    //     out string -> IntPtr (converted to string with Marshal.PtrToStringAnsi)
    //     return string -> IntPtr (converted to string with Marshal.PtrToStringAnsi)
    //     inout struct -> IntPtr (converted from/to struct with Marshal.PtrToStructure/Marshal.PtrToStructure
    //     return struct -> IntPtr (converted to struct with Marshal.PtrToStructure)
    [DllImport("stddemolib.dll")]
    public static extern int std_add(int a, int b);
    [DllImport("stddemolib.dll")]
    public static extern int std_sum(int n, int[] a);
    [DllImport("stddemolib.dll")]
    public static extern IntPtr std_dup(string s);
    [DllImport("stddemolib.dll")]
    public static extern void std_get_out(out int iv, out IntPtr sv);
    [DllImport("stddemolib.dll")]
    public static extern IntPtr std_get_ret();
    [DllImport("stddemolib.dll")]
    public static extern void std_chg(IntPtr p);
    [DllImport("stddemolib.dll")]
    public static extern void std_free(IntPtr s);
    public static void Main(string[] args) 
    {
        int a = 123;
        int b = 456;
        int c = std_add(a, b);
        Console.WriteLine(c);
        //
        int[] six = { 1, 2, 3, 4, 5, 6 };
        int allsix = std_sum(six.Length, six);
        Console.WriteLine(allsix);
        //
        string s = "ABC";
        IntPtr temps = std_dup(s);
        string s2 = Marshal.PtrToStringAnsi(temps);
        Console.WriteLine(s2);
        std_free(temps);
        //
        int iv;
        IntPtr tempsv;
        std_get_out(out iv, out tempsv);
        string sv = Marshal.PtrToStringAnsi(tempsv);
        Console.WriteLine("{0} {1}", iv, sv);
        std_free(tempsv);
        //
        IntPtr p1 = std_get_ret();
        MgData1 d1 = (MgData1)Marshal.PtrToStructure(p1, typeof(MgData1));
        std_free(p1);
        Console.WriteLine("{0} {1}", d1.iv, d1.sv);
        //
        MgData2 d2 = new MgData2();
        d2.iv1 = 123;
        d2.sv1 = "ABC";
        IntPtr p2 = Marshal.AllocHGlobal(Marshal.SizeOf(d2));
        Marshal.StructureToPtr(d2, p2, true);
        std_chg(p2);
        d2 = (MgData2)Marshal.PtrToStructure(p2, typeof(MgData2));
        Console.WriteLine("{0} {1}", d2.iv2, d2.sv2);
    }
}

Output:

579
21
ABCABC
123 ABC
123 ABC
124 ABCX

When I first learned to use DllImport then I were a bit skeptical, but it actually works well. And since it is seeing some usage out in the real world, then it is well tested and easy to get help to.

C++/CLI mixed mode:

MSVC++ is a very advanced C++ compiler. It supports 3 different modes:

  1. Pure native code (standard C++)
  2. Pure managed code (in the language C++/CLI that are defined in ECMA 372 standard)
  3. Mixed with both native and managed code

C++/CLI mixed mode is an convenient way to get from managed code to native code.

Data classes in MgData.cs:

public struct MgData1
{
    public int iv;
    public string sv;
};

public struct MgData2
{
    public int iv1;
    public string sv1;
    public int iv2;
    public string sv2;
}

C++/CLI wrapper code in mixdemolib.cpp:

// standard C++
#include <cstring>
#include <cstdlib>

// standard .NET
#using <mscorlib.dll>
using namespace System;
using namespace System::Runtime::InteropServices;

// the C functions
extern "C"
{
#include "demolib.h"
}

// the data classes
#using "MgData.dll"

// class with wrapper methods
public ref class Mix
{
    public:
        static int MixAdd(int a, int b);
        static int MixSum(array<int>^ a);
        static String^ MixDup(String^ s);
        static void MixGetOut([out] int% iv, String^% sv);
        static MgData1 MixGetRet();
        static void MixChg(MgData2% d);
};

int Mix::MixAdd(int a, int b)
{
    return add(a, b);
}

int Mix::MixSum(array<int>^ a)
{
    pin_ptr<int> p = &a[0];
    return sum(a->Length, p);
}

String^ Mix::MixDup(String^ s)
{
    IntPtr p = Marshal::StringToHGlobalAnsi(s);
    char *s1 = static_cast<char *>(p.ToPointer());
    char *s2 = dup(s1);
    String^ res = gcnew String(s2);
    Marshal::FreeHGlobal(p);
    free(s2);
    return res;
}

void Mix::MixGetOut(int% iv, String^% sv)
{
    int tempiv;
    char *tempsv;
    get_out(&tempiv, &tempsv);
    iv = tempiv;
    sv = gcnew String(tempsv);
    free(tempsv);
}

MgData1 Mix::MixGetRet()
{
    Data1 *d = get_ret();
    MgData1 res;
    res.iv = d->iv;
    res.sv = gcnew String(d->sv);
    return res;
}

void Mix::MixChg(MgData2% d)
{
    struct Data2 nd;
    nd.iv1 = d.iv1;
    IntPtr p = Marshal::StringToHGlobalAnsi(d.sv1);
    strcpy(nd.sv1, static_cast<char *>(p.ToPointer()));
    Marshal::FreeHGlobal(p);
    chg(&nd);
    d.iv2 = nd.iv2;
    d.sv2 = gcnew String(nd.sv2);
}

TestMix.cs:

using System;
using System.Runtime.InteropServices;
using System.Text;

public class TestMix
{
    public static void Main(string[] args) 
    {
        int a = 123;
        int b = 456;
        int c = Mix.MixAdd(a, b);
        Console.WriteLine(c);
        //
        int[] six = { 1, 2, 3, 4, 5, 6 };
        int allsix = Mix.MixSum(six);
        Console.WriteLine(allsix);
        //
        string s = "ABC";
        string s2 = Mix.MixDup(s);
        Console.WriteLine(s2);
        //
        int iv;
        string sv;
        Mix.MixGetOut(out iv, out sv);
        Console.WriteLine("{0} {1}", iv, sv);
        //
        MgData1 d1 = Mix.MixGetRet();
        Console.WriteLine("{0} {1}", d1.iv, d1.sv);
        //
        MgData2 d2 = new MgData2();
        d2.iv1 = 123;
        d2.sv1 = "ABC";
        Mix.MixChg(out d2); // note out even though it is really ref
        Console.WriteLine("{0} {1}", d2.iv2, d2.sv2);
    }
}
Build:
csc /t:library MgData.cs
cl /c /MD demolib.c
cl /clr /LD mixdemolib.cpp demolib.obj
csc /r:MgData.dll /r:mixdemolib.dll TestMix.cs

Output:

579
21
ABCABC
123 ABC
123 ABC
124 ABCX

I think C++/CLI mixed mode is a very impressive technology. And I believe it is the most flexible way to integrate managed code and native code in .NET. But I must admit that it seems a little bit like black magic to me and I am not super comfortable relying on it.

Python:

ctypes:

Python comes with a module to call C functions in DLL/so. The module is named ctypes.

pydemolib.py:

from ctypes import *
from numpy import *

# define native structs
class Data1(Structure):
    _fields_ = [('iv', c_int), ('sv', c_char * 50)]

class Data2(Structure):
    _fields_ = [('iv1', c_int), ('sv1', c_char * 50), ('iv2', c_int), ('sv2', c_char * 50)]

# load library
cdll.LoadLibrary('stddemolib.dll')

# define function prototypes
demolib = cdll.stddemolib
demolib.std_add.argtypes = [c_int, c_int]
demolib.std_add.restype = c_int
demolib.std_sum.argtypes = [c_int, ctypeslib.ndpointer(dtype=int32)]
demolib.std_sum.restype = c_int
demolib.std_dup.argtypes = [c_char_p]
demolib.std_dup.restype = c_char_p
demolib.std_get_out.argtypes = [POINTER(c_int), POINTER(c_char_p)]
demolib.std_get_out.restype = None
demolib.std_get_ret.argtypes = []
demolib.std_get_ret.restype = POINTER(Data1)
demolib.std_chg.argtypes = [POINTER(Data2)]
demolib.std_chg.restype = None

# handling data types:
#     in int as is
#     out int wrap with byref function and use value
#     return int as is
#     in int array wrap with array function
#     in string wrap with str function and encode to byte array
#     out string wrap with byref function, use value and decode from byte array
#     return string decode from byte array
#     inout struct wrap with byref function
#     return struct use contents
def add(a, b):
    return demolib.std_add(a, b)

def sum(a):
    return demolib.std_sum(len(a), array(a))

def dup(s):
    return demolib.std_dup(str(s).encode('UTF-8')).decode('UTF-8')

def get_out(): 
    iv = c_int()
    sv = c_char_p()
    demolib.std_get_out(byref(iv), byref(sv))
    return (iv.value, sv.value.decode('UTF-8'))

def get_ret():
    d = demolib.std_get_ret()
    return (d.contents.iv, d.contents.sv.decode('UTF-8'))

def chg(d):
    nd = Data2();
    nd.iv1 = d.iv1
    nd.sv1 = str(d.sv1).encode('UTF-8')
    demolib.std_chg(byref(nd))
    d.iv2 = nd.iv2
    d.sv2 = nd.sv2.decode('UTF-8')
    return d

Change cdll.LoadLibrary('stddemolib.dll') to cdll.LoadLibrary('stddemolib.so') on *nix and to cdll.LoadLibrary('stddemolib') with a logical on VMS.

TestPy.py:

import pydemolib

class PyData1(object):
    def __init__(self, _iv = 0, _sv = ''):
        self.iv = _iv
        self.sv = _sv

class PyData2(object):
    def __init__(self, _iv1 = 0, _sv1 = '', _iv2 = 0, _sv2 = ''):
        self.iv1 = _iv1
        self.sv1 = _sv1
        self.iv2 = _iv2
        self.sv2 = _sv2
        
a = 123
b = 456
c = pydemolib.add(a, b)
print(c)

six = [1, 2, 3, 4, 5, 6]
allsix = pydemolib.sum(six)
print(allsix)

s = 'ABC'
s2 = pydemolib.dup(s)
print(s2)

(iv, sv) = pydemolib.get_out()
print('%d %s' % (iv, sv))

d = PyData1()
(d.iv, d.sv) = pydemolib.get_ret()
print('%d %s' % (d.iv, d.sv))

d = PyData2()
d.iv1 = 123
d.sv1 = 'ABC'
pydemolib.chg(d)
print('%d %s' % (d.iv2, d.sv2))

Output:

579
21
ABCABC
123 ABC
123 ABC
124 ABCX

Windows COM:

Concept:

COM/DCOM/COM+/ActiveX is a core Windows technology.

It covers a lot of different functionality:

Here we will focus on the two first aspects.

At the very lowest level a COM class is just a class implementing the interface IUnknown and following some conventions.

COM classes that need to support dynamic API explorartion must also implement the interface IDispatch. This is needed to support script languages without strong type system.

But this article will not focus on the lower levels - instead it will focus on how it practically can be used for cross language calls.

For more details on COM see the COM article.

Server:

Writing a COM component in C++ is not that simple.

Basically there are two approaches:

Since this article is about using COM not understanding the internals, then it will take the second approach as it is significant less code to write - and live with the fact that it is far from standard C++.

Here comes an example.

First we define th einterface in IDL (Interface Definition Language).

SNat.idl:

import "oaidl.idl";

// IS
[object, uuid(CBF5E889-BFBA-41FC-83DF-98156E999C21), oleautomation, dual, pointer_default(unique)]
interface IS : IDispatch
{
    [id(1)] HRESULT Add([in] long a, [in] long b, [out,retval] long *c);
    [id(2)] HRESULT Dup([in] BSTR s, [out,retval]BSTR *s2);
    [id(3),propget] HRESULT Iv([out, retval] long *iv);
    [id(3),propput] HRESULT Iv([in] long iv);
    [id(4),propget] HRESULT Sv([out, retval] BSTR *sv);
    [id(4),propput] HRESULT Sv([in] BSTR sv);
    [id(5)] HRESULT M();
}

// S library
[uuid(170E2745-A02D-416D-A3FA-4160ACE70F41), version(1.0)]
library SLibrary
{
    importlib("stdole32.tlb");
    [uuid(DB83A5B9-D6CC-4807-9755-BBD02755DD67)]
    coclass CoS
    {
        [default] interface IS;
    };
};

We compile that with the MIDL compiler:

midl SNat.idl

and we get a bunch of files including SNat.h with a C++ interface IS and a SNat.tlb type library, that will be used by languages with strong type system.

We define a class that use stuff from the generated files and the builtin support for COM.

SNatEx.h:

#ifndef SNAT_H
#define SNAT_H

#include <windows.h>
#include <tchar.h>

#include <atlbase.h>
#include <atlcom.h>

#include "SNat_i.c"
#include "SNat.h"

class ATL_NO_VTABLE CCoS : public CComObjectRootEx<CComSingleThreadModel>, public CComCoClass<CCoS, &CLSID_CoS>, public IDispatchImpl<IS, &IID_IS, &LIBID_SLibrary>
{
private:
    long m_iv;
    BSTR m_sv;
public:
    CCoS();
    virtual ~CCoS();
    BEGIN_COM_MAP(CCoS)
       COM_INTERFACE_ENTRY(IS)
       COM_INTERFACE_ENTRY2(IDispatch, IS)
    END_COM_MAP()
    // IS
    STDMETHODIMP Add(long a, long b, long *c);
    STDMETHODIMP Dup(BSTR s, BSTR *s2);
    STDMETHODIMP get_Iv(long *iv);
    STDMETHODIMP put_Iv(long iv);
    STDMETHODIMP get_Sv(BSTR *sv);
    STDMETHODIMP put_Sv(BSTR sv);
    STDMETHODIMP M();
    static HRESULT WINAPI UpdateRegistry(BOOL b)
    {
        return _Module.UpdateRegistryClass(CLSID_CoS, _T("COM.SNat.1"), _T("COM.SNat"), 0U, THREADFLAGS_APARTMENT, b);
    }
};

#endif

We provide the implementation for that class and some registration functions.

SNat.cpp:

#include <windows.h>
#include <oleauto.h>
#include <initguid.h>

#include <atlbase.h>
extern CComModule _Module;
#include <atlcom.h>

#include "SNatEx.h"

CComModule _Module;

BEGIN_OBJECT_MAP(ObjectMap)
    OBJECT_ENTRY(CLSID_CoS, CCoS)
END_OBJECT_MAP()

CCoS::CCoS()
{
    m_iv = 0;
    m_sv = SysAllocString(L"");
}

CCoS::~CCoS()
{
    SysFreeString(m_sv);
}

STDMETHODIMP CCoS::Add(long a, long b, long *c)
{
    *c = a + b;
    return S_OK;
}

STDMETHODIMP CCoS::Dup(BSTR s, BSTR *s2)
{
    OLECHAR *buf = new OLECHAR[2 * SysStringLen(s) + 1];
    wcscpy(buf, s);
    wcscat(buf, s);
    *s2 = SysAllocString(buf);
    delete[] buf;
    return S_OK;
}

STDMETHODIMP CCoS::get_Iv(long *iv)
{
    *iv = m_iv;
    return S_OK;
}

STDMETHODIMP CCoS::put_Iv(long iv)
{
    m_iv = iv;
    return S_OK;
}

STDMETHODIMP CCoS::get_Sv(BSTR *sv)
{
    *sv = SysAllocString(m_sv);
    return S_OK;
}

STDMETHODIMP CCoS::put_Sv(BSTR sv)
{
    SysReAllocString(&m_sv, sv);
    return S_OK;
}

STDMETHODIMP CCoS::M()
{
    m_iv = m_iv + 1;
    OLECHAR *buf = new OLECHAR[SysStringLen(m_sv) + 1 + 1];
    wcscpy(buf, m_sv);
    wcscat(buf, L"X");
    SysFreeString(m_sv);
    m_sv = SysAllocString(buf);
    delete[] buf;
    return S_OK;
}

BOOL WINAPI DllMain(HINSTANCE hInstance, DWORD dwReason, LPVOID lpReserved)
{
    if(dwReason == DLL_PROCESS_ATTACH)
    {
        _Module.Init(ObjectMap, hInstance, &LIBID_SLibrary);
    }
    else if(dwReason == DLL_PROCESS_DETACH)
    {
        _Module.Term();
    }
    return TRUE;
}

STDMETHODIMP DllGetClassObject(REFCLSID rclsid, REFIID riid, void **ppv)
{
    return _Module.GetClassObject(rclsid, riid, ppv);
}

STDMETHODIMP DllCanUnloadNow()
{
    return (_Module.GetLockCount() == 0) ? S_OK : S_FALSE;
}

STDMETHODIMP DllRegisterServer()
{
    return _Module.RegisterServer(TRUE);
}

STDMETHODIMP DllUnregisterServer()
{
    return _Module.UnregisterServer(TRUE);
}

Build:

cl /LD SNat.cpp SNat.def oleaut32.lib

Registration:

regsvr32 SNat.dll

Be sure to register it correctly as either a 64 bit or 32 bit COM component.

Client:

We will see how to call the above COM component from C#, VB.NET, VBScript and JScript.

using System;

namespace COM
{
    public class MainClass
    {
        public static void Main(string[] args)
        {
            CoS cosnat = new CoSClass();
            IS snat = (IS)cosnat;
            int a = 123;
            int b = 456;
            int c = snat.Add(a, b);
            Console.WriteLine(c);
            string s = "ABC";
            string s2 = snat.Dup(s);
            Console.WriteLine(s2);
            snat.Iv = 123;
            snat.Sv = "ABC";
            snat.M();
            int iv = snat.Iv;
            string sv = snat.Sv;
            Console.WriteLine(iv + " " + sv);
        }
    }
}

Build:

tlbimp /out=SNatWrap.dll /namespace:COM SNat.tlb
csc /r:SNatWrap.dll C.cs

Output:

579
ABCABC
124 ABCX
Imports System

Namespace COM
    Public Class MainClass
        Public Shared Sub Main(args As String())
            Dim cosnat As CoS = New CoSClass()
            Dim snat As [IS] = DirectCast(cosnat, [IS])
            Dim a As Integer = 123
            Dim b As Integer = 456
            Dim c As Integer = snat.Add(a, b)
            Console.WriteLine(c)
            Dim s As String = "ABC"
            Dim s2 As String = snat.Dup(s)
            Console.WriteLine(s2)
            snat.Iv = 123
            snat.Sv = "ABC"
            snat.M()
            Dim iv As Integer = snat.Iv
            Dim sv As String = snat.Sv
            Console.WriteLine(iv & " " & sv)
        End Sub
    End Class
End Namespace

Build:

tlbimp /out=SNatWrap.dll /namespace:COM SNat.tlb
vbc /r:SNatWrap.dll C.vb

Output:

579
ABCABC
124 ABCX
using System;

namespace COM
{
    public class MainClass
    {
        public static void Main(string[] args)
        {
            dynamic sdn = Activator.CreateInstance(Type.GetTypeFromProgID("COM.SNat"));
            int a = 123;
            int b = 456;
            int c = sdn.Add(a, b);
            Console.WriteLine(c);
            string s = "ABC";
            string s2 = sdn.Dup(s);
            Console.WriteLine(s2);
            sdn.Iv = 123;
            sdn.Sv = "ABC";
            sdn.M();
            int iv = sdn.Iv;
            string sv = sdn.Sv;
            Console.WriteLine(iv + " " + sv);
        }
    }
}

Build:

csc CDyn.cs

Output:

579
ABCABC
124 ABCX
Option Strict Off

Imports System

Namespace COM
    Public Class MainClass
        Public Shared Sub Main(args As String())
            Dim sdn As Object = Activator.CreateInstance(Type.GetTypeFromProgID("COM.SNat"))
            Dim a As Integer = 123
            Dim b As Integer = 456
            Dim c As Integer = sdn.Add(a, b)
            Console.WriteLine(c)
            Dim s As String = "ABC"
            Dim s2 As String = sdn.Dup(s)
            Console.WriteLine(s2)
            sdn.Iv = 123
            sdn.Sv = "ABC"
            sdn.M()
            Dim iv As Integer = sdn.Iv
            Dim sv As String = sdn.Sv
            Console.WriteLine(iv & " " & sv)
        End Sub
    End Class
End Namespace

Build:

vbc CDyn.vb

Output:

579
ABCABC
124 ABCX
Set sdn = CreateObject("COM.SNat")
a = 123
b = 456
c = sdn.Add(a, b)
WScript.Echo CStr(c)
s = "ABC"
s2 = sdn.Dup(s)
WScript.Echo s2
sdn.Iv = 123
sdn.Sv = "ABC"
call sdn.M
iv = sdn.Iv
sv = sdn.Sv
WScript.Echo CStr(iv) & " " & sv 
Set sdn = Nothing

Run:

cscript c.vbs

Output:

579
ABCABC
124 ABCX
var sdn = new ActiveXObject("COM.SNat");
var a = 123;
var b = 456;
var c = sdn.Add(a, b);
WScript.Echo(c);
var s = "ABC";
var s2 = sdn.Dup(s);
WScript.Echo(s2);
sdn.Iv = 123;
sdn.Sv = "ABC";
sdn.M();
var iv = sdn.Iv;
var sv = sdn.Sv;
WScript.Echo(iv + " " + sv); 
sdn = null;

Run:

cscript c.js

Output:

579
ABCABC
124 ABCX

The old Microsoft Java came with support for COM.

Other Java implementations and that include all recent does not support COM. But an open source project Jacob adds the capability.

Strictly speaking Jacob does not call COM from Java. Strictly speaking Jacob use JNI to call some C/C++ code in a DLL that then call COM. But from a practical developer perspective Jacob "calls" COM.

Jacob provide two classes for calling COM:

Dispatch
procedural style
ActiveXCompnent
object orieneted style

Both as names indicateuse IDispatch and dynamic API.

This is Dispatch:

import com.jacob.com.Dispatch;
import com.jacob.com.Variant;

public class C1 {
    public static void main(String[] args) {
        Dispatch snat = new Dispatch("COM.SNat");
        int a = 123;
        int b = 456;
        int c = Dispatch.callN(snat, "Add", new Variant(a), new Variant(b)).getInt();
        System.out.println(c);
        String s = "ABC";
        String s2 = Dispatch.callN(snat, "Dup", new Variant(s)).getString();
        System.out.println(s2);
        Dispatch.put(snat, "Iv", new Variant(123));
        Dispatch.put(snat, "Sv", new Variant("ABC"));
        Dispatch.callN(snat, "M");
        int iv = Dispatch.get(snat, "Iv").getInt();
        String sv = Dispatch.get(snat, "Sv").getString();
        System.out.println(iv + " " + sv);
    }
}

Build and run:

javac -cp %JACOBDIR%\jacob.jar C1.java
java -Djava.library.path=%JACOBDIR% -cp .;%JACOBDIR%\jacob.jar C1

Output:

579
ABCABC
124 ABCX

The old Microsoft Java came with support for COM.

Other Java implementations and that include all recent does not support COM. But an open source project Jacob adds the capability.

Strictly speaking Jacob does not call COM from Java. Strictly speaking Jacob use JNI to call some C/C++ code in a DLL that then call COM. But from a practical developer perspective Jacob "calls" COM.

Jacob provide two classes for calling COM:

Dispatch
procedural style
ActiveXCompnent
object orieneted style

Both as names indicate use IDispatch and dynamic API.

This is ActiveXComponent:

import com.jacob.activeX.ActiveXComponent;
import com.jacob.com.Variant;

public class C2 {
    public static void main(String[] args) {
        ActiveXComponent snat = new ActiveXComponent("COM.SNat");
        int a = 123;
        int b = 456;
        int c = snat.invoke("Add", new Variant(a), new Variant(b)).getInt();
        System.out.println(c);
        String s = "ABC";
        String s2 = snat.invoke("Dup", new Variant(s)).getString();
        System.out.println(s2);
        snat.setProperty("Iv", 123);
        snat.setProperty("Sv", "ABC");
        snat.invoke("M");
        int iv = snat.getPropertyAsInt("Iv");
        String sv = snat.getPropertyAsString("Sv");
        System.out.println(iv + " " + sv);
    }
}

Build and run:

javac -cp %JACOBDIR%\jacob.jar C2.java
java -Djava.library.path=%JACOBDIR% -cp .;%JACOBDIR%\jacob.jar C2

Output:

579
ABCABC
124 ABCX

Article history:

Version Date Description
1.0 February 2nd 2020 Initial version
1.1 February 9th 2020 Add Windows COM
1.2 February 22nd 2020 Add Java Jacob

Other articles:

See list of all articles here

Comments:

Please send comments to Arne Vajhøj