Skip to Content

tag

Tag Archives: .NET

Obstacles when communicating with TwinCAT from F# (plus Workarounds)

I took a detour from main goal of project, which is type provider, in order to provide with lightweight fsharpish library for ADS communication. To my surprise a lot of drawbacks popped out, here you may find a descriptions of them, plus workarounds for some.

Marshalling structures with BOOL arrays.

On one hand, according to Infosys BOOL type has size of 1 byte (where 1 is true, 0 is false, while any other value is incorrect as it can’t be interpreted correctly), same 1 byte size for bool is in .NET. On the other hand, we have Win32 BOOL, which is 4 byte size (more detailed on SO).  Which is scenario we can easily handle in C#, but not in F#.

[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct BoolArrSample
{
  [MarshalAs(UnmanagedType.ByValArray, SizeConst = 10, ArraySubType = UnmanagedType.I1)]
  public readonly bool[] values;
}
.class public sequential ansi sealed beforefieldinit Structs.BoolArrSample
  extends [mscorlib]System.ValueType
{
  .pack 1
  .size 0

  .field public initonly marshal(fixed array[10] int8( bool[] values
}
[<Struct;StructLayout(LayoutKind.Sequential, Pack = 1)>]
type BoolArrSample =
  [<MarshalAs(UnmanagedType.ByValArray, SizeConst = 10, ArraySubType = UnmanagedType.I1)>]
  val  values: BOOL[]
.class public sequential ansi sealed serializable NuSoft.Ads.Experimental.Samples.BoolArrSample
  extends [mscorlib]System.ValueType
  implements class [mscorlib]System.IEquatable`1<valuetype NuSoft.Ads.Experimental.Samples.BoolArrSample>,
             [mscorlib]System.Collections.IStructuralEquatable,
             class [mscorlib]System.IComparable`1<valuetype NuSoft.Ads.Experimental.Samples.BoolArrSample>,
             [mscorlib]System.IComparable,
             [mscorlib]System.Collections.IStructuralComparable
{
  .custom instance void [FSharp.Core]Microsoft.FSharp.Core.StructAttribute::.ctor() = (
    01 00 00 00
  )
  .custom instance void [FSharp.Core]Microsoft.FSharp.Core.CompilationMappingAttribute::.ctor(valuetype [FSharp.Core]Microsoft.FSharp.Core.SourceConstructFlags) = (
    01 00 03 00 00 00 00 00
  )
  .pack 1
  .size 0

  .field assembly marshal(fixed array[10]( bool[] values@

  .property instance bool[] values()
  {
    .custom instance void [FSharp.Core]Microsoft.FSharp.Core.CompilationMappingAttribute::.ctor(valuetype [FSharp.Core]Microsoft.FSharp.Core.SourceConstructFlags,  int32) = (
      01 00 04 00 00 00 00 00 00 00 00 00
    )
    .get instance bool[] NuSoft.Ads.Experimental.Samples.BoolArrSample::get_values()
    {
      IL_0000: ldarg.0
      IL_0001: ldfld bool[] NuSoft.Ads.Experimental.Samples.BoolArrSample::values@
      IL_0006: ret
    }
  }
}

as you may notice, there’s no ArraySubType value in F# sample IL output. If you look at a visual fsharp sources you’ll notice that ArraySubType is disregarded. Click here for open issue.

Workaround for marhsalling byte arrays in  F#

Other than fixing the issue, one can use Struct array instead of  bool array, though it’s not much convenient.

[<AutoOpen>]
module Workarounds =
  open System.Runtime.InteropServices
  
  [<Struct;StructLayout(LayoutKind.Sequential, Pack=1)>]
  type BOOLSTRUCT = 
    val value: BOOL

    new(value) = {
      value=value
    }
    
  let inline BOOL (x: BOOL) = new BOOLSTRUCT(x)
  let inline BOOLSTRUCT (x: BOOLSTRUCT) = x.value

Marshalling TwinCAT TOD, DATETIME, TIME, DATE types

In TwinCAT time and date types are all 4 bytes, closest out-of-the-box equivalents in .Net world are DateTime for DATE_AND_TIME and DATE, and  TimeSpan for TIME_OF_DAY and TIME, problem is both of them are 8 bytes long.

Marshalling DateTime/TimeSpan values

If you read trough Infosys, you’ll find Reading and writing of TIME/DATE variables, it utilizes AdsStream and AdsBinaryReader/AdsBinaryWriter, but personally prefer to use other methods provided by TcAdsClient.dll.

[<AutoOpen>]
module Ext =
  type TcAdsClient with 
        member self.ReadDateTime(variableHandle) = 
          self.ReadAny(variableHandle, typedefof<uint32>) 
          :?> uint32
          |> int64 
          |> TwinCAT.Ads.Internal.PlcOpenDateConverterBase.ToDateTime

        member self.ReadTimeSpan(variableHandle) = 
          self.ReadAny(variableHandle, typedefof<uint32>) 
          :?> uint32
          |> int64 
          |> TwinCAT.Ads.Internal.PlcOpenTimeConverter.ToTimeSpan

        member self.WriteAny(variableHandle, value) = 
          let v = 
            value 
            |> TwinCAT.Ads.Internal.PlcOpenDateConverterBase.ToTicks
            |> uint32
          self.WriteAny(variableHandle, v) 
        
        member self.WriteAny(variableHandle, value) = 
          let v = 
            value 
            |> TwinCAT.Ads.Internal.PlcOpenTimeConverter.ToTicks
            |> uint32
          self.WriteAny(variableHandle, v) 

you can always use code from Infosys as an extension methods as well.

Marshalling DateTime/TimeSpan structure fields

When it comes to workaround for structure fields there is no workaround that I would like, at least not among those that I know to work. I had few approaches to solutions that do not work, and as I learned couldn’t even possible work. May say one learns by failing. Let’s start with one notable failure, as it also comes with F# missing functionality.

Custom marshallers

At first I thought that it would be possible to marshal one filed with usage of custom marshaller, well it doesn’t work that way.

I may be wrong at this, but Custom marshaller is not an option even if we invoked native method from tcadsdll.dll, as the signature

[DllImport("tcadsdll.dll", CharSet=CharSet.None, ExactSpelling=false)]
public static extern unsafe AdsErrorCode AdsSyncReadReq(byte* addr, uint indexGroup, uint indexOffset, int length, void* data);

expects void pointer not actual structure

Workaround

[<Struct;StructLayout(LayoutKind.Sequential, Pack = 1)>]
type DateTimeArraySample =
  [<MarshalAs(UnmanagedType.ByValArray, SizeConst = 10, ArraySubType = UnmanagedType.U4)>]
  val values: uint32[]

  member self.Values =
    self.values 
    |> Array.map (int64 >> TwinCAT.Ads.Internal.PlcOpenDateConverterBase.ToDateTime)

what I do not like about is obfuscating structure with additional properties, and even if we’d use extension properties/methods, it still involves duplicating every single DT/TOD/D/T fields.

Note on custom marshallers in F#

Putting it simply, they cannot be used ATM, SO question. Still not sure why, but actual code for read and write IL, is present in sources, yet when parsing MarshalAsAttribute exception is thrown explicitly, though it doesn’t seem that hard to fix it, I have it working on modified sources. If I only knew how to test it properly…

Extending AutoFixture with custom ISpecimenBuilder

At times AutoFixture out-of-a-box behavior and capabilities, fall to short when we require more customized test data. Such was my case. In previous post [click] I  mentioned that I prefer not to provide objects with invalid data, as a matter of fact, it’s not only a case of personal preference. If we declare array of certain length via MarshalAsAttribute, and then set different one, will end up with exception. Thus the idea of generating test data based on MarshalAsAttribute, since we already have access to it, shouldn’t be a problem. One more thing worth mentioning. At some point I’ll be mocking ADS client, since CI build server won’t have access to actual TwinCAT runtime. Default AutoFixture behavior throws exception if we try to generate structure with default constructor only, if we only go with suggested SupportMutableValueTypesCustomization we’ll end up with structure which has all fields set with default values, isn’t the thing we need, is it?

Implementing ISpecimenBuilder

First thing we ought to do is implementing ISpecimenBuilder and it’s quite simple as it only has one method.

let MarshalAsTypeBuilder =
  {
    new ISpecimenBuilder with
    member self.Create(request, context) =
      
      match request with
        | : ? Type as t when 
          t.IsValueType 
          && t.IsPrimitive |> not
          && Array.contains t 
            [|
              typedefof<string>;
              typedefof<decimal>;
              typedefof<DateTime>;
              typedefof<DateTimeOffset>;
              typedefof<TimeSpan>
            |] |> not ->
              let instance = Activator.CreateInstance(t)
              instance
        | _ -> new NoSpecimen() :> obj
  }

What we’re doing here is verifying if the current context is type of our interest. If not we fall back to default behavior on line 20. As you may noticed we still only have an instance with default values.

Providing data for struct fields

before we proceed, let’s take a look on code compiled from F#.

[<StructLayout(LayoutKind.Sequential, Pack=1)>]
type StringTestStruct =
  struct
    [<MarshalAs(UnmanagedType.ByValTStr, SizeConst=81)>]
    val string80Var:string
    [<MarshalAs(UnmanagedType.ByValTStr, SizeConst=2)>]
    val string1Var:string
    [<MarshalAs(UnmanagedType.ByValTStr, SizeConst=257)>]
    val string256Var:string
  end

[CompilationMapping(SourceConstructFlags.ObjectType)]
[Serializable]
public struct StringTestStruct : IEquatable<Tests.StringTestStruct>, IStructuralEquatable, IComparable<Tests.StringTestStruct>, IComparable, IStructuralComparable
{
    internal string string80Var@;

    internal string string1Var@;

    internal string string256Var@;

    [CompilationMapping(SourceConstructFlags.Field, 1)]
    public string string1Var
    {
        get
        {
            return this.string1Var@;
        }
    }

    [CompilationMapping(SourceConstructFlags.Field, 2)]
    public string string256Var
    {
        get
        {
            return this.string256Var@;
        }
    }

    [CompilationMapping(SourceConstructFlags.Field, 0)]
    public string string80Var
    {
        get
        {
            return this.string80Var@;
        }
    }
}

F# hides struct fields behind properties. Properties itself are of no interest for us as they do not store MarshalAsAttribute metadata, thus we’ll be iterating over non-public instance fields.

let instance = Activator.CreateInstance(t)

t.GetFields(BindingFlags.Instance ||| BindingFlags.NonPublic ||| BindingFlags.Public)
|> Array.map (fun fi -> fi,context.Resolve(fi))
|> Array.iter (fun (fi, value) -> 
  fi.SetValue(instance, value)
)
instance

At this point we have are supplied with struct instance with default AutoFixture data, which do not correspond to MarshalAsAttribute data.

Assuring valid data

As said in beginning, we have to provide data for arrays and strings accordingly to information we provided via MarshalAsAttribute.

Thus we’ll have to override default behavior for those as well.

Strings not longer that SizeConst

| : ? FieldInfo as fi when fi.FieldType = typedefof<string> ->
  let attr = fi.GetMarshalAsAttribute()
  match attr with
    | Some(maa) when maa.Value = UnmanagedType.ByValTStr && maa.SizeConst > 0 ->
        let result = context.Resolve(typedefof<string>).ToString()
        result.Substring(0,Math.Min(result.Length,maa.SizeConst-1)) :> obj
    | _ -> new NoSpecimen() :> obj

if field is string, and it has MarshalAsAttribute we use AutoFixture default string generator, only difference is, if it’s longer than expected we take substring of it, so it’s length would correspond to SizeConst

–1 is used only due to ADS string null terminator.

Arrays length equal to SizeConst

| : ? FieldInfo as fi when fi.FieldType.IsArray -> 
  let attr = fi.GetMarshalAsAttribute()
  match attr with
    | Some(maa) -> 
      let result = Array.CreateInstance(fi.FieldType.GetElementType(), maa.SizeConst)
      Array.Copy(Array.init maa.SizeConst (fun _ -> context.Resolve(fi.FieldType.GetElementType())), result,maa.SizeConst)
      result :> obj
    | _ -> new NoSpecimen() :> obj

again we override default behavior, this time first we create array with length based on SizeConst, then we fill with values generated with AutoFixture (that includes our overrides, if we have nested types)

Marshalling records – dead end

It turns out idea to use records in ADS client instead of structs wasn’t a good idea. I expected some issues, mostly with lack of parameterless constructor, but things turned out worse than expected.

let’s consider simple struct example

[<StructLayout(LayoutKind.Sequential, Pack=1)>]
type SimpleStruct =
  struct
    val byteVar: BYTE
  end
  
[<StructLayout(LayoutKind.Sequential, Pack=1)>]
type ContainingStruct =
  struct
    [<field: MarshalAs(UnmanagedType.Struct)>]
    val nested: SimpleStruct
  end
[<StructLayout(LayoutKind.Sequential, Pack=1)>]
type ContainingStructArray =
  struct
    [<field: MarshalAs(UnmanagedType.ByValArray, SizeConst=10,ArraySubType=UnmanagedType.Struct)>]
    val nested: SimpleStruct array
  end

Assert.Equal(1, typedefof<SimpleStruct> |> Marshal.SizeOf)
Assert.Equal(1, typedefof<ContainingStruct> |> Marshal.SizeOf)
Assert.Equal(10, typedefof<ContainingStructArray> |> Marshal.SizeOf)

all passes like a charm, now let’s rewrite our structs to records

[<StructLayout(LayoutKind.Sequential, Pack=1)>]
type SimpleRecord = {
  byteVar: BYTE
}
  
[<StructLayout(LayoutKind.Sequential, Pack=1)>]
type ContainingRecord = {
  [<field: MarshalAs(UnmanagedType.Struct)>]
  nested: SimpleRecord
}
[<StructLayout(LayoutKind.Sequential, Pack=1)>]
type ContainingRecordArray = {
  
  [<field: MarshalAs(UnmanagedType.ByValArray, SizeConst=10,ArraySubType=UnmanagedType.Struct)>]
  nested: SimpleRecord array
}

Assert.Equal(1, typedefof<SimpleRecord> |> Marshal.SizeOf) //success
Assert.Equal(1, typedefof<ContainingRecord> |> Marshal.SizeOf) //success
Assert.Equal(10, typedefof<ContainingRecordArray> |> Marshal.SizeOf) //failure, actual size is 40

note that it only doesn’t work on array of structs, not sure how it works under the hood, but somehow it always defaults to 4 bytes, as if we used sizeof<SimpleRecord>, thus array of 10 items, each of them 4 bytes.

Solution?

I’m aware of none yet, beside writing custom code to serialize/deserialize records to/from byte arrays. Which itself would be that bad idea (if we ignore performance) as I could handle endianness conversions if by any chance communication with PLC would occur from big endian machine. 

Accessing MarshalAsAttribute

While playing with AutoFixture to verify if F# Records can be marshalled as nicely as Structs, to my surprise I noticed MarshalAsAttribute cannot be accessed by standard GetCustomAttributes method. Turns out that it’s one of pseudo custom attributes that are treated differently by compiler.

Why would I need to access MarshalAsAttribute?

AutoFixture by default generates test data for string as memberName+GUID, which is nice for most cases, but if a filed is marked as having 10chars in string, why provide with incorrect data?

type SampleRecord = {
  [<field: MarshalAs(UnmanagedType.I1)>]boolVar:BOOL
  byteVar: BYTE
  dwordVar: DWORD
  lrealVar: LREAL
  [<field: MarshalAs(UnmanagedType.ByValTStr, SizeConst=81)>]
  string80Var: string
  [<field: MarshalAs(UnmanagedType.ByValTStr, SizeConst=11)>]
  string10Var: string
}

Possible solution would be usage of additional attribute like [<field: StringLength(10)>], but I’m not a big fan of providing same information, over and over again, if I already did it once, plus sometimes I’m stubborn, well, quite often if I’m positive something can be done the way I want it.

How to access MarshalAsAttribute?

If you look into .Net Reference Source you’ll find internal static GetCustomAttribute(RuntimeFieldInfo field) method, which returns the very same MarshalAsAttribute as we annotated our field with. Few pretty self-explanatory lines of code and we’re home.

module Extensions

open System.Reflection
open System.Runtime.InteropServices

let maa = typedefof<MarshalAsAttribute>
  
let GetCustomAttribute = 
  maa.GetMethods(BindingFlags.Static ||| BindingFlags.NonPublic) 
  |> List.ofArray 
  |> List.tryFind (fun mi -> 
    let parameters = mi.GetParameters()
    mi.Name = "GetCustomAttribute" 
    && parameters.Length = 1 
    && parameters.[0].ParameterType.Name = "RuntimeFieldInfo"
    )


type FieldInfo with
  member self.GetMarshalAsAttribute () =
    match GetCustomAttribute with
      | Some(gcaMi) ->
        let attr = gcaMi.Invoke(null, [| self |])
        Some(attr :?> MarshalAsAttribute)
      | None -> None

F# ADS Toolkit | Tools review

Why F#?

For the development of ADS Toolkit I have considered two .NET based languages, F# due to Type Providers, and Nemrele due to powerful metaprogramming capabilities. Unfortunately Nemerle, which is great language, I strongly encourage you to try it, lacks mature support in Visual Studio.

Let’s focus on few F# features that’ll be useful for this particular project.

Type aliases

As mentioned in previous post, TwinCAT implements IEC 61131-3 specification, thus data types naming is different from the one we know from .NET world.  Type aliases are a little thing, yet they make communication a lot simpler, due to allowing both parties (.NET Dev and PLC Dev) usage of same naming convention. Besides, it’s easier, and faster, to say double word instead of  32bit unsigned integer.

Units of Measure and strong typing

Yet another little thing, but it makes values less abstract, plus it makes types safe, if by any chance we’ll by trying to add, or compare,  A (current) to V (voltage) we’ll end up with compile time error. Also, typical solution involves precise measurement thus F# lack of dynamic casting between numeric is another plus.

Type providers

An F# type provider is a component that provides types, properties, and methods for use in your program. – MSDN

I want to skip as much manual work as possible when creating software, call me lazy, thus major part of this project is providing types, required to communicate with PLC, based on *.tpy file generated by TwinCAT.

F# itself

On a first few projects in area of Industrial Automation I used C# exclusively, but recently, say one year ago, I decided to take my chances and try F# for the part of solution responsible for communication with PLC, LIN (Local Interconnect Network – an Automotive network) systems and third party web service. Since that time I cannot imagine myself using non-functional language for such appliances. Time required for development was at least cut 40%, moreover I wasn’t distracted by expressive C-like syntax. As a side note, code metrics is way shorter, yet it’s not a valid criterion to judge software.

Testing

AppVeyor

I always wanted to give those fancy hosted CI build servers a shot. I picked AppVeyor as it’s free for OSS projects (same as Travis-CI), comes with nice badges (again, same as Travis-CI) and is Windows based. Later, if I get to the point at which I replace Beckhoff’s TwinCAT.Ads.dll with own, fully managed .NET client, I’ll be testing it on mono/linux.

FoQ, AutoFixture, xUnit, FsUnit

I admit, I used none of them before, only nUnit, MSTest and JustMock, no wonder I want to try something different, although JustMock has some nice features. DSP is great moment to try new tools, and I’m curious if xUnit is as much extensible as it is described, if AutoFixture saves as much time as its authors boast, and moreover, how all of them play together.

Why I’ll need mocking

As said earlier, I’ll be using AppVeyor, hosted CI build server. It means I can forget about testing against actual TwinCAT instance, whether VM or actual PLC, adding routes to running TwinCAT instance can sometimes be pain in the ass, and I don’t want to check how it can be done on AppVeyor, it probably can’t be done anyway. I’m going to wrap TcAdsClient in  my class (and mock it for test). It’s always good idea to create wrapper for TcAdsClient anyway, as AdsException does not contain information regarding on which particular symbolic/handle operation has failed.

F# ADS Toolkit

Blog registered for “Daj się poznać” contest so I can start countdown until it begins as well as work on project.

To begin with let me explain what I’ll be working on and why I find it useful.

Background

I won’t go into detail what ADS (Automation Device System) is, you can read that at Infosys. What matters is, that it is a protocol that allows you to communicate with TwinCAT. Note that I’ll only focus on access via symbolic variable names.

If we were to read/write, let’s say uint16 variable, it would look like

use client = new TcAdsClient()

//connecto do ADS device
client.Connect(amsNetId,801)

//create variable handle
let myuintHandle = client.CreateVariableHandle ".myuint"

//read value and cast to short
let myuintValue = client.ReadAny(myuintHandle, Operators.typedefof<uint16>) :?> uint16

//write incremented value
client.WriteAny(myuintHandle,myuintValue + 1us)

(* some other logic *)

//remove handle
client.DeleteVariableHandle myuintHandle

//dispose client
client.Dispose()

If you look at highlighted lines, you’ll notice that this code is prone to errors. First of all we pass symbolic name as a string, if we make typo, in runtime we end up with “0x710 symbol not found” exception. Sure we can make it more convenient by wrapping it into generic read methods, but still we have to pay a lot of attention to naming and types. Yep, type names in TwinCAT follow IEC61131-3 specification . Our .NET uint16 is WORD/UINT in TwinCAT, for other types refer to Infosys.

Another tricky thing we can encounter are structures. Sure we can marshal whole structures, but as always, layout is important.

STRUCT
	lrealVal: LREAL := 1.23;
	dintVal1: DINT := 120000;
END_STRUCT
END_TYPE

TYPE TComplexStruct :
STRUCT
	intVal : INT:=1200;
	dintArr: ARRAY[0..3] OF DINT:= 1,2,3,4;
	boolVal: BOOL := FALSE;
	byteVal: BYTE:=10;
	stringVal : STRING(5) := 'hallo';
	simpleStruct1: TSimpleStruct;
END_STRUCT
END_TYPE
[<StructLayout(LayoutKind.Sequential, Pack=1)>]
type SimpleStruct =
  struct
    val lrealVal: double
    val dintVal1: int
  end

[<StructLayout(LayoutKind.Sequential, Pack=1)>]
type ComplexStruct =
  struct
    val intVal: int16
    [<MarshalAs(UnmanagedType.ByValArray, SizeConst=4)>]
    val dintArr: int array
    [<MarshalAs(UnmanagedType.I1)>]
    val boolVal: bool
    val byteVal: byte
    [<MarshalAs(UnmanagedType.ByValTStr, SizeConst=6)>]
    val stringVal: string
    val simpleStruct1: SimpleStruct
  end

Beside basically repeating same code on different sides, imagine situation where PLC developer slightly refactors his code and changes order of fields, no exceptions thrown but invalid values as a result.

Notice highlighted lines, when declaring .NET counterparts, you have to declare length 1 byte longer then TwinCAT due to string null terminator. That’s yet another thing you have to keep in mind when doing things manually.

I could go on with things we can mess up and end up debugging if we do things manually, but above are main problems I aim to address with this project.

First milestone – Convenient syntax and base classes

As a first milestone we should end up with set of base classes that we can extend manually to suit particular project we work on, and use it in convenient way

Second milestone – Type provider based on *.tpy file

Second milestone is to generate ADS client based on *.tpy file, it is generated when TwinCAT project is compiled, at least that’s default setting. It contains custom types definitions and variables. Format is XML based.