I'm using LMDB to store objects serialized by the excellent NESM library. Unfortunately, LMDB wants its data to be of type void*, and my serialized objects are all strings. At first, this wasn't a concern - I just set the data field to

addr myString

Unfortunately though, this doesn't work. I was able to get farther by creating a fixed array of characters and then filling that in with the information in my string like so:

 proc asArray*(s: string, arr: var openarray[char]): void =
   if s.len < arr.len:
     for i, c in s:
       arr[i] = c
     quit("Not enough space available for that!", -1)

but this assumes that whatever array I'm referencing is of a fixed length. While this approach does work, I would rather not create a one size fits all array[5000, char] if most of the time my strings are quite small.

I know there has to be an easy approach, I'm just not familiar enough with Nim and how it interfaces with C to invoke the correct magic via string or cstring alone. Any help would be greatly appreciated!

2017-07-12 22:02:40
pointer is Nim representation of void* in C. (CMIIW)
2017-07-13 02:41:21

say you have a C function:

void store(void* data);

then you can using Nim ffi like this:

proc store(data: pointer) {.importc, dynlib: "shared lib name".}
or pretend that the argument is a cstring

proc store(data: cstring) {.importc, dynlib: "shared lib name".}

how to use the first one?

var mydata = "something"
store(mydata[0].addr) #get the underlying char* address

store(mydata.cstring) # implicit cast from char* to void*

store(cast[pointer](mydata.cstring)) #unnecessary, but ok too

how to use the second one?

var mydata = "something"
store(mydata) #use Nim's automatic conversion from string to cstring

store(mydata.cstring) #unnecessary, but it's ok too.

if your C function is something like:

void store(void* data, int len);

then wrap it like this:

proc c_store(data: pointer, len: cint) {.importc:"store", dynlib: "shared lib name".}

# use ordinary wrapper
proc store(data: string) =
  c_store(data.cstring, data.len.cint)

#or use template to minimize overhead
template store(data: string) =
  c_store(data.cstring, data.len.cint)

#or use inline proc
proc store(data: string) {.inline.} =
  c_store(data.cstring, data.len.cint)

2017-07-13 03:29:45

Wow, that was an excellent response and the simple myString.cstring seems to be doing exactly what I needed. But now I'm looking to turn a void* into a string. To this end I've tried var myString = cast[ptr string](data)[], but this generates a segment fault:

Program received signal SIGSEGV, Segmentation fault.
0x000000000040cf9c in copyString (src=0x1) at /home/steve/nim-0.17.0/lib/system/sysstr.nim:93
93          if (src.reserved and seqShallowFlag) != 0:

I feel like I'm really close to resolving this. If anyone can point me toward what I'm missing, I would really appreciate it!

2017-07-13 15:28:07


var myString = $ cast[cstring](data)

2017-07-13 16:06:03

This solution looks like it may do the trick. Unfortunately, when I had serialized my object to its string format, it had some non-standard content:

0x7ffff6dd4fee: 1       0       0       0       0       0       0       0
0x7ffff6dd4ff6: 42      0       0       0       0       0       96      106
0x7ffff6dd4ffe: 71

Is there a good way (apart from building a sequence up byte by byte and then converting it to a string) to tell Nim that this 17 byte buffer should be read as a string?

2017-07-13 17:07:32

Okay, this seems to work great for converting pointers to Nim strings:

proc fromCString(p: pointer, len: int): string =
  result = newStringOfCap(len)
  var lowLevel = cast[cstring](p)
  for i in 0 ..< len:
    var character: char = lowLevel[i]
    add(result, character)

Many thanks to the individuals who contributed in this thread!

2017-07-13 17:33:19

variations to convert C pointer to Nim string, and perhaps faster too:

proc fromCString(p: pointer, len: int): string =
  result = newString(len)
  for i in 0.. <len:
    result[i] = cast[cstring](p)[i]


proc fromCString(p: pointer, len: int): string =
  result = newString(len)
  copyMem(result.cstring, p, len)

2017-07-14 06:26:01

Wow Jangko, I might just have to use that second example you provided. I can't imagine coming up with code that would do the job more efficiently!

It's kind of funny that in your first example you used cast[cstring](p)[i]. I was trying to get something similar to work for me but must have been slightly off on my syntax. This thread has been a very valuable one for me, and I'm thankful that the community is so knowledgeable and friendly. Thank you!

2017-07-14 22:31:15